Is it possible To scale down a kubernetes jobs parallelism value but leave any running jobs up until they have run to completion.

I have a specific scenario where I am running vsts build agents in containers and need to design the solution in such a way that there is always one agent available at all times. Scaling up is easy as this can just be done at the start of every build however because I need to clean up the agent before the pod finishes I cant scale down in the same way at the end of the build. If there were a way to set the target parallelism but not delete any pods until one completed this would be perfect but any other way for me to achieve this or pointers on where to look would be great.