I have a GKE setup running KubeFlow on the latest versions with Kustomize. The master TFJob pulls a Docker image of the full model and runs it. I'm running into a simple issue where I wish to increase the amount of CPU usage but can't seem to do it.
Is this something I need to specify in the Docker model code, in the Kubernetes configuration, in the TFJob yamls, or somewhere else?
I've tried specifying a minimum size & limit for cpu resources for the container. I also tried specifying those for the master pod but it doesn't seem to update.
There also isn't much documentation or many similar questions on this.
Just refer to Customizing the TFJob
And yes, it is set in your TFJob yaml. the code snippets:
resources:
limits:
cpu: '1'