I'm currently running two Kubernetes clusters one on Google cloud and one on IBM cloud. To manage them I use kubectl. I've made a script that executes some commands on one of the clusters then switches to the other and does some other work there. This works fine as long as the script only runs in one process, however when run in parallel the credentials are sometimes overwritten by one process when in use by another and this obviously causes issues.
I therefore want to know if I can supply kubectl with a credentials file for every call, instead of storing it in a environmental variable with kubectl config set-credentials.
Any help/solution is much appreciated.
You need to use Kubedef in order to manage multiple clusters.
It will take one cluster as the main one, and execute all the same requests to the second cluster.
If I need to work with multiple clusters using kubectl I am splitting my terminal and setting KUBECONFIG for each split: For my first split:
export KUBECONFIG=~/.kube/cluster1For the second split
export KUBECONFIG=~/.kube/cluster2It is working pretty well, but this approach has one issue: If you are using some kind of prompt with the current Kubernetes context it will give you different output and it might be missing leading.
For scripts, I am just changing value of KUBECONFIG in for loop, to loop over each cluster.