An operator I'm building needs to talk to other Kubernetes clusters, are there any best practices on how to do that from within the operator that runs on Kubernetes?
Using k8s.io/client-go/tools/clientcmd package I can call BuildConfigFromFlags method passing masterURL and kubeconfig location. This works fine from outside Kubernetes, but within Kubernetes, can any assumptions be made about kubeconfig location? Or should some other API be used?
As a side note: I'm using controller-runtime's Client API for talking to Kubernetes.
Question: Are these two clusters on GKE or are they managed on other environments by you manually?
A Kubernetes service is basically a collection of pods which can be your controller.
One way to accomplish this is by exposing your service to outside the cluster. On GKE you can do this by using an ILB, ILB will help you expose the service to other k8s clusters you might want to connect to in the same VPC. But if the other services you need to connect to are in some other VPC you should either:
Does this help?
From "Programming Kubernetes", ed. Oreilly (which I suggest as reading):
"When running a binary inside of a pod in a cluster, the kubelet will automatically mount a service account into the container at /var/run/secrets/kubernetes.io/serviceaccount. It replaces the kubeconfig file just mentioned and can easily be turned into a rest.Config via the rest.InClusterConfig() method."
Required config is taken with this line:
cfg, err := rest.InClusterConfig()Take a look at the code here:
https://github.com/programming-kubernetes/cnat/blob/master/cnat-client-go/main.go
Turns out it's quite easy to do, just call the following with the master URL and the token to access it:
cfg, err := clientcmd.BuildConfigFromFlags(os.Getenv("MASTERURL"), os.Getenv("KUBECONFIG"))
cfg.BearerToken = os.Getenv("BEARERTOKEN")It might also require:
cfg.Insecure = trueA kubernetes operator is, in the end, a pod running inside of Kubernetes. Nicola Ben`s answer is the way to go if you want to access the current K8S cluster's API.
However, you're saying your operator "needs to talk to other Kubernetes clusters" (my emphasis), and that it works locally.
Then, one simple approach is to create a ConfigMap containing the kubeconfig file, and mount that to a known location in the operator pod. Then you can simply use that the same way you do it locally.