Unable to initiate Kubernetes Federation in Google Cloud Platform

4/11/2018

I'm trying to setup kubernetes federation. When I run the following commands I get an error:

# Set the project and domain name
export PROJECT=abc-9102-1sf
export DNS_ZONE=echo.examplefed.com.

# Create the clusters
gcloud container clusters create west-cluster --zone us-west1-c --scopes "cloud-platform,storage-ro,logging-write,monitoring-write,service-control,service-management,https://www.googleapis.com/auth/ndev.clouddns.readwrite" --preemptible &
gcloud container clusters create east-cluster --zone us-east1-b --scopes "cloud-platform,storage-ro,logging-write,monitoring-write,service-control,service-management,https://www.googleapis.com/auth/ndev.clouddns.readwrite" --preemptible &

# Workaround for RBAC error
# https://github.com/kubernetes/kubernetes/issues/42559
gcloud config set container/use_client_certificate True
export CLOUDSDK_CONTAINER_USE_CLIENT_CERTIFICATE=True

# Get credentials
gcloud container clusters get-credentials west-cluster --zone=us-west1-c
gcloud container clusters get-credentials east-cluster --zone=us-east1-b

# Aliases
kubectl config set-context east --cluster=gke_${PROJECT}_us-east1-b_east-cluster --user=gke_${PROJECT}_us-east1-b_east-cluster
kubectl config set-context west --cluster=gke_${PROJECT}_us-west1-c_west-cluster --user=gke_${PROJECT}_us-west1-c_west-cluster

# Initiate Federation. After this step I get the error.
kubefed init kfed  --host-cluster-context=east  --dns-zone-name=${DNS_ZONE}  --dns-provider=google-clouddns

Creating a namespace federation-system for federation system components...Error from server (Forbidden): namespaces is forbidden: User "client" cannot create namespaces at the cluster scope: Unknown user "client".

The versions for kubefed and kubectl are 1.8.8:

gcp@user:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.8", GitCommit:"2f73858c9e6ede659d6828fe5a1862a48034a0fd", GitTreeState:"clean", BuildDate:"2018-02-09T21:30:57Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"8+", GitVersion:"v1.8.8-gke.0", GitCommit:"6e5b33a290a99c067003632e0fd6be0ead48b233", GitTreeState:"clean", BuildDate:"2018-02-16T18:26:58Z", GoVersion:"go1.8.3b4", Compiler:"gc", Platform:"linux/amd64"}
gcp@user:~$ kubefed version
Client Version: version.Info{Major:"1", Minor:"8", GitVersion:"v1.8.8", GitCommit:"2f73858c9e6ede659d6828fe5a1862a48034a0fd", GitTreeState:"clean", BuildDate:"2018-02-09T21:30:57Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"8+", GitVersion:"v1.8.8-gke.0", GitCommit:"6e5b33a290a99c067003632e0fd6be0ead48b233", GitTreeState:"clean", BuildDate:"2018-02-16T18:26:58Z", GoVersion:"go1.8.3b4", Compiler:"gc", Platform:"linux/amd64"}
-- NEO
google-cloud-platform
google-kubernetes-engine
kubectl
kubernetes

1 Answer

4/11/2018

After spending several hours on this I have managed to fix it with the following command. Note that it must be executed for each context you have:

kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin --user $(gcloud config get-value account) --context east

kubectl create clusterrolebinding cluster-admin-binding \
--clusterrole cluster-admin --user $(gcloud config get-value account) --context west

And DO NOT run this:

# Workaround for RBAC error
# https://github.com/kubernetes/kubernetes/issues/42559
gcloud config set container/use_client_certificate True
export CLOUDSDK_CONTAINER_USE_CLIENT_CERTIFICATE=True
-- NEO
Source: StackOverflow