Kubernetes cluster disappeared from Google Container Engine

2/15/2017

I've been running an app on Kubernetes very smoothly for the past few weeks, until today I found that my Kubernetes master is no longer available.

But the application is still up, seemingly because the cluster's nodes are still available in Google Compute Engine:

gcloud compute engine nodes

(We don't use any compute instances aside from our K8s cluster.)

My gcloud CLI has all the right configuration:

$ gcloud config list
Your active configuration is: [default]

[compute]
...
[container]
use_client_certificate = True
[core]
account = {email}
disable_usage_reporting = True
project = {myproject}

Even my kubectl CLI is configured to use the cluster that once existed:

$ kubectl config current-context
gke_{myproject}_us-central1-a_{clustername}

But any kubectl call results in a timeout.

$ kubectl get pods
Unable to connect to the server: dial tcp {ipv4 address}:443: i/o timeout

gcloud and the Google Container Engine web UI both agree that there are no clusters:

$ gcloud container clusters list
*crickets*
-- glifchits
google-cloud-platform
google-kubernetes-engine
kubernetes

1 Answer

3/1/2017

I got in touch with Google Support to solve this problem. The cluster's master instance was erroneously marked as abusive, and the master was taken down. Google was able to bring it back up. After that, there was a process to return the cluster to its normal state.

I suppose the solution here is to contact support.

-- glifchits
Source: StackOverflow