gcloud credentials using wrong cluster

2/20/2019

I'm using container builder to with a cloudbuild.yaml, my problem an old cluster name is being used which doesn't exist. I have tried deleting my service key and creating it again to no avail.

Starting Step #3
Step #3: Already have image (with digest): gcr.io/cloud-builders/kubectl
Step #3: Running: gcloud container clusters get-credentials --project="amx-instance-1" --zone="australia-southeast1-a" "amx-cluster-au9"
Step #3: Fetching cluster endpoint and auth data.
Step #3: ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission(s) for "projects/amx-instance-1/zones/australia-southeast1-a/clusters/amx-cluster-au9". See https://cloud.google.com/kubernetes-engine/docs/troubleshooting#gke_service_account_deleted for more info.

Cluster name amx-cluster-au9 is an old cluster that no longer exist. What is causing this issue and how can I fix it?

edit: cloudbuild.yaml file

steps:
- name: gcr.io/cloud-builders/wget
  args: [
    "-O",
    "go-cloud-debug",
    "https://storage.googleapis.com/cloud-debugger/compute-go/go-cloud-debug"
  ]
- name: 'gcr.io/cloud-builders/go'
  args: ["install", "-gcflags=-N", "-gcflags=-l", ".", ]
  env: ['PROJECT_ROOT=github.com/amalexpress/amx-server', 'CGO_ENABLED=0', 'GOOS=linux']
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '--tag=gcr.io/$PROJECT_ID/amx-img:$SHORT_SHA', '.']
- name: 'gcr.io/cloud-builders/kubectl'
  args:
  - set
  - image
  - deployment
  - echoserver
  - echoserver=gcr.io/$PROJECT_ID/amx-img:$SHORT_SHA
  env:
  - 'CLOUDSDK_COMPUTE_ZONE=australia-southeast1-a'
  - 'CLOUDSDK_CONTAINER_CLUSTER=amx-cluster-au-2'
images: ['gcr.io/$PROJECT_ID/amx-img:$SHORT_SHA']

Basically I don't know why it keeps referencing a cluster that I've deleted and no longer use.

Here are the logs if it might help.

-- robertson
gcloud
google-container-builder
google-kubernetes-engine

1 Answer

2/21/2019

The issue was resolved by deleting the repository in Cloud Source Repository. I have no idea why this fixed the issue. I should note that I deleted the github repo and re-initialised it. Still seems like a bug though as there is no indication of the root cause at all. Furthermore while the above fixed the cluster name issue, I had to follow instructions here to to give proper role access to the cluster:

PROJECT="$(gcloud projects describe \
    $(gcloud config get-value core/project -q) --format='get(projectNumber)')"

gcloud projects add-iam-policy-binding $PROJECT \
    --member=serviceAccount:$PROJECT@cloudbuild.gserviceaccount.com \
    --role=roles/container.developer
-- robertson
Source: StackOverflow