K
Q

When creating a node pool (multi-zone) with gcloud, the nodes do not register on the kubernetes master

4/5/2018

When running the following gcloud command on my existing GKE cluster

SCOPES=(
https://www.googleapis.com/auth/compute
https://www.googleapis.com/auth/devstorage.read_write
https://www.googleapis.com/auth/monitoring.write
https://www.googleapis.com/auth/logging.write
https://www.googleapis.com/auth/monitoring
https://www.googleapis.com/auth/pubsub
https://www.googleapis.com/auth/servicecontrol
https://www.googleapis.com/auth/service.management
https://www.googleapis.com/auth/sqlservice.admin
https://www.googleapis.com/auth/trace.append
https://www.googleapis.com/auth/cloud_debugger
https://www.googleapis.com/auth/cloud-platform
)

gcloud beta container node-pools create $POOL_NAME \
--machine-type $MACHINE_TYPE \
--disk-size $DISK_SIZE \
--enable-autorepair \
--enable-autoscaling \
--min-nodes 1 --max-nodes 4 \
--cluster $CLUSTER \
--zone $ZONE \
--num-nodes 1 \
--scopes $(printf ",%s" "${SCOPES[@]}")

(Note this is regional, so using the beta command - I suspect this doesn't happen with non-regional clusters) I get the following error:

    Creating node pool pool-alpha...done.                                                                                                                                                                                                         
ERROR: (gcloud.beta.container.node-pools.create) Operation [<Operation
 endTime: u'2018-03-29T08:56:14.989660264Z'
 name: u'operation-1522311735033-87b12027'
 operationType: OperationTypeValueValuesEnum(CREATE_NODE_POOL, 7)
 selfLink: u'https://container.googleapis.com/v1beta1/projects/xxxxxxxxx/zones/europe-west1-d/operations/operation-1522311735033-87b12027'
 startTime: u'2018-03-29T08:22:15.03391313Z'
 status: StatusValueValuesEnum(DONE, 3)
 statusMessage: u'All cluster resources were brought up, but the cluster API is reporting that only 0 nodes out of 3 have registered. Cluster may be unhealthy.'
 targetLink: u'https://container.googleapis.com/v1beta1/projects/xxxxxxxxxx/zones/europe-west1-d/clusters/digibet-prod/nodePools/pool-alpha'
 zone: u'europe-west1-d'>] finished with error: All cluster resources were brought up, but the cluster API is reporting that only 0 nodes out of 3 have registered. Cluster may be unhealthy.

And indeed, the nodes are created but aren't registered on the cluster. A bug with GKE?

-- Stan Bondi
gcloud
google-cloud-platform
google-kubernetes-engine
kubernetes

Similar Questions

Openshift Admin Token
aks can't create a kubernetes replica set
Kafka Compacted Topic Related Questions
.net core 2.0 logging inside kubernetes pod console

1 Answer

4/5/2018

I reproduced your situation but looks like that is not a GKE issue.

I think the problem is in the $(printf ",%s" "${SCOPES[@]}") function, which creates a list starting from a comma, and it somehow affects available scopes.

I also found that https://www.googleapis.com/auth/cloud_debugger is not in the list of available scopes for node-pools create command. But, based on Cloud Debugger documentation, you can use cloud-platform scope instead of cloud_debugger, and everything will be OK.

So, I checked the command with manually added scopes, it works fine:

gcloud beta container node-pools create $POOL_NAME \
--machine-type $MACHINE_TYPE \
--disk-size $DISK_SIZE \
--enable-autorepair \
--enable-autoscaling \
--min-nodes 1 --max-nodes 4 \
--cluster $CLUSTER \
--zone $ZONE \
--num-nodes 1 \
--scopes logging-write,monitoring-write,service-management,compute-rw,storage-rw,monitoring,pubsub,service-acontrol,service-management,sql-admin,trace,cloud-platform
-- Anton Kostenko
Source: StackOverflow