I'm trying to create a Kubernetes Cluster via the NodeJS client on google App engine. The Kubernetes cluster is on a separate project to where the app engine project is hosted, say "my-node-project" & "my-k8-project".

"my-node-project" has the relevant service account(Owner level access) for the kubernetes project.

I make the cluster create call as follows:

var client = new container.v1.ClusterManagerClient({
    projectId: projectId,
    key: serviceAccount
});

var zone = 'us-central1-b';

var password = "<some password>";

var clusterConfig = {
        "name": clusterName,
        "description": "api created cluster",
        "initialNodeCount": 3,
        "nodeConfig": {
                "oauthScopes": [
                    "https://www.googleapis.com/auth/compute",
                 "https://www.googleapis.com/auth/devstorage.read_only"
                ]
        }
        ,
        "masterAuth": {
            "username": "admin",
            "password": password
        },
        "zone": zone
    };

var request = {
projectId: projectId,
zone: zone,
cluster: clusterConfig,
};
return client.createCluster(request)
.then(responses => {
    var response = responses[0];
    console.log("response: ", response);
    return response;
})
.catch(err => {
    console.error(err);
    return err;
});

In the above code the serviceAccount variable is a json object containing the service account, with all the private key, project id fields etc.

The strange thing is that when I run the code locally, i.e. call the endpoint that runs the above function, the request goes through just fine, i.e. the clusters are created and I can even add workloads via the api.

However, after I deploy the nodejs project to app engine standard and call the same endpoint running on app engine, I get the error:

Error: 7 PERMISSION_DENIED: Required "container.clusters.create" permission(s) for "projects/my-k8-project". See https://cloud.google.com/kubernetes-engine/docs/troubleshooting#gke_service_account_deleted for more info. at Object.exports.createStatusError (/srv/node_modules/grpc/src/common.js:91:15) at Object.onReceiveStatus (/srv/node_modules/grpc/src/client_interceptors.js:1204:28) at InterceptingListener._callNext (/srv/node_modules/grpc/src/client_interceptors.js:568:42) at InterceptingListener.onReceiveStatus (/srv/node_modules/grpc/src/client_interceptors.js:618:8) at callback (/srv/node_modules/grpc/src/client_interceptors.js:845:24) code: 7, metadata: Metadata { _internal_repr: {} }, details: 'Required "container.clusters.create" permission(s) for "projects/my-k8-project". See https://cloud.google.com/kubernetes-engine/docs/troubleshooting#gke_service_account_deleted for more info.' }

Since I got that troubleshooting link, I tried to create a new service account and use that. In addition I tried disabling and enabling the both the kubernetes and compute APIs. I also tried to place the service account in the root directory of the project and refer to the service account that way.

Unfortunately everything I tried resulted in exactly the same error. But still worked when running from localhost.

Is there a whitelist somewhere I'm missing? Perhaps localhost is whitelisted by default and "my-node-project" app engine project isn't on the list?

Any tips, hints or pointing in the right direction would be very much appreciated.