I'm working on a Python(3.6) project in which I have implemented Google cloud apis, I have setup credentials for Google Cloud via service account. Here's what I have tried:
Obtain Credentials:
def prepare_credentials(cred_file):
from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_file(cred_file)
return credentials
And then I have tried to run gcloud container clusters get-credentials $CLUSTER_NAME --zone $ZONE --project $PROJECT
as below:
Run gcloud command via Python's Subprocess:
print(subprocess.call(
'gcloud container clusters get-credentials ' + data['cluster_name'] + ' --zone '
+ data['zone_region']
+ ' --project ' + data['project_id'],
shell=True))
it returns this error:
Fetching cluster endpoint and auth data.
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission for "projects/brain-183103/zones/europe-west1-d/clusters/numpy".
1
W0629 04:12:00.776926 2222 factory_object_mapping.go:423] Failed to download OpenAPI (Get https://104.197.10.169/swagger-2.0.0.pb-v1: dial tcp 104.197.10.169:443: i/o timeout), falling back to swagger Unable to connect to the server: dial tcp 104.197.10.169:443: i/o timeout
Service Account credentials are working well as I'm using these credentials for various other Google Cloud API calls and also I have set the owner
permission to the project for this service_account.
Have I configured something wrong?
Help me, please!
Thank You, Abdul
This approach won't work.
Python's subprocess
provides a way to run e.g. gcloud
commands and communicate between your Python process and e.g. gcloud
using I/O (and error) pipes.
In your code, you show the creation of a Python credential
object representing your service account and then your run gcloud
in a sub-process. There's no credential sharing between these two processes and so gcloud
is using whatever it has in context, probably the user's $HOME/.config/gcloud
.
It's further complicated by the fact that gcloud container clusters get-credentials
performs a bunch of magic with the user's gcloud
credentials and the cluster specified, to configure $HOME/.kube/config
so that subsequent kubectl commands authenticate using Open ID Connect.
Given that you're using a service account, I assume, because this requires distributing the service account's key to authenticate, that you don't intend to have many|dynamic copies of your code performing this authentication. If that's correct, then you may wish to consider performing the gcloud container cluster get-credentials
outside of (and before) your Python code (either manually or via a shell script) and then (once you have a correctly configured $HOME/.kube/config
for your cluster), use Kubernetes' Python client (!) to perform whatever work you'd like to do on the cluster.
Although not a solution to this problem specifically, I recommend you use Application Default Credentials (ADCs) in your code. ADCs result in one way to authenticate regardless of whether your code runs off-or-on Google Cloud Platform and regardless of whether you use user or service accounts.
Here's a Python sample using the Cloud Client Library and Application Default Credentials. The library does not (appear) to solve the challenge in effectively converting GCP service account credentials into credentials (via ~/.kube/config) that you may use to access the Kubernetes cluster. But, you can apply operations to clusters using this library:
requirements.txt:
google-cloud-container==0.1.1
python.py:
from google.cloud import container_v1
client = container_v1.ClusterManagerClient()
project_id = "[[YOUR-PROJECT-ID]]"
zone = "[[YOUR-ZONE]]"
cluster_id = "[[YOUR-CLUSTER-ID]]"
cluster = {
"name": cluster_id,
"initial_node_count": 1,
"master_auth":{
#"username": "admin",
"client_certificate_config": {
},
},
#logging_service = "logging.googleapis.com",
#monitoring_service = "monitoring.googleapis.com",
"initial_cluster_version": "1.10.5-gke.3",
}
response = client.create_cluster(project_id, zone, cluster)
response = client.list_clusters(project_id, zone)
response = client.delete_cluster(project_id, zone, cluster_id)
NB username is not present in the dict to disable username|password. client_certificate_config
is empty ({}
) to not issue a client cert. You may wish to change these. I think there's a bug in the JSON response of the create cluster too that I'll report.