Getting "Insufficient Permission" error while storing data into GCS from docker container in kubernetes engine

5/15/2020

I have configured gcloud with the project, service account and other configurations. Enabled google api and runs the application on GKE clusters.

I am getting the following error logs,

Exception in thread "Thread-10" com.google.cloud.storage.StorageException: Insufficient Permission
    at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:227)
    at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:308)
    at com.google.cloud.storage.StorageImpl$3.call(StorageImpl.java:203)
    at com.google.cloud.storage.StorageImpl$3.call(StorageImpl.java:200)
    at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
    at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
    at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
    at com.google.cloud.storage.StorageImpl.internalCreate(StorageImpl.java:199)
    at com.google.cloud.storage.StorageImpl.create(StorageImpl.java:161)
    at java.lang.Thread.run(Thread.java:748)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
POST https://storage.googleapis.com/upload/storage/v1/b/my-bucket/o?projection=full&uploadType=multipart
{
  "code" : 403,
  "errors" : [ {
    "domain" : "global",
    "message" : "Insufficient Permission",
    "reason" : "insufficientPermissions"
  } ],
  "message" : "Insufficient Permission"
}
    at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:555)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:475)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:592)
    at com.google.cloud.storage.spi.v1.HttpStorageRpc.create(HttpStorageRpc.java:305)
    ... 12 more
-- SST
gcloud
google-kubernetes-engine

1 Answer

5/15/2020

IIUC, have a look at this:

https://cloud.google.com/kubernetes-engine/docs/tutorials/authenticating-to-cloud-platform

You need to create a Kubernetes (!) secret representing the GCP service account credentials.

You then volume-mount the secret into the containers that need to use the secret and set GOOGLE_APPLICATION_CREDENTIALS referencing the account in the container's environment.

-- DazWilkin
Source: StackOverflow