EDIT:

I now think the issue is with my Golang pod communicating with the proxy pod via localhost, as in the second error message.

I added the service account credentials JSON file to my Docker image's GOOGLE_APPLICATION_CREDENTIALS environment variable. After doing that, using my-project:us-central1:my-instance as connName below works.

However, when I try using the DB_HOST environment variable in the container as connName, I still get the 404 error below.

ORIGINAL POST

I'm following this guide to connect to Google Cloud SQL from a pod on Kubernetes Engine. The pod is running two containers: one with the Cloud SQL proxy image and another with a Golang service to do the actual database queries.

I'm getting the following error when my Golang service tries to initiate a connection:

ensure that the Cloud SQL API is enabled for your project (https://console.cloud.google.com/flows/enableapi?apiid=sqladmin). Error during createEphemeral for my-project:us-central1:my-instance: googleapi: Error 403: Insufficient Permission, insufficientPermission

I've looked at a few threads here and elsewhere and here's what I've done so far:

• Ensured the Cloud SQL API is in fact enabled.
• Added the Editor role to the service account I'm using.
• Removed and re-added the Cloud SQL Client role on the service account I'm using.
• Verified the correct secrets were created, with the same namespace as the pods.

Below is a snippet of the Golang code I'm using, which was taken from here:

cfg := mysql.Cfg(connName, dbUser, dbPassword)
cfg.DBName = dbName
db, err := mysql.DialCfg(cfg)

if err != nil {
    log.Println(err)
    return c.NoContent(http.StatusInternalServerError)
}

connName is the same string that shows up in the error: my-project:us-central1:my-instance. I've tried changing that to 127.0.0.1:3306 instead, but I then get this error below:

ensure that the account has access to "127.0.0.1:3306" (and make sure there's no typo in that name). Error during createEphemeral for 127.0.0.1:3306: googleapi: got HTTP response code 404 with body: Not Found

Also, here is a snippet of the yaml file I'm using to deploy the pods.

    env: 
    - name: DB_HOST
      value: 127.0.0.1:3306
    - name: DB_USER
      valueFrom:
        secretKeyRef:
          name: cloudsql-db-credentials
          key: username
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: cloudsql-db-credentials
          key: password
  - name: cloudsql-proxy
    image: gcr.io/cloudsql-docker/gce-proxy:1.11
    command: ["/cloud_sql_proxy",
              "-instances=my-project:us-central1:my-instance=tcp:3306",
              "-credential_file=/secrets/cloudsql/credentials.json"]
    volumeMounts:
      - name: cloudsql-instance-credentials
        mountPath: /secrets/cloudsql
        readOnly: true
  volumes:
    - name: cloudsql-instance-credentials
      secret:
        secretName: cloudsql-instance-credentials

I've also verified that the Cloud SQL proxy starts without issue:

2018/04/21 20:41:19 Listening on 127.0.0.1:3306 for my-project:us-central1:my-instance
2018/04/21 20:41:19 Ready for new connections

I'm not sure what else to try here. Any help is appreciated.