We have 2 clusters on GKE, a staging and production cluster. We setup our production cluster to use cloud-sql-proxy as a pod attached to our app pods to connect to the cloud sql servers.
When we stood up the staging cluster, everything worked except that our apps cloud-sql proxies are throwing this error, all the secrets and associated paths have been changed for the staging cluster.
couldn't connect to " project-name:us-central1:name-db-staging": x509: certificate is valid for project-name:name-db, not project-name:name-db-staging
This is happening for two separate projects, both are saying the production ssl cert is being used, but the db it is trying to connect to is the staging server
The workaround for this was to run this
gcloud sql instances reset-ssl-config instance-name
and then destroy the pods and let the deployment bring new ones back up.