The biggest problem I understand here is managing IPs. The rails pods IPs will be dynamic so will not be able to whitelist them. So how do I get Rails access to the Google Cloud SQL database in a secure way without knowing the IPs of the Rails containers?
You can run the Cloud SQL Proxy in your pod which will allow you to connect to Cloud SQL via a local UNIX socket: https://github.com/GoogleCloudPlatform/cloudsql-proxy#to-use-from-kubernetes