Kubernetes Engine Cloud SQL proxy Lost connection to MySQL server at 'reading initial communication packet'

3/22/2019

I have Django application deployed on Kubernetes Engine following this tutorial - https://cloud.google.com/python/django/kubernetes-engine. When I'm trying to make database request I'm getting this error:

OperationalError at /admin/login/
(2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0")
Request Method: POST
Request URL:    [MY_URL]
Django Version: 2.1.5
Exception Type: OperationalError
Exception Value:    
(2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0")
Exception Location: /env/lib/python3.6/site-packages/MySQLdb/connections.py in __init__, line 164
Python Executable:  /env/bin/python3
Python Version: 3.6.8
Python Path:    
['/home/vmagent/app',
 '/env/bin',
 '/env/lib/python36.zip',
 '/env/lib/python3.6',
 '/env/lib/python3.6/lib-dynload',
 '/opt/python3.6/lib/python3.6',
 '/env/lib/python3.6/site-packages']
Server time:    Fri, 22 Mar 2019 14:41:35 +0000

There're a lot of similar questions, however, I've tried everything and haven't found exactly my case.

  • I can connect using very same Cloud SQL proxy on my local PC and it works perfectly.
  • I can add 0.0.0.0/0 to Cloud SQL allowed networks and connect directly from Django and it works perfectly.

Here's my django settings for database:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'polls',
        'USER': os.getenv('DATABASE_USER'),
        'PASSWORD': os.getenv('DATABASE_PASSWORD'),
        'HOST': '127.0.0.1',
        'PORT': '3306',
    }
}

And my k8s deployment yaml:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "3"
  creationTimestamp: 2019-03-22T01:02:36Z
  generation: 4
  labels:
    app: polls
  name: polls
  namespace: default
  resourceVersion: "305840"
  selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/polls
  uid: 2ead6fa6-4c3e-11e9-9e47-42010a800102
spec:
  progressDeadlineSeconds: 2147483647
  replicas: 3
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: polls
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: polls
        revision: "1"
    spec:
      containers:
      - env:
        - name: DATABASE_USER
          valueFrom:
            secretKeyRef:
              key: username
              name: cloudsql
        - name: DATABASE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: password
              name: cloudsql
        image: gcr.io/fcul-vova/polls
        imagePullPolicy: Always
        name: polls-app
        ports:
        - containerPort: 8080
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      - command:
        - /cloud_sql_proxy
        - --dir=/cloudsql
        - -instances=fcul-vova:europe-west1:polls-instance=tcp:0.0.0.0:3306
        - -credential_file=/secrets/cloudsql/credentials.json
        image: gcr.io/cloudsql-docker/gce-proxy:1.12
        imagePullPolicy: IfNotPresent
        name: cloudsql-proxy
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /secrets/cloudsql
          name: cloudsql-oauth-credentials
          readOnly: true
        - mountPath: /etc/ssl/certs
          name: ssl-certs
        - mountPath: /cloudsql
          name: cloudsql
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      volumes:
      - name: cloudsql-oauth-credentials
        secret:
          defaultMode: 420
          secretName: cloudsql-oauth-credentials
      - hostPath:
          path: /etc/ssl/certs
          type: ""
        name: ssl-certs
      - emptyDir: {}
        name: cloudsql
status:
  availableReplicas: 3
  conditions:
  - lastTransitionTime: 2019-03-22T01:04:53Z
    lastUpdateTime: 2019-03-22T01:04:53Z
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 4
  readyReplicas: 3
  replicas: 3
  updatedReplicas: 3
-- Володимир Кельбель
cloud-sql-proxy
google-cloud-platform
google-cloud-sql
google-kubernetes-engine

1 Answer

3/23/2019
-- Володимир Кельбель
Source: StackOverflow