High latency using gunicorn with django in EKS (Kubernetes) and ELB

6/2/2019

I'm running DjangoRestAPI in a Kubernetes and notice some really poor performance when I use gunicorn on the same deployment to serve the DjangoRestAPI. On average it takes ~6000ms for a simple HttpResponse. Without gunicorn, serving by python manage.py runserver, the same request would only take 50ms.

Here is my deployment for djangoRestAPI and gunicorn by running the command gunicorn api.wsgi --bind 0.0.0.0:8000

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: djangoapi
    type: web
  name: djangoapi
  namespace: "default"
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: djangoapi
        type: web
    spec:
      containers:
      - name: djangoapi
        image: <repo>/app:v0.9a
        imagePullPolicy: Always
        args:
        - gunicorn
        - api.wsgi
        - --bind
        - 0.0.0.0:8000
        envFrom:
        - configMapRef:
            name: djangoapi-config
        ports:
        - containerPort: 8000
      imagePullSecrets:
        - name: regcred
      restartPolicy: Always

---
apiVersion: v1
kind: Service
metadata:
  name: djangoapi-svc
  namespace: "default"
  labels:
    app: djangoapi
spec:
  ports:
  - port: 8000
    protocol: TCP
    targetPort: 8000
  selector:
    app: djangoapi
    type: web
  type: NodePort  

If I replace the arguments with the following, the response time immediately drops down to 50ms

        - python
        - manage.py
        - runserver
        - 0.0.0.0:8000

Just in case, here are my yaml files for ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: api-ingress
  namespace: "default"
  labels:
    app: api-ingress
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:eu-west-1:<>:certificate/<>
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/healthcheck-interval-seconds: '15'
    alb.ingress.kubernetes.io/healthcheck-timeout-seconds: '8'
    alb.ingress.kubernetes.io/healthy-threshold-count: '2'
    alb.ingress.kubernetes.io/healthcheck-path: '/'
    alb.ingress.kubernetes.io/healthcheck-protocol: HTTPS
    alb.ingress.kubernetes.io/success-codes: "200"
spec:
  rules:
  - host: test.mydomain.com
    http:
      paths:
      - path: /*
        backend:
          serviceName: echoheaders
          servicePort: 80
  - host: dev.mydomain.com
    http:
      paths:
      - path: /*
        backend:
          serviceName: djangoapi-svc
          servicePort: 8000

and my alb-ingress-controller is based upon https://github.com/kubernetes-sigs/aws-alb-ingress-controller

I'm wondering if the way that I'm deployment gunicorn is wrong or if there's any other way to resolve the latency issue.

-- Ivan
amazon-eks
amazon-elb
gunicorn
kubernetes

0 Answers