Is there a point of having pod-level nginx when using nginx ingress?

11/2/2018

I was wondering if I should have the pod level nginx in the implementations below: I was previously using a normal ingress and kube-lego after migrating from VMs and now I am using cert-manager and GKE. My Ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: myapp-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: myapp-static-ip
    kubernetes.io/ingress.class: nginx
    kubernetes.io/ingress.allow-http: "false"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    ingress.kubernetes.io/rewrite-target: /
    certmanager.k8s.io/cluster-issuer: letsencrypt
  namespace: default
spec:
  tls:
  - hosts:
    - myapp.com
    secretName: myapp-crt
  rules:
  - host: 
    http:
      paths:
      - path: /
        backend:
          serviceName: myapp
          servicePort: http

My service:

apiVersion: v1
kind: Service
metadata:
  name: myapp
  labels:
    app: myapp
spec:
  type: NodePort
  ports:
  - port: 80
    targetPort: 80
    nodePort: 32111
    protocol: "TCP"
    name: http
  selector:
    app: myapp

My Deployment

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: myapp
  labels:
    app: myapp
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: gcr.io/myapp-1/myapp:latest
        imagePullPolicy: Always
        env:
            - name: DB_USER
              valueFrom:
                secretKeyRef:
                  name: cloudsql
                  key: username
            - name: DB_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: cloudsql
                  key: password
            -
              name: STATIC_ROOT
              value: https://storage.googleapis.com/myapp-api/static/
            -
              name: STATIC_URL
              value: https://storage.googleapis.com/myapp-api/static/
            -
              name: MEDIA_ROOT
              value: /myapp/media
            -
              name: MEDIA_URL
              value: http://myapp.com/media/

      -
        name: nginx
        image: nginx
        command: [nginx, -g,'daemon off;']

        imagePullPolicy: Always
        volumeMounts:
          -
            name: api-nginx-config
            mountPath: /etc/nginx/nginx.conf
            subPath: nginx.conf
          -
            name: myapp-media
            mountPath: /myapp/media/

        ports:
        - containerPort: 80

      - image: b.gcr.io/cloudsql-docker/gce-proxy:1.05
        name: cloudsql-proxy
        command: ["/cloud_sql_proxy", "--dir=/cloudsql",
                  "-instances=myapp-1:europe-west1:myapp-api=tcp:5432",
                  "-credential_file=/secrets/cloudsql/credentials.json"]
        volumeMounts:
          - name: cloudsql-oauth-credentials
            mountPath: /secrets/cloudsql
            readOnly: true
          - name: ssl-certs
            mountPath: /etc/ssl/certs
          - name: cloudsql
            mountPath: /cloudsql
          - name: myapp-media
            mountPath: /myapp/media

      volumes:
        - name: cloudsql-oauth-credentials
          secret:
            secretName: cloudsql-oauth-credentials
        - name: cloudsql
          emptyDir:
        - name: api-nginx-config
          configMap:
            name: api-nginx-config
        -
          name: myapp-media
          persistentVolumeClaim:
            claimName: myapp-media

my nginx conf:

apiVersion: v1
kind: ConfigMap
metadata:
  name: api-nginx-config
data:
  nginx.conf: |
    events {
      worker_connections  1024;
    }
    http {
      upstream api {
        server 127.0.0.1:8080 fail_timeout=0;
      }
      server {
        access_log /var/log/nginx/http-access.log;
        error_log /var/log/nginx/http-error.log;
        listen 80;
        listen [::]:80;
        server_name myapp.com;

        location /media/ {
        alias   /myapp/media;
        }

       location = /favicon.ico {
        access_log off;
        log_not_found off;
        }

        location / {
          proxy_set_header X-Real-IP $remote_addr;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header Host $host;
          proxy_set_header X-NginX-Proxy true;
          proxy_pass http://127.0.0.1:8080/;
        }
      }
    }

Is it serving any major purpose given I could directly map the myapp/media directly to /media in the volume mount and my tls is handled by ingress. My major concern is with the pod-level nginx as I highlighted earlier, is it useless in this case? Is it just a baggage I am carrying over from previous implementations?

-- ArdentLearner
google-kubernetes-engine
kubernetes
kubernetes-ingress
nginx

2 Answers

11/2/2018

Generally, there is not really a point to have an extra nginx pod. If you do, you would have something of a double ingress. An nginx ingress controller pod already has nginx in it and you can scale that up/down.

One reason you would want to keep it is for backward compatibility, if for example, you want to use an ingress but want to gradually roll it out in this sort of fashion: create new nginx ingress -> flip traffic from your own nginx only through the new nginx ingress and your own nginx, until you have flipped all your pods -> remove your own nginx gradually until you have removed them all.

Another reason is to support a very specific nginx configuration that is not supported by the nginx ingress controller yet.

-- Rico
Source: StackOverflow

11/3/2018

You may need to run your own nginx as deployment , for the reasons listed in the above answer , plus , you may need to scale the nginx deployment , let say 10 replicas. you cant scale ingress like that. But in any case , you just need one of them.

-- Ijaz Ahmad Khan
Source: StackOverflow