Kubernetes on AWS: Preserving Client IP with nginx-ingress + cert-manager

12/20/2018

We have setup Kubernetes with nginx-ingress combined with cert-manager to automatically obtain and use SSL certificates for ingress domains using LetsEncrypt using this guide: https://medium.com/@maninder.bindra/auto-provisioning-of-letsencrypt-tls-certificates-for-kubernetes-services-deployed-to-an-aks-52fd437b06b0. The result is that each Ingress defines its own SSL certificate that is automatically provisioned by cert-manager.

This all works well but for one problem, the source IP address of the traffic is lost to applications in Pods.

There is an annotation that is advised to use to apply to the nginx-ingress controller service service.beta.kubernetes.io/aws-load-balancer-backend-protocol: '*'. This has the effect of preserving source IP addresses. However, doing it breaks SSL:

An error occurred during a connection to {my.domain.com}. SSL received a record that exceeded the maximum permissible length. Error code: SSL_ERROR_RX_RECORD_TOO_LONG

My head is starting to spin. Does anyone know of any approaches to this (it seems to me that this would be a common requirement)?

Ingress configuration:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-http-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
spec:
  rules:
  - host: my.host.com
    http:
      paths:
      - path: /
        backend:
          serviceName: my-http-service
          servicePort: 80
  tls:
  - hosts:
    - "my.host.com"
    secretName: malcolmqa-tls
-- dom_watson
amazon-eks
kubernetes
ssl

2 Answers

12/28/2018

As @dom_watson mentioned in the comments, adding parameter controller.service.externalTrafficPolicy=Local to Helm install configuration solved the issue due to the fact that Local value preserves the client source IP, thus the network traffic will reach target Pod in Kubernetes cluster. Find more information in the official Kubernetes guidelines.

-- mk_sta
Source: StackOverflow

7/29/2019
helm upgrade my-nginx stable/nginx-ingress --set rbac.create=true --set controller.service.externalTrafficPolicy=Local
-- user6625186
Source: StackOverflow