Istio Ingress Controller does not preserves client original IP

6/12/2018

We are using Istio 0.7 with istio ingress controller to handle outside world's request. We use "externalIPs" property of the "ClusterIP" type of service to expose ingress controller outside the kubernetes.

Istio Ingress Controller Yaml:

apiVersion: v1
kind: Service
metadata:
  name: istio-ingress-3
  namespace: istio-system
  labels:
    istio: ingress
spec:
  ports:
  - port: 80
    name: http
  - port: 443
    name: https
  externalIPs:
  - 192.168.X.X
  selector:
    istio: ingress-3
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: istio-ingress-3
  namespace: istio-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        istio: ingress-3
      annotations:
        sidecar.istio.io/inject: "false"
    spec:
      nodeSelector:
        kubernetes.io/hostname: node3.example.com
      serviceAccountName: istio-ingress-service-account
      containers:
      - name: istio-ingress
        image: docker.io/istio/proxy:0.7.1
        args:
        - proxy
        - ingress
        - --discoveryAddress
        - istio-pilot:8080 #--controlPlaneAuthPolicy
        - --discoveryRefreshDelay
        - '1s' #discoveryRefreshDelay
        - --drainDuration
        - '45s' #drainDuration
        - --parentShutdownDuration
        - '1m0s' #parentShutdownDuration
        - --connectTimeout
        - '10s' #connectTimeout
        - --serviceCluster
        - istio-ingress
        - --zipkinAddress
        - zipkin:9411
        - --statsdUdpAddress
        - istio-mixer:9125
        - --proxyAdminPort
        - "15000"
        - --controlPlaneAuthPolicy
        - NONE #--controlPlaneAuthPolicy
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
        - containerPort: 443
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        volumeMounts:
        - name: istio-certs
          mountPath: /etc/certs
          readOnly: true
        - name: ingress-certs
          mountPath: /etc/istio/ingress-certs
          readOnly: true
      volumes:
      - name: istio-certs
        secret:
          secretName: istio.istio-ingress-service-account
          optional: true
      - name: ingress-certs
        secret:
          secretName: istio-ingress-certs
          optional: true

The issue is every time when we check the logs of the ingress controller after firing the request it gives the same ip that we have given in the external ip (host ip). But we want the client original information like client original IP

What you expected to happen:

We want the original client IP rather then the IP given in the externalIP ie (192.168.X.X).

Additional Information

I have checked on the internet and found that using "externalTrafficPolicy: Local" flag we can preserve the client ip, but this flag is only valid in the NodePort type services. And we do not want to use NodePort service, because if we use it it will open our ingress controller for all the interface for the host that is private as well as public.

Kubernetes version

Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.1", GitCommit:"d4ab47518836c750f9949b9e0d387f20fb92260b", GitTreeState:"clean", BuildDate:"2018-04-12T14:26:04Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3+coreos.0", GitCommit:"f1b890dbbf11abe58280b3ffe17d67749f5ae70e", GitTreeState:"clean", BuildDate:"2018-05-21T17:27:17Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

OS (e.g. from /etc/os-release)

NAME="Container Linux by CoreOS" ID=coreos VERSION=1745.5.0 VERSION_ID=1745.5.0 BUILD_ID=2018-05-31-0701 PRETTY_NAME="Container Linux by CoreOS 1745.5.0 (Rhyolite)" ANSI_COLOR="38;5;75" HOME_URL="https://coreos.com/" BUG_REPORT_URL="https://issues.coreos.com" COREOS_BOARD="amd64-usr"

Kernel (e.g. uname -a)

Linux node1.example.com 4.14.44-coreos-r1 #1 SMP Thu May 31 06:04:02 UTC 2018 x86_64 Intel(R) Xeon(R) CPU L5640 @ 2.27GHz GenuineIntel GNU/Linux

Can somebody help us?

-- Ronak Pandya
istio
kubernetes
kubernetes-ingress

1 Answer

6/13/2018

Based on this GitHub issue it appears that Istio is smart enough to include the X-Forwarded-For header in its upstream requests, so I would expect the destination Pod needs to be updated to consult that header rather than $HTTP_REMOTE or whatever mechanism it is using to obtain the IP now.

-- mdaniel
Source: StackOverflow