Trying to set up an ingress with tls and open to some IPs only on GKE

1/15/2019

I'm having trouble setting up an ingress open only to some specific IPs, checked docs, tried a lot of stuff and an IP out of the source keep accessing. that's a Zabbix web interface on an alpine with nginx, set up a service on node-port 80 then used an ingress to set up a loadbalancer on GCP, it's all working, the web interface is working fine, but how can I make it accessible only to desired IPs? my firewall rules are ok and it's only accessible through load balancer IP

Also, I have a specific namespace for this deploy.

Cluster version 1.11.5-gke.5 EDIT i'm using GKE standard ingress GLBC

My template is config as follow can someone help enlighten me on what is missing:

    apiVersion: v1
    kind: ReplicationController
    metadata:
      name: zabbix-web
      namespace: zabbix-prod
      labels:
        app: zabbix
        tier: frontend
    spec:
      replicas: 1
      template:
        metadata:
          labels:
            name: zabbix-web
            app: zabbix
        spec:
          volumes:
          - name: cloudsql-instance-credentials
            secret:
              defaultMode: 420
              secretName: cloudsql-instance-credentials
          containers:
            - command:
              - /cloud_sql_proxy
              - -instances=<conection>
              - -credential_file=/secrets/cloudsql/credentials.json
              image: gcr.io/cloudsql-docker/gce-proxy:1.11
              imagePullPolicy: IfNotPresent
              name: cloudsql-proxy
              resources: {}
              securityContext:
                allowPrivilegeEscalation: false
                runAsUser: 2
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
              volumeMounts:
              - mountPath: /secrets/cloudsql
                name: credentials
                readOnly: true
            - name: zabbix-web
              image: zabbix/zabbix-web-nginx-mysql:alpine-3.2-latest
              ports:
              - containerPort: 80
              env:
              - name: MYSQL_USER
                valueFrom:
                  secretKeyRef:
                    key: <user>
                    name: <user>
              - name: MYSQL_PASSWORD
                valueFrom:
                  secretKeyRef:
                    key: <pass>
                    name: <pass>
              - name: DB_SERVER_HOST
                value: 127.0.0.1
              - name: MYSQL_DATABASE
                value: <db>
              - name: ZBX_SERVER_HOST
                value: <db>
            readinessProbe:
              failureThreshold: 3
              httpGet:
                path: /index.php
                port: 80
                scheme: HTTP
              periodSeconds: 10
              successThreshold: 1
              timeoutSeconds: 30
---
apiVersion: v1
kind: Service
metadata:
  name: "zabbix-web-service"
  namespace: "zabbix-prod"
  labels:
    app: zabbix
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    name: "zabbix-web"
  type: "NodePort"
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: zabbix-web-ingress
  namespace: zabbix-prod
  annotations:
    ingress.kubernetes.io/service.spec.externalTrafficPolicy: local
    ingress.kubernetes.io/whitelist-source-range: <xxx.xxx.xxx.xxx/32>
spec:
  tls:
  - secretName: <tls-cert>
  backend:
    serviceName: zabbix-web-service
    servicePort: 80
-- Liz Neto
containers
google-cloud-platform
google-kubernetes-engine
kubernetes
kubernetes-ingress

3 Answers

1/16/2019

AFAIK, you can't restrict IP addresses through GLBC or on GCP L7 Load Balancer itself. Note that GLBC is also a work in progress as of this writing.

ingress.kubernetes.io/whitelist-source-range works great but when you are using something like an nginx ingress controller because nginx itself can restrict IP addresses.

The general way to restrict/whitelist IP addresses is using VPC Firewall Rules (which seems like you are doing already). Essentially you can restrict/whitelist the IP addresses to the network where your K8s nodes are running on.

-- Rico
Source: StackOverflow

1/17/2019

One of the best options to accomplish your goal is using firewall rules since you can't restrict IP addresses through the Global LB or on GCP L7 LB itself. However, another option if you are using Ingress on your Kubernetes cluster, it is possible to restrict access to your application based on dedicated IP addresses.

One possible use case would be that you have a development setup and don’t want to make all the fancy new features available to everyone, especially competitors. In such cases, IP whitelisting to restrict access can be used.

This can be done with specifying the allowed client IP source ranges through the ingress.kubernetes.io/whitelist-source-range annotation.

The value is a comma separated list of CIDR block.

For example:

10.0.0.0/24, 1.1.1.1/32.

Please get more information here.

-- Galo
Source: StackOverflow

5/21/2019

For anyone who stumbles on this question via Google like I did, there is now a solution. You can implement this via a BackendConfig from the cloud.google.com Kubernetes API in conjunction with a GCE CloudArmor policy.

https://cloud.google.com/kubernetes-engine/docs/how-to/cloud-armor-backendconfig

-- JSchwerberg
Source: StackOverflow