Loadbalancing with reserved IPs in Google Container Engine

12/29/2015

I want to host a website (simple nginx+php-fpm) on Google Container Engine. I built a replication controller that controls the nginx and php-fpm pod. I also built a service that can expose the site.

How do I link my service to a public (and reserved) IP Address so that the webserver sees the client IP addresses?

I tried creating an ingress. It provides the client IP through an extra http header. Unfortunately ingress does not support reserved IPs yet:

kind: Ingress
metadata:
  name: example-ingress
spec:
  rules:
  - host: example.org
    http:
      paths:
      - backend:
          serviceName: example-web
         servicePort: 80
        path: /

I also tried creating a service with a reserved IP. This gives me a public IP address but I think the client IP is lost:

apiVersion: v1
kind: Service
metadata:
  name: 'example-web'
spec:
  selector:
    app: example-web
  ports:
    - port: 80
      targetPort: 80
  loadBalancerIP: "10.10.10.10"
  type: LoadBalancer

I would setup the HTTP Loadbalancer manually, but I didn't find a way to configure a cluster IP as a backend for the loadbalancer.

This seems like a very basic use case to me and stands in the way of using container engine in production. What am I missing? Where am I wrong?

-- tback
google-kubernetes-engine
kubernetes

1 Answer

12/29/2015

As you are running in google-container-engine you could set up a Compute Engine HTTP Load Balancer for your static IP. The Target proxy will add X-Forwarded- headers for you.

Set up your kubernetes service with type NodePort and add a nodePort field. This way nodePort is accessible via kubernetes-proxy on every nodes IP address regardless of where the pod is running:

apiVersion: v1
kind: Service
metadata:
 name: 'example-web'
spec:
 selector:
   app: example-web
 ports:
   - nodePort: 30080
     port: 80
     targetPort: 80
 type: NodePort

Create a backend service with HTTP health check on port 30080 for your instance group (nodes).

-- jayme
Source: StackOverflow