GKE Ingress to services on same container with 2 exposed ports

3/1/2020

I have a GKE cluster, a static ip, and a container/pod which exposes 2 ports: 8081(UI https) and 8082 (WSS https). I must connect to the "UI Https" and "WSS Https" on the same IP. The "WSS Https" service does not have a health check endpoint.

Do i need to use Isito, Consul, Nginx ingress or some service mesh to allow these connections on the same IP with different ports?

Is this even possible? enter image description here

Things i have tried:

  1. GCP global lb with 2 independent ingress services. The yaml for the second service never works correctly but i can add another backed service via the UI. The ingress always reverts to the default health check for the "WSS Https" service and it always unhealthy.
  2. Changed Service type from NodePort to LoadBalancer with a static ip. This will not allow me to change the readiness check and always reverts back.
  3. GCP GLIB with 1 ingress and 2 backend gives me the same healthcheck failure as above
  4. TCP Proxy - Does not allow me to set the same instance group.

Below are my ingress, service, and deployment.

Ingress

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app-ingress
  namespace: myappnamespace
  annotations:
    kubernetes.io/ingress.global-static-ip-name: global-static-ip-name
  labels:
    app: appname
spec:
  backend:
    serviceName: ui-service
    servicePort: 8081
  tls:
  - hosts:
    - my-host-name.com
    secretName: my-secret
  rules:
  - host: my-host-name.com
    http:
      paths:
        - backend:
            serviceName: ui-service
            servicePort: 8081
  - host: my-host-name.com
    http:
      paths:
        - backend:
            serviceName: app-service
            servicePort: 8082

Services

---
apiVersion: v1
kind: Service
metadata:
  labels:
    name: ui-service
  name: ui-service
  namespace: myappnamespace
  annotations:
    cloud.google.com/app-protocols: '{"ui-https":"HTTPS"}'
    beta.cloud.google.com/backend-config: '{"ports":{"8081":"cloud-armor"}}'
spec:
  selector:
      app: appname
  ports:
  - name: ui-https
    port: 8081
    targetPort: "ui"
    protocol: "TCP"
  selector:
    name: appname
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  labels:
    name: app-service
  name: app-service
  namespace: myappnamespace
  annotations:
    cloud.google.com/app-protocols: '{"serviceport-https":"HTTPS"}'
    beta.cloud.google.com/backend-config: '{"ports":{"8082":"cloud-armor"}}'
spec:
  selector:
      app: appname
  ports:
  - name: serviceport-https
    port: 8082
    targetPort: "service-port"
    protocol: "TCP"
  selector:
    name: appname
  type: NodePort
---

Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
    name: appname
    namespace: myappnamespace
    labels:
        name: appname
spec:
    replicas:1
    selector:
        matchLabels:
            name: appname
    strategy:
        type: Recreate
    template:
        metadata:
            name: appname
            namespace: appnamespace
            labels:
                name: appname
        spec:
            restartPolicy: Always
            serviceAccountName: myserviceaccount
            containers:
            - name: my-container
              image: image
              ports:
              - name: service-port
                containerPort: 8082
              - name: ui
                containerPort: 8081
              readinessProbe:
              failureThreshold: 3
              httpGet:
               path: /api/health
                 port: 8081
                 scheme: HTTPS
        livenessProbe:
          exec:
            command:
              - cat
              - /version.txt
        [......]
-- rubio
google-cloud-platform
google-kubernetes-engine
kubernetes-ingress

1 Answer

3/2/2020

A Service exposed through an Ingress must respond to health checks from the load balancer.

External HTTP(S) Load Balancer that GKE Ingress creates only supports port 443 for https traffic.

In that case you may want to:

  1. Use two separate Ingress resources to route traffic for two different host names on the same IP address and port:

    • ui-https.my-host-name.com

    • wss-https.my-host-name.com

  2. Opt to use Ambassador or Istio Virtual Service.

  3. Try Multi-Port Services.

Please let me know if that helped.

-- OhHiMark
Source: StackOverflow