K
Q

Question

Cross namespace communication in GKE ingress

4/7/2020

We have two different applications deployed in two different namespaces in GKE, I am trying to expose these applications using GKE Ingress which is in the default namespace. I have SSL certificates for this applications.

The issue we are facing now is external load balancer is not able to communicate with our applications because the namespaces are different.

Referring to github issue no #17088 I found that Nginx ingress can be used in this case, but my use-case restricts use of Nginx or any third party tool, Hence can you help me to find a solution or any work-around for this using GKE ingress only.

I have tried above test-case using Nginx ingress controller and Istio service mesh.It is working well with Nginx.

But in case of Istio, I am able to hit only one application and getting 404 error on second application. I am referring this documentation for multi-host tls configuration. I have created two secrets one is istio-ingressgateway-certs in normal ingressgateway-certs directory and another one is istio-ingressgateway-tmcoaching-certs in custom directory named ingressgateway-tmcoaching-certs and also updated the istio-ingressgateway deployment by applying the json file as suggested in the docs. The config files are as follows.

Gateway and Virtual services of applications.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: tm-coaching-vs
  namespace: tm-coaching
spec:
  hosts:
  - q-tm-coaching.q-appliedai.com
  gateways:
  - cross-namespace-gateway
  http:
  - match:
    - uri:
       exact: /
    route:
    - destination:
        host: demo-tm-coaching.tm-coaching.svc.cluster.local
        port:
          number: 81

---

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: case-doc-vs
  namespace: case-doc
spec:
  hosts:
  - case-doc.q-appliedai.com
  gateways:
  - cross-namespace-gateway
  http:
  - match:
    - uri:
        exact: /
    route:
    - destination:
        host: demo-case-doc.case-doc.svc.cluster.local
        port:
          number: 80

---

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: cross-namespace-gateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: demo-case-doc
      protocol: HTTPS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-certs/tls.key
    hosts:
    - case-doc.q-appliedai.com
  - port:
      number: 443
      name: demo-tm-coaching
      protocol: HTTPS
    tls:
      mode: SIMPLE
      serverCertificate: /etc/istio/ingressgateway-tmcoaching-certs/tls.crt
      privateKey: /etc/istio/ingressgateway-tmcoaching-certs/tls.key
    hosts:
    - q-tm-coaching.q-appliedai.com

The sidecar is injected inside a pods

ankita_shinde@cloudshell:~/istio-tls-cross-namespace/istio-cross-namespace-codebase/istio-config (qp-ihg-rpa-2020-03)$ kubectl get pods -n case-doc -o=custom-columns=NameSpace:.metadata.namespace,NAME:.metadata.name,CONTAINERS:.spec.containers[*].name
NameSpace   NAME                             CONTAINERS
case-doc    demo-case-doc-55db5ddcf4-ppnrq   demo-case-doc,istio-proxy
ankita_shinde@cloudshell:~/istio-tls-cross-namespace/istio-cross-namespace-codebase/istio-config (qp-ihg-rpa-2020-03)$ kubectl get pods -n tm-coaching -o=custom-columns=NameSpace:.metadata.namespace,NAME:.metadata.name,CONTAINERS:.spec.containers[*].name
NameSpace     NAME                               CONTAINERS
tm-coaching   demo-tm-coaching-54bd45d68-lqh2l   demo-tm-coaching,istio-proxy

The logs are

[2020-04-15T19:33:24.369Z] "GET / HTTP/2" 404 - "-" "-" 0 0 0 - "10.128.15.226" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" "0b41dd7b-7bbe-9db1-95b1-df47386c5e8b" "q-tm-coaching.q-appliedai.com" "-" - - 10.4.4.10:443 10.128.15.226:12688 case-doc.q-appliedai.com default
[2020-04-15T19:33:27.719Z] "GET / HTTP/2" 404 - "-" "-" 0 0 0 - "10.128.15.226" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" "3e6a51b3-d370-946e-a90f-259f17c4cea7" "case-doc.q-appliedai.com" "-" - - 10.4.4.10:443 10.128.15.226:12688 case-doc.q-appliedai.com default

-- Ankita Shinde

google-cloud-platform

google-kubernetes-engine

istio

kubernetes

nginx-ingress

1 Answer

4/7/2020

Unfortunately there is no possibility to configure GKE Ingress to support services in different namespaces.

Please take a look on below citing from a github page of ingress-gce.

Are Ingress controllers namespaced?
Ingress is namespaced, this means 2 Ingress objects can have the same name in 2 namespaces, and must only point to Services in its own namespace. An admin can deploy an Ingress controller such that it only satisfies Ingress from a given namespace, but by default, controllers will watch the entire Kubernetes cluster for unsatisfied Ingress.
-- Github.com: Kubernetes: Ingress-gce: Are ingress controllers namespaced

Additionally, it is confirmed by user liggitt in the github issue that you included in your post:

It would be good to be able to address services in any namespace.
It was intentionally avoided. Cross namespace references would be a prime source of privilege escalation attacks.
-- https://github.com: Kubernetes Issues: 17088

The possible workarounds could be:

Spawning both Deployments in the same namespace.
Using custom Ingress controller like for example NGINX Ingress.
Using a service mesh like Istio.
Spawning 2 GKE Ingress resources for each namespace (each will have a different IP address). 1

1 : It will generate additional costs

Please let me know if you have questions in that.

There is official documentation about best practices with namespaces: Cloud.google.com: Kubernetes best practices organizing with namespaces

-- Dawid Kruk

Source: StackOverflow

KQ

Cross namespace communication in GKE ingress

Similar Questions

1 Answer

Are Ingress controllers namespaced?

K
Q