For the past two weeks, I've been working on deploying a gRPC service (also with a gRPC gateway). I got stuck a lot at many points and it still doesn't work... Why? I am not sure to know.

I first used Ingress from Google but it wasn't working with gRPC, so I moved to Ingress NGINX, but I still have issues...

I installed NGINX from https://kubernetes.github.io/ingress-nginx/deploy/#gce-gke

I added some annotations:

kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
nginx.ingress.kubernetes.io/grpc-backend: "true"

service.yaml (containing my Service, Deployment, Ingress (gRPC) and Ingress (REST))

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: gkegrpcservice
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      run: gkegrpcservice
  template:
    metadata:
      labels:
        run: gkegrpcservice
    spec:
      containers:
        - name: gkegrpcservice
          image: gcr.io/<PROJECT_ID>/gkegrpcservice:latest
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
            - containerPort: 8081
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    cloud.google.com/app-protocols: '{"grpc":"HTTP2"}'
  name: gkegrpcservice
  namespace: default
spec:
  type: NodePort
  selector:
    run: gkegrpcservice
  ports:
    # Port that accepts gRPC and JSON/HTTP2 requests over HTTP.
    - port: 8080
      targetPort: 8080
      protocol: TCP
      name: grpc
    # Port that accepts REST requests over HTTP.
    - port: 8081
      targetPort: 8081
      protocol: TCP
      name: rest
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gkegrpcservice-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
    nginx.ingress.kubernetes.io/grpc-backend: "true"
  namespace: default
spec:
  rules:
    - http:
        paths:
          - path: /GKEgRPCService.GKEgRPCService/*
            backend:
              serviceName: gkegrpcservice
              servicePort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gkegrpcservice-gateway-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - http:
        paths:
          - path: /rest/*
            backend:
              serviceName: gkegrpcservice
              servicePort: 8081

I also created a github repository, so you can access the whole code/configuration but also the Dockerfile and the service.yaml: https://github.com/Emixam23/GKE-gRPC-Service-Ingress

based on the above, I then describe my Ingress:

MacBook-Pro-de-Emixam23:src emixam23$ kubectl describe ingress
Name:             gkegrpcservice-gateway-ingress
Namespace:        default
Address:          35.228.118.83
Default backend:  default-http-backend:80 (10.4.2.7:8080)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *     
        /rest/*   gkegrpcservice:8081 (10.4.1.25:8081)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/ssl-redirect":"false"},"name":"gkegrpcservice-gateway-ingress","namespace":"default"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"gkegrpcservice","servicePort":8081},"path":"/rest/*"}]}}]}}

  kubernetes.io/ingress.class:               nginx
  nginx.ingress.kubernetes.io/ssl-redirect:  false
Events:
  Type    Reason  Age                 From                      Message
  ----    ------  ----                ----                      -------
  Normal  CREATE  26m                 nginx-ingress-controller  Ingress default/gkegrpcservice-gateway-ingress
  Normal  UPDATE  115s (x3 over 26m)  nginx-ingress-controller  Ingress default/gkegrpcservice-gateway-ingress


Name:             gkegrpcservice-ingress
Namespace:        default
Address:          35.228.118.83
Default backend:  default-http-backend:80 (10.4.2.7:8080)
Rules:
  Host  Path  Backends
  ----  ----  --------
  *     
        /GKEgRPCService.GKEgRPCService/*   gkegrpcservice:8080 (10.4.1.25:8080)
Annotations:
  nginx.ingress.kubernetes.io/backend-protocol:      GRPC
  nginx.ingress.kubernetes.io/grpc-backend:          true
  nginx.ingress.kubernetes.io/ssl-redirect:          false
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/backend-protocol":"GRPC","nginx.ingress.kubernetes.io/grpc-backend":"true","nginx.ingress.kubernetes.io/ssl-redirect":"false"},"name":"gkegrpcservice-ingress","namespace":"default"},"spec":{"rules":[{"http":{"paths":[{"backend":{"serviceName":"gkegrpcservice","servicePort":8080},"path":"/GKEgRPCService.GKEgRPCService/*"}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:
  Type    Reason  Age    From                      Message
  ----    ------  ----   ----                      -------
  Normal  CREATE  6m43s  nginx-ingress-controller  Ingress default/gkegrpcservice-ingress
  Normal  UPDATE  6m6s   nginx-ingress-controller  Ingress default/gkegrpcservice-ingress

From here, 35.228.118.83/rest/health_check should return

200 {} But only a 404 get returned...

However, this (to me) doesn't make sense, even by taking out /rest/, I should get a response, but OK, it doesn't work.

I then move to my ingress-controller freshly created from the link above. The logs I get are:

I don't really understand all the logs but it doesn't seem to display much about me pinging...

Do someone has any idea about what is really happening and if I have any way to maybe debug something? Because right now, based on the Internet, I am good to go, but in reality, nothing seems to work at all...