I deployed a gRPC service (spring boot docker image) in my on-premise kubernetes cluster. I followed this documentation to configure correctly deployment, service and ingress kubernetes manifests.
I tried to test my service using grpcurl :
grpcurl -insecure fortune-teller.mydomain.cloud:443 build.stack.fortune.FortuneTeller/Predictand the request still stuck for minutes.
In the ingress logs (debug enabled), I see client timeout :
client timed out (110: Connection timed out), client: 1.2.3.4, server: _, request: "POST /grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo HTTP/2.0", host: "fortune-teller.mydomain.cloud:443"After 4 timeouts in the ingress logs, the command ends at the client side with :
Error invoking method "build.stack.fortune.FortuneTeller/Predict": failed to query for service descriptor "build.stack.fortune.FortuneTeller": rpc error: code = Internal desc = stream terminated by RST_STREAM with error code: PROTOCOL_ERRORI use nginx/1.13.12 with rancher/nginx-ingress-controller:0.16.2-rancher1 image.
At the annotation level, I tested :
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/backend-protocol: "GRPC"and
nginx.ingress.kubernetes.io/grpc-backend: "true"and both have the same result (timeout).
Note : I bypassed the ingress testing another gRPC service directly using the kubernetes service DNS and it works.
Any idea of what's going wrong ?
I found a problem in my ingress resource.
Despite the tls configuration, the nginx config didn't listen on 443. Something wrong with my secret
I fixed this and I retested my different configuration.
The working annotations are :
metadata:
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/grpc-backend: "true"Which are not the annotations used in the kubernetes documentation.