I trying to debug it for few hours to no effect. I either do know how to google it or it unusual issue.

I have Kubernetes Engine cluster on google cloud. To test it I deploy simple app: https://github.com/iniside/uSpark It is composed from three console apps:

• GreeterServer - Frontend with gRPC and Google Endpoints
• Greeter.Backend - Backend with gRPC service.
• uSpark - client console app.

All of them are using .NET Core 2.1, gRPC 1.12, and C#. Kubernetes Engine is 1.10

I already tried to see if the service dns is resolved inside cluster. I also tried to call service directly by it's IP when creating Channel.

Either of those end up with:

Grpc.Core.RpcException: 'Status(StatusCode=Unavailable, Detail="Name resolution failure")'

GreeterServer is trying to talk to Greeter.Backend:

public override Task<HelloReply> SayHelloAgain(HelloRequest request, ServerCallContext context)
{
    Channel channel = new Channel("grpc-greeter-backend.default.svc.cluster.local", 9000, ChannelCredentials.Insecure);
    var backendClient = new GreeterBackend.GreeterBackendClient(channel);

    var reply = backendClient.SayHelloFromBackend(new BackendHelloRequest { Name = "iniside" });
    channel.ShutdownAsync().Wait(); //not neeed to wait, but easier to debug now.
    return Task.FromResult<HelloReply>(new HelloReply { Message = "Hello Again " + request.Name + " " + reply.Message });
}

And this is code of my backend:

class GreeterBackendImpl : GreeterBackend.GreeterBackendBase
{
    public override Task<BackendHelloReply> SayHelloFromBackend(BackendHelloRequest request, ServerCallContext context)
    {
        return Task.FromResult<BackendHelloReply>(new BackendHelloReply { Message = "Hello " + request.Name + " From Backend"});
    }
}

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine("Hello World!");

        Server server = new Server
        {
            Services = { GreeterBackend.BindService(new GreeterBackendImpl()) },
            Ports = { new ServerPort("127.0.0.1", 9000, ServerCredentials.Insecure) }
        };

        server.Start();

        Console.WriteLine("Gretter server is linstening on port 50051");
        Console.WriteLine("Press any key to stop server");

        int read = Console.Read();
        while (read < 0)
        {

        }
    }
}

Configuration of Frontend:

apiVersion: v1
kind: Service
metadata:
  name: esp-grpc-greeter
spec:
  ports:
  # Port that accepts gRPC and JSON/HTTP2 requests over HTTP.
  - port: 80
    targetPort: 9100
    protocol: TCP
    name: http2
  selector:
    app: esp-grpc-greeter
  type: LoadBalancer
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: esp-grpc-greeter
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: esp-grpc-greeter
    spec:
      containers:
      - name: esp
        image: gcr.io/endpoints-release/endpoints-runtime:1
        args: [
          "--http2_port=9100",
          "--service=hellohorld3.endpoints.sa-game-206414.cloud.goog",
          "--rollout_strategy=managed",
          "--backend=grpc://127.0.0.1:9000"
        ]
        ports:
          - containerPort: 9100
      - name: greeter
        image: eu.gcr.io/sa-game-206414/greeter-service:v1
        ports:
          - containerPort: 8000

Backend Configuration:

apiVersion: v1
kind: Service
metadata:
  name: grpc-greeter-backend
spec:
  ports:
  # Port that accepts gRPC and JSON/HTTP2 requests over HTTP.
  - port: 8080
    targetPort: 9000
    protocol: TCP
    name: http2
  - port: 9000
    targetPort: 9000
    protocol: TCP
    name: http2900
  selector:
    app: grpc-greeter-backend
  type: ClusterIP
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: grpc-greeter-backend
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: grpc-greeter-backend
    spec:
      containers:
      - name: greeter-backend
        image: eu.gcr.io/sa-game-206414/greeter-backend:b1
        ports:
          - containerPort: 9000