I have a GRPC service written in C# using .net core deployed in Kubernetes with flannel as CNI. When I try to connect to this GRPC service on from another container deployed on same cluster using service:9000, I am getting Status(StatusCode=Unavailable, Detail=\"Connect Failed\").

I checked locally on my machine without containers and k8s, everything works fine.

My deployment yaml for this service looks like this:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  finalizers:
  - kubernetes.io/pvc-protection
  name: service-pvc
  namespace: services
spec:
  accessModes:
  - ReadWriteOnce
  - ReadOnlyMany
  - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: ""
  volumeName: nfs-service
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: service-deployment
  labels:
    app: service
  name: service
  namespace: services
spec:
  selector:
    matchLabels:
      app: service
  replicas: 1
  template:
    metadata:
      labels:
        app: service
    spec:
      containers:
        image: <ImageUrl>
        imagePullPolicy: Always
        name: service  
        ports:
        - containerPort: 9000
          name: service
          protocol: TCP
        securityContext:
          allowPrivilegeEscalation: false
          privileged: false
          readOnlyRootFilesystem: false
          runAsNonRoot: false
        stdin: true
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        tty: true
        volumeMounts:
        - mountPath: /var/certs/
          name: service-vol1
          readOnly: true
          subPath: service
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      terminationGracePeriodSeconds: 30
      imagePullSecrets:
      - name: unstable      
      volumes:
      - name: service-vol1
        persistentVolumeClaim:
          claimName: service-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: service
  namespace: services
spec:
  selector:
    app: service
  ports:
  - port: 9000
    targetPort: service

Please help.

\=======

An observation: When I ran netstat -tulpn on container shell, it give duplicate entries in listen state.

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:9000            0.0.0.0:*               LISTEN      1/dotnet
tcp        0      0 0.0.0.0:9000            0.0.0.0:*               LISTEN      1/dotnet
tcp        0      0 0.0.0.0:9000            0.0.0.0:*               LISTEN      1/dotnet
tcp        0      0 0.0.0.0:9000            0.0.0.0:*               LISTEN      1/dotnet
root@service-67d57dfb6d-wsvvf:/app#