Why can't my service pass traffic to a pod with a named port on minikube?

2/3/2020

I'm having trouble with the examples in section 5.1.1 Using Named Ports of Kubernetes In Action by Marko Luksa. The example goes like this:

First - Create

I'm creating a pod with a named port that runs a Node.js container that responds with You've hit <hostname> when it's hit:

apiVersion: v1
kind: Pod
metadata:
  name: named-port-pod
  labels:
    app: named-port
spec: 
  containers:
  - name: kubia
    image: michaellundquist/kubia
    ports:
    - name: http
      containerPort: 8080

And a service like this (note, this is a simplified version of the original example which also doesn't work.:

apiVersion: v1
kind: Service
metadata:
  name: named-port-service
spec:
  ports:
  - name: http
    port: 80
    targetPort: http
  selector:
    app: named-port

Second - Verify

$ kubectl get po -o wide --show-labels
NAME             READY   STATUS    RESTARTS   AGE   IP           NODE       NOMINATED NODE   READINESS GATES   LABELS
named-port-pod   1/1     Running   0          45m   172.17.0.7   minikube   <none>           <none>            app=named-port


$ kubectl get services
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes           ClusterIP   10.96.0.1       <none>        443/TCP   53m
named-port-service   ClusterIP   10.96.115.108   <none>        80/TCP    19m

$ kubectl describe service named-port-service 
Name:              named-port-service
Namespace:         default
Labels:            <none>
Annotations:       <none>
Selector:          app=named-port
Type:              ClusterIP
IP:                10.96.115.108
Port:              http  80/TCP
TargetPort:        http/TCP
Endpoints:         172.17.0.7:8080
Session Affinity:  None
Events:            <none>

Third - Test (Failing)

$ kubectl exec named-port-pod -- curl named-port-pod:8080
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    26    0    26    0     0   5494      0 --:--:-- --:--:-- --:--:--  6500
You've hit named-port-pod

$ kubectl exec named-port-pod -- curl --max-time 20 named-port-service
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:19 --:--:--     0curl: (28) Connection timed out after 20001 milliseconds
command terminated with exit code 28

As you can see, everything works when I hit named-port-pod:8080, but fails when I hit named-port-service. I'm pretty sure I have the mapping correct because kubectl describe service named-port-service has the correct endpoint I think minikube can use named ports but my service can't pass connections to my pod. Why?

p.s here's my minikube version:

$ minikube version
minikube version: v1.6.2
commit: 54f28ac5d3a815d1196cd5d57d707439ee4bb392
-- mikeLundquist
kubernetes
kubernetes-service
minikube

1 Answer

2/3/2020

This is known issue with minikube. Pod cannot reach itself via service IP. You can try accesing your service from a different pod or use the following workaround to fix this.

minikube ssh
sudo ip link set docker0 promisc on

Open issue: https://github.com/kubernetes/minikube/issues/1568

-- Shashank V
Source: StackOverflow