Pod cannot connect to itself via service IP (kubernetes v1.10)

4/27/2018

I have a kubernetes cluster (v1.10) and flannel as cni. Using the default settings, the setup works fine (using kubeadm) but a pod cannot connect to itself via service IP.

Tried setting hairpin-mode as "promiscuous-bridge" but kubelet complains:

Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"

Not sure what's going on. Went through this https://kubernetes.io/docs/tasks/debug-application-cluster/debug-service/#a-pod-cannot-reach-itself-via-service-ip but no help. Any pointers ?

-- Phagun Baya
flannel
kubernetes

2 Answers

4/28/2018

The default flannel configuration does not set HairpinMode to true.

https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml

Setting hairpinMode to true in flannel config resolved the issue. The config change is as below:

kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-system
  labels:
    tier: node
    app: flannel
data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "type": "flannel",
      "delegate": {
        "hairpinMode": true,
        "isDefaultGateway": true
      }
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }

Complete flannel config is here:

https://gist.githubusercontent.com/phagunbaya/2a53519a9427ba0623244f1680a5b5ff/raw/13ada0d6dd92388c8c5aae93bfb1ccaf9c79f60b/flannel-0.9.1.yaml

Instead of the default kubectl command to apply flannel cni use following command:

kubectl apply -f https://gist.githubusercontent.com/phagunbaya/2a53519a9427ba0623244f1680a5b5ff/raw/13ada0d6dd92388c8c5aae93bfb1ccaf9c79f60b/flannel-0.9.1.yaml
-- Phagun Baya
Source: StackOverflow

8/24/2018

In my case the issue was not disabling firewalld service as mentioned below. My core-dns pods kept crashing with error connect - no route to host

https://github.com/rancher/rancher/issues/6139

Run the below commands on each of your nodes as mentioned

  • systemctl stop firewalld
  • systemctl disable firewalld
-- fatcook
Source: StackOverflow