KubeDNS no route to host for kubernetes.default

7/1/2021

I am trying to setup RabbitMQ Cluster on K8S Cluster (CentOS 7) but it errored with

2021-07-01 16:30:10.048 [error] <0.273.0> Failed to fetch a list of nodes from Kubernetes API: {failed_connect,[{to_address,{"kubernetes.default",443}}, {inet,[inet],nxdomain}]}

I tried debugging the issue but I am not sure what's missing

nslookup output

[root@re-devk8s-control01 tmp]# nslookup kubernetes.default
Server:         174.25.1.14
Address:        174.25.1.14#53

** server can't find kubernetes.default: NXDOMAIN

dnsutils output

[root@re-devk8s-control01 tmp]#  kubectl exec -i -t dnsutils -- cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local awstruewind.local
options ndots:5

/etc/resolv.conf

[root@re-devk8s-control01 tmp]# cat /etc/resolv.conf
# Generated by NetworkManager
search ourdomain.local
nameserver 174.25.1.14

KubeDNS log and I see few errors in it

[root@re-devk8s-control01 tmp]# kubectl logs --namespace=kube-system -l k8s-app=kube-dns
.:53
[INFO] plugin/reload: Running configuration MD5 = db32ca3650231d74073ff4cf814959a7
CoreDNS-1.8.0
linux/amd64, go1.15.3, 054c9ae
[ERROR] plugin/errors: 2 610092700101479936.1385205460227937641. HINFO: read udp 10.244.0.31:47645->174.25.1.14:53: read: no route to host
[ERROR] plugin/errors: 2 610092700101479936.1385205460227937641. HINFO: read udp 10.244.0.31:60675->174.25.1.14:53: read: no route to host

Though there were many similar posts in SO, I was unable to find out the root cause

-- Sathish Kumar
centos
kubernetes
rabbitmq

1 Answer

7/1/2021

I found the following link from SO which helped me to resolve the issue KubeDNS Masquerade

firewall-cmd --list-all
  ... 
  masquerade: yes

Enable if it's "no":

firewall-cmd --add-masquerade --permanent
firewall-cmd --reload
-- Sathish Kumar
Source: StackOverflow