kubernetes coredns resolution is failing systematically

5/18/2020

DNS revolution is systematically failing in my 3 node cluster. I have reviewed the official debugging kubernetes documentation and did not find a resolution. My cluster nodes are CentOS 7 vm's and I'm using a Weave net CNI for Multicast support. Using sniff I was able to get a tcpdump of the DNS messages. What can I do to resolve the DNS failure?

enter image description here

kube-system   coredns-66bff467f8-22f6v                      1/1     Running   20         23d    10.32.0.2    control.example.com   <none>           <none>
kube-system   coredns-66bff467f8-kx9l4                      1/1     Running   20         23d    10.32.0.3    control.example.com   <none>           <none>

kube-system   kube-dns         ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP          23d

Log from coredns pod

[INFO] 10.44.0.4:50257 - 14336 "AAAA IN nmos-registry-67795b97d-jfgsz.example.com. udp 59 false 512" NXDOMAIN qr,rd,ra 135 0.025346947s
[INFO] 10.44.0.4:50257 - 14336 "A IN nmos-registry-67795b97d-jfgsz.example.com. udp 59 false 512" NXDOMAIN qr,rd,ra 135 0.023372789s
[INFO] 10.44.0.4:50257 - 14336 "AAAA IN nmos-registry-67795b97d-jfgsz.cluster.local. udp 61 false 512" NXDOMAIN qr,aa,rd 154 0.000156607s
[INFO] 10.44.0.4:50257 - 14336 "A IN nmos-registry-67795b97d-jfgsz.svc.cluster.local. udp 65 false 512" NXDOMAIN qr,aa,rd 158 0.000051891s
[INFO] 10.44.0.4:50257 - 14336 "A IN nmos-registry-67795b97d-jfgsz.cluster.local. udp 61 false 512" NXDOMAIN qr,aa,rd 154 0.000037125s
[INFO] 10.44.0.4:50257 - 14336 "A IN nmos-registry-67795b97d-jfgsz.example.com. udp 59 false 512" NXDOMAIN qr,aa,rd,ra 135 0.000041387s
[INFO] 10.44.0.4:50257 - 14336 "AAAA IN nmos-registry-67795b97d-jfgsz.default.svc.cluster.local. udp 73 false 512" NXDOMAIN qr,aa,rd 166 0.000068773s
[INFO] 10.44.0.4:50257 - 14336 "AAAA IN nmos-registry-67795b97d-jfgsz.svc.cluster.local. udp 65 false 512" NXDOMAIN qr,aa,rd 158 0.000037855s
[INFO] 10.44.0.4:50257 - 14336 "AAAA IN nmos-registry-67795b97d-jfgsz.example.com. udp 59 false 512" NXDOMAIN qr,aa,rd,ra 135 0.000036585s
-- dcrearer
kubernetes
kubernetes-pod

0 Answers