Kubernetes.default nslookup not able to resolve from different namespaces

2/7/2019

I'm facing a problem resolving kubernetes.default.svc.cluster.local from outside default namespace

I'm running two busybox:1.30 pods on each namespace and the name successfully resolves from the default namespace only

    [admin@devsvr3 ~]$ kubectl exec -n default -ti busybox -- nslookup kubernetes
    Server:    10.96.0.10
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    Name:      kubernetes
    Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local
    [admin@devsvr3 ~]$ kubectl exec -n namespace-dev -ti busybox -- nslookup kubernetes
    Server:         10.96.0.10
    Address:        10.96.0.10:53
    ** server can't find kubernetes.namespace-dev.svc.cluster.local: NXDOMAIN
    *** Can't find kubernetes.svc.cluster.local: No answer
    *** Can't find kubernetes.cluster.local: No answer
    *** Can't find kubernetes.namespace-dev.svc.cluster.local: No answer
    *** Can't find kubernetes.svc.cluster.local: No answer
    *** Can't find kubernetes.cluster.local: No answer
    [admin@devsvr3 ~]$ 

I'm running CentOS 7 kubernetes cluster on an air-gaped environment and using weave net CNI add-on and this is my CoreDNS config

    apiVersion: v1
    data:
      Corefile: |
        .:53 {
            log
            errors
            health
            kubernetes cluster.local in-addr.arpa ip6.arpa {
              pods insecure
              upstream
              fallthrough in-addr.arpa ip6.arpa
            }
            prometheus :9153
            proxy . /etc/resolv.conf
            cache 30
            reload
            loadbalance
        }
    kind: ConfigMap
    metadata:
      creationTimestamp: "2019-01-28T10:59:25Z"
      name: coredns
      namespace: kube-system
      resourceVersion: "1177652"
      selfLink: /api/v1/namespaces/kube-system/configmaps/coredns
      uid: c6b5ddae-22eb-11e9-8689-0017a4770068
-- Amr Nassar
centos
coredns
kube-apiserver
kubernetes
weave

1 Answer

4/3/2019

Following your steps indeed I approached the same issue. But if you create the pod using this yaml it works correctly. Changing the busybox image seems to end up with your described error. Will try to find out why. But for now this is the solution.

apiVersion: v1 kind: Pod metadata: name: busybox namespace: namespace-dev spec: containers: - name: busybox image: busybox:1.28 command: - sleep - "3600" imagePullPolicy: IfNotPresent restartPolicy: Always

and then: kubectl exec -ti -n=namespace-dev busybox -- nslookup kubernetes.default it works as intended and explained here.

/ # nslookup kubernetes
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

nslookup: can't resolve 'kubernetes'
/ # nslookup kubernetes.default
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes.default
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local
-- aurelius
Source: StackOverflow