K
Q

DNS resolves externally but not local to kube cluster (keys ARE located in etcd)

1/20/2016

DNS can resolve to sites external to the cluster etcd is modified correctly for new containers, services, nodes, etc

here are some details:

[fedora@kubemaster ~]$ kubectl logs kube-dns-v10-q9mlb -c kube2sky --namespace=kube-system
I0118 17:42:24.639508 1 kube2sky.go:436] Etcd server found: http://127.0.0.1:4001
I0118 17:42:25.642366 1 kube2sky.go:503] Using https://10.254.0.1:443 for kubernetes master
I0118 17:42:25.642772 1 kube2sky.go:504] Using kubernetes API 
[fedora@kubemaster ~]$

Showing that etcd is being properly populated:

[fedora@kubemaster ~]$ kubectl exec -t busybox -- nslookup kubelab.local
Server: 10.254.0.10
Address 1: 10.254.0.10

nslookup: can't resolve 'kubelab.local'

error: error executing remote command: Error executing command in container: Error executing in Docker Container: 1

fedora@kubemaster ~]$ etcdctl ls --recursive
/kubelab.local
/kubelab.local/network
/kubelab.local/network/config
/kubelab.local/network/subnets
/kubelab.local/network/subnets/172.16.46.0-24
/kubelab.local/network/subnets/172.16.12.0-24
/kubelab.local/network/subnets/172.16.70.0-24
/kubelab.local/network/subnets/172.16.21.0-24
/kubelab.local/network/subnets/172.16.54.0-24
/kubelab.local/network/subnets/172.16.71.0-24

To help a little further:

[fedora@kubemaster ~]$ kubectl exec --namespace=kube-system kube-dns-v10-6krfm -c skydns ps
PID   USER     COMMAND
    1 root     /skydns -machines=http://127.0.0.1:4001 -addr=0.0.0.0:53 -ns-rotate=false -domain=kubelab.local.
   11 root     ps
[fedora@kubemaster ~]$

I DID change cluster.local to kubelab.local, but I also made the changes prior to my kubenodes:

KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubelet.kubeconfig --config=/etc/kubernetes/manifests --cluster-dns=10.254.0.10 --cluster-domain=kubelab.local"

/etc/resolv.conf appears to be ok on a testhost (in this case, busybox per DNS documentation example):

[fedora@kubemaster ~]$ kubectl exec busybox -c busybox -i -t -- cat /etc/resolv.conf
search default.svc.kubelab.local svc.kubelab.local kubelab.local openstacklocal kubelab.com
nameserver 10.254.0.10
nameserver 192.168.1.70
options ndots:5
[fedora@kubemaster ~]$

Results = still a little frustrating:

[fedora@kubemaster ~]$ kubectl exec -t busybox -- nslookup kubelab.local
 Server:    10.254.0.10
 Address 1: 10.254.0.10

 nslookup: can't resolve 'kubelab.local'
 error: error executing remote command: Error executing command in container: Error executing in Docker Container: 1
[fedora@kubemaster ~]$
-- v1k0d3n
kubernetes

Similar Questions

Kubernetes: Docker pod starting fails except one
Accessing kubernetes running inside docker in a guest OS from the host OS
why run exec command?
Ways to manage pods in different executing result

2 Answers

1/20/2016

fedora@kubemaster ~]$ etcdctl ls --recursive /kubelab.local /kubelab.local/network /kubelab.local/network/config /kubelab.local/network/subnets /kubelab.local/network/subnets/172.16.46.0-24 /kubelab.local/network/subnets/172.16.12.0-24 /kubelab.local/network/subnets/172.16.70.0-24 /kubelab.local/network/subnets/172.16.21.0-24 /kubelab.local/network/subnets/172.16.54.0-24 /kubelab.local/network/subnets/172.16.71.0-24

This is showing flannel config, not skydns.

-- Tim Hockin
Source: StackOverflow
1/21/2016

you show the Replication controller info, but do you also have a Service setup?

apiVersion: v1
kind: Service
metadata:
  name: kube-dns
  namespace: kube-system
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: "KubeDNS"
spec:
  selector:
    k8s-app: kube-dns
  clusterIP: 10.3.0.10
  ports:
  - name: dns
    port: 53
    protocol: UDP
  - name: dns-tcp
    port: 53
    protocol: TCP
-- MrE
Source: StackOverflow