I´m setting up kubernetes on GKE as described in Kelsey Hightowers https://github.com/kelseyhightower/kubernetes-the-hard-way/
Everything works fine except for setting up the DNS ClusterAddon https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/12-dns-addon.md
When I start kube-dns like that:
kubectl create -f https://storage.googleapis.com/kubernetes-the-hard-way/kube-dns.yaml
I do get the expected output :
serviceaccount "kube-dns" created
configmap "kube-dns" created
service "kube-dns"
created deployment "kube-dns" createdBut checking state of the pods and the output of the kube-dns container I see errors:
kubectl get po -n kube-system
NAME READY STATUS RESTARTS AGE
kube-dns-6c857864fb-cpvvr 2/3 CrashLoopBackOff 63 2hand in the container log:
I0115 13:22:35.272492 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0115 13:22:35.772476 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0115 13:22:36.272406 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0115 13:22:36.772356 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
I0115 13:22:37.272386 1 dns.go:173] Waiting for services and endpoints to be initialized from apiserver...
E0115 13:22:37.273178 1 reflector.go:201] k8s.io/dns/pkg/dns/dns.go:147: Failed to list *v1.Endpoints: Get https://10.32.0.1:443/api/v1/endpoints?resourceVersion=0: dial tcp 10.32.0.1:443: i/o timeout
E0115 13:22:37.273340 1 reflector.go:201] k8s.io/dns/pkg/dns/dns.go:150: Failed to list *v1.Service: Get https://10.32.0.1:443/api/v1/services?resourceVersion=0: dial tcp 10.32.0.1:443: i/o timeoutThe URL https://10.32.0.1:443 in the container log seems to be wrong, but I cannot find any place where I can specify a different URL and neither the place where this URL is set in the config file https://storage.googleapis.com/kubernetes-the-hard-way/kube-dns.yaml
I was using kubespray (release v2.5.0) to try to setup a kubernetes (version 1.10.4) cluster on Openstack and got problem with exactly same error messages. Google led me here but didn't provide a solution for this problem.
My final solution is change kube_proxy_mode option in inventory/mycluster/group_vars/kube-cluster.yml, from default value of 'iptables' to 'ipvs':
# Kube-proxy proxyMode configuration.
# Can be ipvs, iptables
kube_proxy_mode: ipvs
After that re-run the ansible playbook command, then this issue is gone and all services/pods are running as expected. Hope it's helpful for those who try with the same toolchain to setup a kubernetes cluster.
The URL comes from internal kubernetes information (service account token) and it should be fine (it should point to the first IP in the range assigned to service network, and that should be the kubernetes.default service. What you need to check is if your pod-to-pod networking and kube-proxy (it implements service ClusterIPs) works as expected.
If you do kubectl get svc kubernetes -o yaml you should see a kubernetes service with that 10.32.0.1 IP so confirm that as well (apiserver registers its own IP for this svc so doing ksp get endpoints kubernetes should give you API IP/PORT)