Just started with kubernetes. I have three physical machines: 1 master, 2 nodes. I did basically getting started configuration. Everything seems up and running, nodes can communicate with master, but when I try to install a sample application (see in following kubernetes guide) I get this warning over and over again:
kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
And I can't access the app in: master_ip:30001
Any idea in what is going on and how to fix it?
Here is the configuration:
$ kubectl get nodes
NAME STATUS AGE VERSION
master-precision-t1600 Ready 19h v1.7.4
node2-precision-t1600 Ready 19h v1.7.4
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system etcd-master-precision-t1600 1/1 Running 1 19h
kube-system kube-apiserver-master-precision-t1600 1/1 Running 1 19h
kube-system kube-controller-manager-master-precision-t1600 1/1 Running 1 19h
kube-system kube-dns-2425271678-xrlp4 3/3 Running 3 19h
kube-system kube-proxy-58qm6 1/1 Running 1 19h
kube-system kube-proxy-tjskl 1/1 Running 1 19h
kube-system kube-scheduler-master-precision-t1600 1/1 Running 1 19h
kube-system kubernetes-dashboard-3313488171-7n56j 1/1 Running 0 38m
kube-system weave-net-1hjxl 2/2 Running 2 19h
kube-system weave-net-lwk8r 2/2 Running 2 19h
sock-shop carts-2469883122-h8f4n 1/1 Running 0 1h
sock-shop carts-db-1721187500-pkpk0 1/1 Running 0 1h
sock-shop catalogue-4293036822-hpkgp 1/1 Running 0 1h
sock-shop catalogue-db-1846494424-xlb8m 1/1 Running 0 1h
sock-shop front-end-2337481689-s8bkm 1/1 Running 0 1h
sock-shop orders-733484335-n7h4c 1/1 Running 0 1h
sock-shop orders-db-3728196820-12rt8 1/1 Running 0 1h
sock-shop payment-3050936124-kwqfs 1/1 Running 0 1h
sock-shop queue-master-2067646375-n8sgj 1/1 Running 0 1h
sock-shop rabbitmq-241640118-dqh6p 1/1 Running 0 1h
sock-shop shipping-2463450563-g01sw 1/1 Running 0 1h
sock-shop user-1574605338-kwqmp 1/1 Running 0 1h
sock-shop user-db-3152184577-w3f39 1/1 Running 0 1h
$ kubectl describe nodes
Name: master-precision-t1600
Role:
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/hostname=master-precision-t1600
node-role.kubernetes.io/master=
Annotations: node.alpha.kubernetes.io/ttl=0
volumes.kubernetes.io/controller-managed-attach-detach=true
Taints: node-role.kubernetes.io/master:NoSchedule
CreationTimestamp: Tue, 22 Aug 2017 17:05:06 +0200
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
OutOfDisk False Wed, 23 Aug 2017 12:26:45 +0200 Tue, 22 Aug 2017 17:05:02 +0200 KubeletHasSufficientDisk kubelet has sufficient disk space available
MemoryPressure False Wed, 23 Aug 2017 12:26:45 +0200 Tue, 22 Aug 2017 17:05:02 +0200 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Wed, 23 Aug 2017 12:26:45 +0200 Tue, 22 Aug 2017 17:05:02 +0200 KubeletHasNoDiskPressure kubelet has no disk pressure
Ready True Wed, 23 Aug 2017 12:26:45 +0200 Tue, 22 Aug 2017 17:05:02 +0200 KubeletReady kubelet is posting ready status. AppArmor enabled
Addresses:
InternalIP: xxx.xxx.xxx.215
Hostname: master-precision-t1600
Capacity:
alpha.kubernetes.io/nvidia-gpu: 0
cpu: 8
memory: 8127968Ki
pods: 110
Allocatable:
alpha.kubernetes.io/nvidia-gpu: 0
cpu: 8
memory: 8025568Ki
pods: 110
System Info:
Machine ID: d718aa59fbe54581a9b058eb453ca453
System UUID: 4C4C4544-005A-4410-805A-C4C04F32354A
Boot ID: 687c603a-aad9-477a-a398-dfffeeaa4cd0
Kernel Version: 4.10.0-32-generic
OS Image: Ubuntu 16.04.3 LTS
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://1.11.2
Kubelet Version: v1.7.4
Kube-Proxy Version: v1.7.4
ExternalID: master-precision-t1600
Non-terminated Pods: (8 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits
--------- ---- ------------ ---------- --------------- -------------
kube-system etcd-master-precision-t1600 0 (0%) 0 (0%) 0 (0%) 0 (0%)
kube-system kube-apiserver-master-precision-t1600 250m (3%) 0 (0%) 0 (0%) 0 (0%)
kube-system kube-controller-manager-master-precision-t1600 200m (2%) 0 (0%) 0 (0%) 0 (0%)
kube-system kube-dns-2425271678-xrlp4 260m (3%) 0 (0%) 110Mi (1%) 170Mi (2%)
kube-system kube-proxy-58qm6 0 (0%) 0 (0%) 0 (0%) 0 (0%)
kube-system kube-scheduler-master-precision-t1600 100m (1%) 0 (0%) 0 (0%) 0 (0%)
kube-system kubernetes-dashboard-3313488171-7n56j 0 (0%) 0 (0%) 0 (0%) 0 (0%)
kube-system weave-net-1hjxl 20m (0%) 0 (0%) 0 (0%) 0 (0%)
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
CPU Requests CPU Limits Memory Requests Memory Limits
------------ ---------- --------------- -------------
830m (10%) 0 (0%) 110Mi (1%) 170Mi (2%)
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
39m 30s 36 kubelet, master-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "kubernetes-dashboard-3313488171-7n56j_kube-system(1ed597d4-87e8-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
Name: node2-precision-t1600
Role:
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
kubernetes.io/hostname=node2-precision-t1600
Annotations: node.alpha.kubernetes.io/ttl=0
volumes.kubernetes.io/controller-managed-attach-detach=true
Taints: <none>
CreationTimestamp: Tue, 22 Aug 2017 17:10:43 +0200
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
OutOfDisk False Wed, 23 Aug 2017 12:26:49 +0200 Wed, 23 Aug 2017 11:42:43 +0200 KubeletHasSufficientDisk kubelet has sufficient disk space available
MemoryPressure False Wed, 23 Aug 2017 12:26:49 +0200 Wed, 23 Aug 2017 11:42:43 +0200 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Wed, 23 Aug 2017 12:26:49 +0200 Wed, 23 Aug 2017 11:42:43 +0200 KubeletHasNoDiskPressure kubelet has no disk pressure
Ready True Wed, 23 Aug 2017 12:26:49 +0200 Wed, 23 Aug 2017 11:42:43 +0200 KubeletReady kubelet is posting ready status. AppArmor enabled
Addresses:
InternalIP: 129.241.110.167
Hostname: node2-precision-t1600
Capacity:
alpha.kubernetes.io/nvidia-gpu: 1
cpu: 8
memory: 8127968Ki
pods: 110
Allocatable:
alpha.kubernetes.io/nvidia-gpu: 1
cpu: 8
memory: 8025568Ki
pods: 110
System Info:
Machine ID: d701c70173f547168978ca276552bb88
System UUID: 4C4C4544-005A-4410-805A-B5C04F32354A
Boot ID: 827de455-66cb-481d-a362-557a17db11f4
Kernel Version: 4.10.0-32-generic
OS Image: Ubuntu 16.04.3 LTS
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://1.11.2
Kubelet Version: v1.7.4
Kube-Proxy Version: v1.7.4
ExternalID: node2-precision-t1600
Non-terminated Pods: (15 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits
--------- ---- ------------ ---------- --------------- -------------
kube-system kube-proxy-tjskl 0 (0%) 0 (0%) 0 (0%) 0 (0%)
kube-system weave-net-lwk8r 20m (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop carts-2469883122-h8f4n 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop carts-db-1721187500-pkpk0 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop catalogue-4293036822-hpkgp 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop catalogue-db-1846494424-xlb8m 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop front-end-2337481689-s8bkm 100m (1%) 0 (0%) 100Mi (1%) 0 (0%)
sock-shop orders-733484335-n7h4c 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop orders-db-3728196820-12rt8 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop payment-3050936124-kwqfs 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop queue-master-2067646375-n8sgj 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop rabbitmq-241640118-dqh6p 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop shipping-2463450563-g01sw 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop user-1574605338-kwqmp 0 (0%) 0 (0%) 0 (0%) 0 (0%)
sock-shop user-db-3152184577-w3f39 0 (0%) 0 (0%) 0 (0%) 0 (0%)
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
CPU Requests CPU Limits Memory Requests Memory Limits
------------ ---------- --------------- -------------
120m (1%) 0 (0%) 100Mi (1%) 0 (0%)
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
44m 44m 1 kubelet, node2-precision-t1600 Normal NodeReady Node node2-precision-t1600 status is now: NodeReady
44m 44m 3 kubelet, node2-precision-t1600 Normal NodeHasSufficientDisk Node node2-precision-t1600 status is now: NodeHasSufficientDisk
44m 44m 3 kubelet, node2-precision-t1600 Normal NodeHasSufficientMemory Node node2-precision-t1600 status is now: NodeHasSufficientMemory
44m 44m 3 kubelet, node2-precision-t1600 Normal NodeHasNoDiskPressure Node node2-precision-t1600 status is now: NodeHasNoDiskPressure
44m 44m 1 kubelet, node2-precision-t1600 Normal NodeAllocatableEnforced Updated Node Allocatable limit across pods
44m 44m 1 kubelet, node2-precision-t1600 Warning Rebooted Node node2-precision-t1600 has been rebooted, boot id: 827de455-66cb-481d-a362-557a17db11f4
44m 44m 1 kubelet, node2-precision-t1600 Normal Starting Starting kubelet.
44m 44m 1 kube-proxy, node2-precision-t1600 Normal Starting Starting kube-proxy.
43m 11m 9 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "rabbitmq-241640118-dqh6p_sock-shop(79e3bb08-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
44m 5m 24 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "orders-db-3728196820-12rt8_sock-shop(79ca1e21-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
44m 2m 22 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "payment-3050936124-kwqfs_sock-shop(79cb96f4-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
44m 2m 28 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "shipping-2463450563-g01sw_sock-shop(79fa9dd4-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
43m 2m 22 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "carts-2469883122-h8f4n_sock-shop(79bbf964-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
44m 1m 30 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "user-db-3152184577-w3f39_sock-shop(7a303582-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
44m 1m 16 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "user-1574605338-kwqmp_sock-shop(7a11a937-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
44m 1m 20 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "catalogue-db-1846494424-xlb8m_sock-shop(79c24789-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
44m 54s 26 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "queue-master-2067646375-n8sgj_sock-shop(79d46bb2-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
44m 45s 30 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "front-end-2337481689-s8bkm_sock-shop(79c49a6c-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
44m 24s 11 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "carts-db-1721187500-pkpk0_sock-shop(79bd1f99-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
43m 11s 25 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "catalogue-4293036822-hpkgp_sock-shop(79bf628c-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
43m 3s 25 kubelet, node2-precision-t1600 Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "orders-733484335-n7h4c_sock-shop(79c6f31c-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.
43m 0s 228 kubelet, node2-precision-t1600 Warning MissingClusterDNS (combined from similar events): kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. pod: "payment-3050936124-kwqfs_sock-shop(79cb96f4-87e4-11e7-ab01-782bcba630bb)". Falling back to DNSDefault policy.As Mike told me in his answer I got the DNS IP with : kubectl get services --namespace=kube-system. Things I've tried so far to change the command-line flags:
Both without results
The kubelet service needs a command-line flag to set the cluster DNS IP - it looks like you're running kube-dns, so you can get that IP by either running kubectl get services --namespace=kube-system or grabbing the IP from the "ClusterIP" field on the kube-dns service YAML or JSON config.
Once you have the IP, you'll have to set the --cluster-dns command-line flag for kubelet.
I haven't used kubeadm to setup a cluster, so I'm not sure how it runs the services and can't say how to change the command-line flags - hopefully somebody who knows can provide input for that piece.