I'm currently trying to learn Kubernetes by creating a Vagrant/Ansible project that allows me to bring up a master and node(s) and I've run into some problems. I'm currently use kubeadm
to run the init
and join
function. For the init
, I'm parsing the command line output to get the token (which I know is terrible; is there a more standard way to do this?):
---
- command: kubeadm init
args:
creates: /etc/kubernetes/kubelet.conf
register: kube_init
- local_action: copy content={{ kube_init.stdout_lines[-1].split()[-2] }} dest=../kube.token
when: kube_init is defined
- local_action: copy content={{ kube_init.stdout_lines[-1].split()[-1] }} dest=../kube.master_ip
when: kube_init is defined
But it works. The trouble is, I can't get the node to connect:
TASK: [node | command kubeadm join --token=1a3131.637de46bfdebadd4 10.0.2.15] ***
failed: [node1] => {"changed": true, "cmd": ["kubeadm", "join", "--token=1a3131.637de46bfdebadd4", "10.0.2.15"], "delta": "0:00:00.079545", "end": "2016-10-25 01:54:51.858545", "rc": 1, "start": "2016-10-25 01:54:51.779000", "warnings": []}
stderr: error: <node/discovery> failed to request cluster info [Get http://10.0.2.15:9898/cluster-info/v1/?token-id=1a3131: dial tcp 10.0.2.15:9898: getsockopt: connection refused]
stdout: <util/tokens> validating provided token
<node/discovery> created cluster info discovery client, requesting info from "http://10.0.2.15:9898/cluster-info/v1/?token-id=1a3131"
FATAL: all hosts have already failed -- aborting
I can ping my master VM from the node VM and I can even ssh to it. It appears as if the kubernetes services are all running on the master too:
$ systemctl status kubelet.service
● kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: active (running) since Tue 2016-10-25 01:51:35 UTC; 13h ago
Docs: http://kubernetes.io/docs/
Main PID: 5726 (kubelet)
Tasks: 15
Memory: 45.5M
CPU: 11min 37.179s
CGroup: /system.slice/kubelet.service
├─5726 /usr/bin/kubelet --kubeconfig=/etc/kubernetes/kubelet.conf --require-kubeconfig=true --pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin --cluster-
└─5740 journalctl -k -f
Oct 25 15:43:35 ubuntu-xenial kubelet[5726]: I1025 15:43:35.532171 5726 factory.go:108] Factory "systemd" can handle container "/system.slice/var-lib-kubelet-pods-9d0e93cd\\x2d9a55\\x2d11e6\\x2db2dd\\x2d02558d159192-volumes-kubernetes.io\\x7esecret-default\\
Oct 25 15:43:35 ubuntu-xenial kubelet[5726]: I1025 15:43:35.532175 5726 manager.go:843] ignoring container "/system.slice/var-lib-kubelet-pods-9d0e93cd\\x2d9a55\\x2d11e6\\x2db2dd\\x2d02558d159192-volumes-kubernetes.io\\x7esecret-default\\x2dtoken\\x2ddn3r9.m
Oct 25 15:43:35 ubuntu-xenial kubelet[5726]: I1025 15:43:35.532184 5726 factory.go:104] Error trying to work out if we can handle /system.slice/run-docker-netns-default.mount: invalid container name
Oct 25 15:43:35 ubuntu-xenial kubelet[5726]: I1025 15:43:35.532187 5726 factory.go:115] Factory "docker" was unable to handle container "/system.slice/run-docker-netns-default.mount"
Oct 25 15:43:35 ubuntu-xenial kubelet[5726]: I1025 15:43:35.532190 5726 factory.go:108] Factory "systemd" can handle container "/system.slice/run-docker-netns-default.mount", but ignoring.
Oct 25 15:43:35 ubuntu-xenial kubelet[5726]: I1025 15:43:35.532194 5726 manager.go:843] ignoring container "/system.slice/run-docker-netns-default.mount"
Oct 25 15:43:35 ubuntu-xenial kubelet[5726]: I1025 15:43:35.532316 5726 factory.go:104] Error trying to work out if we can handle /system.slice/var-lib-docker-overlay-6d625f0d241bc6ae9eb243b430a71f078912a76147fc51e41e8e6e5f3226c6e9-merged.mount: error inspec
Oct 25 15:43:35 ubuntu-xenial kubelet[5726]: I1025 15:43:35.532321 5726 factory.go:115] Factory "docker" was unable to handle container "/system.slice/var-lib-docker-overlay-6d625f0d241bc6ae9eb243b430a71f078912a76147fc51e41e8e6e5f3226c6e9-merged.mount"
Oct 25 15:43:35 ubuntu-xenial kubelet[5726]: I1025 15:43:35.532325 5726 factory.go:108] Factory "systemd" can handle container "/system.slice/var-lib-docker-overlay-6d625f0d241bc6ae9eb243b430a71f078912a76147fc51e41e8e6e5f3226c6e9-merged.mount", but ignoring.
Oct 25 15:43:35 ubuntu-xenial kubelet[5726]: I1025 15:43:35.532330 5726 manager.go:843] ignoring container "/system.slice/var-lib-docker-overlay-6d625f0d241bc6ae9eb243b430a71f078912a76147fc51e41e8e6e5f3226c6e9-merged.mount"
Why is my master not listening for connections? Here is the exact commit I'm using:
https://gitlab.com/djsumdog/skube/tree/0e25fca668e1067a8aa4af59b019411303838e86
If you have ansible and vagrant installed, you should be able to run vagrant up
and get the same result.
It turns out this was due to my Vagrant setup. Both VMs were getting the same IP address. I needed to specify a :private_network
and an ip:
in my Vagrantfile:
...
config.vm.define "master" do |master|
master.vm.network :private_network, ip: 10.10.30.10
...
...
config.vm.define "node1" do |node1|
node1.vm.network :private_network, :ip: 10.10.30.50
...
Try executing following commands after kubeadm init and before kubeadm join
kubectl taint nodes --all dedicated-
kubectl apply -f <add-on.yaml>
Refer this page http://kubernetes.io/docs/getting-started-guides/kubeadm/