I installed K8s on my two Ubuntus in VirtualBox (Master and Node01). After installation (I proceeded according K8s doc site) I typed kubectl get nodes and got bot servers in status Ready. But after restart systems I got this:
# kubectl get nodes
The connection to the server localhost:8080 was refused - did you specify the
right host or port? I checked kubelet service and it is running:
# systemctl status kubelet
kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: active (running) since Mon 2017-04-24 10:01:51 CEST; 15min ago
Docs: http://kubernetes.io/docs/
Main PID: 13128 (kubelet)
Tasks: 21
Memory: 48.2M
CPU: 58.014s
CGroup: /system.slice/kubelet.service
├─13128 /usr/bin/kubelet --kubeconfig=/etc/kubernetes/kubelet.conf --require-kubeconfig=true --pod-manifest-path=/etc/kubernetes/manifests --allow-privileged=true --cluster-dns=10.96.0.10 --cluster-domain=cluster.local
└─13164 journalctl -k -f
Apr 24 10:16:40 master kubelet[13128]: I0424 10:16:40.204156 13128 kuberuntime_manager.go:752] Back-off 5m0s restarting failed container=weave pod=weave-net-5qgvz_kube-system(4b7bb2f0-2691-11e7-bfb6-080027229776)
Apr 24 10:16:40 master kubelet[13128]: E0424 10:16:40.204694 13128 pod_workers.go:182] Error syncing pod 4b7bb2f0-2691-11e7-bfb6-080027229776 ("weave-net-5qgvz_kube-system(4b7bb2f0-2691-11e7-bfb6-080027229776)"), skipping: fail
Apr 24 10:16:42 master kubelet[13128]: I0424 10:16:42.972302 13128 operation_generator.go:597] MountVolume.SetUp succeeded for volume "kubernetes.io/secret/2b59d0d9-2692-11e7-bfb6-080027229776-default-token-h3v7c" (spec.Name: "
Apr 24 10:16:48 master kubelet[13128]: I0424 10:16:48.949731 13128 operation_generator.go:597] MountVolume.SetUp succeeded for volume "kubernetes.io/secret/2bb42bc1-2692-11e7-bfb6-080027229776-default-token-h3v7c" (spec.Name: "
Apr 24 10:16:51 master kubelet[13128]: I0424 10:16:51.978663 13128 operation_generator.go:597] MountVolume.SetUp succeeded for volume "kubernetes.io/secret/2b023c31-2692-11e7-bfb6-080027229776-default-token-h3v7c" (spec.Name: "
Apr 24 10:16:52 master kubelet[13128]: I0424 10:16:52.909589 13128 operation_generator.go:597] MountVolume.SetUp succeeded for volume "kubernetes.io/secret/4b7bb2f0-2691-11e7-bfb6-080027229776-default-token-gslqd" (spec.Name: "
Apr 24 10:16:53 master kubelet[13128]: I0424 10:16:53.186057 13128 kuberuntime_manager.go:458] Container {Name:weave Image:weaveworks/weave-kube:1.9.4 Command:[/home/weave/launch.sh] Args:[] WorkingDir: Ports:[] EnvFrom:[] Env:
Apr 24 10:16:53 master kubelet[13128]: I0424 10:16:53.188091 13128 kuberuntime_manager.go:742] checking backoff for container "weave" in pod "weave-net-5qgvz_kube-system(4b7bb2f0-2691-11e7-bfb6-080027229776)"
Apr 24 10:16:53 master kubelet[13128]: I0424 10:16:53.188717 13128 kuberuntime_manager.go:752] Back-off 5m0s restarting failed container=weave pod=weave-net-5qgvz_kube-system(4b7bb2f0-2691-11e7-bfb6-080027229776)
Apr 24 10:16:53 master kubelet[13128]: E0424 10:16:53.189136 13128 pod_workers.go:182] Error syncing pod 4b7bb2f0-2691-11e7-bfb6-080027229776 ("weave-net-5qgvz_kube-system(4b7bb2f0-2691-11e7-bfb6-080027229776)"), skipping: fail Here is systemd log file with restarted kubelet: Google Drive.
... I'm not sure what I missed in doc or what happend with kubelet. Can I ask you for help? :]
• Ubuntu version
cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.2 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.2 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial • Kernel
# uname -a
Linux ubuntu 4.4.0-72-generic #93-Ubuntu SMP Fri Mar 31 14:07:41 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux • Kubectl version
# kubectl version
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.1", GitCommit:"b0b7a323cc5a4a2019b2e9520c21c7830b7f708e", GitTreeState:"clean", BuildDate:"2017-04-03T20:44:38Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.0", GitCommit:"fff5156092b56e6bd60fff75aad4dc9de6b6ef37", GitTreeState:"clean", BuildDate:"2017-03-28T16:24:30Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} • Kubeadm version
# kubeadm version
kubeadm version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.1", GitCommit:"b0b7a323cc5a4a2019b2e9520c21c7830b7f708e", GitTreeState:"clean", BuildDate:"2017-04-03T20:33:27Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} • Kubelet version
# kubelet --version
Kubernetes v1.6.1 • Docker version
# docker version
Client:
Version: 1.11.2
API version: 1.23
Go version: go1.5.4
Git commit: b9f10c9
Built: Wed Jun 1 22:00:43 2016
OS/Arch: linux/amd64
Server:
Version: 1.11.2
API version: 1.23
Go version: go1.5.4
Git commit: b9f10c9
Built: Wed Jun 1 22:00:43 2016
OS/Arch: linux/amd64 I had the same problem with kubernetes 1.12.3 and ubuntu 16.04.05. I then looked at the kubernetes log by running the command
$ journalctl -u kubelet
and then in the log i saw that k8s was complaining (exiting with status 255) about swap being on.
So i then turned swap off by running
$ swapoff -a
Then i edited fstab and commented out the entry for swap
$ vi /etc/fstab
#comment out line with swapand then rebooting the system. After the system came back up, i verified that swap was disabled by running
$ free -m
and checking whether the row for swap has 0. It did.
Then i verified that kubeapi service had successfully started by executing
$ systemctl status kubelet
It has successfully started. I verified by also re-checking journalctl logs. Did not see swap error this time.
I verified k8s node status by running
$ kubectl get nodes
which was now working and showing expected output.
NOTE: I had KUBECONFIG set in my .bash_profile file as well, previously.
root@k8s-master:~# cat .bash_profile
export KUBECONFIG="/etc/kubernetes/admin.conf"I had bad exported variable KUBECONFIG which is needed by kubelet (history details are in comment under question). To ~/.zprofile I saved KUBECONFIG=$HOME/admin.conf which solved my problem.
After reloading ENV variables is kubelet working:
# kubectl get nodes
NAME STATUS AGE VERSION
master Ready 5d v1.6.1
node01 NotReady 5d v1.6.1As the comment there, you really need check whether apiserver is started, because kubectl will talk to apiserver. While from you description and version of kubeadm, I believe this is a duplicate question I just answered, so I just copy the answer to here.
In current version of kubeadm(v1.6.1), insecure port of ApiServer is abandoned by default, you can verify this by checking api-server yaml file in /etc/kubernetes/manifests/kube-apiserver.yaml, there is kube-apiserver parameter --insecure-port=0.
You can
Correct this in a running cluster:
$ mv kube-apiserver.yaml ../kube-apiserver.yaml
// edit ../kube-apiserver.yaml to remove --insecure-port=0
// or change it to --insecure-port=<WHATERER_YOUR_LIKE>
$ mv ../kube-apiserver.yaml kube-apiserver.yaml
Do it right at startup. You need a kubeadm config file to do this. A simple one would like:
apiVersion: kubeadm.k8s.io/v1alpha1
kind: MasterConfiguration
apiServerExtraArgs:
insecure-port: 8080 //or whatever you like
// Then you can start a master node use `kubeadm init --config=<this-configure-file-path>`