The following sections are: the errors, the configuration and the kubernetes version and the etcd version.
[root@xt3 kubernetes]# for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do systemctl restart $SERVICES; systemctl enable $SERVICES; systemctl status $SERVICES ; done
etcd.service - Etcd Server
Loaded: loaded (/usr/lib/systemd/system/etcd.service; enabled)
Active: active (running) since Fri 2016-03-25 11:11:25 CST; 58ms ago
Main PID: 6382 (etcd)
CGroup: /system.slice/etcd.service
й╕йд6382 /usr/bin/etcd
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: data dir = /var/lib/etcd/default.etcd
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: member dir = /var/lib/etcd/default.etcd/member
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: heartbeat = 100ms
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: election = 1000ms
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: snapshot count = 10000
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: advertise client URLs = http://localhost:2379,http://localhost:4001
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: loaded cluster information from store: default=http://localhost:2380,default=http://localhost:7001
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 etcdserver: restart member ce2a822cea30bfca in cluster 7e27652122e8b2ae at commit index 10686
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 raft: ce2a822cea30bfca became follower at term 8
Mar 25 11:11:25 xt3 etcd[6382]: 2016/03/25 11:11:25 raft: newRaft ce2a822cea30bfca [peers: [ce2a822cea30bfca], term: 8, commit: 10686, applied: 10001, lastindex: 10686, lastterm: 8]
Job for kube-apiserver.service failed. See 'systemctl status kube-apiserver.service' and 'journalctl -xn' for details.
kube-apiserver.service - Kubernetes API Server
Loaded: loaded (/usr/lib/systemd/system/kube-apiserver.service; enabled)
Active: activating (auto-restart) (Result: exit-code) since Fri 2016-03-25 11:11:35 CST; 58ms ago
Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 6401 (code=exited, status=255)
Mar 25 11:11:35 xt3 systemd[1]: Failed to start Kubernetes API Server.
Mar 25 11:11:35 xt3 systemd[1]: Unit kube-apiserver.service entered failed state.
kube-controller-manager.service - Kubernetes Controller Manager
Loaded: loaded (/usr/lib/systemd/system/kube-controller-manager.service; enabled)
Active: active (running) since Fri 2016-03-25 11:11:35 CST; 73ms ago
Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 6437 (kube-controller)
CGroup: /system.slice/kube-controller-manager.service
й╕йд6437 /usr/bin/kube-controller-manager --logtostderr=true --v=0 --master=http://127.0.0.1:8080
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.954951 6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.ReplicationController: Get http://127.0.0.1:8080/api/v1/replicationcontrollers: dia... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955075 6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.PersistentVolume: Get http://127.0.0.1:8080/api/v1/persistentvolumes: dial tcp 127.... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955159 6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods: dial tcp 127.0.0.1:8080: getsockopt: connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955222 6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.PersistentVolume: Get http://127.0.0.1:8080/api/v1/persistentvolumes: dial tcp 127.... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955248 6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Namespace: Get http://127.0.0.1:8080/api/v1/namespaces: dial tcp 127.0.0.1:8080: ge... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955331 6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.PersistentVolumeClaim: Get http://127.0.0.1:8080/api/v1/persistentvolumeclaims: dia... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955379 6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Namespace: Get http://127.0.0.1:8080/api/v1/namespaces: dial tcp 127.0.0.1:8080: ge... connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955430 6437 resource_quota_controller.go:62] Synchronization error: Get http://127.0.0.1:8080/api/v1/resourcequotas: dial tcp 127.0.0.1:8080: getsockopt: connection refused (&url....or)(0xc8204f2000)})
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955576 6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods: dial tcp 127.0.0.1:8080: getsockopt: connection refused
Mar 25 11:11:35 xt3 kube-controller-manager[6437]: E0325 11:11:35.955670 6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.ServiceAccount: Get http://127.0.0.1:8080/api/v1/serviceaccounts?fieldSelector=meta... connection refused
Hint: Some lines were ellipsized, use -l to show in full.
kube-scheduler.service - Kubernetes Scheduler Plugin
Loaded: loaded (/usr/lib/systemd/system/kube-scheduler.service; enabled)
Active: active (running) since Fri 2016-03-25 11:11:36 CST; 71ms ago
Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 6466 (kube-scheduler)
CGroup: /system.slice/kube-scheduler.service
й╕йд6466 /usr/bin/kube-scheduler --logtostderr=true --v=0 --master=http://127.0.0.1:8080
Mar 25 11:11:36 xt3 systemd[1]: Started Kubernetes Scheduler Plugin.
Mar 25 11:11:36 xt3 kube-scheduler[6466]: E0325 11:11:36.031318 6466 reflector.go:180] pkg/scheduler/factory/factory.go:194: Failed to list *api.ReplicationController: Get http://127.0.0.1:8080/api/v1/replicationcontrollers: dial tcp 127.0.0.1:...: connection refused
Mar 25 11:11:36 xt3 kube-scheduler[6466]: E0325 11:11:36.031421 6466 reflector.go:180] pkg/scheduler/factory/factory.go:189: Failed to list *api.Service: Get http://127.0.0.1:8080/api/v1/services: dial tcp 127.0.0.1:8080: getsockopt: connection refused
Mar 25 11:11:36 xt3 kube-scheduler[6466]: E0325 11:11:36.031564 6466 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%21%3D: dial tcp 127....: connection refused
Mar 25 11:11:36 xt3 kube-scheduler[6466]: E0325 11:11:36.031644 6466 reflector.go:180] pkg/scheduler/factory/factory.go:184: Failed to list *api.Node: Get http://127.0.0.1:8080/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse: dial tcp 127...: connection refused
Mar 25 11:11:36 xt3 kube-scheduler[6466]: E0325 11:11:36.031677 6466 reflector.go:180] pkg/scheduler/factory/factory.go:177: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%3D: dial tcp 127.0.0.1:8080:...: connection refused
Hint: Some lines were ellipsized, use -l to show in full.
[root@xt3 kubernetes]#
[root@xt3 kubernetes]#
The error details are the following.
[root@xt3 kubernetes]# journalctl -xn
-- Logs begin at Sat 2016-03-19 15:30:07 CST, end at Fri 2016-03-25 11:11:42 CST. --
Mar 25 11:11:41 xt3 kube-controller-manager[6437]: E0325 11:11:41.958470 6437 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.ServiceAccount: Get http://127.0.0.1:8080/api/v1/serviceaccounts?fieldSelector=metadata.name%3Ddefault: d
Mar 25 11:11:42 xt3 kube-scheduler[6466]: E0325 11:11:42.034315 6466 reflector.go:180] /usr/lib/golang/src/runtime/asm_amd64.s:1696: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%21%3D: dial tcp 127.0.0.1:8080: getsockopt:
Mar 25 11:11:42 xt3 kube-scheduler[6466]: E0325 11:11:42.034325 6466 reflector.go:180] pkg/scheduler/factory/factory.go:184: Failed to list *api.Node: Get http://127.0.0.1:8080/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse: dial tcp 127.0.0.1:8080: getsockopt
Mar 25 11:11:42 xt3 kube-scheduler[6466]: E0325 11:11:42.034324 6466 reflector.go:180] pkg/scheduler/factory/factory.go:189: Failed to list *api.Service: Get http://127.0.0.1:8080/api/v1/services: dial tcp 127.0.0.1:8080: getsockopt: connection refused
Mar 25 11:11:42 xt3 kube-scheduler[6466]: E0325 11:11:42.034413 6466 reflector.go:180] pkg/scheduler/factory/factory.go:194: Failed to list *api.ReplicationController: Get http://127.0.0.1:8080/api/v1/replicationcontrollers: dial tcp 127.0.0.1:8080: getsockopt: conne
Mar 25 11:11:42 xt3 kube-scheduler[6466]: E0325 11:11:42.034434 6466 reflector.go:180] pkg/scheduler/factory/factory.go:177: Failed to list *api.Pod: Get http://127.0.0.1:8080/api/v1/pods?fieldSelector=spec.nodeName%3D: dial tcp 127.0.0.1:8080: getsockopt: connection
Mar 25 11:11:42 xt3 kube-apiserver[6487]: E0325 11:11:42.206743 6487 reflector.go:180] pkg/admission/namespace/lifecycle/admission.go:95: Failed to list *api.Namespace: Get http://127.0.0.1:8080/api/v1/namespaces: dial tcp 127.0.0.1:8080: getsockopt: connection refus
Mar 25 11:11:42 xt3 kube-apiserver[6487]: E0325 11:11:42.206767 6487 reflector.go:180] pkg/admission/limitranger/admission.go:102: Failed to list *api.LimitRange: Get http://127.0.0.1:8080/api/v1/limitranges: dial tcp 127.0.0.1:8080: getsockopt: connection refused
Mar 25 11:11:42 xt3 kube-apiserver[6487]: E0325 11:11:42.206816 6487 reflector.go:180] pkg/admission/namespace/exists/admission.go:89: Failed to list *api.Namespace: Get http://127.0.0.1:8080/api/v1/namespaces: dial tcp 127.0.0.1:8080: getsockopt: connection refused
Mar 25 11:11:42 xt3 kube-apiserver[6487]: E0325 11:11:42.206831 6487 reflector.go:180] pkg/admission/resourcequota/admission.go:59: Failed to list *api.ResourceQuota: Get http://127.0.0.1:8080/api/v1/resourcequotas: dial tcp 127.0.0.1:8080: getsockopt: connection ref
[root@xt3 kubernetes]# The configurations are the following:
[root@xt3 kubernetes]# pwd
/etc/kubernetes
[root@xt3 kubernetes]# cat config
###
# kubernetes system config
#
# The following values are used to configure various aspects of all
# kubernetes services, including
#
# kube-apiserver.service
# kube-controller-manager.service
# kube-scheduler.service
# kubelet.service
# kube-proxy.service
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false"
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://127.0.0.1:8080"
[root@xt3 kubernetes]#
[root@xt3 kubernetes]#
[root@xt3 kubernetes]# cat apiserver
###
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#
# The address on the local server to listen to.
KUBE_API_ADDRESS="--insecure-bind-address=127.0.0.1"
# The port on the local server to listen on.
# KUBE_API_PORT="--port=8080"
# Port minions listen on
# KUBELET_PORT="--kubelet-port=10250"
# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:2379"
# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
# default admission control policies
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"
# Add your own!
KUBE_API_ARGS=""
[root@xt3 kubernetes]#
[root@xt3 kubernetes]#
[root@xt3 kubernetes]# ls
apiserver apiserver.rpmsave config config.rpmsave controller-manager kubelet proxy scheduler
[root@xt3 kubernetes]# cat controller-manager
###
# The following values are used to configure the kubernetes controller-manager
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS=""
[root@xt3 kubernetes]#
[root@xt3 kubernetes]# cat kubelet
###
# kubernetes kubelet (minion) config
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=127.0.0.1"
# The port for the info server to serve on
# KUBELET_PORT="--port=10250"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=127.0.0.1"
# location of the api-server
KUBELET_API_SERVER="--api-servers=http://127.0.0.1:8080"
# Add your own!
KUBELET_ARGS=""
[root@xt3 kubernetes]#
[root@xt3 kubernetes]# cat proxy
###
# kubernetes proxy config
# default config should be adequate
# Add your own!
KUBE_PROXY_ARGS=""
[root@xt3 kubernetes]#
[root@xt3 kubernetes]#
[root@xt3 kubernetes]#
[root@xt3 kubernetes]# cat scheduler
###
# kubernetes scheduler config
# default config should be adequate
# Add your own!
KUBE_SCHEDULER_ARGS=""The versions of the kubernetes and etcd:
[root@xt3 kubernetes]# rpm -qa | grep kuber
kubernetes-node-1.1.0-0.4.git2bfa9a1.el7.x86_64I do all the configurations as the kubernetes sites told.(http://kubernetes.io/docs/getting-started-guides/centos/centos_manual_config/)
kubernetes-client-1.1.0-0.4.git2bfa9a1.el7.x86_64
kubernetes-1.1.0-0.4.git2bfa9a1.el7.x86_64
kubernetes-master-1.1.0-0.4.git2bfa9a1.el7.x86_64
[root@xt3 kubernetes]# rpm -qa | grep etcd
etcd-2.0.9-1.el7.x86_64I look forward to replying for the answers. Please contact me. Thanks very much.
the log you provided is not enough. you can see all the detaile log in by use
tail -n 1000 /var/log/messagesIn my case , I stop the kube-apiserver service first and then start it ,and to search in the /var/log/messages.And got my error is a file is removed because I restart the machine.It's may not be your reason, but you can find it use the command I said.
good luck.
Running the following commands should make it work as the kubelet may be stuck in a crashloop:
sudo systemctl stop kubelet
sudo rm -rf /var/lib/kubelet/pods/*
sudo systemctl start kubelet
I had a very similar issue, in my case I fixed it changing the KUBE_API_PORT= to another one avaliable on my system like KUBE_API_PORT=9090
Try to curl 127.0.0.1:8080 to figure out if that port is being used by another service