Kubernetes API server is failing to start: api-server(master-host) host not found
12/21/2019
I had set-up the Kubernetes cluster using kubeadm and it was working fine. I was able to deploy pods and service. I had tried restarting the cluster and it used to come up without any issues. Suddenly, after a few days, I had to restart my machine(hosting clusters on vms) and the cluster started not coming up. Not able to understand why is this started happening now. Can somebody help me please in troubleshooting this issue?
This is how my cluster looks:
Node Hostname IP
master master-kube 192.168.50.10
worker-node-1 node-1-kube 192.168.50.11
worker-node-2 node-2-kube 192.168.50.12 All nodes(ubuntu bionic) hosted on a virtual box.
More details on cluster:
kubelet = 1.16.0-00
kubeadm = 1.16.0-00
kubectl = 1.16.0-00
docker = 18.06.2-ce, build 6d37f41
pod-network = calico-latest
command used to init cluster = kubeadm init --apiserver-advertise-address="192.168.50.10" --apiserver-cert-extra-sans="192.168.50.10" --pod-network-cidr=172.16.0.0/16I would like to highlight the errors I am getting currently to restart the kubelet:
kubelet[19946]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubel
kubelet[19946]: Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet
kubelet[19946]: Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubel
kubelet[19946]: Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet
kubelet[19946]: I1221 05:37:49.631489 19946 server.go:410] Version: v1.16.0
kubelet[19946]: I1221 05:37:49.631636 19946 plugins.go:100] No cloud provider specified.
kubelet[19946]: I1221 05:37:49.631649 19946 server.go:773] Client rotation is on, will bootstrap in background
kubelet[19946]: I1221 05:37:49.633624 19946 certificate_store.go:129] Loading cert/key pair from "/var/lib/kubelet/pki/kubelet-client-current.pem".
kubelet[19946]: I1221 05:37:49.658641 19946 server.go:644] --cgroups-per-qos enabled, but --cgroup-root was not specified. defaulting to /
kubelet[19946]: I1221 05:37:49.659118 19946 container_manager_linux.go:265] container manager verified user specified cgroup-root exists: []
kubelet[19946]: I1221 05:37:49.659269 19946 container_manager_linux.go:270] Creating Container Manager object based on Node Config: {RuntimeCgroupsName: SystemCgroupsName: KubeletCgroupsName: ContainerRuntime
kubelet[19946]: I1221 05:37:49.659508 19946 fake_topology_manager.go:29] [fake topologymanager] NewFakeManager
kubelet[19946]: I1221 05:37:49.659604 19946 container_manager_linux.go:305] Creating device plugin manager: true
kubelet[19946]: I1221 05:37:49.659735 19946 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider: &{kubelet.sock /var/lib/kubelet/device-plugins/ map[] {0 0} <nil> {{} [0 0 0]} 0x
kubelet[19946]: I1221 05:37:49.659850 19946 state_mem.go:36] [cpumanager] initializing new in-memory state store
kubelet[19946]: I1221 05:37:49.660030 19946 state_mem.go:84] [cpumanager] updated default cpuset: ""
kubelet[19946]: I1221 05:37:49.660127 19946 state_mem.go:92] [cpumanager] updated cpuset assignments: "map[]"
kubelet[19946]: I1221 05:37:49.660220 19946 fake_topology_manager.go:39] [fake topologymanager] AddHintProvider HintProvider: &{{0 0} 0x799d338 10000000000 0xc0009ee9c0 <nil> <nil> <nil> <nil> map[memory:{{1
kubelet[19946]: I1221 05:37:49.660363 19946 kubelet.go:287] Adding pod path: /etc/kubernetes/manifests
kubelet[19946]: I1221 05:37:49.660502 19946 kubelet.go:312] Watching apiserver
kubelet[19946]: E1221 05:37:49.663227 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
kubelet[19946]: E1221 05:37:49.663829 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
kubelet[19946]: I1221 05:37:49.665047 19946 client.go:75] Connecting to docker on unix:///var/run/docker.sock
kubelet[19946]: I1221 05:37:49.665066 19946 client.go:104] Start docker client with request timeout=2m0s
kubelet[19946]: E1221 05:37:49.665772 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
kubelet[19946]: W1221 05:37:49.667035 19946 docker_service.go:563] Hairpin mode set to "promiscuous-bridge" but kubenet is not enabled, falling back to "hairpin-veth"
kubelet[19946]: I1221 05:37:49.667144 19946 docker_service.go:240] Hairpin mode set to "hairpin-veth"
kubelet[19946]: I1221 05:37:49.694036 19946 docker_service.go:255] Docker cri networking managed by cni
kubelet[19946]: I1221 05:37:49.701825 19946 docker_service.go:260] Docker Info: &{ID:OF2B:RLPC:I6RY:MIMU:H5S4:3GBP:QB6T:IXQF:TMTB:7JBZ:XY2O:ICQO Containers:28 ContainersRunning:6 ContainersPaused:0 Containers
kubelet[19946]: I1221 05:37:49.702214 19946 docker_service.go:273] Setting cgroupDriver to cgroupfs
kubelet[19946]: I1221 05:37:49.711797 19946 remote_runtime.go:59] parsed scheme: ""
kubelet[19946]: I1221 05:37:49.711813 19946 remote_runtime.go:59] scheme "" not registered, fallback to default scheme
kubelet[19946]: I1221 05:37:49.711835 19946 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/run/dockershim.sock 0 <nil>}] <nil>}
kubelet[19946]: I1221 05:37:49.711841 19946 clientconn.go:577] ClientConn switching balancer to "pick_first"
kubelet[19946]: I1221 05:37:49.711856 19946 remote_image.go:50] parsed scheme: ""
kubelet[19946]: I1221 05:37:49.711860 19946 remote_image.go:50] scheme "" not registered, fallback to default scheme
Dec 21 05:37:49 master-kube kubelet[19946]: I1221 05:37:49.711866 19946 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{/var/run/dockershim.sock 0 <nil>}] <nil>}
kubelet[19946]: I1221 05:37:49.711870 19946 clientconn.go:577] ClientConn switching balancer to "pick_first"
kubelet[19946]: E1221 05:37:49.916339 19946 aws_credentials.go:77] while getting AWS credentials NoCredentialProviders: no valid providers in chain. Deprecated.
kubelet[19946]: For verbose messaging see aws.Config.CredentialsChainVerboseErrors
kubelet[19946]: I1221 05:37:49.920329 19946 kuberuntime_manager.go:207] Container runtime docker initialized, version: 18.06.2-ce, apiVersion: 1.38.0
kubelet[19946]: I1221 05:37:49.925361 19946 server.go:1065] Started kubelet
Dec 21 05:37:49 master-kube kubelet[19946]: E1221 05:37:49.925583 19946 kubelet.go:1302] Image garbage collection failed once. Stats initialization may not have completed yet: failed to get imageFs info: unable to find data in memory ca
kubelet[19946]: E1221 05:37:49.927007 19946 event.go:246] Unable to write event: 'Post https://192.168.50.10:6443/api/v1/namespaces/default/events: dial tcp 192.168.50.10:6443: connect: connection refused' (m
kubelet[19946]: I1221 05:37:49.928287 19946 server.go:145] Starting to listen on 0.0.0.0:10250
kubelet[19946]: I1221 05:37:49.931484 19946 server.go:354] Adding debug handlers to kubelet server.
kubelet[19946]: I1221 05:37:49.935058 19946 fs_resource_analyzer.go:64] Starting FS ResourceAnalyzer
kubelet[19946]: I1221 05:37:49.935868 19946 status_manager.go:156] Starting to sync pod status with apiserver
kubelet[19946]: I1221 05:37:49.936434 19946 kubelet.go:1822] Starting kubelet main sync loop.
kubelet[19946]: I1221 05:37:49.936965 19946 kubelet.go:1839] skipping pod synchronization - [container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]
kubelet[19946]: I1221 05:37:49.943766 19946 volume_manager.go:249] Starting Kubelet Volume Manager
kubelet[19946]: E1221 05:37:49.955051 19946 controller.go:135] failed to ensure node lease exists, will retry in 200ms, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-l
kubelet[19946]: I1221 05:37:49.960391 19946 desired_state_of_world_populator.go:131] Desired state populator starts to run
kubelet[19946]: E1221 05:37:49.960555 19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
kubelet[19946]: E1221 05:37:49.960939 19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
kubelet[19946]: I1221 05:37:49.977365 19946 clientconn.go:104] parsed scheme: "unix"
kubelet[19946]: I1221 05:37:49.977587 19946 clientconn.go:104] scheme "unix" not registered, fallback to default scheme
kubelet[19946]: I1221 05:37:49.977801 19946 passthrough.go:48] ccResolverWrapper: sending update to cc: {[{unix:///run/containerd/containerd.sock 0 <nil>}] <nil>}
kubelet[19946]: I1221 05:37:49.977941 19946 clientconn.go:577] ClientConn switching balancer to "pick_first"
kubelet[19946]: W1221 05:37:49.988778 19946 cni.go:328] CNI failed to retrieve network namespace path: **cannot find network namespace for the terminated container "2fe393eaa9c070dcc24651a27fa01e421ada7a8b20a4d**
kubelet[19946]: W1221 05:37:49.991163 19946 docker_sandbox.go:394] **failed to read pod IP from plugin/docker: networkPlugin cni failed on the status hook for pod "coredns-5644d7b6d9-j8f7f_kube-system": CNI fai**
kubelet[19946]: W1221 05:37:49.994413 19946 docker_sandbox.go:394] failed to read pod IP from plugin/docker: networkPlugin cni failed on the status hook for pod "coredns-5644d7b6d9-j8f7f_kube-system": CNI fai
kubelet[19946]: I1221 05:37:50.039352 19946 kubelet.go:1839] skipping pod synchronization - container runtime status check may not have completed yet
kubelet[19946]: E1221 05:37:50.045922 19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: I1221 05:37:50.046091 19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.047491 19946 kubelet_node_status.go:72] Attempting to register node master-kube
kubelet[19946]: E1221 05:37:50.047805 19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
kubelet[19946]: I1221 05:37:50.048908 19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.050080 19946 cpu_manager.go:161] [cpumanager] starting with none policy
kubelet[19946]: I1221 05:37:50.050184 19946 cpu_manager.go:162] [cpumanager] reconciling every 10s
kubelet[19946]: I1221 05:37:50.050283 19946 policy_none.go:42] [cpumanager] none policy: Start
kubelet[19946]: I1221 05:37:50.052514 19946 plugin_manager.go:116] Starting Kubelet Plugin Manager
kubelet[19946]: E1221 05:37:50.054259 19946 eviction_manager.go:246] eviction manager: failed to get summary stats: failed to get node info: node "master-kube" not found
kubelet[19946]: 2019-12-21 05:37:50.059 [INFO][20107] k8s.go 228: Using Calico IPAM
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.155714 19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: E1221 05:37:50.156669 19946 controller.go:135] failed to ensure node lease exists, will retry in 400ms, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-l
kubelet[19946]: I1221 05:37:50.254696 19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.254929 19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: E1221 05:37:50.257253 19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: I1221 05:37:50.264235 19946 kubelet_node_status.go:72] Attempting to register node master-kube
kubelet[19946]: E1221 05:37:50.267581 19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
kubelet[19946]: I1221 05:37:50.278488 19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.289042 19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: I1221 05:37:50.298498 19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
kubelet[19946]: W1221 05:37:50.307965 19946 pod_container_deletor.go:75] Container "6d39199a688542a9336b33ed6e7d370ea6320ad47a2b17da800b0954bc985bec" not found in pod's containers
kubelet[19946]: W1221 05:37:50.308035 19946 pod_container_deletor.go:75] Container "d97a9bcd4aa63b11db422d92fed678a90878ab94528b3e6c1c7fd99200c972a7" not found in pod's containers
kubelet[19946]: E1221 05:37:50.361156 19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: I1221 05:37:50.362961 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-local-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/68aa1
kubelet[19946]: I1221 05:37:50.363626 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91b546a2d4f3
kubelet[19946]: I1221 05:37:50.364328 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/68aa12790236de85ba80bddfd434
kubelet[19946]: I1221 05:37:50.364504 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "k8s-certs" (UniqueName: "kubernetes.io/host-path/68aa12790236de85ba80bddfd43
kubelet[19946]: I1221 05:37:50.364802 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/68aa1279023
kubelet[19946]: I1221 05:37:50.365575 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etc-ca-certificates" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91
kubelet[19946]: I1221 05:37:50.366108 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/4e1bd6e5b41d60d13135315758
kubelet[19946]: I1221 05:37:50.366614 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/f15776993ab
kubelet[19946]: I1221 05:37:50.367265 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-certs" (UniqueName: "kubernetes.io/host-path/cf6cdea881ad085c694fbe5bf3
kubelet[19946]: I1221 05:37:50.367499 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etcd-data" (UniqueName: "kubernetes.io/host-path/cf6cdea881ad085c694fbe5bf37
kubelet[19946]: I1221 05:37:50.368086 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "ca-certs" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91b546a2d4f36
kubelet[19946]: I1221 05:37:50.368745 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "kubeconfig" (UniqueName: "kubernetes.io/host-path/f15776993ab30bc91b546a2d4f
kubelet[19946]: I1221 05:37:50.369349 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "usr-local-share-ca-certificates" (UniqueName: "kubernetes.io/host-path/f1577
kubelet[19946]: I1221 05:37:50.369922 19946 reconciler.go:207] operationExecutor.VerifyControllerAttachedVolume started for volume "etc-ca-certificates" (UniqueName: "kubernetes.io/host-path/68aa12790236de85b
kubelet[19946]: E1221 05:37:50.461604 19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: E1221 05:37:50.598762 19946 kubelet.go:2267] node "master-kube" not found
kubelet[19946]: E1221 05:37:50.598807 19946 controller.go:135] failed to ensure node lease exists, will retry in 800ms, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-l
kubelet[19946]: E1221 05:37:50.667616 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.667813 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:50 master-kube kubelet[19946]: I1221 05:37:50.669813 19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.670999 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:50 master-kube kubelet[19946]: I1221 05:37:50.678853 19946 kubelet_node_status.go:72] Attempting to register node master-kube
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.680121 19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.701314 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.802348 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.903471 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.994146 19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:50 master-kube kubelet[19946]: E1221 05:37:50.994146 19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.004535 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.105943 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.207194 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.320505 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.401092 19946 controller.go:135] failed to ensure node lease exists, will retry in 1.6s, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-le
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.424687 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: I1221 05:37:51.485948 19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
Dec 21 05:37:51 master-kube kubelet[19946]: I1221 05:37:51.493301 19946 kubelet_node_status.go:72] Attempting to register node master-kube
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.493752 19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.524956 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.625893 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.669061 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.671281 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.672457 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.726204 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.827363 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.929008 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.996179 19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:51 master-kube kubelet[19946]: E1221 05:37:51.997820 19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.035831 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.136937 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.238053 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.338740 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.439243 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.549008 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.649594 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.672587 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.673694 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.674519 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.749925 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.850243 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.950601 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.997582 19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:52 master-kube kubelet[19946]: E1221 05:37:52.999199 19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.002688 19946 controller.go:135] failed to ensure node lease exists, will retry in 3.2s, error: Get https://192.168.50.10:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-le
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.051016 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: I1221 05:37:53.094012 19946 kubelet_node_status.go:286] Setting node annotation to enable volume controller attach/detach
Dec 21 05:37:53 master-kube kubelet[19946]: I1221 05:37:53.097332 19946 kubelet_node_status.go:72] Attempting to register node master-kube
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.098210 19946 kubelet_node_status.go:94] Unable to register node "master-kube" with API server: Post https://192.168.50.10:6443/api/v1/nodes: dial tcp 192.168.50.10:6443: connect
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.152044 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.253059 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.353765 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.454334 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.554929 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.655913 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.675733 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.678185 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.678624 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.756190 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.856636 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:53 master-kube kubelet[19946]: E1221 05:37:53.965559 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.004697 19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.RuntimeClass: Get https://192.168.50.10:6443/apis/node.k8s.io/v1beta1/runtimecl
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.005008 19946 reflector.go:123] k8s.io/client-go/informers/factory.go:134: Failed to list *v1beta1.CSIDriver: Get https://192.168.50.10:6443/apis/storage.k8s.io/v1beta1/csidriver
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.070989 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.171403 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.272286 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.372573 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.474665 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.574953 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.675738 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.677084 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:450: Failed to list *v1.Service: Get https://192.168.50.10:6443/api/v1/services?limit=500&resourceVersion
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.680293 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: Get https://192.168.50.10:6443/api/v1/pods?fieldSelector=spec.nodeNa
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.681313 19946 reflector.go:123] k8s.io/kubernetes/pkg/kubelet/kubelet.go:459: Failed to list *v1.Node: Get https://192.168.50.10:6443/api/v1/nodes?fieldSelector=metadata.name%3Dm
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.782388 19946 kubelet.go:2267] node "master-kube" not found
Dec 21 05:37:54 master-kube kubelet[19946]: E1221 05:37:54.883757 19946 kubelet.go:2267] node "master-kube" not founddocker ps command on master node: 
-- Sudhir Kumar
calico
devops
kubeadm
kubelet
kubernetes