i'm trying to build a Kubernetes cluster V1.9.3 with RBAC authz and admission-control options.
I've already do it with V1.6 and all works good, when my nodes starts the kubelet it use bootstrap file to get client certificate for kubelet. It get 5 files : kubelet.crt, kubelet.key, kubelet-client.key, kubelet-client.crt and the kubelet.conf.
In V1.9 i'am using the same configuration on all components, i just adapted them to the update 1.6->1.9. apiserver, scheduler and controller-manager starts without any issue.
When my nodes starts the kubelet, it use the bootstrap too and get kubelet.crt kubelet.key and kubelet-client.key, but neither kubelet-client.crt nor kubelet.conf file.
I see no issue in logs of all components, all seems OK, but my kubelet don't get it's config file
Here my apiserver config in 1.9:
--admission-control=NodeRestriction,NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds
--authorization-mode=Node,RBAC
--secure-port=6443
--bind-address=0.0.0.0
--advertise-address=10.0.0.8
--insecure-port=0
--insecure-bind-address=127.0.0.1
--client-ca-file=/etc/kubernetes/pki/ca.crt
--etcd-servers=http://127.0.0.1:2379
--service-account-key-file=/etc/kubernetes/pki/sa.pub
--service-cluster-ip-range=172.30.100.0/24
--tls-cert-file=/etc/kubernetes/pki/kube-apiserver.crt
--tls-private-key-file=/etc/kubernetes/pki/kube-apiserver.key
--enable-bootstrap-token-auth
--kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt
--kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key
--kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
--requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt
--requestheader-username-headers=X-Remote-User
--requestheader-group-headers=X-Remote-Group
--requestheader-allowed-names=front-proxy-client
--requestheader-extra-headers-prefix=X-Remote-Extra-
Kubelet config in 1.9 :
--bootstrap-kubeconfig=/etc/kubernetes/kubeconfig
--kubeconfig=/etc/kubernetes/kubelet.conf
--pod-manifest-path=/etc/kubernetes/manifests
--allow-privileged=true
--network-plugin=cni
--cni-conf-dir=/etc/cni/net.d
--cni-bin-dir=/opt/cni/bin
--cluster-dns=172.30.0.10
--cluster-domain=cluster.local
--authorization-mode=Webhook
--client-ca-file=/etc/kubernetes/pki/ca.crt
--cadvisor-port=0
--rotate-certificates=true
--cert-dir=/var/lib/kubelet/pki
--cgroup-driver=systemd
systemctl status -l kubelet :
Feb 23 17:24:43 kubenode1 systemd[1]: Started Kubernetes Kubelet.
Feb 23 17:24:43 kubenode1 systemd[1]: Starting Kubernetes Kubelet...
Feb 23 17:24:43 kubenode1 kubelet[12489]: I0223 17:24:43.664804 12489 feature_gate.go:226] feature gates: &{{} map[]}
Feb 23 17:24:43 kubenode1 kubelet[12489]: I0223 17:24:43.664956 12489 controller.go:114] kubelet config controller: starting controller
Feb 23 17:24:43 kubenode1 kubelet[12489]: I0223 17:24:43.664960 12489 controller.go:118] kubelet config controller: validating combination of defaults and flags
Feb 23 17:24:44 kubenode1 kubelet[12489]: W0223 17:24:44.078716 12489 cni.go:171] Unable to update cni config: No networks found in /etc/cni/net.d
Feb 23 17:24:44 kubenode1 kubelet[12489]: I0223 17:24:44.082194 12489 server.go:182] Version: v1.9.3
Feb 23 17:24:44 kubenode1 kubelet[12489]: I0223 17:24:44.082228 12489 feature_gate.go:226] feature gates: &{{} map[]}
Feb 23 17:24:44 kubenode1 kubelet[12489]: I0223 17:24:44.082310 12489 plugins.go:101] No cloud provider specified.
ls /var/lib/kubelet/pki :
kubelet-client.key
kubelet.crt
kubelet.key
How can i force kubelet to get kubelet.conf file in v1.9 ? Someone have any idea ?
I think I found the problem, the controller-manager doesn't auto approve the kubelet request anymore. I have to use the following commands to approve it :
$ kubectl -n kube-system get csr
NAME AGE REQUESTOR CONDITION
node-csr-a1NZL_Efc2eDyrtSNZcIL1NxuHTvrsCO-ObfFFRUh8U 27s system:bootstrap:61bb3c Pending
node-csr-co5i3DRYeYOwCBff5XYA0rfSIqniy05VtdiVTmarQQA 5h system:bootstrap:61bb3c Approved,Issued
$ kubectl certificate approve node-csr-IoN3CNTJmJHGKUWso1OXLwd1H3KLOon4oVCz3uVYfqw
certificatesigningrequest "node-csr-a1NZL_Efc2eDyrtSNZcIL1NxuHTvrsCO-ObfFFRUh8U" approved
I followed this doc thinking that would enable the auto-approve, but it's seems not :/ https://kubernetes.io/docs/admin/kubelet-tls-bootstrapping/#kubectl-approval