everyone
Recently I upgraded my k8s cluster to v1.10.3, then I rolled it back to v1.9.8, then to v1.8.12. After that I found something I can't understand.
I can list deployment in my default namespace:
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
dunking-hedgehog-helmet 1 1 1 1 197d
kube-system-tomcat-official 1 1 1 1 197d
mongodb 1 1 1 1 152d
smelly-pig-mysql 1 1 1 1 204dBut can't in my kube-system namespace:
# kubectl get deploy -nkube-system
Error from server: no kind "Deployment" is registered for version "apps/v1"Also, the logs of apiserver startup:
E0530 10:47:09.511854 1 cacher.go:277] unexpected ListAndWatch error: storage/cacher.go:/daemonsets: Failed to list *extension
s.DaemonSet: no kind "DaemonSet" is registered for version "apps/v1"
E0530 10:47:09.534114 1 cacher.go:277] unexpected ListAndWatch error: storage/cacher.go:/daemonsets: Failed to list *extension
s.DaemonSet: no kind "DaemonSet" is registered for version "apps/v1"
E0530 10:47:09.577678 1 cacher.go:277] unexpected ListAndWatch error: storage/cacher.go:/replicasets: Failed to list *extensio
ns.ReplicaSet: no kind "ReplicaSet" is registered for version "apps/v1"
E0530 10:47:09.580008 1 cacher.go:277] unexpected ListAndWatch error: storage/cacher.go:/deployments: Failed to list *extensio
ns.Deployment: no kind "Deployment" is registered for version "apps/v1"
E0530 10:47:09.580234 1 cacher.go:277] unexpected ListAndWatch error: storage/cacher.go:/deployments: Failed to list *extensio
ns.Deployment: no kind "Deployment" is registered for version "apps/v1"We all know API version apps/v1 is added since v1.9.0, so why does v1.8.12 try to register Deployment for version "apps/v1" ?
In 1.10, objects in the apps API group began persisting in etcd in apps/v1 format (introduced in 1.9).
Rolling back to 1.9.x from 1.10.x is safe
If you want to roll back further to 1.8.x, you must first round trip all the apps/v1 resources (daemonsets, deployments, replicasets, statefulsets) to ensure they are persisted in etcd in a format that 1.8 can read.
The error you are getting indicates there is apps/v1 content in etcd which the kubernetes 1.8 apiserver cannot decode (since apps/v1 didn't exist in 1.8). The solution is to upgrade to 1.9.x, get/put all existing apps/v1 resources before downgrading to kube 1.8 again.