Trying to build k8s cluster on bare metal. I use CoreOS as host OS for my nodes. And I'm a bit confusing with the way I should install flannel for cluster networking.
I see from docs that I can either download it to my host machine and start it using systemd
or use a k8s DaemonSet manifest.
Personally I like the idea of DaemonSet because k8s will take care of running flannel on each node. But are there any disadvantages in running flannel as k8s-DaemonSet?
Documentation also says that I should run flannel before any other pods. That's why my first idea was to put it onto the path specified for kubelet --pod-manifest
parameter. But since the yaml for flannel contains not Pod
but DaemonSet
and some RBAC-related entities I get the following error using --pod-manifest
dir for deploying flannel:
Can't process manifest file "/etc/kubernetes/manifests/flannel.yaml": /etc/kubernetes/manifests/flannel.yaml: couldn't parse as pod(invalid pod: &rbac.ClusterRole{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"flannel", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Initializers:(*v1.Initializers)(nil), Finalizers:[]string(nil), ClusterName:""}, Rules:[]rbac.PolicyRule{rbac.PolicyRule{Verbs:[]string{"get"}, APIGroups:[]string{""}, Resources:[]string{"pods"}, ResourceNames:[]string(nil), NonResourceURLs:[]string(nil)}, rbac.PolicyRule{Verbs:[]string{"list", "watch"}, APIGroups:[]string{""}, Resources:[]string{"nodes"}, ResourceNames:[]string(nil), NonResourceURLs:[]string(nil)}, rbac.PolicyRule{Verbs:[]string{"patch"}, APIGroups:[]string{""}, Resources:[]string{"nodes/status"}, ResourceNames:[]string(nil), NonResourceURLs:[]string(nil)}}, AggregationRule:(*rbac.AggregationRule)(nil)}), please check manifest file.
So if I want to execute my flannel yaml-manifest before other pods on particular node, where should I put it? Or the only option is to execute kubectl apply -f flannel.yaml
?
That manifest includes a ClusterRole, a ClusterRoleBinding, a ConfigMap, and the DaemonSet resources (which implicitly defines the pod resources), so it can't be run with --pod-manifest, which only wants pod resource definitions.
I don't have comparative experience running flannel under systemd vs as a daemonset.
I would look at the AddOn manager as a way to ensure kubernetes objects are considered cluster services and are started early in cluster lifecycle:
https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/addon-manager/README.md