I've setup a standalone k8s cluster using kops, so I want all pods running on only 1 master too. The cluster started well but kube-dns and kube-autoscaler pods does not running, it's in pending status. This is node describe
kubectl describe nodes
Name: ip-10-0-3-184.ap-southeast-1.compute.internal
Roles: master
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/instance-type=m3.large
beta.kubernetes.io/os=linux
failure-domain.beta.kubernetes.io/region=ap-southeast-1
failure-domain.beta.kubernetes.io/zone=ap-southeast-1a
kops.k8s.io/instancegroup=master-ap-southeast-1a
kubernetes.io/hostname=ip-10-0-3-184.ap-southeast-1.compute.internal
kubernetes.io/role=master
node-role.kubernetes.io/master=
Annotations: node.alpha.kubernetes.io/ttl=0
volumes.kubernetes.io/controller-managed-attach-detach=true
Taints: node-role.kubernetes.io/master:NoSchedulebut /var/log/kube-scheduler.log is:
I0526 12:49:54.629475 1 scheduler.go:191] Failed to schedule pod: kube-system/kube-dns-autoscaler-787d59df8f-8jgn7
I0526 12:49:54.629570 1 factory.go:1251] Updating pod condition for kube-system/kube-dns-autoscaler-787d59df8f-8jgn7 to (PodScheduled==False)
I0526 12:50:09.706382 1 scheduler.go:191] Failed to schedule pod: kube-system/kube-dns-7785f4d7dc-rqzdq
I0526 12:50:09.706484 1 factory.go:1251] Updating pod condition for kube-system/kube-dns-7785f4d7dc-rqzdq to (PodScheduled==False)
I0526 12:50:10.632285 1 scheduler.go:191] Failed to schedule pod: kube-system/kube-dns-autoscaler-787d59df8f-8jgn7
I0526 12:50:10.632371 1 factory.go:1251] Updating pod condition for kube-system/kube-dns-autoscaler-787d59df8f-8jgn7 to (PodScheduled==False)
I0526 12:50:41.709687 1 scheduler.go:191] Failed to schedule pod: kube-system/kube-dns-7785f4d7dc-rqzdq
I0526 12:50:41.709796 1 factory.go:1251] Updating pod condition for kube-system/kube-dns-7785f4d7dc-rqzdq to (PodScheduled==False)
I0526 12:50:42.635260 1 scheduler.go:191] Failed to schedule pod: kube-system/kube-dns-autoscaler-787d59df8f-8jgn7
I0526 12:50:42.635361 1 factory.go:1251] Updating pod condition for kube-system/kube-dns-autoscaler-787d59df8f-8jgn7 to (PodScheduled==False)I can fix it manually by using command kubectl taint nodes --all node-role.kubernetes.io/master- but I want it is enable when I create the cluster with kops create cluster
Thanks
The Taints are set in kops templates for all types of Kubernetes networking add-ons.
It is not possible for now to enable/disable that part of the template with command line options.
But, there is a possibility to configure the taints for an instance group:
kops edit ig master-us-west-1cInstance Groups
By default, a cluster has:
- An instance group called nodes spanning all the zones; these instances are your workers.
- One instance group for each master zone, called master- (e.g. master-us-east-1c). These normally have minimum size and maximum size = 1, so they will run a single instance. We do this so that the cloud will always relaunch masters, even if everything is terminated at once. We have an instance group per zone because we need to force the cloud to run an instance in every zone, so we can mount the master volumes - we cannot do that across zones.
Adding Taints or Labels to an Instance Group
If you're running Kubernetes 1.6.0 or later, you can also control taints in the InstanceGroup. The taints property takes a list of strings. The following example would add two taints to an IG, using the same edit -> update -> rolling-update process as above.Additionally, nodeLabels can be added to an IG in order to take advantage of Pod Affinity. Every node in the IG will be assigned the desired labels. For more information see the labels documentation.
metadata:
creationTimestamp: "2016-07-10T15:47:14Z"
name: nodes
spec:
machineType: m3.medium
maxSize: 3
minSize: 3
role: Node
taints:
- dedicated=gpu:NoSchedule
- team=search:PreferNoSchedule
nodeLabels:
spot: "false"