Pod is not getting selected by Deployment selector

2/13/2021

I have this Deployment object:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-webserver-nginx
  annotations:
    description: This is a demo deployment for nginx webserver 
  labels:
    app: deployment-webserver-nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: deployment-webserver-pods
  template:
    metadata:
      labels:
        app: deployment-webserver-pods
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80

My understanding on this Deployment object is that any Pod with app:deployment-webserver-pods label will be selected. Of course, this Deployment object is creating 3 replicas, but I wanted to add one more Pod explicitly like this, so I created a Pod object and had its label as app:deployment-webserver-pods, below is its Pod definition:

apiVersion: v1
kind: Pod
metadata:
  name: deployment-webserver-nginx-extra-pod
  labels:
    app: deployment-webserver-pods
spec:
  containers:
  - name: nginx-alpine-container-1
    image: nginx:alpine
    ports:
      - containerPort: 81

My expectation was that continuously running Deployment Controller will pick this new Pod, and when I do kubectl get deploy then I will see 4 pods running. But that didn't happen.

I even tried to first create this pod with this label, and then created my Deployment and thought that maybe now this explicit Pod will be picked but still that didn't happen.

Doesn't Labels and Selectors work like this? I know I can scale by deployment to 4 Replicas, but I am trying to understand how Pods / other Kubernetes objects are selected using Labels and Selectors.

-- hagrawal
google-kubernetes-engine
kubernetes
kubernetes-deployment
kubernetes-pod

4 Answers

6/21/2021

Just to add in the above explanations that if we are trying to manually create pod and manage then what is the purpose of having controllers in K8s.

My expectation was that continuously running Deployment Controller will pick this new Pod, and when I do kubectl get deploy then I will see 4 pods running. But that didn't happen.

As per your yaml replicas:3 was already set so deployment would not take a new pod as the 4th replica.

-- Deepak Mourya
Source: StackOverflow

2/14/2021

From the official docs:

Note: You should not create other Pods whose labels match this selector, either directly, by creating another Deployment, or by creating another controller such as a ReplicaSet or a ReplicationController. If you do so, the first Deployment thinks that it created these other Pods. Kubernetes does not stop you from doing this.

As described further in docs, it is not recommended to scale replicas of the deployments using the above approach.

<hr> Another important point to note from same section of docs:

If you have multiple controllers that have overlapping selectors, the controllers will fight with each other and won't behave correctly.

-- Krishna Chaurasia
Source: StackOverflow

2/13/2021

My expectation was that continuously running Deployment Controller will pick this new Pod, and when I do kubectl get deploy then I will see 4 pods running. But that didn't happen.

The Deployment Controller does not work like that, it listen for Deployment-resources and "drive" them to desired state. That typically means, if any change in the template:-part, then a new ReplicaSet is created with the number of replicas. You cannot add a Pod to a Deployment in another way than changing replicas: - each instance is created from the same Pod-template and is identical.

Doesn't Labels and Selectors work like this?

... but I am trying to understand how Pods / other Kubernetes objects are selected using Labels and Selectors.

Yes, Labels and Selectors are used for many things in Kubernetes, but not for everything. When you create a Deployment with a label, and a Pod with the same label and finally a Service with a selector - then the traffic addressed to that Service will distribute traffic to your instances of your Deployment as well as to your extra Pod.

Example:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: deployment-webserver-pods
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

Labels and Selector are also useful for management when using e.g. kubectl. You can add labels for Teams or e.g. App, then you can select all Deployments or Pods belonging to that Team or App (e.g. if the app consist of App-deployment and a cache-deployment), e.g:

kubectl get pods -l team=myteam,app=customerservice
-- Jonas
Source: StackOverflow

2/13/2021

My expectation was that continuously running Deployment Controller will pick this new Pod, and when I do kubectl get deploy then I will see 4 pods running. But that didn't happen.

Kubernetes is a system that operates "Declaratively" and not "Imperatively" which means you write down the desired state of the application in the cluster typically through a YAML file, and these declared desired states define all of the pieces of your application.

If a cluster were to configured imperatively like the way you are expecting it to be, it would have been very difficult to understand and replicate how the cluster came to be in that state.

-- Benjamin
Source: StackOverflow