Why does Kubernetes pod Affinity not work?

5/12/2020

I have following pods

NAME                                       READY   STATUS             NODE                          LABELS
bss-a-0                                    5/5     Running            aks-bss-0   tcb.segment=bss,tcb.zone=centralus-1
four-6mxdk                                 0/1     Pending            <none>      job-name=four
rat-a-0                                    7/7     Running            aks-rat-1   tcb.segment=rat,tcb.zone=centralus-1
rat-b-0                                    7/7     Running            aks-rat-2   tcb.segment=rat,tcb.zone=centralus-2
seagull-0                                  1/1     Running            aks-bss-0   tcb.segment=rat

four-6mxdk is pod with following pod affinity section:

  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: tcb.segment
            operator: In
            values:
            - rat
          - key: tcb.zone
            operator: In
            values:
            - centralus-1
        namespaces:
        - default
        topologyKey: k8s.io/hostname

Node detailes:

apiVersion: v1
kind: Node
metadata:
  annotations:
    node.alpha.kubernetes.io/ttl: "0"
    volumes.kubernetes.io/controller-managed-attach-detach: "true"
  labels:
    agentpool: rat
    beta.kubernetes.io/arch: amd64
    beta.kubernetes.io/instance-type: Standard_D32s_v3
    beta.kubernetes.io/os: linux
    failure-domain.beta.kubernetes.io/region: centralus
    failure-domain.beta.kubernetes.io/zone: centralus-1
    kubernetes.azure.com/cluster: cluster
    kubernetes.azure.com/role: agent
    kubernetes.io/arch: amd64
    kubernetes.io/hostname: aks-rat-1
    kubernetes.io/os: linux
    kubernetes.io/role: agent
    node-role.kubernetes.io/agent: ""
    storageprofile: managed
    storagetier: Premium_LRS
  name: aks-rat-1
  allocatable:
    attachable-volumes-azure-disk: "32"
    cpu: 31580m
    ephemeral-storage: "93492541286"
    hugepages-1Gi: "0"
    hugepages-2Mi: "0"
    memory: 121466760Ki
    pods: "30"
  nodeInfo:
    architecture: amd64
    containerRuntimeVersion: docker://3.0.8
    kernelVersion: 4.15.0-1069-azure
    kubeProxyVersion: v1.15.7
    kubeletVersion: v1.15.7
    operatingSystem: linux
    osImage: Ubuntu 16.04.6 LTS

My expectation: pod will be scheduled on node: aks-rat-1 due to it's only node where both labels are matched.

Actual result: 3 node(s) didn''t match pod affinity/anti-affinity.

Is it bug? or i do something wrong?

-- Yaroslav
kubernetes

1 Answer

5/12/2020

Looks like aks uses kubernetes.io/hostname instead of k8s.io/hostname or since some version of K8 it was changed and most of documents weren't updated. Anyway topologyKey was an issue here.

-- Yaroslav
Source: StackOverflow