Kubernetes Pod Affinity Topology Key for custom label

2/17/2021

We're trying to deploy some Keda Scaled Jobs to our Kubernetes Cluster. We always want one node per job. I'm trying to configure a podAntiAffinity rule to enforce that a job doesn't get deployed to a node where another job (pod) with the given label is running. Per the documentation and samples, you need a topologyKey, which is

the key for the node label that the system uses to denote such a topology domain;

In our case we want to use an actual label that we specify, as opposed to, say, the hostname. How would we configure our rule in that case?

apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: my-scaled-job-name
  namespace: my-namespace
  labels:
    # multiple jobs may have this label
    my-special-label: foo 
spec:
  ...
  jobTargetRef:
    ...
    template:
      spec:
        nodeSelector:
          "beta.kubernetes.io/os": linux
          agentpool: my-agent-pool
        affinity:
          # This anti-affinity rule makes sure a new pod won't start on a node where another one is already running
          podAntiAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
            # Unclear whether this is the right topology key, or if I even need one?   
            - topologyKey: kubernetes.io/hostname 
              labelSelector:
                matchLabels:
                  - my-special-label: foo
        containers:
        - name: my-container
          ...
  # This is the trigger definition - watches the queue for incoming messages
  triggers:
    ...
-- Chuck Spencer
keda
keda-scaledjob
kubernetes
kubernetes-pod

0 Answers