K
Q

Question

Identifying a node's zone from inside a Kubernetes pod/container

9/20/2018

I have a use case with elastic search for rack awareness, which requires me to identify the zone that a pod has been scheduled in.

I've seen many requests for this outside of SO, such as:

https://github.com/kubernetes/kubernetes/issues/40610

The zone is not exposed in the downward API due to it being a node label, not a pod label.

The two "answers" I have come up with are:

a) Curl request to the google metadata endpoint to retrieve the node from the compute engine metadata

b) Identify the node name via the downward API, and make a call to the Kube API with the node name to retrieve the node object, and use a tool such as JQ to filter the JSON response to get the zone.

I don't like option B due to it being more or less hardcoded against the API call, and I would need to provision a custom docker image with JQ and curl included. Option A feels a bit 'hacky' given it's not Kube native.

Are there any better solutions that I've missed?

Thanks in advance,

-- dwillams782

google-kubernetes-engine

kubernetes

3 Answers

9/20/2018

I don't particularly like doing it this way but I've yet to find a better answer, and none of the feature requests or bug reports on github seem to be going anywhere.

I opted to use a config map with a bash script which would do the curl request and some string manipulation, and then mount this into the container with volumes/volumeMounts and set a custom entry point to execute the script; injecting values into elasticsearch.yml, and then execute ES itself:

apiVersion: v1
kind: ConfigMap
metadata:
  name: elastic
data:
  elastic.sh: |-
    set -e
    elasticConfig="/usr/share/elasticsearch/config/elasticsearch.yml"

    #this gives the us the node zone in the format projects/{projectnumber}/zones/{zone}
    zone=$(curl -sS http://metadata.google.internal/computeMetadata/v1/instance/zone -H 'Metadata-Flavor: Google')

    #Split and retain the text after the last /
    zone=${zone##*/}

    #Append it to elasticsearch.yml

    echo "\nnode.attr.zone: ${zone}" >> $elasticConfig
    echo "\ncluster.routing.allocation.awareness.attributes: zone" >> $elasticConfig

    echo "\ncluster.routing.allocation.awareness.force.zone.values: {{ .Values.elastic.shardAwareness.zones }}" >> $elasticConfig

I don't particularly like this solution as it adds unnecessary overhead and isn't kube native. Querying the kube API is possible but has its own set of complications and hacks. I hope one day either the downward API will expose zone / region labels, or that maybe I'm missing something and there is a better way than this after all.

If nothing else, perhaps this will help someone else and stop them wasting time googling for answers that don't seem to be out there!

-- dwillams782

Source: StackOverflow

6/11/2019

My case is Google Kubernetes Engine, and I needed to have zone/region labels on pods.

What i did was combined @dwillams782 approach in https://stackoverflow.com/a/52428782/8547463 of getting zone via

curl -sS http://metadata.google.internal/computeMetadata/v1/instance/zone -H 'Metadata-Flavor: Google'`

and set it as labels to pods via approach in https://github.com/devth/k8s-labeler. Basically i called the zone detection API from initContainer of pod.

-- Anton Fedorov

Source: StackOverflow

9/20/2018

I think you can use pod anti-affinity to do this, here is a example:

spec:
  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: security
              operator: In
              values:
              - S2
          topologyKey: failure-domain.beta.kubernetes.io/zone

More detail refer to kubernetes docs

-- Kun Li

Source: StackOverflow

KQ

Identifying a node's zone from inside a Kubernetes pod/container

Similar Questions

3 Answers

K
Q