What's the relationship between deployment and ConfigMap?

3/6/2019

I'm curious about how ConfigMap and Deployment works in kubernetes.

I wanted to use the values in ConfigMap as arguments to my deployment pods. I've tried this with different images and found different behaviours when passing ConfigMap value as command arguments between containers that use sh as entry point and other commands as entry point.

Here is an example configuration to better illustrate my case:

configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-envs
data:
  key: "value"
  BUCKET_NAME: "gs://bucket-name/"
  OUTPUT_PATH: "/data"

deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
spec:
  template:
    containers:
    - name: firstContainer
      image: busybox
      command: ["sh"]
      args: 
      - c
      - |
        echo $key
        echo ${BUCKET_NAME}
        echo $(OUTPUT_PATH)
      envFrom:
      - configMapRef:
          name: app-envs
    - name: secondContainer
      image: someImage
      args: [ "cmd", "${BUCKET_NAME}", "${OUTPUT_DATA}", "${key}" ] 
      envFrom:
      - configMapRef:
          name: app-envs
    - name: thirdContainer
      image: someImage
      args: [ "cmd", "$(BUCKET_NAME)", "$(OUTPUT_DATA)", "$(key)" ] 
      envFrom:
      - configMapRef:
          name: app-envs

someImage is a docker image, which has certain bash script as its entry point that prints the environment values.


The firstContainer and thirdContainer are able to print all the ConfigMap values correctly, meaning, all value, gs://bucket-name/ and /data are received as input arguments.

However, the secondContainer is unable to print these values correctly. I tried to echo the received arguments, and it turned out that it receives:

${BUCKET_NAME}, ${OUTPUT_DATA}, and ${key} literally as input arguments instead of the actual values from ConfigMaps.

So after observing the above behaviours, here are my questions:

  1. What's the relationship between deployment and ConfigMap? Is there some kind of order which specify how resources are created in a k8s pod/deployment (e.g., whether ConfigMap is loaded first, then the volumeMounts, and then the container or some kind of orderings)?

  2. What's the difference between ${} and $()? Why does the ConfigMap values are received as literal strings when using ${} to a container that has different entry point than bash or sh?

Thank you. Your help would be appreciated.

-- dekauliya
bash
docker
google-kubernetes-engine
kubernetes
shell

1 Answer

3/6/2019

Kubernetes only directly understands environment variable references in parentheses $(VAR); see for example the note in Define a Command and Arguments for a Container.

args: [ "cmd", "$(BUCKET_NAME)", "$(OUTPUT_DATA)", "$(key)" ] 

Kubernetes itself knows what the environment variables are and does the substitution, so the container is launched as cmd gs://bucket-name/ /data key.

command: ["sh"]
args: 
- c
- |
  echo $key
  echo ${BUCKET_NAME}
  echo $(OUTPUT_PATH)

Kubernetes expands $(OUTPUT_PATH) but doesn't understand any other form of braces, so the other strings get sent on as-is. Since you're explicitly running this through a shell, though, both $key and ${BUCKET_NAME} are standard shell variable expansions, so the shell expands these values.

args: [ "cmd", "${BUCKET_NAME}", "${OUTPUT_DATA}", "${key}" ] 

Kubernetes doesn't expand things in curly braces, and there's no shell or anything else to expand these variables, so the variable strings (and not their contents) get passed along as-is.

-- David Maze
Source: StackOverflow