ConfigMap that can reference current Namespace

4/13/2021

I'm working with a Pod (Shiny Proxy) that talks to Kubernetes API to start other pods. I'm wanting to make this generic, and so don't want to hardcode the namespace (because I intend to have multiple of these, deployed probably as an OpenShift Template or similar).

I am using Kustomize to set the namespace on all objects. Here's what my kustomization.yaml looks like for my overlay:

bases:
- ../../base

namespace: shiny
commonAnnotations:
  technical_contact: A Local Developer <somedev@example.invalid>

Running Shiny Proxy and having it start the pods I need it to (I have service accounts and RBAC already sorted) works, so long as as in the configuration for Shiny Proxy I specify (hard-code) the namespace that the new pods should be generated in. The default namespace that Shiny Proxy will use is (unfortunately) 'default', which is inappropriate for my needs.

Currently for the configuration I'm using a ConfigMap (perhaps I should move to a Kustomize ConfigMapGenerator)

The ConfigMap in question is currently like the following:

apiVersion: v1
kind: ConfigMap
metadata:
  name: shiny-proxy
data:
  application_yml: |
      ...
      container-backend: kubernetes
      kubernetes:
        url: https://kubernetes.default.svc.cluster.local:443
        namespace: shiny
      ...

The above works, but 'shiny' is hardcoded; I would like to be able to do something like the following:

        namespace: { .metadata.namespace }

But this doesn't appear to work in a ConfigMap, and I don't see anything in the documentation that would lead to believe that it would, or that a similar thing appears possible within the ConfigMap machinery.

Looking over the Kustomize documentation doesn't fill me with clarity either, particularly as the configuration file is essentially plain-text (and not a YAML document as far as the ConfigMap is concerned). I've seen some use of Vars, but https://github.com/kubernetes-sigs/kustomize/issues/741 leads to believe that's a non-starter.

Is there a nice declarative way of handling this? Or should I be looking to have the templating smarts happen within the container, which seems kinda wrong to me, but I am still new to Kubernetes (and OpenShift)

I'm using CodeReady Containers 1.24 (OpenShift 4.7.2) which is essentially Kubernetes 1.20 (IIRC). I'm preferring to keep this fairly well aligned with Kubernetes without getting too OpenShift specific, but this is still early days.

Thanks, Cameron

-- Cameron Kerr
kubernetes
kustomize

2 Answers

4/13/2021

If you don't want to hard-code a specific data in your manifest file, you can consider using Kustomize plugins. In this case, the sedtransformer plugin may be useful. This is an example plugin, maintained and tested by the kustomize maintainers, but not built-in to kustomize.
As you can see in the Kustomize plugins guide:

Kustomize offers a plugin framework allowing people to write their own resource generators and transformers.

For more information on creating and using Kustomize plugins, see Extending Kustomize.


I will create an example to illustrate how you can use the sedtransformer plugin in your case.

Suppose I have a shiny-proxy ConfigMap:
NOTE: I don't specify a namespace, I use namespace: NAMESPACE instead.

$ cat cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: shiny-proxy
data:
  application_yml: |
      container-backend: kubernetes
      kubernetes:
        url: https://kubernetes.default.svc.cluster.local:443
        namespace: NAMESPACE
      something_else:
        something: something

To use the sedtransformer plugin, we first need to create the plugin’s configuration file which contains a YAML configuration object:
NOTE: In argsOneLiner: I specify that NAMESPACE should be replaced with shiny.

$ cat sedTransformer.yaml
apiVersion: someteam.example.com/v1
kind: SedTransformer
metadata:
  name: sedtransformer
argsOneLiner: s/NAMESPACE/shiny/g

Next, we need to put the SedTransformer Bash script in the right place.

When loading, kustomize will first look for an executable file called $XDG_CONFIG_HOME/kustomize/plugin/${apiVersion}/LOWERCASE(${kind})/${kind}

I create the necessary directories and download the SedTransformer script from the Github:
NOTE: The downloaded script need to be executable.

$ mkdir -p $HOME/.config/kustomize/plugin/someteam.example.com/v1/sedtransformer
$ cd $HOME/.config/kustomize/plugin/someteam.example.com/v1/sedtransformer
$ wget https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/plugin/someteam.example.com/v1/sedtransformer/SedTransformer
$ chmod a+x SedTransformer

Finally, we can check if it works as expected:
NOTE: To use this plugin, you need to provide the --enable-alpha-plugins flag.

$ tree
.
├── cm.yaml
├── kustomization.yaml
└── sedTransformer.yaml

0 directories, 3 files

$ cat kustomization.yaml
resources:
 - cm.yaml

transformers:
  - sedTransformer.yaml
  
$ kustomize build --enable-alpha-plugins .
apiVersion: v1
data:
  application_yml: |
    container-backend: kubernetes
    kubernetes:
      url: https://kubernetes.default.svc.cluster.local:443
      namespace: shiny
    something_else:
      something: something
kind: ConfigMap
metadata:
  name: shiny-proxy

Using the sedtransformer plugin can be especially useful if you want to replace NAMESPACE in a number of places.

-- matt_j
Source: StackOverflow

4/14/2021

I found the easiest way of doing this was to use an entrypoint script in the container that harvested the downward API (?) service credentials (specifically the namespace secret) that get mounted in the container, and exposes that as an environment variable.

export SHINY_K8S_NAMESPACE=`cat /run/secrets/kubernetes.io/serviceaccount/namespace`

cd /opt/shiny-proxy/working
exec java ${JVM_OPTIONS} -jar /opt/shiny-proxy/shiny-proxy.jar

Within the application configuration (shiny-proxy), it supports the use of environment variables in its configuration file, so I can refer to the pod's namespace using ${SHINY_K8S_NAMESPACE}

Although, I've just now seen the idea of a fieldRef (from https://docs.openshift.com/enterprise/3.2/dev_guide/downward_api.html), and would be generalisable to things other than just namespace.

apiVersion: v1
kind: Pod
metadata:
  name: dapi-env-test-pod
spec:
  containers:
  - name: env-test-container
    image: gcr.io/google_containers/busybox
    command: ["/bin/sh", "-c", "env"]
    env:
    - name: MY_NAMESPACE
      valueFrom:
        fieldRef:
          fieldPath: metadata.namespace
    restartPolicy: Never
-- Cameron Kerr
Source: StackOverflow