K
Q

Passing minified yaml as an argument to a kubernetes job

10/12/2020

Here's a simplified version of a kubernetes job YAML config I use commonly:

apiVersion: batch/v1
kind: Job
metadata:
  name: myjob
spec:
  template:
    spec:
      containers:
      - name: mycontainer
        image: me/mycontainer:latest
        command: ["bash", "-c"]
        args:
          - python -u myscript.py
              --param1 abc
              --param2 xyz

The above works great, and is easy to maintain and read. But now one of my parameters needs some minified YAML:

apiVersion: batch/v1
kind: Job
metadata:
  name: myjob
spec:
  template:
    spec:
      containers:
      - name: mycontainer
        image: me/mycontainer:latest
        command: ["bash", "-c"]
        args:
          - python -u myscript.py
              --param_minified_yaml "{key: value}"

This bit of embedded minified yaml is being parsed by kubectl and causing: error: error parsing STDIN: error converting YAML to JSON: yaml: line 26: mapping values are not allowed in this context

How can the embedded yaml in args: be escaped such that it's passed as a pure text argument?

-- David Parks
kubectl
kubernetes
yaml

Similar Questions

Unable to curl ingress from inside of the machine: 404
How to stop a self-built kubernetes cluster
Helm simple for loop
How to deploy a single node elasticsearch cluster in Google Kubernetes engine?
How do I add log_output_level argument to istio-sidecar-injector on GKE?

3 Answers

10/12/2020

The bash -c wrapper forces the entire actual command into a single argument, and then leads to problems around escaping and quoting. Unless you actually need to invoke a shell (to interpolate environment variables perhaps) it's better to remove it. Then the command: is a list of words; you are responsible for breaking up the command into individual words, but conversely, you can use any valid YAML syntax for each individual word.

So here I might write:

<!-- language: lang-yaml -->
command:
  - python
  - -u
  - myscript.py
  - --param_minified_yaml
  - '{key: value}'         # in quotes, so it is a YAML string

Any of the syntactic variations suggested in other answers work, and you can mix and match on a per-argument basis.

<!-- language: lang-yaml -->
command:
  - ...
  - --param_minified_yaml
  - >-                     # block scalar syntax, again makes a string
      {key: value}
-- David Maze
Source: StackOverflow
10/12/2020

If the minified yaml (or the args string in general) does not include single quotes, you can wrap the whole command line in them:

apiVersion: batch/v1
kind: Job
metadata:
  name: myjob
spec:
  template:
    spec:
      containers:
      - name: mycontainer
        image: me/mycontainer:latest
        command: ["bash", "-c"]
        args:
          - 'python -u myscript.py
              --param_minified_yaml "{key: value}"'

If the arg string contains includes single quotes, the args string can be passed as a YAML multiline string:

apiVersion: batch/v1
kind: Job
metadata:
  name: myjob
spec:
  template:
    spec:
      containers:
      - name: mycontainer
        image: me/mycontainer:latest
        command: ["bash", "-c"]
        args:
          - >-
            python -u myscript.py
            --param_minified_yaml "{key: 'value'}"
-- Lauri Koskela
Source: StackOverflow
10/12/2020

You can use a block scalar here:

        args:
          - >
            python -u myscript.py
              --param_minified_yaml "{key: value}"

Try the different scalar formats in a yaml validator or converter to learn about their differences.

-- webwurst
Source: StackOverflow