Pass in Dynamic Formatted Datetime to K8s Container Config

10/1/2019

I have a CronJob that runs a process in a container in Kubernetes.

This process takes in a time window that is defined by a --since and --until flag. This time window needs to be defined at container start time (when the cron is triggered) and is a function of the current time. An example running this process would be:

$ my-process --since=$(date -v -1H +"%Y-%m-%dT%H:%M:%SZ") --until=$(date -v +1H +"%Y-%m-%dT%H:%M:%SZ")

So for the example above, I would like the time window to be from 1 hour ago to 1 hour in the future. Is there a way in Kubernetes to pass in a formatted datetime as a command argument to a process?

An example of what I am trying to do would be the following config:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: my-process
spec:
  schedule: "*/2 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: my-process
            image: my-image
            args:
            - my-process
            - --since=$(date -v -1H +"%Y-%m-%dT%H:%M:%SZ")
            - --until=$(date -v +1H +"%Y-%m-%dT%H:%M:%SZ")

When doing this, the literal string "$(date -v -1H +"%Y-%m-%dT%H:%M:%SZ")" would be passed in as the --since flag.

Is something like this possible? If so, how would I do it?

-- northsideknight
containers
kubernetes
kubernetes-cronjob

1 Answer

10/1/2019

Note that in your CronJob you don't run bash or any other shell and command substitution is a shell feature and without one will not work. In your example only one command my-process is started in the container and as it is not a shell, it is unable to perform command substitution.

This one:

$ my-process --since=$(date -v -1H +"%Y-%m-%dT%H:%M:%SZ") --until=$(date -v +1H +"%Y-%m-%dT%H:%M:%SZ")

will work properly because it is started in a shell so it may take advantage of shell features such as mentioned command substitution

One thing: date -v -1H +"%Y-%m-%dT%H:%M:%SZ" doesn't expand properly in bash shell with default GNU/Linux date implementation. Among others -v option is not recognized so I guess you're using it on MacOSX or some kind of BSD system. In my examples below I will use date version that works on Debian.

So for testing it on GNU/Linux it will be something like this:

date --date='-1 hour' +"%Y-%m-%dT%H:%M:%SZ"

For testing purpose I've tried it with simple CronJob from this example with some modifications:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: debian
            env:
            - name: FROM
              value: $(date --date="-1 hour" +"%Y-%m-%dT%H:%M:%SZ")
            - name: TILL
              value: $(date --date="+1 hour" +"%Y-%m-%dT%H:%M:%SZ")
            args:
            - /bin/sh
            - -c
            - date; echo from $(FROM) till $(TILL)
          restartPolicy: OnFailure

It works properly. Below you can see the result of CronJob execution:

$ kubectl logs hello-1569947100-xmglq
 Tue Oct  1 16:25:11 UTC 2019
 from 2019-10-01T15:25:11Z till 2019-10-01T17:25:11Z

Apart from the example with use of environment variables I tested it with following code:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: debian
            args:
            - /bin/sh
            - -c
            - date; echo from $(date --date="-1 hour" +"%Y-%m-%dT%H:%M:%SZ") till $(date --date="+1 hour" +"%Y-%m-%dT%H:%M:%SZ")
          restartPolicy: OnFailure

and as you can see here command substitution also works properly:

$ kubectl logs hello-1569949680-fk782
Tue Oct  1 17:08:09 UTC 2019
from 2019-10-01T16:08:09Z till 2019-10-01T18:08:09Z

It works properly because in both examples first we spawn bash shell in our container and subsequently it runs other commands as simple echo provided as its argument. You can use your my-process command instead of echo only you'll need to provide it in one line with all its arguments, like this:

args:
- /bin/sh
- -c
- my-process --since=$(date -v -1H +"%Y-%m-%dT%H:%M:%SZ") --until=$(date -v +1H +"%Y-%m-%dT%H:%M:%SZ")

This example will not work as there is no shell involved. echo command not being a shell will not be able to perform command substitution which is a shell feature:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: debian
            args:
            - /bin/echo
            - from $(date --date="-1 hour" +"%Y-%m-%dT%H:%M:%SZ") till $(date --date="+1 hour" +"%Y-%m-%dT%H:%M:%SZ")
          restartPolicy: OnFailure

and the results will be a literal string:

$ kubectl logs hello-1569951180-fvghz
from $(date --date="-1 hour" +"%Y-%m-%dT%H:%M:%SZ") till $(date --date="+1 hour" +"%Y-%m-%dT%H:%M:%SZ")

which is similar to your case as your command, like echo isn't a shell and it cannot perform command substitution.

To sum up: The solution for that is wrapping your command as a shell argument. In first two examples echo command is passed along with other commands as shell argument.

Maybe it is visible more clearly in the following example with a bit different syntax:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: debian
            command: ["/bin/sh","-c"]
            args: ["FROM=$(date --date='-1 hour' +'%Y-%m-%dT%H:%M:%SZ'); TILL=$(date --date='+1 hour' +'%Y-%m-%dT%H:%M:%SZ') ;echo from $FROM till $TILL"]
          restartPolicy: OnFailure

man bash says:

-c If the -c option is present, then commands are read from the first non-option argument command_string.

so command: ["/bin/sh","-c"] basically means run a shell and execute following commands which then we pass to it using args. In bash commands should be separated with semicolon ; so they are run independently (subsequent command is executed no matter what was the result of executing previous command/commands).

In the following fragment:

args: ["FROM=$(date --date='-1 hour' +'%Y-%m-%dT%H:%M:%SZ'); TILL=$(date --date='+1 hour' +'%Y-%m-%dT%H:%M:%SZ') ;echo from $FROM till $TILL"]

we provide to /bin/sh -c three separate commands:

FROM=$(date --date='-1 hour' +'%Y-%m-%dT%H:%M:%SZ')

which sets FROM environment variable to result of execution of date --date='-1 hour' +'%Y-%m-%dT%H:%M:%SZ' command,

TILL=$(date --date='+1 hour' +'%Y-%m-%dT%H:%M:%SZ')

which sets TILL environment variable to result of execution of date --date='+1 hour' +'%Y-%m-%dT%H:%M:%SZ' command

and finally we run

echo from $FROM till $TILL

which uses both variables.

Exactly the same can be done with any other command.

-- mario
Source: StackOverflow