K8s cronjob is not scheduled correctly several times a day

9/26/2019

I’ve a cluster with cronjob which is running Ok, I schedule the cronjob to run on every 3 min and I notice that sometimes the jobs are not running at all for 6-9 min , (two or three intervals ) . This happen several times a day and I'm not sure why , how can I check what is the problem can be ? is there a way to overcome this ? we use k8s 1.14.7

This is the cronjob

I try to change also the interval to 10 min and still I see this pattern , i.e. several times a day (for 20/30 min - 2-3 intervals )the job is not running .

the job execution time is just 30 sec and does not run in parallel (it run like a singleton job )

the logs (for the running jobs) doesn't show anything

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: fdc-job
  namespace: {{required "Mon namespace variable '(.Values.mon.namespace)' is required" .Values.mon.namespace}}
spec:
  suspend: false
  schedule: "*/3 * * * *"
  successfulJobsHistoryLimit: 1
  failedJobsHistoryLimit: 1
  startingDeadlineSeconds: 10
  jobTemplate:
    spec:
      backoffLimit: 1
      template:
        spec:
          serviceAccountName: mon-sa
          containers:
            - name: cluster-check
              image: {{required "A valid .Values.artifactory.host entry required!" .Values.artifactory.host}}/{{(.Values.mon.image.repository)}}:{{(.Values.mon.image.tag)}}
              args: [“fdc"]
          restartPolicy: Never
          activeDeadlineSeconds: 100
          imagePullSecrets:
            - name: docker-images-secret

update

running command: kubectl get pods return the following for the last 40 min ...

cluster-1569490560-b78nw   0/1     Completed   0          39m
cluster-1569490740-8gcwl    0/1     Completed   0          36m
cluster-1569490920-t9hwj     0/1     Completed   0          33m
cluster-1569491280-qz5sp    0/1     Completed   0          27m
cluster-1569491460-r2dwv    0/1     Completed   0          24m
cluster-1569491640-qn7r8    0/1     Completed   0          21m
cluster-1569492180-vkxcs    0/1     Completed   0          12m
cluster-1569492360-ksn7s    0/1     Completed   0          9m41s
cluster-1569492540-qqwwc   0/1     Completed   0          6m40s
cluster-1569492720-v2dr2     0/1     Completed   0          3m40s

as you can see the job run every 3 min and you can see that 15 / 18/ 30 min doesn't shown as they don't executed , any idea ?

in addition, i've upgraded the version of k8s to 1.15.4 which doesn't solve the problem

The command kubectl get cronjobs returns

NAME                   SCHEDULE      SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cluster               */3 * * * *   False     0        2m49s           10d

Any clue or direction will be very helpful...

-- Rayn D
amazon-web-services
cron
go
google-cloud-platform
kubernetes

0 Answers