A cronjob we scheduled at night was started correctly but the image itself run much later than scheduled. We expect that there was a problem either pulling the image or requesting the resources from the cluster. Usually I can see such errors in the events section of the kubectl describe job-name
output. In this section I can see events such as pull image
create container
etc. But after the job is finished, no events are shown anymore.
Is it possible the see those events for a finished job?
Or is there another way to investigate such a problem?
The problem with storing events is wider than just cronjobs. Events in Kubernetes by default are stored only for 1 hour (--event-ttl flag for kube-apiserver). That means that if your cronjob was run two hours ago - you will not see events in "kubectl describe".
In order to save events for later investigations you need to export them somewhere. For example, Google Kubernetes Engine stores events into Stackdriver. For vanilla Kubernetes you may store events in Prometheus using event_exporter or to Elasticsearch. Does that answer your question?