I am running GKE k8s deployments/jobs that take a long time to execute - from days to weeks (Machine Learning). Default GKE Events expire after 1h, which is not enough to debug problems that can occur during training, like OOMKilling, etc.
kube-apiserver
isn't exposed in GKE, so I'd like to find a way to access and change a property like event-ttl
.
How can I change event-ttl
for an already launched cluster, or how to specify event-ttl
at a cluster creation? For example if I would like all my events in a cluster to be available for 24 hours. Is there a kubectl
command for that (the kubernetes
svc is accessible)?
Thanks in advance!
It's currently not possible. As you said, the kube-apiserver
flags are currently not user-configurable on GKE. There are plans to expose more such knobs to the users but I don’t know when would this be available.