We are investigating an issue that is related to slow pod startups on Azure Kubernetes Services. Once the pod startup is complete, we are unable to see historical events. The latest event log appears as <none>.

Here is an example:

sshuser@sandbox-ubuntu-vm:~$ kubectl get pods
NAME                                            READY   STATUS      RESTARTS   AGE
my-service-58bb6868c-rfrgr               1/1     Running     0          4h51m
sshuser@sandbox-ubuntu-vm:~$ kubectl describe pod my-service-58bb6868c-rfrgr
Name:           my-service-58bb6868c-rfrgr
Namespace:      default
Priority:       0
Node:           aks-myproject-41067869-1/10.0.1.163
Start Time:     Thu, 20 Feb 2020 05:25:36 -0500
Labels:         app.kubernetes.io/instance=my-service
                app.kubernetes.io/name=my-service
                pod-template-hash=58bb6868c
Annotations:    <none>
Status:         Running
IP:             10.0.1.183
IPs:            <none>
Controlled By:  ReplicaSet/my-service-58bb6868c
Containers:
  my-service:
    Container ID:   docker://b366fc6459f7dc03287d62da81c9fb1ed2c30ec0ccb4b601d2eea62e644de9f9
    Image:          myacr.azurecr.io/myproject/microservices/my-service:0.0.48
    Image ID:       docker-pullable://myacr.azurecr.io/myproject/microservices/my-service@sha256:cca04f3499271ac5ae383ddccba8c3f716b8f95cd0c06042ef4d7f69626beb8d
    Port:           3000/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Thu, 20 Feb 2020 05:32:38 -0500
    Ready:          True
    Restart Count:  0
    Liveness:       http-get http://:http/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Readiness:      http-get http://:http/ delay=0s timeout=1s period=10s #success=1 #failure=3
    Environment:
      PORT:                          3000
    Mounts:
      /kvmnt from my-service-kv-volume (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from my-service-token-lbfjg (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  my-service-kv-volume:
    Type:       FlexVolume (a generic volume resource that is provisioned/attached using an exec based plugin)
    Driver:     azure/kv
    FSType:
    SecretRef:  &LocalObjectReference{Name:kvcreds,}
    ReadOnly:   false
    Options:    map[REMOVED]
  my-service-token-lbfjg:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  my-service-token-lbfjg
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>
sshuser@sandbox-ubuntu-vm:~$ kubectl get events --all-namespaces
No resources found
sshuser@sandbox-ubuntu-vm:~$

Is there a periodic purge for the event logs? We are able to see events when the pod status is ContainerCreating, but then the events are not visible shortly after (unknown how long) the pod goes to a Running state.

The Kubernetes version is 1.14.8.

Any help is greatly appreciated.

Thanks.