I'm trying to verify that shutdown is completing cleanly on Kubernetes, with a .NET Core 2.0 app.

I have an app which can run in two "modes" - one using ASP.NET Core and one as a kind of worker process. Both use Console and JSON-which-ends-up-in-Elasticsearch-via-Filebeat-sidecar-container logger output which indicate startup and shutdown progress.

Additionally, I have console output which writes directly to stdout when a SIGTERM or Ctrl-C is received and shutdown begins.

Locally, the app works flawlessly - I get the direct console output, then the logger output flowing to stdout on Ctrl+C (on Windows).

My experiment scenario:

• App deployed to GCS k8s cluster (using helm, though I imagine that doesn't make a difference)
• Using kubectl logs -f to stream logs from the specific container
• Killing the pod from GCS cloud console site, or deleting the resources via helm delete
• Dockerfile is FROM microsoft/dotnet:2.1-aspnetcore-runtime and has ENTRYPOINT ["dotnet", "MyAppHere.dll"], so not wrapped in a bash process or anything
• Not specifying a terminationGracePeriodSeconds so guess it defaults to 30 sec
• Observing output returned

Results:

• The API pod log streaming showed just the immediate console output, "[SIGTERM] Stop signal received", not the other Console logger output about shutdown process
• The worker pod log streaming showed a little more - the same console output and some Console logger output about shutdown process
• The JSON logs didn't seem to pick any of the shutdown log output

My conclusions:

1. I don't know if Kubernetes is allowing the process to complete before terminating it, or just issuing SIGTERM then killing things very quick. I think it should be waiting, but then, why no complete console logger output?
2. I don't know if console output is cut off when stdout log streaming at some point before processes finally terminates?
3. I would guess that the JSON stuff doesn't come through to ES because filebeat running in the sidecar terminates even if there's outstanding stuff in files to send

I would like to know:

• Can anyone advise on points 1,2 above?
• Any ideas for a way to allow a little extra time or leeway for the sidecar to send stuff up, like a pod container termination order, delay on shutdown for that container, etc?