As logs go away as soon as pods crashes, I would like to store them directly on my local machine. I don't want to use GCE. Also I would have multiple nodes of a service, so HostPath will not be of any use.
kubectl logs <pod-name> > log.txt will just capture a snapshot. I want the complete logs to be persistent on my local machine.
Logs are already on nodes in /var/lib/docker/containers/CONTAINER_ID/CONTAINER_ID-json.log
For collecting them you can use fluent-bit. The easiest approach would be using kubernetes to manage fluent-bit - run daemon set with host path volumes.
Here is helm chart that can do it for you: https://github.com/helm/charts/tree/master/stable/fluent-bit