I want to see logs on stack driver/google logging per http request. Currently, I get all the logs but with so many pods I can't correlate with which log belongs to which request.
On appengine every log entry is per http request by default and contains nested logs from the same request.
I am using gunicorn with python if that's helping.
If that's helping, that's how i write logs:
def set_logging_env(app):
logging.basicConfig(format='', level=logging.INFO)
if __name__ != '__main__':
gunicorn_logger = logging.getLogger('gunicorn.info')
app.logger.handlers = gunicorn_logger.handlers
app.logger.setLevel(gunicorn_logger.level)There are some option to customize your logging patterns. First I would suggest getting familiar with the basics from the official documentations here.
Than there is a general guide regarding logging with stackdriver.
There we have:
Stackdriver Logging agent attaches metadata to each log entry, for you to use later in queries to select only the messages you’re interested in: for example, the messages from a particular pod.
Which is one of the things you seek I suppose.
Finally you can use follow this guide to know how to view logs and later this one to setup advanced filters:
This guide shows you how to write advanced logs filters, which are expressions that can specify a set of log entries from any number of logs. Advanced logs filters can be used in the Logs Viewer, the Stackdriver Logging API, or the command-line interface.
Also if you want to check logs from the running pods from the Kubernetes level you can use this cheatsheet.
kubectl logs my-pod # dump pod logs (stdout)
kubectl logs -l name=myLabel # dump pod logs, with label name=myLabel (stdout)
kubectl logs my-pod --previous # dump pod logs (stdout) for a previous instantiation of a container
kubectl logs my-pod -c my-container # dump pod container logs (stdout, multi-container case)
kubectl logs -l name=myLabel -c my-container # dump pod logs, with label name=myLabel (stdout)
kubectl logs my-pod -c my-container --previous # dump pod container logs (stdout, multi-container case) for a previous instantiation of a container
kubectl logs -f my-pod # stream pod logs (stdout)
kubectl logs -f my-pod -c my-container # stream pod container logs (stdout, multi-container case)
kubectl logs -f -l name=myLabel --all-containers # stream all pods logs with label name=myLabel (stdout)I hope I understood you correctly and my answer would be valuable. Please let me know if that helped.