how container logging works?

4/29/2020

I've seen in a lot of docker containers that it's a common practice to create a log file and log like this

/var/log/container.log
...

tail  -f /var/log/container.log

And somehow the output seems to be taken by the container runtime log driver. So if I run

docker container logs container

This will show the output of the file container.log

I know the best practice for any container is to log on stdout and stderr, but I can't understand why the first option works (create a file and run tail on it). Is it because the output of the tail is taken by the driver log and it merges it with the contents of stdout?

Really appreciate your help. Thanks.

-- Ivan Hernandez
docker
kubernetes
logging

1 Answer

4/29/2020

The tail -f approach isn't really a best practice.

As you note, the right way to run a process and produce logs is to have it write its output directly to stdout and stderr, and docker logs will capture them. You can configure Docker's logging layer to send them somewhere else; if you move to a clustered infrastructure like Kubernetes, you can configure its logging layer to capture logs as well, without modifying your application code.

#!/bin/sh
# I am the script that runs the main container process.
# Run a single process, as a foreground process; when that
# process ends, the container exits.
exec some_server --no-daemon --log-file /dev/stdout

When you see tail -f it generally looks like this:

#!/bin/sh
# Start something as a background process.
something --log-file /var/log/something.log &
# Hackily keep the container from exiting.
tail -f /var/log/something.log

The main container process is the tail process. When tail exits, the container will stop, and if you docker stop the container, the signal will go to tail. Oh, by the way, as a side effect, there's a something process running, but if it fails Docker has no way of noticing it. Still, since the tail process is the main process and tail -f will never exit, this keeps a container running, and it will make its file-based logs visible to the container's stdout.

Another interesting approach here is what the Docker Hub httpd image does: it sets its logging to point at a file, but the default content of that file is a symlink to /dev/stdout.

RUN mkdir /var/log/something \
 && ln -s /dev/stdout /var/log/something/something.log
CMD ["something", "--log-file", "/var/log/something/something.log"]

This is interesting because the default behavior will be to write things to that log file, but it's really the /dev/stdout special device, so it will go to the process's stdout (of the main container process). If you want to capture this in a file instead, you can bind-mount your own directory over /var/log/something which will hind the symlink.

-- David Maze
Source: StackOverflow