Docker | awslogs driver | Setting meaning stream name

1/20/2020

I'm telling containerd to use awslogs using /etc/docker/daemon.json file as proposed in the documentation in https://docs.docker.com/config/containers/logging/awslogs/

By default, the aws stream name is set to a randomly generated container-id which is meaningless when you list the streams inside a group.

awslogs driver has an option to set awslogs-stream to a specific name but that won't satisfy my needs since I want different containers to use different streams.

I guess what I want to do is to tell docker to compose the stream-id from the image name and the container-id, but I couldn't find an option for that.

Theoretically, I can set the stream name directly in the docker run command, but that is not good enough because I use Kubernetes to launch those containers so I'm not sure how to set the stream_name from the application yml file.

Any ideas how to accomplish my needs?

-- Eytan Naim
amazon-cloudwatchlogs
containers
docker
kubernetes

1 Answer

1/20/2020

You're correct, there's no sign of --log-opt being implemented into Kubernetes, since dockerd is deprecated.

Instead of specifying the awslogs-stream have you tried to set a tag?

From Docker documentation link you posted:

Specify tag as an alternative to the awslogs-stream option. tag interprets Go template markup, such as {{.ID}}, {{.FullID}} or {{.Name}} docker.{{.ID}}. See the tag option documentation for details on all supported template substitutions.

The other viable approach is using a sidecar container daemon to process the logs and then forward to awslogs but tag is a cleaner, simplier solution.

Here's the process with the fluentd:

How to Send Kubernetes Logs to AWS Cloudwatch

-- willrof
Source: StackOverflow