How can I save k8s pod logs to host disk

4/13/2020

i'm stack at k8s log storage.we have logs that can't output to stdout,but have to save to dir.we want to save to glusterfs shared dir like /data/logs/./xxx.log our apps are written in java ,how can we do that

-- tennis wang
fluentd
kubernetes

2 Answers

4/13/2020

This is mostly up to your CRI plugin, usually Docker command line options. They already do write to local disk by default, you just need to mount your volume at the right place (probably /var/log/containers or similar, look at your Docker config).

-- coderanger
Source: StackOverflow

4/13/2020

I had the same problem with one 3rd party application. It was writing logs in a log file and I wanted Fluentd to be able to get them so I wanted somehow to print them on the stdout. I found a workaround with one additional container running alongside with the app container in the same pod.

Lets say the 3rd party app is writing logs in the following file:

/some/folders/logs/app_log_file.log

The following pod will run two containers, one with the app and the other with busybox image which we will use to fetch the logs from the app container.

apiVersion: v1
kind: Pod
metadata:
  name: application-pod
spec:
  containers:
  - name: app-container
    image: <path-to-app-image>
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: log-volume
      mountPath: /some/folders/logs
  - name: log-fetcher-container
    image: busybox
    args: [/bin/sh, -c, 'sleep 60 && tail -n+1 -f /var/log/app_log_file.log']
    volumeMounts:
    - name: log-volume
      mountPath: /var/log
  volumes:
  - name: log-volume
    emptyDir: {}

As you can see this manifest is creating a empty volume and mounting volume to the /some/folders/logs folder in the app container and to the /var/log folder in the log fetcher container. Now every file that the app container writes to /some/folders/logs will be visible in /var/log also. That's why the busybox image is running a shell command:

sleep 60 && tail -n+1 -f /var/log/app_log_file.log

First we wait 60 seconds because the app container must have time to start up and create the log file and then the tail command is going to print every new line in the log file to the stdout of the log fetcher container.

And now the fluentd will be able to get the logs from the log file of the app container getting the stdout logs of the log fetcher container.

-- aliench0
Source: StackOverflow