Circleci and many other ci tools show real time logs during the job running. It like the tail -f
in linux but it also show all the previous logs.
I am trying to sync specific logs from kubernetes to s3 and then move the update to the browse it that possible?
I am trying to sync specific logs from kubernetes to s3 and then move the update to the browse it that possible?
IMHO you would want to have a "tee" mechanism to divert the log streams to each destination, since those two destinations have vastly different access (and retry!) mechanisms. AFAIK kubernetes allows unlimited(?) numbers of kubectl logs -f
connections, so you would want one process that connects to the pod and relays those bytes out to the browser, and a separate process that connects and relays the bytes to S3. You could have one process that does both, but that runs the risk of a single bug wiping out both streams, making everyone unhappy.
I used kubectl logs -f
as a "shortcut" for this answer, but I am super positive there's an API endpoint for doing that, I just don't remember what it is offhand -- kubectl --v=100 logs -f $pod
will show it to you.