I'm looking for a way to stream ("tail") a binary file on a Kubernetes pod to my local machine.
I've tried this:
kubectl exec -it app-service-58697cf7c9-nnzgh -c tcpdumper -- tail -f -c +0 /output.pcap
(tcpdumper is just a thin wrapper around tcpdump which runs as a helper container in the pod).
This almost works. I'm able to view a stream of binary data on my local machine when I run this command.
The end goal of what I'm trying to do here is that I'd like to take this binary stream of pcap data and pipe it to Wireshark running on my machine. That's what doesn't work, and it's because the data isn't exactly what's being written on the pod.
What's relevant though isn't that this is Kubernetes, or that it's packet capture. The issue appears to be with how I'm streaming this data using tail; when I do this in this fashion, tail appears to add newline characters. I presume this is because tail is not intended to handle binary data.
If I run tcpdump directly on the pod, write it to a .pcap file there, and then transfer that file using kubectl cp, and then load that file into Wireshark, it works and I can view the network traffic. I can see using a hex editor that the difference between this method and the tail method above is that there are extraneous characters ("0xD", which is the newline character) every so often.
Any ideas?
I'm not sure what's causing tail to randomly insert newline chars, and I'm not in a position to reproduce your issue but I do have a couple of ideas:
1: base64 encode the file on the server as it's being written to, and base64 decode it as you read it on your local machine.
2: Instead of writing to a file, try writing to a fifo (mkfifo), which you can simply cat, rather than tailing.
Without knowing your situation better I can't really talk intelligently on whether or not these are applicable to your situation or not, but thought they'd be worth mentioning.