How to trace traffic to GCP kubernetes Services

11/19/2019

How to trace traffic of GCP kubernetes service? I would like to see who access the service and volume of traffic.

-- Nipu
google-kubernetes-engine
http-trace
kubernetes

1 Answer

11/19/2019

Make sure your service that is exposing your pods is configured with externalTrafficPolicy: local to ensure that requests that reach you pod maintain their clientIP.

Next, ensure your application logs incoming connections (nginx does this by default). Ensure that you have logging enabled (either stackdriver or a 3rd party tool like ELK) so that you can export the logs (the k8s cluster won't hold the container logs for very long).

If you are using Stackdriver, you can then create a log-based metric based off of the logs you collected. As long as you collect the logs by container, the metric should aggregate logs from each of your pods. You can then use Stackdriver Monitoring to watch the network load the pods receive and Stackdriver Logging to review the source IPs of the requests.

If you can't get your application to log connections, an alternative would be to create a DaemonSet that runs a TCPdump to stream connections to your service (how you configure this sepcifically will depend on which service you are using).

-- Patrick W
Source: StackOverflow