Simple hello world demo for writing custom OpenCensus metrics to StackDriver on GKE?

8/26/2019

Is there a simple hello world sample for creating and writing custom metrics in a GKE application to StackDriver?

I see there’s a couple larger demo projects that seem to use it, like:

https://github.com/GoogleCloudPlatform/microservices-demo https://github.com/rghetia/microservices-demo/tree/oc_agent

But they seem to be different. The latter one has a DaemonSet for the oc-agent, whereas the first one appears to not have one. Also, there’s the beta Stackdriver Kubernetes Engine Monitoring, and I’m not sure if it would be done differently that way? It seems already set up a metrics server pod, as well as a prometheus-to-sd prod.

-- jacob
google-cloud-platform
google-cloud-stackdriver
google-kubernetes-engine
opencensus
stackdriver

3 Answers

8/27/2019

You can use Prometheus to send custom metric to Stackdriver. You have Go, Java, Python and Ruby, and a lot of Unofficial third-party client libraries.

Here you have a guide and some examples here and here.

-- David C
Source: StackOverflow

8/27/2019

If you're looking to specifically write custom OpenCensus metrics to Stackdriver, there are supported Stats/Metrics and Trace exporters for Stackdriver available for OpenCensus. Examples can be found at https://opencensus.io/exporters/ .

There is example code for stats/metrics implementation with Stackdriver specifically for Node.js, Go, and JAVA.

And here's an example from the GCP Documentation: https://cloud.google.com/monitoring/custom-metrics/open-census

-- AlphaPapa
Source: StackOverflow

8/28/2019

AlphaPapa answer above is good.

I've written a couple of simple examples too:

https://medium.com/google-cloud/opencensus-and-firestore-native-34bc1e3a962f

Some background|guidance:

After Google acquired Stackdriver, client libraries were provided that enabled developers to ship metrics to Stackdriver. A challenge with this tightly-coupled approach is that, customers also use Prometheus, Datadog, Azure, AWS etc. for metrics and Zipkin, Jaeger, Stackdriver and others for tracing and customers should require flexibility in the systems they use today and may use tomorrow.

Google internally uses a system called Census that decouples metrics producers (e.g. your code) from metrics consumers (e.g. Stackdriver). This mechanism was open-sourced as OpenCensus. OpenCensus decouples metrics producers from consumers and trace producers from consumers. This permits you to write code metrics|trace client code once and it permits your SRE team to switch monitoring|tracing systems with impunity. OpenCensus Agent|Collector is the zenith of this approach and should be your default choice when using OpenCensus.

OpenCensus represents probably the best approach and it is mostly excellent except OpenCensus and OpenTracing are merging into OpenTelemetry and -- IMO -- this has caused a speed bump.

I recommend you:

  • consider OpenCensus but do so knowing that it's entered a period of significant change.
  • prefer using OpenCensus Agent|Collector to decouple your client code and services
  • configure the Agent to use e.g. Stackdriver and other metric|trace services
  • use Application Default Credentials on the Agent with appropriate IAM roles:
    • write metrics to Stackdriver
    • write traces to Stackdriver Trace

In my example, I used the Stackdriver Exporter. You should use the OpenCensus Agent Exporter instead. The only caveat with this is that not all the OpenCensus language SDKs have been extended to include the OpenCensus Agent. The Agent is availabe in e.g. Golang. My example shows how to deploy to Kubernetes, using a Google Service Account mapped through a Kubernetes Secret. You could add the Agent as a sidecar (preferable) and then just rejigger the Kubernetes configuration to provide the service account secret to the Agent.

HTH!

-- DazWilkin
Source: StackOverflow