I am running a standalone Apache Spark cluster in a Kubernetes environment.

There is a need to export metrics to Prometheus and then finally display them in Grafana.

I found that installing a Graphite exporter was the simplest solution to do this since I am experiencing some trouble with getting all Spark metrics while only using the JMX exporter.

The thing I am having trouble with is to create mappings from graphite output to output that is parsable by prometheus templating.

For instance, I want to be able to parse

app_20200120105608_0736_0_executor_threadpool_completeTasks

so that it matches to something similar to this:

- match: '*.*.threadpool.*.*'
  name: app_data
  labels:
    application: $1 // app_20200120105608_0736
    executor_id: $2 // 0
    type: $3 // threadpool
    qty: $4 // completeTasks

I am not convinced that this will be the best solution overall so any other suggestions are welcome (For instance how this could be done in a proper way with using a JMX exporter only while also getting Spark app data.)