I have a few celery workers running in containers under kubernetes. They are not auto-scaled by celery and each run in a single process (i.e. no multiprocessing). I would like to get a bunch of different metrics from them into prometheus. I've looked at celery-prometheus-exporter (unmaintained) and celery-exporter but they are focused on metrics at celery level rather than app metrics inside of the celery workers.
It looks like two options would be either to find some hacky way to get app level metrics to celery-prometheus-exporter which then would make them available to prometheus OR to use pushgateway.
Which is better, or maybe there's another option I missed?
Just use the default client and let it run the http server in a thread.
You can expose any custom app metrics in prometheus format at /metrics endpoint of the app and then scrape /meetrics endpoint from prometheus.