Prometheus logo PromCon 2017

The Prometheus conference — August 17 - 18 in Munich

Talk abstract

The Uninstrumentable; Getting Apache Spark and Prometheus to Play Nicely

Speaker 1: Dan Rathbone

Speaker 2: Joe Stringer

Instrumenting your code with Prometheus is simple and easy. Or so we thought until we tried to instrument our Python application running under Apache Spark… The distributed nature of Spark presents some interesting challenges when it comes to instrumenting your code effectively, for example a lack of global state, transient processes and no control over the execution profile.

We’ll talk about our myriad failed attempts at instrumenting under Spark and our journey to finally getting something working effectively, without DOSing Prometheus with millions of time series! :)

Video link - Slides

Back to schedule