From 80ccf68f55dbb70d7e5ed52ee95b3c9d1b6ce264 Mon Sep 17 00:00:00 2001 From: Jasper Ras Date: Thu, 20 Mar 2025 11:07:49 +0100 Subject: vault backup: 2025-03-20 11:07:48 --- .../Backup service metrics/Prometheus metrics.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 4 Archives/Backup service metrics/Prometheus metrics.md (limited to '4 Archives/Backup service metrics/Prometheus metrics.md') diff --git a/4 Archives/Backup service metrics/Prometheus metrics.md b/4 Archives/Backup service metrics/Prometheus metrics.md new file mode 100644 index 0000000..d41a80c --- /dev/null +++ b/4 Archives/Backup service metrics/Prometheus metrics.md @@ -0,0 +1,22 @@ +--- +tags: + - gobs + - metrics + - prometheus +--- +Prometheus' main thing is to scrape endpoints for metrics. Thus instead of exposing alot of data via the API that keystone would query and create relevant metrics, we can just expose a /metrics endpoint ourselves. This is alot faster because we can directly query all data structures and we have all the domain knowledge here to know what is relevant. + +A hack to clear gauges in an older version of prometheus_client: +```python + def update_gauge(self, gauge, stats): + # Ensure that if a stat combination is gone, because it is resolved, we actually remove it from the Gauge. + # Same as clear() which is available in a later version of prometheus_client but not in the one available + # through apt at the time of writing. + with gauge._lock: + gauge._metrics = {} + + for stat, count in stats.items(): + gauge.labels(*stat).set(count) +``` + +According to Rutger labels aren't deleted on the Thanos/Prometheus side, so we don't want to be "too flexible" with our labelling. Rather create specific metrics. This is exactly the cardinality problem of labels. \ No newline at end of file -- cgit v1.2.3