summaryrefslogtreecommitdiff
path: root/4 Archives/Backup service metrics/Prometheus metrics.md
blob: d41a80c0341ce81b15e9ee64279409b24c6bd735 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
---
tags:
  - gobs
  - metrics
  - prometheus
---
Prometheus' main thing is to scrape endpoints for metrics. Thus instead of exposing alot of data via the API that keystone would query and create relevant metrics, we can just expose a /metrics endpoint ourselves. This is alot faster because we can directly query all data structures and we have all the domain knowledge here to know what is relevant.

A hack to clear gauges in an older version of prometheus_client:
```python
    def update_gauge(self, gauge, stats):
        # Ensure that if a stat combination is gone, because it is resolved, we actually remove it from the Gauge.
        # Same as clear() which is available in a later version of prometheus_client but not in the one available
        # through apt at the time of writing.
        with gauge._lock:
            gauge._metrics = {}

        for stat, count in stats.items():
            gauge.labels(*stat).set(count)
```

According to Rutger labels aren't deleted on the Thanos/Prometheus side, so we don't want to be "too flexible" with our labelling. Rather create specific metrics. This is exactly the cardinality problem of labels.