--- tags: - gobs - metrics - prometheus --- Prometheus' main thing is to scrape endpoints for metrics. Thus instead of exposing alot of data via the API that keystone would query and create relevant metrics, we can just expose a /metrics endpoint ourselves. This is alot faster because we can directly query all data structures and we have all the domain knowledge here to know what is relevant. A hack to clear gauges in an older version of prometheus_client: ```python def update_gauge(self, gauge, stats): # Ensure that if a stat combination is gone, because it is resolved, we actually remove it from the Gauge. # Same as clear() which is available in a later version of prometheus_client but not in the one available # through apt at the time of writing. with gauge._lock: gauge._metrics = {} for stat, count in stats.items(): gauge.labels(*stat).set(count) ``` According to Rutger labels aren't deleted on the Thanos/Prometheus side, so we don't want to be "too flexible" with our labelling. Rather create specific metrics. This is exactly the cardinality problem of labels.