Skip to content

Metric model redesign for cardinality, scalability, and Prometheus efficiency #34

@darthfork

Description

@darthfork

Summary

The current metric model exposes several high-cardinality labels directly from GitHub webhook payloads, including branch names, runner names, job names, workflow names, commit authors, and commit author emails. This is likely to cause Prometheus series explosion in larger repositories or organizations, increasing memory usage, scrape cost, and query latency.

This issue tracks a broader redesign of metric dimensions so the exporter remains useful at higher event volume and repository count.

Why this matters

  • Prometheus cost scales heavily with time series cardinality.
  • Ephemeral runners, feature branches, and user-identifying labels create unbounded dimensions.
  • Large push events and diverse workflows can make the exporter expensive enough to be impractical in production.

Goals

  • Reduce unbounded label cardinality.
  • Preserve useful operational insights.
  • Make high-cardinality dimensions configurable instead of always-on.
  • Normalize labels where appropriate.

Suggested scope

  • Audit all exported labels by boundedness and operational value.
  • Remove or gate the highest-risk labels.
  • Introduce normalization or bucketing for branches/runners/workflows where useful.
  • Revisit push metrics so they do not create excessive churn per author/email.

Child issues

  • Remove or make optional high-cardinality labels in core metrics.
  • Redesign push-event metrics to avoid per-author/per-email series explosion.
  • Add configurable label normalization and filtering.
  • Revisit histogram and metric shape choices for long-running CI workloads.

Acceptance criteria

  • A default deployment has bounded, production-safe metric cardinality.
  • Optional high-cardinality labels are disabled by default and documented.
  • Metric naming and semantics remain clear for operators.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions