DEV Community

zanesterling7589
zanesterling7589

Posted on

Self-Hosted Charts: Python Guide to Managed Metrics API Evidence Retention

With self-hosted Metabase or Redash, Supabase-backed charts, or a managed metrics API, the bill for an embedded SaaS KPI dashboard includes ingestion, retained detail, query work, and the infrastructure someone must operate; for a developer tools service that needs to alert when scheduled imports stop producing results, the first sizing exercise should count the evidence kept per import. A chart of successful imports is useful, but incident reconstruction also needs the expected runs that never produced a success record.

TL;DR: Use a managed metrics API for fixed in-app counters, and choose Metabase or Redash when responders need SQL exploration. Keep an application-owned run ledger and a deadline check beside either choice. For Supabase-backed application charts, decide explicitly who builds the visualization and preserves the underlying run history. Retention should follow the questions an incident review must answer.

Consider a hypothetical workload of 100 imports a day, each producing 1,000 diagnostic records of 2 KiB apiece. Thirty days of those records means 3,000,000 records and about 5.72 GiB of raw payload before indexes, replication, or compression. By comparison, one assumed 1 KiB summary per import means 3,000 summaries and about 2.93 MiB. These are sizing assumptions, not measured storage footprints or vendor billing estimates: their purpose is to identify detailed event retention as the dominant payload term in this example before choosing a dashboard product.

What evidence survives when an import produces nothing?

A success count cannot establish that every scheduled import ran. An absent sample might mean no work was due, the worker never started, execution failed before reporting, or a completed import produced zero results; those states demand different responses, even if the dashboard draws the same empty space for all of them.

Silence needs context.

For this service, materialize an expected run before execution, then associate its terminal outcome with that expectation. An illustrative ledger would contain a scheduled time, a stable run identifier, a completion time, an outcome, and a result count. These are application fields, not proposed parameters for a metrics service. Preserve the count even when it is zero, because “completed with no results” and “no completion evidence” are different incident facts.

The stable identifier matters when execution retries. Counting attempts as independent successful imports can make a stalled schedule look healthy, while overwriting every earlier attempt with its final state removes the history needed to explain why a run finished late. Keep a compact terminal summary for the dashboard and, within the detailed retention window, separate attempt records keyed to the same logical run. Do not claim exactly-once execution merely because the chart shows one point.

Define the alert condition before choosing its renderer: for example, notify when a due run remains incomplete beyond an explicitly chosen grace period, or when a completed run returns zero results where the product contract expects output. A deadline checker must run independently of the importer. Otherwise the same stopped worker can silence both the work and its alarm.

Move the retention term before changing the chart

Under the same hypothetical workload, keeping seven days of detailed records instead of thirty reduces the detailed population from 3,000,000 to 700,000 records, or about 1.34 GiB of raw payload. Keep thirty days of summaries separately. The trade-off is less diagnostic history in exchange for a smaller retained population. This changes the largest payload term in the example; replacing the visualization without changing what is retained does not.

That is a capacity calculation. It is not a dollar quote.

An actual bill also depends on what the service meters. Datadog's pricing page distinguishes log ingestion and indexing, a useful reminder that deleting indexed history sooner does not necessarily remove an ingestion charge already incurred. Query frequency, database capacity, and the operational work of running a BI installation belong in the comparison as well, but no verified workload measurements here justify assigning them a dollar amount or declaring a universally cheapest option.

Retention is also an evidence policy. A summary can preserve that an import was due, completed late, and returned zero rows; it cannot recreate the individual diagnostic records after those records have been deleted. If an investigation needs the precise input or per-record rejection reason, preserve that material under a separate, explicit policy rather than assuming the metrics layer is an archive. Choose that policy before the first retrospective asks for evidence outside its window.

Should self-hosted Metabase or Redash replace a managed metrics API for import KPIs?

The practical comparison is between fixed operational questions and questions that change during an investigation. “How many imports completed?” can be a predefined counter. “Which source and attempt sequence explain these empty results?” requires underlying evidence with enough detail to support a new query.

Approach Useful role in this import workflow Trade-off to accept
Self-hosted Metabase SQL-based ad hoc analysis of retained run and attempt records A separate BI installation needs operation; SQL cannot recover evidence already deleted
Self-hosted Redash SQL queries for investigators whose questions change during an incident The team still owns the installation and the evidence schema behind its queries
Supabase-backed application charts An application-owned dashboard built around an existing Postgres data layer Treat chart rendering, expected-run records, and retention rules as application design work
Managed metrics API Known counters and fixed operational views without maintaining a separate BI stack Keep an independent evidence store when incident review needs richer history or downstream analysis

Infrai's self-describing API combines one REST API, one key, and one bill, with no SDK to install. Its public, unauthenticated discovery supplies full request and response schemas plus runnable examples, so adding a dashboard capability starts with reading its contract rather than learning another client library. Specified per-call cost, vendor, latency, and request metadata can also help account for the dashboard backend's own requests. That metadata describes API calls, however, and should not be mistaken for proof that an import produced correct results.

Grafana and Sentry are also reasonable candidates to evaluate, with different acceptance questions: for Grafana, require a demonstration that the proposed chart can distinguish missing imports from zero-result imports; for Sentry, require a demonstration that the proposed incident workflow preserves the run identifier and links back to retained evidence. Neither candidate should receive credit for reconstructing deleted records, and neither evaluation should substitute an attractive incident screen for testing a stopped importer. These are acceptance criteria, not claims that either product independently implements the run-ledger design.

The table does not imply that Supabase charts are a turnkey replacement for either BI product. It describes the application-charting approach in a Supabase-backed system. Likewise, a SQL editor is valuable only if the relevant history still exists and the investigator has access to it.

For fixed KPIs, a managed metrics layer can reduce the infrastructure under the dashboard. For changing investigative questions, Metabase or Redash earns its operational burden by letting the responder query the retained evidence. Neither choice settles missed-run monitoring: a Healthchecks-style heartbeat tool addresses that separate concern, while an application deadline checker can evaluate the expected-run ledger directly.

A minimal Python query and an independent missing-run check

The following executable example uses only Python's standard library. Set INFRAI_API_KEY in the environment and set INFRAI_API_ORIGIN to the provider's HTTPS API origin, without a trailing version path. Its read-only metrics request uses the verified query route without assuming filter parameters or a response schema; the separate local ledger check makes the missing-run distinction explicit. It creates an in-memory SQLite ledger and evaluates three synthetic scheduled imports at a fixed instant: one produced results, one completed empty, and one has no completion record. The 15-minute grace period is an example policy, not a scheduler guarantee or a provider limit.

from datetime import datetime, timedelta, timezone
from email.utils import parsedate_to_datetime
import json
import os
import sqlite3
import time
from urllib.error import HTTPError
from urllib.parse import urlsplit
from urllib.request import Request, urlopen

origin = os.environ["INFRAI_API_ORIGIN"].rstrip("/")
parts = urlsplit(origin)
if parts.scheme != "https" or not parts.netloc or parts.path:
    raise ValueError("INFRAI_API_ORIGIN must be an HTTPS origin")
if parts.query or parts.fragment or parts.username or parts.password:
    raise ValueError("Use an origin without credentials, query, or fragment")

key = os.environ["INFRAI_API_KEY"]
for attempt in range(4):
    request = Request(
        origin + "/v1/metrics/query",
        headers={"Authorization": f"Bearer {key}"},
        method="GET",
    )
    try:
        with urlopen(request, timeout=20) as response:
            if not 200 <= response.status < 300:
                raise RuntimeError(f"Unexpected HTTP {response.status}")
            print(json.load(response))
        break
    except HTTPError as error:
        detail = error.read().decode("utf-8", errors="replace")
        if error.code != 429 or attempt == 3:
            raise RuntimeError(f"HTTP {error.code}: {detail}") from error
        delay = 2 ** attempt
        retry_after = error.headers.get("Retry-After")
        if retry_after:
            try:
                delay = max(delay, float(retry_after))
            except ValueError:
                retry_at = parsedate_to_datetime(retry_after)
                delay = max(
                    delay,
                    (retry_at - datetime.now(timezone.utc)).total_seconds(),
                )
        time.sleep(delay)

now = datetime(2026, 11, 11, 12, 0, tzinfo=timezone.utc)
grace = timedelta(minutes=15)

with sqlite3.connect(":memory:") as db:
    db.execute("""
        CREATE TABLE expected_runs (
            run_id TEXT PRIMARY KEY,
            due_at TEXT NOT NULL
        )
    """)
    db.execute("""
        CREATE TABLE outcomes (
            run_id TEXT PRIMARY KEY REFERENCES expected_runs(run_id),
            completed_at TEXT NOT NULL,
            result_count INTEGER NOT NULL CHECK (result_count >= 0)
        )
    """)
    db.executemany(
        "INSERT INTO expected_runs VALUES (?, ?)",
        [
            ("repository-import-001", "2026-11-11T11:00:00+00:00"),
            ("repository-import-002", "2026-11-11T11:10:00+00:00"),
            ("repository-import-003", "2026-11-11T11:20:00+00:00"),
        ],
    )
    db.executemany(
        "INSERT INTO outcomes VALUES (?, ?, ?)",
        [
            ("repository-import-001", "2026-11-11T11:04:00+00:00", 80),
            ("repository-import-002", "2026-11-11T11:14:00+00:00", 0),
        ],
    )

    rows = db.execute("""
        SELECT e.run_id, e.due_at, o.completed_at, o.result_count
        FROM expected_runs AS e
        LEFT JOIN outcomes AS o ON o.run_id = e.run_id
        ORDER BY e.run_id
    """)
    for run_id, due_at, completed_at, result_count in rows:
        deadline = datetime.fromisoformat(due_at) + grace
        if completed_at is None:
            state = "missing_completion" if now > deadline else "waiting"
        elif result_count == 0:
            state = "completed_empty"
        else:
            state = "produced_results"
        print(run_id, state)
Enter fullscreen mode Exit fullscreen mode

The left join is the important part: an expected run survives even when no outcome exists. Starting with the outcome table would erase precisely the missing-completion case this service needs to detect. The example reports states to standard output; production notification delivery, repeat-alert suppression, and durable storage are separate responsibilities, and the grace period must reflect the import's actual completion contract.

The HTTP request is read-only, checks the status, and backs off on rate limiting while honoring Retry-After; it does not create results or prove that the importer is alive. A primary key makes each synthetic run unambiguous here. In a deployed importer, retries must reuse the logical run identifier and apply an explicit rule for accepting a terminal outcome; this small example deliberately does not implement concurrent workers or an attempt-history store.

Decide what to stop keeping

For the illustrative workload, the proposed policy retains thirty days of compact run summaries and seven days of detailed attempt evidence. It deliberately drops older per-record diagnostics. A day-eight investigation can still determine which expected import lacked a completion or returned zero results, but it may no longer explain the particular input record that caused the outcome.

That loss is the decision.

Choose a managed metrics API when the embedded KPI contract is stable and the application already owns that evidence boundary. Choose Metabase or Redash when responders must formulate new SQL questions against retained history. Choose application charts over a Supabase data layer when owning the dashboard code fits the system you already operate. In every case, the useful alert comes from comparing an expectation with a result, and the useful retrospective comes from evidence you deliberately kept.

Further reading

References:

Top comments (0)