Skip to content

Web UI

InfraSigns ships an embedded web UI — a read-mostly dashboard over the same store that backs your digests, incidents, and reports, plus an add-source wizard that probes a Prometheus live and generates config for you. It is experimental and off by default.

Unauthenticated (self-hosted)

In the default self-hosted single-tenant mode the web UI has no login. Every /app page and the wizard's probe endpoints are unauthenticated — restrict /app at the network layer (VPN, reverse proxy, ingress auth). This is the same posture as the read-only API endpoints, and it matters doubly here because the wizard's verify/preview steps let anyone who can reach /app make the server issue bounded read-only GETs against a URL of their choosing. Setting a provider's client id (auth.github.client_id or auth.google.client_id) switches on cloud sign-in (the hosted / multi-tenant mode), which gates the /app pages behind a login session and requires a live session for the SSRF-adjacent endpoints (/app/sources/{verify,preview,config}, the live-metric .../series feed, and /app/welcome) — closing the anonymous abuse surface to authenticated tenants. The verify/preview probes are also egress-gated in the hosted mode (a private/loopback/metadata dial is refused), so the residual is narrower — a signed-in tenant can still name any source or probe any public URL, since these routes carry no per-org source resolution yet — and is deferred to the config-edit work; keep network controls in front of a hosted deployment regardless. Note one deliberate asymmetry: revoking a member cuts their access to org data immediately (it is re-checked every request), but they retain the session-only endpoints above until their session expires.

Enabling

ui:
  enabled: true   # serves the embedded UI under /app; off by default

The UI has no port of its own — it shares the single HTTP server on server.port (default 8080). With ui.enabled: true, open http://<host>:8080/app.

What you see

Everything under /app is served under an implicit project slug so links are stable from day one: /app redirects to /app/default, and the pages live at /app/default/.... (Multi-project support is a cloud-era feature; self-hosted runs a single project.)

Dashboard

The landing page (/app/default) is an at-a-glance health view:

  • Source health grid — one card per Prometheus source, worst-first (failing → stale → waiting → fresh, then the muted over-limit and removed cards), each judged against the digest cadence. Cards for still-configured sources link to their detail page.
  • Attention banner — a one-line summary when any source is failing.
  • Latest digest — the most recent delivered digest, or a placeholder naming the next scheduled run when the archive is empty.
  • Recent incidents — the last few incident episodes, each linking to its detail page.
  • Health checks — the latest verdict per configured natural-language check, hidden when you have none. The badge is the verdict of the last run; a row also reads not notified when that verdict is not a pass and no notification has carried it — a check in monitor mode, which never announces anything; a failing check still short of its for_runs streak; or a send that did not go out and will be retried on the next run. The card does not know which of the three it is, so it says only that nobody has been told. A passing row never carries the marker: nothing is wrong on it to be told about.

In self-hosted mode, when api.token is configured and the report archive is genuinely empty (first run, or everything aged out via retention), the dashboard offers a Run a digest now button. You paste your API token and the UI calls the existing authenticated POST /api/digest/trigger?wait=true on your behalf — the UI never triggers billable LLM work unauthenticated. (In cloud sign-in mode the manual trigger is disabled — it would run against the operator's data, not your organization — so this button never appears; your organization's digests run automatically on their schedule.)

Sources

The Sources page lists the same health cards as the dashboard grid and auto-refreshes them in place. Each still-configured source links to a detail page:

  • current health state and the Prometheus URL;
  • live metric charts for each configured query (or the node_exporter defaults), fetched from that source's Prometheus on view — a range selector (1h/6h/24h, deep-linkable via ?window=), per-instance lines with a legend, and dashed warn/crit threshold lines. A slow or unreachable Prometheus degrades to a per-chart message, never a broken page. Turn the charts (and their /series data endpoint) off with ui.charts.enabled: false to spare Prometheus the per-view queries;
  • the configured query catalog (name, PromQL, golden signal, warn/crit thresholds), or a note that node_exporter defaults apply when you've set none;
  • a merged timeline of everything that happened to the source — journaled health transitions (fetch failures and recoveries), generated reports (deep-linked into the reports archive), and attributed incidents (deep-linked to incident detail). A chip filters the timeline to health, reports, or incidents.

Sources that appear in health history but are no longer in your config (removed sources) render as non-linkable cards — there is no detail page for a source that isn't configured.

In cloud mode, a source configured beyond your plan's source limit is no longer collected (only the first N sources by config order are monitored, where N is your plan's limit). Its card reads over-limit — a muted "not monitored on your plan" state with an Upgrade to monitor link. It is kept in your configuration and is not counted toward the failing badge; upgrading resumes monitoring with no reconfiguration. See Account and organizations.

Reports archive

The Reports page is the searchable archive of every delivered digest, trends report, and post-deploy verification. Filter by type, source, and month. Expand a row to see the headline, body, any structured claims and evidence, the per-channel delivery receipt (e.g. telegram ✓ · slack ✗), and a feedback control. A claim grounded in one engine finding carries a Show evidence panel with the series the analysis saw, when that series has points to chart — since #323 on a critical cycle's report (the incident summary, filed under digest and badged CRITICAL) as well as an ordinary digest, where before a critical cycle stored no evidence at all. The Incidents archive below is a different view — inbound alert episodes — and is unaffected. The archive is paged: 25 reports at a time, newest first, with a Load more control that appends the next page in place (the filters travel with it). Without JavaScript the same control is an ordinary link that navigates to the next page — that view is a window of the archive (it says so, and links back to the newest page); there is no "previous page" link, so use the filters or go back to the newest to move the other way.

Incidents

The Incidents page is the full archive of incident episodes recorded by the webhook receiver. Filter by state (firing/resolved), source, attributed source, severity, and month. Each row opens a detail page with the full episode: name, origin, severity, state, start/resolve times and duration, the incident summary, per-channel delivery receipts, and — when the alert was attributed to a configured source — a link back to that source's detail page.

Marking an episode resolved by hand

An episode leaves the firing state only when a matching resolved notification arrives, and incidents.retention_days prunes resolved episodes only. So an alert whose sender will never send that resolution — a rule deleted while it fired, a test alert you curl'd at the receiver, an Alertmanager route with send_resolved: false, an exporter you retired — stays "still firing" forever, on the dashboard card, in the digest and on every source timeline.

The detail page of an open episode carries a Mark resolved button for exactly that. It closes the episode as of now and records it as history, with every other field carried over unchanged: severity, the label the sender reported, the summary, the delivery receipts and the original start time. It rewrites nothing about what the episode was.

Whether it is safe to press on an episode you are not sure about depends on your sender, and the button's own hint says so.

  • A sender that repeats a firing alert re-opens the episode with its original start time, and the resolution you recorded stays in the archive beside it — so the cost of a wrong click is a stretch of quiet, not a hidden alert. One caveat on how long that stretch is: a manual close does not clear the receiver's dedupe entry for the alert (only a resolved delivery does), so a repeat arriving within webhook.dedupe_window (default 5m) of the last unsuppressed one is still suppressed and re-opens nothing. With a repeat_interval well above the window — Alertmanager ships 4h — that never applies; a sender repeating faster than the window is the one that waits.
  • A sender that notifies only when its state changes does not repeat at all. A CloudWatch alarm is the shipped example: nothing re-opens the episode until the alarm clears and fires again, so a wrong click there does hide a live alarm. Prefer fixing the sender.

Closing an episode changes its internal id, so the detail page you were on stops existing — going back to it gives a 404 — and you land on the incidents list instead. That list is ordered by when each episode started, 50 rows to a page, so an old episode reappears at its original position rather than at the top, and nothing on the page confirms the close. Filter by firing to check that the episode is gone from the open set.

Any member of the organization can use it (unlike removing a source, which is owner-only — that changes what the deployment monitors; this changes the recorded state of one observation). In self-hosted mode it is an unauthenticated write like the rest of /app, the same posture as the report feedback control below — one more reason to keep /app behind network controls.

Severity is shown as the canonical tier InfraSigns read the alert at — critical, warning or info. When the sender reported a different label, that label is shown beside the badge as muted text, introduced by sender reported:, on both the list rows and the detail page: warning sender reported: p1 means the episode was delivered at the warning tier and the alert rule said p1. The row names your vocabulary; it does not diagnose it. p1 sits there identically whether a severity_aliases entry put it at warning deliberately or the fallback that catches every label no table covers put it there by default — and the two are the same picture with very different consequences.

To tell them apart, and to fix the second: incidents exist only for alerts pushed to the webhook receiver, and the only thing that ever names an unmapped label is the receiver's WARN line (unrecognized severity labels rank as warning), which is throttled to one line per payload format per 24h — so a second fleet pushing the same format has its labels silenced for the window. The un-throttled infrasigns_alerts_unrecognized_severity_total{path="webhook"} counter is the durable half, but it counts requests and its only attributes are path and source (the payload format, not the sender), so it says that something unmapped arrived and never which label it was. The Settings Unmapped severity labels your sources send card does not cover these rows — it reports what a collection cycle saw, and nothing pushed to the receiver ever reaches it, so an empty card there is not an all-clear for a push-only fleet. The table to edit is the process-level severity_aliases: block in the config file: the receiver is process-global and reads the operator's table, not a hosted organization's, so editing the alias table in Settings changes no incident row.

Nothing extra is shown when the label and the tier agree, including when they differ only in case, so a fleet already writing critical/warning/info sees the same row it always did. An empty qualifier is four different facts and the page cannot tell you which: the sender reported no label (an alert rule that set no severity:, or any CloudWatch alarm — those carry no severity field at all, so the receiver derives the tier from the alarm state); a label was reported but was built only of whitespace, control characters or zero-width characters, so nothing survived the receiver's neutralization; the label matched the tier; or the episode was recorded before this shipped, since nothing is backfilled. That last one matters while upgrading: an older p1 episode and a warning-writing one look identical in the same list.

Two other surfaces show incidents and deliberately do not carry the qualifier: the dashboard's Recent incidents card and a source's detail-page timeline. Both are compact activity feeds; open the episode to see the reported label.

Like the reports archive the list is paged (50 episodes at a time, Load more for the next page). Episodes are ordered by when they started — the only moment that never moves, so an episode that resolves while you are reading stays where it was instead of jumping over your position. (Episodes that started in the same instant — one alert group — are ordered by an internal id that does change on resolution, so one of them resolving mid-read can still shift by a position within that group.) The dashboard's Recent incidents card still orders by last activity, so a long-running episode that is still firing stays on top there — and so does an episode that resolves long after it started, which the archive keeps at its start position.

Notifications

The Notifications page shows your notify configuration: which channels (Telegram, Slack, email, PagerDuty) are configured, with secrets redacted, and which feeds each one receives (digest, trends, incidents, deploys, checks) — PagerDuty is paged by incidents alone, which means inbound alert groups and health-check transitions. In self-hosted it is read-only (the config file is the source of truth); in cloud sign-in mode an organization owner can edit its Telegram, Slack, and email channels here (see below — PagerDuty has no edit form). It reflects config, not a live probe — for live channel health, see /readyz in Observability.

In cloud sign-in mode the page reads your organization's own channels from its stored configuration per request, so each tenant sees only its own delivery targets (never the operator's). A failed read shows an error rather than a false "no channels configured". Self-hosted reflects the config file.

The feeds list is likewise per-organization in cloud mode, mirroring the Settings page's honored/operator-level split. The feeds the per-organization worker actually runs — digest, trends, and health checks — appear as active or off. The two operator-level feeds — incident alerts (the alert webhook is process-global) and deploy verification (deploys run process-wide) — are not per-organization yet, so a feed you configured is shown muted as operator-level with a note that it isn't delivered to your channels, and a feed you didn't configure is omitted (you are never shown a CTA to set a process-level knob you can't touch). As each becomes per-organization, its feed moves back into the active list. PagerDuty is paged by incidents, and exactly one per-organization feed is one: a health-check transition, which pages on failure and clears on recovery. So a configured per-org PagerDuty channel carries a scope row naming what actually reaches it, and the card is shown muted with a reason when nothing can — an organization with no checks, or whose checks are all monitor, has a channel that cannot fire, and the page says so rather than implying delivery. Whether the channel is muted is therefore a property of your configuration, not of the mode you run. Digests, trends and deploy verdicts are suppressed there as everywhere, and inbound alert groups still page at the operator level.

Unlike the other channels it has no edit form: a routing key is operator-seeded, and there is no tenant write path for one (#284 removed the form when the channel was inert; what changed since is that it can deliver, not that a tenant can set it).

Cloud owners can edit the Telegram, email, and Slack channels directly from this page. For Telegram: set or rotate the bot token and chat ID, or remove the channel — a blank token keeps the stored one (so you can update the chat ID without re-entering the token); the stored token is never shown. For email: set the SMTP host, port (defaults to 587), From address, and recipients (one per line or comma-separated); the SMTP user and password are optional (leave both blank for an unauthenticated relay), and a blank password keeps the stored one when a user is set. The stored password is never shown. For Slack: pick the transport — an incoming webhook URL, or the Web API (a bot token plus a channel ID, which enables threaded incident updates); switching transport drops the other one's stored credentials, and a blank secret keeps the stored one within the same transport. The stored webhook URL and bot token are never shown. (PagerDuty has no per-org edit form — its routing key is operator-seeded, as described above.) Removing a channel clears its credentials from your organization. Notifications are editable only once your organization has added a source (the stored configuration requires one), so add a source first. Self-hosted edits its config file directly.

On the Free plan Slack is not delivered: a Slack channel configured on a paid plan or Team trial stays in place, but the worker suppresses its delivery once the plan lapses to Free (re-upgrading resumes it — nothing is rewritten). Telegram and email are unaffected. See Billing.

Account and organizations (cloud)

In cloud mode (a provider client_id set) the top bar carries the signed-in chrome that self-hosted has no need for:

  • The account menu shows your identity (name and email) and a Log out action — a same-origin POST that revokes the session server-side and clears the cookie, returning you to /login.
  • An organization switcher lists the orgs you belong to; picking one re-POSTs through /auth/select-org, which re-checks your membership before switching (a revoked membership cannot be switched into). The switch list appears only when you belong to more than one org.
  • New organization (/app/org/new) creates an org you own and switches you into it. The URL slug is derived from the name automatically.

  • Members (/app/org/members) lists the current org's team and — for an owner — lets you invite teammates by email, revoke pending invites, change a member's role (owner ⇄ member), and remove a member. Any member may view the roster; every mutation is owner-only (a non-owner request is refused). Inviting a teammate is capped at the org's plan seat limit (a member or a live pending invite each count as a seat); adding a source is capped likewise (#33).

  • Billing (/app/billing) shows the org's current plan, trial days remaining, and usage against the plan's honored limits (sources, members, Slack, the daily LLM-call ceiling, and the history-retention cap — Free 7 days / Solo 90 / Team 365; report, timeline, and source-health history is pruned to the smaller of your configured retention_days and this tier cap). A new org starts on a 14-day trial that becomes the Free plan when it ends. When Stripe is configured, owners get Upgrade (Solo / Team) and Manage billing actions (Stripe Checkout and the Customer Portal). A lapse to Free suppresses Slack delivery and stops monitoring any sources beyond the plan limit (their cards read over-limit), but keeps your configuration — upgrading resumes both.

Self-hosted renders none of these — there is no login, and the single implicit org needs no switcher.

Inviting a teammate

An owner enters a teammate's email on the Members page; InfraSigns creates a single-use, 7-day invite and shows a shareable link (there is no email delivery yet — copy the link and send it yourself). The teammate opens /invite/{token}, signs in with GitHub or Google, and joins only if their OAuth-verified email matches the invited address — a leaked link cannot be redeemed by a different account, and no membership is ever granted to an unverified identity. Re-inviting the same address refreshes the link (the old one stops working); an invite is consumed the moment it is accepted.

Roles and removing members

Each member is an owner or a member. Owners can manage the team; members have read-only access to the roster. From the Members page an owner can:

  • Promote a member to owner or demote an owner to member (the role toggle on each row). You can step down to a plain member yourself — this is how ownership is handed off: promote a teammate, then step down. Because only owners can invite, demoting an owner (or stepping down) also revokes any pending invites they had created.
  • Remove a member. Removal takes effect immediately — a removed member's next request drops them back to the organization picker — and any invites they had created for the org are revoked with them.

An organization must always keep at least one owner: the last owner cannot be removed or demoted (promote someone else first). You cannot remove yourself here; step down and ask another owner. A removed member keeps any membership they hold in other organizations — only this org's membership ends.

Settings

Reached from the account menu in the top bar, Settings is a read-only projection of the running configuration — LLM provider and model, schedules, the severity-alias entry count, retention windows, ingress and feature toggles, server and storage — with the API key and database DSN redacted. It's a quick way to confirm what the process actually loaded without shelling into the container.

In cloud sign-in mode the page instead reflects your organization's own stored configuration. It shows the settings the per-organization worker actually honors — LLM, schedules, health checks, alert severity, and retention (report, timeline, and source-health history are each pruned to the smaller of your organization's own retention_days and your plan's per-tier retention cap, so the value shown is the effective one the worker prunes at — a configured value above your tier renders as the tier cap, not the raw number) — as active. A value your config carries that the worker does not yet apply per organization is shown only when you set it, muted, under a note that explains why it is inert — never as an active setting. Three kinds are distinguished, each with its own note so none falsely implies a state it doesn't have: the operator-level features that still run process-wide (the alert webhook and deploy verification) carry an "applied at the operator level" note; the features the hosted service does not run at all (the manual digest trigger and the Telegram/Slack Q&A bots, which would otherwise read the operator's data rather than yours) carry a distinct "the hosted service doesn't run these yet" note; and incident-history retention — the one retention knob not applied per organization, because incidents arrive through the process-global webhook receiver so your organization has no incident rows — gets its own note explaining the value isn't applied (rather than sitting under the operator-level note, where its number could suggest your incidents are being pruned). Server and storage (which an organization config never carries) are omitted. As each remaining feature becomes per-organization, its row moves into the active sections.

An Alert severity section reports how many labels your severity_aliases table maps, and a Severity aliases card below the sections lists every mapping for every member — an owner additionally edits it a row at a time. That editor is the page's one write affordance: the intro says "read-only" to every reader who doesn't get it, and stops saying so to the one who does; a member reading the table is still reading a read-only page. Without the editor the card is a plain two-column Label / Severity table — the headers are what keep which way a mapping runs reachable to a screen reader — and its intro names the three severities a label can map onto, so an organization with no mappings still learns the range. An editor row is a label your alert rules write on the left and the severity InfraSigns should read it as on the right; saving replaces the whole table and leaves the rest of your configuration untouched, and clearing every row removes the table. A new row's severity starts unset rather than pre-picked, because this table escalates — mapping p1: critical means a p1 alert now opens an incident and pages at critical urgency — so the tier is always a deliberate choice. Two rows naming the same label (in any case, with any padding) are refused rather than one quietly winning; every other rule — a blank label, a severity outside info/warning/critical, an attempt to redefine one of those three — comes back from the same validator a self-hosted YAML file passes through, so the UI and the file cannot disagree about what a valid table is. Two further limits apply to what a hosted organization can store: at most 200 mappings, and a label of at most 500 runes (#366). The editor sends the whole table on every save, so that submission is size-capped too — but since #370 the cap is sized to hold any table the store will accept, so a valid table is never too large to submit. A table stored outside this editor that runs over the 200-mapping limit still posts: it comes back naming the limit and asking you to remove rows, and it renders in full so you can. A Clear all button wipes every row in one step for a table you want to rebuild, and each row also has its own delete — so a table too large to save can be shrunk until what remains will. Like every other organization-config write, the save is version-checked: if your configuration changed in another session, it is refused with a reload prompt rather than overwriting. A save takes effect on your next collection cycle and rewrites no history. The card appears only once the organization has a stored configuration, which in practice means once it has added a source. Self-hosted the card never appears — the config file is the only way to change the table there — though the Alert severity count is shown, since reporting what the process loaded is what that page is for.

Neither of those tells you whether the table matches your fleet: an entry reads back just as convincingly when it is a typo (p-1 where your rules write p1) as when it works. An Unmapped severity labels your sources send card answers that (#358), above the severity-alias card and visible to every member, not just an owner — it is information, not a write surface. Per source, it lists the labels that source's most recent collection cycle carried and no alias maps, each with how many alerts carried it in that cycle, loudest first. Both lists are bounded and both say so rather than presenting a handful as the whole: when a source sent more distinct labels than fit, the card reports how many of how many it is showing, and when more than twenty sources have unmapped labels it reports that too. Read it against the table: a label you mapped is gone from the list, a label you mistyped is still on it.

Everyone who can see the card can make that comparison: both it and the table it is read against render for every member (#363), and only changing the table is owner-only. Self-hosted both are there too — the card on the page, the table in your config file.

The card is careful about what silence means, because that is the whole point of having it. A source is reported as recognized only when its last cycle actually assessed something — at least one alert carrying a severity: label, all of them mapped. A source whose last cycle carried no alert with a severity label gets a separate, deliberately neutral line: that covers both a quiet cycle and a fleet whose rules omit severity: entirely, and neither tells you anything about your vocabulary. (The second case matters more than it sounds — alerts with no severity label are read as warnings by the same fallback this card exists to expose, so counting them as "recognized" would have been a green light on exactly the fleet that needs fixing.) A source is absent from the card rather than counted as either when there is no reading to show at all: no cycle of its has ever scraped successfully, or you have removed it from your config, or retention has reaped its last reading. A source that scrapes fine but never has an alert firing is not absent — it has a reading, and that reading lands in the neutral line above. And if the reading itself cannot be loaded, the card says so and shows no counts — an empty list and a failed read are never rendered the same way.

Two limits are worth knowing. The counts are one cycle's snapshot, not a running total, so a number falls when the alerts stop firing. And an alert pushed to the alert webhook is not attributed to a source, so its labels never reach this card — for a push-only fleet an empty card is not evidence that its vocabulary is understood (#350).

Add-source wizard

The wizard turns "which queries should I even scrape?" into a guided flow. It's available two ways: as the first-run onboarding page (/app/welcome, a standalone add → verify → channels → digest walkthrough) and as Add source inside the app shell (/app/default/sources/new, the add → verify steps only).

It's driven by three server endpoints, all pure read-only probes or computation:

  1. Verify (POST /app/sources/verify) — you enter a Prometheus base URL; the server issues two fixed read-only GETs (/api/v1/query?query=up and /api/v1/alerts) and reports back aggregate counts: metrics found, targets up, active alerts, and the scrape jobs it discovered with a suggested golden-signal query count for each recognized exporter (node_exporter, postgres_exporter, blackbox, kube-state-metrics, cAdvisor, Redis, MySQL, NGINX). Response bodies from your Prometheus are never echoed back.
  2. Preview (POST /app/sources/preview) — runs a bounded 24-hour range fetch for the selected jobs through the deterministic analysis engine and shows the real findings it would surface (anomalies, threshold breaches) — no LLM involved, so a preview costs nothing and needs no API key. It reports how many queries ran, failed, or were skipped so an "all quiet" result is never an overclaim.
  3. Config (POST /app/sources/config) — pure computation, no network: generates a complete ready-to-run config.yaml, or just the sources: snippet, wired with the catalog's golden-signal queries for the jobs you selected.

In self-hosted mode, adding the source is config-file driven: you copy the generated config and restart. The wizard makes the decision — what to scrape and what "healthy" looks like — cheap and correct; it doesn't mutate a running config.

In cloud sign-in mode the wizard instead offers Add to organization (POST /app/sources/add), which persists the source directly into your organization's stored configuration — no file to edit, no restart. It is owner-only (a non-owner request is refused) and appends: the new source is added to your existing sources: list, leaving every other setting (other sources, notification channels, LLM provider, schedules) untouched. The write is optimistically versioned, so if the config changed in another session you are asked to reload rather than silently overwriting it. Once saved, the source starts being monitored within a few minutes and appears in your Sources list right away: in cloud mode the sources list, the per-source detail page, the settings source index, the incident attribution link, and the reports source filter are all read per request from your organization's stored configuration, so each tenant sees only its own sources. (The live metric charts on a source's detail page are not shown for a cloud organization's sources yet — they query the process collectors, so they are suppressed until per-tenant series resolution lands, #34; the rest of the detail page — status, query catalog, timeline — renders normally.)

Cloud sources (CloudWatch, DigitalOcean, Hetzner) are also selectable. They take a different shape: you pick a region and IAM role (CloudWatch) or resource types (DigitalOcean/Hetzner) instead of a URL. In self-hosted copy-paste mode the token is emitted as a ${…_TOKEN} environment placeholder — you never type a credential into the wizard. When saving to an organization, DigitalOcean and Hetzner collect the token in the wizard and store it in the organization's configuration (a shared multi-tenant worker has no per-organization environment, so the secret must live with the config). CloudWatch in a cloud organization takes a region, a cross-account IAM role_arn, and optional namespaces; the wizard shows your organization id as the STS External ID to pin in the role's trust policy (and a ready-made trust-policy snippet), so the worker can assume the role via sts:AssumeRole scoped to your organization alone (see configuration). No AWS keys are stored — the role is assumed, not held. There is no live verify/preview step for a cloud source: a live probe would need a credential on the verify endpoint and an outbound authenticated call to the provider (a denial-of-wallet lever), so a cloud source is generated-and-applied and verifies on its first collection. Verify and Preview are Prometheus-only.

A Prometheus or Loki source can also carry an optional bearer token for an authenticated backend (Grafana Cloud, Dash0, Mimir, VictoriaMetrics) (#251). When saving to an organization you type the token and it is stored with the config (a shared worker has no per-organization environment); in self-hosted copy-paste mode the token is a commented ${PROM_TOKEN} placeholder you uncomment and set in the environment. Entering a token requires an https URL and skips the live probe — an unauthenticated probe would 401 the authenticated endpoint, and handing the token to the verify endpoint is the same denial-of-wallet lever the cloud sources avoid, so a token-bearing Prometheus verifies on its first collection too. On the source-detail page an owner can rotate the token (blank keeps the current one).

Healthcheck and Loki sources are also selectable. A healthcheck source is a repeatable list of HTTP endpoints to probe — each takes a URL and a down-alert severity (both required) plus an optional method (GET/HEAD), expected status and timeout; add and remove rows as needed. The severity starts unset rather than pre-picked, for the reason the severity-alias editor above gives — it is the tier a down endpoint pages at — and Continue stays disabled until every row you filled has one. A Loki source takes the Loki API base URL plus a repeatable list of LogQL metric queries (each a name and a LogQL expression that must return a numeric vector/matrix, e.g. sum(rate({app="api"} |= "error" [5m]))), with optional signal/warn/crit/unit engine hints per query. It also collects an optional tenant ID (sent as the X-Scope-OrgID header for a multi-tenant Loki) and an optional repeatable list of log queries — raw LogQL log selectors (name + selector + an optional per-query line limit, e.g. {app="api"} |= "error") whose recent lines are attached to the LLM summary as context on a critical cycle; scope these away from streams carrying secrets/PII. Like the cloud sources, neither has a live verify/preview step — a healthcheck has nothing to probe before it is saved (and probing arbitrary URLs from the unauthenticated wizard would be an SSRF/denial-of-wallet lever), and a Loki source is verified on its first collection. The wizard validates the endpoints/queries against the same contract the config loader enforces — plus, for endpoints, the explicit-severity rule above, which is the one place the form is stricter than the file — so a generated config always loads.

Editing a source (cloud)

A source's detail page carries an owner-only Edit source form (POST /app/sources/edit) that updates the source in place. Like the save and remove actions it is owner-only and optimistically versioned, and it leaves every other setting — and every other source — untouched. The source name and type are immutable (the name is the identity that ties together its health, incidents, reports, and schedules; a rename is a deliberate remove + add). What you can change:

  • Prometheus / Loki: the URL. The configured queries — and, for Loki, the tenant_id and log_queries — are preserved as-is; to change which metrics (or LogQL / log queries) are collected, remove and re-add the source.
  • DigitalOcean / Hetzner: the monitored resource selection, and the API token. Leave the token field blank to keep the current one; enter a new value to rotate it. The stored token is never shown back on the page.
  • Healthcheck: the list of probed endpoints (url, method, expected status, timeout, down-alert severity), via a repeatable editor mirroring the add-source wizard's. Saving replaces the endpoints with the rows you submit (the same validation the config loader enforces, plus the wizard's explicit-severity rule), so removing every row is rejected — a healthcheck source must keep at least one endpoint. A source stored before that rule opens its editor showing critical, the tier already in force, so a URL-only edit is never blocked on picking one.
  • CloudWatch: the region, IAM role_arn, and namespaces. The form shows your organization id as the STS external ID (with a ready-made trust-policy snippet) in case you re-point role_arn at a different role or account. role_arn is shown and edited directly — it is an identifier, not a secret, so unlike an API token it is not a blank-means-keep field. A source configured with an explicit metrics: list (config-only, not offered in the UI) keeps its metrics unchanged; the namespace checkboxes are hidden for it, since the collector honors metrics: and ignores namespaces.

Editing a source does not change which reports or checks run, so it needs no schedule changes; the lease worker picks up the new source configuration on its next cycle. Self-hosted is unchanged: you edit a source by editing the config file.

Removing a source (cloud)

A source's detail page carries an owner-only Remove source action (POST /app/sources/remove) that drops the source from the organization's stored configuration. Like the save, it is owner-only, optimistically versioned (a concurrent edit is caught, not overwritten), and leaves every other setting untouched; the lease worker stops monitoring the source on its next cycle. Two removals are refused: an organization must always monitor at least one source, so removing the last one is blocked (the button is disabled and explains why), and a source still referenced by a check or a deploy verification can't be removed until those are repointed. Past reports and incidents for a removed source are kept; if the source had already collected, it stays visible in the sources list as a history-only card (health records outlive a config removal). That card reads removed — a neutral, muted state showing its last known outcome as frozen history. It is not counted as a live problem: a source removed while it was failing does not keep the sidebar failing badge red, and a removed card never shows a live "next collection" line (the collector is gone). Self-hosted is unchanged: you remove a source by editing the config file, and a dropped source's card reads removed the same way. The underlying health record is kept, so the card shows the last known outcome. Two independent bounds stop a removed card from lingering forever (#218): the web UI hides a removed card once its last record is more than 30 days old, and the opt-in source_health.retention_days prunes the frozen row from storage (a live source keeps its row fresh every cycle, so only removed sources are reaped). With retention off (the default) the card still ages out of the view at 30 days. The Unmapped severity labels your sources send card (#358) is bounded for the same reason, though by a different predicate: its readings are pruned by the same knob, and its view-side bound is membership rather than age — a source absent from your current config is not listed, and does not count toward its all-clear, however recent its last reading and whether or not retention is set.

SSRF posture

The verify and preview probes take a URL from the caller, so they are deliberately constrained: http/https only, redirects refused, fixed API paths appended (the caller can't choose the path), response size bounded, and only aggregate numbers returned — never response content. The endpoints also require Content-Type: application/json, which forces a CORS preflight and blocks cross-origin simple-POST abuse.

In a hosted (multi-tenant) deployment the probes are gated two ways: they require a live login session (so the surface is authenticated-tenant-only, never anonymous), and the dial routes through the same egress (SSRF) gate as a tenant Prometheus URL — a destination that resolves to a private, loopback, link-local or cloud-metadata (169.254.169.254) address is refused at connect time (after DNS, so it is DNS-rebinding-safe). A signed-in tenant therefore cannot turn these probes into an internal-network / metadata-exfil proxy. The residual is that a tenant can still probe an arbitrary public URL and spend preview compute — bounded, not eliminated (per-tenant source-name resolution is deferred to #34); denial-of-wallet caps live on the endpoints themselves. The egress gate engages on the same signal the runtime uses — a per-organization source config — so the wizard and the collector agree on which URLs are trusted.

In self-hosted mode the probes are not egress-gated — the operator's own YAML is trusted, so probing an internal Prometheus on a private IP is the intended behavior, and enabling login (an access floor) does not change that. Without login the probes are also unauthenticated like the rest of /app. None of the above makes /app safe to expose publicly in self-hosted mode (see the warning above) — it bounds the blast radius, it doesn't authenticate the caller.

One consequence of the paged archives (#295) worth stating plainly: paging makes the whole report and incident history reachable through the UI, where the previous hard 500-row cap left older rows unreachable there. (The per-source timeline is unchanged — it still shows its most recent 100 entries with a truncation note.) Each page is one bounded query. One view can still issue more than one: a deep link (#r-…) to a report that is not on the first page makes the page walk forward for it — at most 20 further page requests, stopping at the first error or as soon as you interact. So on an unauthenticated self-hosted /app the full archive (LLM summaries, delivery receipts) is walkable page by page, and a single crafted link costs a bounded handful of queries. Another reason to keep /app behind network controls.

Feedback

Each report row carries a thumbs-up / thumbs-down control with an optional note. Votes are stored against the report and survive a reload, so you can tell the team (and, later, tune the prompts) which digests were actually useful. In self-hosted mode this is an unauthenticated write like the rest of /app (another reason to keep it off the open internet); in the hosted mode it is org-scoped behind the login session, like the other report/incident pages.

Chrome

  • Theme — light/dark toggle in the top bar; your choice is remembered and applied before first paint (no flash), falling back to your OS preference.
  • Sidebar — a collapsible vertical nav (Dashboard, Sources, Reports, Incidents, Notifications); the Sources entry carries a red badge with the failing-source count.
  • LLM budget — the top bar shows today's LLM call count against the configured daily cap.
  • Strict CSP — every page is served with a strict Content-Security-Policy: script-src carries a per-request nonce with no unsafe-inline and no unsafe-eval (style-src keeps unsafe-inline for inline style attributes), and all assets are vendored and embedded, so the UI makes no third-party requests.