Skip to content

Natural-language health checks

Write a health assertion in plain language, put it on a cron schedule, and InfraSigns asks the LLM to verify it against the source's metrics:

checks:
  - name: disk-headroom
    schedule: "*/30 * * * *"
    source: prod            # which configured source's queries to consult
    query: "Is disk usage below 85% on all instances and not growing abnormally?"
    mode: monitor           # monitor (default) = record only; alert = notify
    model: ""               # optional per-check llm.model override
    severity: warning       # info|warning|critical, default warning — tier a FAILING check pages at
    for_runs: 1             # consecutive non-pass runs before the episode opens, default 1
    repeat_interval: "30m"  # re-assert an open failing episode this often; omit = never

Only the first four fields are required — the shortest working check is name/schedule/source/query, and everything below it is optional. The last three are the paging controls, and repeat_interval is the only one shown at something other than its default: see Tuning the paging.

Checks require a real LLM provider: a natural-language assertion needs reasoning, and a mechanical template verdict would mask real failures. In a config file that is a load error — llm.provider: none with checks: configured is rejected at config validate, before the migration pre-hook runs.

In the hosted service the same requirement is answered one layer later, and it has to be: the provider that evaluates an organization's checks is the operator's, not the one in the organization's own stored configuration, so that stored value cannot decide anything. A stored configuration may therefore carry checks: whatever it says about llm.provider; whether they run depends on the service. When the operator's provider is none, the worker schedules no checks for anyone, says so once per organization in its log, and the organization's settings page reads "N configured · not running: the service has no LLM provider" instead of a count that would imply work nobody is doing. The two public pages follow the same rule: the plan comparison prints "Not evaluated by this service" in every tier's Health checks cell, and the landing page drops its health-checks card, because a tier allowance for something nobody runs is not a limit. Nothing is degraded to a template verdict — skipping is the only honest option, because the verdict IS the model's answer. mock counts as a provider here, exactly as it does in a config file: it renders mechanical fail-closed verdicts for demos.

Adding a check in the hosted service

A hosted organization does not edit a YAML file, so its checks live in an editor on the Settings page, under Health checks. It is owner-only, like the severity-alias editor above it: every member can read the list, only an owner can change it. Each row carries the same fields as the YAML above with ONE exception — there is no per-check model, because the hosted worker clears it: the provider and the model are the operator's, so a control for them would be a knob the service discards.

Leave a field empty to keep its default. Checks run in the order listed, and that order is load-bearing under a plan cap (below) — reordering the rows changes which ones run.

A hosted check may run no more often than every 30 minutes. A save carrying a faster cron is refused, naming the check and the interval it actually fires at — and it is the shortest interval between any two of its fires that counts, so */29 * * * * is refused for the two-minute gap it leaves across the hour rather than accepted for its average. Every fire is one billable LLM call from the same daily allowance your digest draws on (Cost); 30 minutes is 48 runs a day. A self-hosted config file has no such floor — there the key is the operator's own.

The floor binds on WRITE and only on write: a configuration already holding a faster check keeps running it, because refusing to LOAD it would stop that organization's digest and its other checks too. What it costs is that every save of that organization's configuration is refused until the check is edited — adding a source, changing a notification channel, anything — and the refusal names the check, not the page the owner was on. Nothing sweeps or reports such rows, so an operator upgrading a service that already has hosted checks should look for them first:

SELECT o.name AS org, c.item ->> 'name' AS check_name, c.item ->> 'schedule' AS schedule
FROM org_configs oc
JOIN organizations o ON o.id = oc.org_id
CROSS JOIN LATERAL jsonb_array_elements(COALESCE(oc.config -> 'checks', '[]'::jsonb)) AS c(item)
ORDER BY o.name;

Read the schedule column: any cron whose two closest fires are less than 30 minutes apart will refuse that organization's next save.

Two things the editor refuses, and both are refusals you can act on from the page you are already on:

  • A source it does not have. A check must name one of the organization's own sources, so the source picker offers exactly those. The mirror also holds: removing a source a check names is REFUSED rather than cascading, with a message naming the reference. Delete or repoint the check first — deleting monitoring nobody asked to delete would be the worse failure, and a refusal costs two clicks.
  • More checks than the plan covers. The Add button stops at the ceiling and says so. If your organization already stores more than the plan covers — after a downgrade — the editor still lists every one of them, because it is the only place they can be removed; Save is held until enough are gone.

If your plan includes no checks at all, the section says so instead of offering an editor whose every save would be refused.

The hosted service also caps how many checks an organization RUNS, by plan: Free 0, Solo 3, Team 10. Free gets none deliberately — a check is one LLM call per fire, so a single check on the */30 * * * * schedule above is 48 calls a day against Free's allowance of 25, and it would starve the daily digest it exists to complement. Store more than your plan covers and the worker schedules the first N in configuration order and no others; the rest stay in your stored configuration untouched, so an upgrade brings them back with nothing to re-enter, and reordering them changes which ones run. The organization's settings page names the count under Plan limit — "1 over-limit: this plan covers 3, so the first 3 in configuration order run — upgrade to run the rest." — beside the ordinary configured/alerting summary. This is the same rule and the same wording as the source cap, whose over-cap cards read "over-limit" too. During a plan-read outage nothing is truncated: the worker RUNS every stored check rather than guess, and the settings page shows no over-limit row. It mints no schedule row it did not already have, though — it cannot tell which tier the organization is on, and a row is durable where a cycle's guess is not — so an organization whose checks were not already scheduled waits for the read to recover.

A check whose SOURCE is beyond the source cap is a different situation with a different remedy, and the two are deliberately not merged: it is inside the check cap, so no over-limit count mentions it, but it has no collector to read, so it does not run either. Its source's card on the health page is the one marked "over-limit". Point the check at a covered source, reorder your sources, or upgrade.

Checks that are configured but not running — for any of the three reasons above — keep whatever check_state they last had: a failing episode stays open rather than being retired, since neither an operator turning the model off nor a plan downgrade is a retirement. Two consequences worth knowing. That open episode is also not re-asserted, so a repeat_interval stops repeating — the last page you received still stands. And in the hosted service resuming is not instant. An organization's schedule row is first written when it SAVES its configuration, and from then on the worker reconciles that organization's rows against what it can actually run every time it rebuilds the organization's runtime — minting the row for a kind that has become runnable and retiring the row for one that has not. A rebuild happens on a configuration save, on a plan transition, on a worker restart, and whenever an organization's cached runtime has aged out — the worker releases one that has not been resolved for about an hour — so a provider the operator configures or a plan upgrade takes effect without anyone saving anything. While the subscription table cannot be READ at all, the worker keeps running every kind it already had but mints no new schedule row: it cannot tell a Team organization from a Free one, and a row it wrote on that guess would outlive the outage.

What it costs is a delay: the reconcile runs when the organization is next resolved, and an organization with no checks scheduled is resolved when its DIGEST next fires, so the wait is at worst that organization's digest cadence — a day, on the default schedule, and a week for an organization whose reports run weekly. The pages say so while it lasts: the Settings Health checks card carries a note that the service has not started running the organization's checks yet; on the dashboard's card a verdict belonging to a check the plan and the model DO cover is marked not scheduled yet; and the Notifications page's Health checks feed row says so too, in the vocabulary of delivery: while an alerting check the service would run has nothing scheduled to run it, the row stops claiming that anything notifies. That is a different marker from no longer evaluated, and deliberately so — this one resolves itself, where the three reasons above each need somebody to act; a check the plan does not cover keeps no longer evaluated throughout. A downgrade is symmetric and was the noisier half before this: its row used to survive and be claimed every poll for a runner that no longer existed.

The count on that same feed row is the number that actually runs: your plan's SOURCE limit is applied as well as its check limit, so a check watching a source beyond that limit is not counted as notifying (#488). When every alerting check your plan covers watches such a source, the row says so and links to the sources page — not straight to checkout, because you have to see WHICH source was dropped first; that source's own card is the one marked over-limit, and it is where the upgrade link lives.

The Settings section's alerting count is the other number, and the two differ for exactly this organization: that one describes what the plan's check limit covers, which is what the Plan limit row beside it explains. So the section names the other cause itself, in a muted Source limit row counting the covered alerting checks whose source the plan does not monitor — the difference between the two pages' numbers, stated on the page that holds the larger one, with a link to the plans. It counts alerting checks only for exactly that reason, and the residual is worth knowing: a monitor check on an over-limit source stops too, and neither page counts it — it notifies nobody, so it is not part of a delivery subtraction. The Billing page's over-limit strip carries the same fact for the whole organization.

Each run fetches the source's configured queries over the last hour, summarizes them (latest/min/max per series), and asks for a verdict. Results land on the dashboard's Health checks card; in alert mode, transitions also notify the channels routed to the checks feed — by default, all of them (feed routing).

On the channels that render structured reports (Telegram, Slack, email, and the plain-text floor) a transition is the same layout a digest gets — PagerDuty gets it as a page instead, see below: the check's name and when it ran, a status-first verdict — Passed / Failed / Could not verify — the model's rationale (collapsed behind a tap on Telegram, shown in full on the other three), then the facts as rows: the state change (Change: pass → fail, or First run: fail when there is no previous result), the source that was read, and the assertion that was checked. If the previous result could not be read at all — the state write failed — the change row is omitted rather than claiming a first run. The time is when the check ran, which is a model round trip before the notification you are reading. A failed or unverifiable check draws a neutral accent on those rows rather than the healthy ✅, which would say the opposite of the verdict above them; a check that passed draws the ✅ without claiming your fleet is healthy — one check was verified, not a fleet.

A query that returns many series (one per instance, pod or node) is not sent in full: the prompt lists at most the first 30 series of each query name individually — fewer when the whole list is long, see below — and folds the rest into one aggregate line whose min and max span every folded series — so an extreme hiding in the remainder still reaches the model, even though the individual series producing it is not named. (The series listed individually carry their own numbers, so the group's overall extreme is the union of the two.) Whenever anything is folded, every query name the prompt lists states its total series count as well — a name dropped by the 40-name ceiling below states nothing, because it is not in the prompt at all. At most 40 distinct query names are listed, in configured order — so on a source that is already past 40, adding a query at the top of queries: pushes the last one out. See LLM providers for the exact shape.

When the prompt could not carry everything

Any of those three limits firing means the verdict was reached on less than the source returned, and the run says so on three surfaces rather than leaving you to infer it from the rationale:

  • a row on the notification — "The verdict was made on reduced metric data. 12 of 52 query names were not sent to the model. 340 of 1200 series were folded into per-name aggregates." — at the warning tier, so it is not drawn as one of the healthy facts beside it. Each clause appears only when that loss actually happened, and each carries its denominator: a drop count on its own does not say how much is left;
  • a log line, check: verdict metric data truncated, once per run with the check name, the source and the same numbers. It is a WARN when a whole query name was dropped — the model never saw that metric, and queries: is the fix — and a DEBUG when all that happened was a fold or a shortened series identity. Those two follow how many series your queries return rather than anything going wrong, so a wide check would otherwise warn on every run, forever, about a shape you cannot clear; the counter and the notification row still carry them;
  • the counter infrasigns_verdict_input_truncated_total, one increment per loss kind per run — the three losses have three different remedies, so they are not summed into one.

The row does not change the verdict. The model still answered; the caveat is about the evidence, so a wide check that passes still reads as passed. What it tells you is which knob to reach for: narrow the source's queries: list, split a compound assertion into two checks, or write the assertion in extremum form (see below) so a fold cannot cost it anything.

Two things it does not say. It names no query — the row states a fraction, not an inventory, so on a source past the 40-name ceiling you still have to compare the prompt's order against your queries: list to see which metric went missing. And it says nothing about a check whose data was complete: no loss, no row, no line, no counter movement.

Health-check verdicts are not archived, so the row travels with the notification only. A deploy verification archives it too — see Deploy verification.

Results are tri-state: pass / fail / error

error means the check could not run — the metrics fetch failed, no data came back, the LLM call failed, or its answer was unparseable. error notifies in alert mode: a monitoring system that cannot run its checks must say so, not go silent. It is deliberately distinct from fail (the LLM verified a problem).

The prompt applies a fail-closed rule: if the data is insufficient to verify the healthy condition, the verdict is fail with the rationale saying what was missing. Absent data entirely is error — the LLM is not even asked.

The series fold above is deliberately not "insufficient data" for an assertion that turns on an extreme — "is anything above X", "is everything up" — because the aggregate line's minimum and maximum span every folded series. A wide check does not start failing just because it is wide, and the prompt says so explicitly. What a fold does destroy is which series holds a value and how many series hold it, so an assertion about one specific labelled series, about a count ("are at least 18 replicas up?"), about a proportion, about an average or about a trend across the series is insufficient data when the series it needs were folded — the prompt routes that case back to the fail-closed rule, and the rationale says the group was folded. An assertion that mixes the two shapes is judged on the part that needs the folded series: the example at the top of this page, "Is disk usage below 85% on all instances and not growing abnormally?", is extremum-shaped in its first half and trend-shaped in its second, so on a folded query it fails closed on the growth half and the rationale says which half could not be checked. Writing check assertions in extremum form is therefore the reliable shape on very wide queries — and splitting a compound assertion into two checks is better still. A dropped query name is the clearest case of all — the metric genuinely is not in the prompt — so a check whose assertion depends on one fails closed and names it.

Alerting on transitions only

Notifications fire when the result changes — pass→fail, fail→pass (recovery), anything→error — never on repeats. A still-failing 30-minute check pages once, not 48 times a day, unless you opt back into a bounded reminder with repeat_interval. State is persisted in Postgres, so transitions survive restarts. On the very first run only a non-pass result notifies — and with for_runs above 1, not even that: nothing notifies until the streak is met.

What changes is the episode state, which by default is the same thing as the run's verdict and stops being the same thing the moment you set for_runs — see Tuning the paging. The dashboard's badge always shows the verdict of the last run either way; a non-passing row reads not notified beside it when no notification has carried that verdict yet.

A transition is an incident episode, keyed on the check itself: a failure opens it, a recovery closes it. That is what threads a check's failure and its recovery together on Telegram and on Slack's api transport — and, if you have configured PagerDuty, what triggers an incident on a failure and resolves it on recovery. A check in alert mode reaches the same pager your inbound alerts do — unless you have routed that channel away from the checks feed, which is how you keep a pager for alert groups alone (see Feed routing). That escape hatch is self-hosted: inbound alert groups are operator-level in the hosted service, so an organization's PagerDuty routing card offers the checks box alone. Clearing it usually parks the channel rather than narrowing it — the exception is an organization whose operator seeded feeds: [incidents], which the card preserves because it cannot offer it, so the channel stays routed at a feed no per-org message carries. monitor mode never pages, which is why every new check should start there.

Three things are worth knowing before you promote a check to alert:

  • An error does not page. "Could not verify" is not "the thing is broken", and one LLM or fetch outage would otherwise open an incident for every alert-mode check at the same moment. Errors still notify your chat channels.
  • A check pages at warning severity unless you raise it. severity: sets the tier per check; see severity for how it interacts with notify.pagerduty.severity.
  • A rename or a delete closes the open failure — it does not claim a recovery. An episode is keyed on the check's name, so nothing runs for a check that is no longer configured and no ordinary run can ever close it. InfraSigns therefore sweeps for orphans separately: on every checks cycle, and again at startup, it compares the stored episodes against the checks you have declared, and for each one you have removed it delivers a Health check RETIRED — <name> message and closes the incident.

The wording is the point. It says "No longer configured", never "Recovered", and it draws a neutral ℹ️ rather than the green ✅ — because nothing was verified as having recovered; the check simply stopped existing. The message carries the last verdict the check recorded so you can see what state it was left in. A rename is a delete plus a create: the old name is retired and the new one opens a fresh episode on its first run.

A check that was quietly passing (or that never notified you at all) is retired silently — its row is dropped and no message is sent, because there was no open incident to close.

Three things it does not do, each on purpose:

  • It never deletes an episode it could not close. If the retirement message reaches no channel — every notifier failed — the stored episode is kept and retried. Dropping it would leave the incident open with the state that could close it gone, which is worse than the problem. Watch infrasigns_checks_retired_total{status="deferred"} for that case, and status="undeleted" for its rarer twin: the stored row could not be deleted after the sweep had settled it, so the row survives and the next sweep repeats the same decision for it — announcing the retirement again if a message went out, or simply retrying the delete for a check that was being retired silently.

    A maintenance window does not delay a retirement, whatever its scope. A retirement is a resolution, and a window never withholds one — swallowing the message that closes a page is how a page gets stranded. (A retirement also carries no source, since the config no longer says which source the check ran against, but that is not what decides it here.)

    How soon "retried" comes round depends on what you still have configured: the next checks cycle if another check is still scheduled, and otherwise — with no checks left, which is the case this whole sweep exists for — the next time the service starts. - It writes no timeline event. The source timeline is keyed by source, and a retired check's source is no longer knowable. Its last real transition is already in the timeline and ages out on the normal retention. - In the hosted service, deleting your last check leaves its orphans until you configure another one. With zero checks there is no checks schedule to run, and the startup sweep covers only the deployment's own organization, not yours. Self-hosted installs are covered either way: restarting the service sweeps regardless of what is configured.

A send that fails is a different matter: a notification no channel took does not advance the stored episode state, so a lost send is re-sent on the check's next run, in both directions. Nothing is silently dropped because a channel was down for one run — the cost is that a send that succeeded but was not recorded (the process died in between) is delivered twice. PagerDuty folds the repeat into the same incident; on chat it is a duplicate line.

Modes

Start every new check in monitor: results are recorded and visible on the dashboard, but nothing pages you while you tune the wording. Promote to alert once the check has been quietly passing (or failing exactly when you expect) for a few days.

A monitor check never announces anything, so a failing one shows its fail badge with the not notified marker beside it for as long as it keeps failing. That is the marker telling the truth, not a fault: on a card that does not show the mode, it is the only thing on screen saying nobody is being told.

Promoting a check that is failing right now pages you on its next run — the episode was never announced, so opening it is a real transition and you get a normal failure notification, not a mid-incident update.

One upgrade note, for a check that was already in monitor mode when you moved to the release that added these fields. The upgrade seeds every check's announced state from its last result — right for alert checks, since it is what stops them all re-paging at once, and wrong for monitor ones, which never announced anything. A monitor check clears it on its first run after the upgrade; until that run a failing one shows no not notified marker. Nothing pages either way, because the mode does not send — but let a promoted check have that one monitor run before you switch it to alert. Promote it inside that window and the runtime sees a failure it believes was already announced: no transition, so no page, until the check recovers and you get a recovery for an incident you never received.

Tuning the paging

Three optional fields shape when and how loudly an alert-mode check reaches you. All three are notification policy and nothing else. The dashboard card still shows the verdict of the last run, infrasigns_checks_evaluated_total still counts every run, and the source timeline still journals every change of the raw verdict — so the flapping that made you reach for for_runs stays visible after for_runs has stopped it waking you.

severity

info, warning or critical; warning when unset. The spelling is case-exact — Critical is rejected at config validate rather than folded.

It sets the tier a failing check pages at, and only that. A recovery is always info — it is good news — and an error is always warning: "could not verify" is not "the thing is broken", and an error cannot page at all, so rendering an LLM outage in critical red would assert something the verdict does not say.

It interacts with notify.pagerduty.severity, which applies only to messages carrying no severity of their own:

  • a check at warning or critical — including one that just took the default — overrides the PagerDuty default and pages at its own tier;
  • a check at an explicit severity: info defers to it, which is how you route every check through one service-wide tier without repeating it per check.

It sets the tier the notification is, not the colour it looks. The verdict band at the top of a structured report is derived from the outcome — a failed check reads as a warning-tier verdict — so a severity: critical check pages at critical urgency under a verdict that is not painted critical red.

for_runs

How many consecutive non-pass runs must land before the episode opens. The default, 1, opens it on the first one — exactly how checks have always behaved. Raise it for a check that is right but twitchy: for_runs: 3 on a 5-minute check means a problem has to survive 15 minutes of it before anyone is woken.

It counts runs, not elapsed time, and the divergence from Alertmanager's for: is deliberate. Alertmanager evaluates on a uniform interval, so a duration there is a run count wearing different clothes; a check's schedule is a 5-field cron that can be as irregular as 0 9 * * 1-5, and a duration threshold could not be honoured without inventing a cadence the config does not have. Multiply by your own schedule to get the wall-clock delay.

An error is a non-pass run, so it counts toward the streak; a pass resets it to zero. With for_runs: 3:

runs what happens
fail, fail, fail the episode opens as fail on run 3, and pages
fail, error, fail also opens as fail on run 3 — an unverifiable run is not a recovery, which is the same reason a fail→error never closes an open page
error, error, error opens as error on run 3: your chat channels hear about the outage, PagerDuty does not, because an error never pages
fail, pass, fail nothing opens — the pass reset the streak, and run 3 starts a fresh one

While a streak is still short of the threshold the dashboard shows the run's own badge with a not notified marker beside it: the run failed, and nobody has been told.

for_runs gates opening an episode and nothing else. A recovery is never damped: the first pass closes an open failure immediately, whatever the threshold is. Delaying good news would leave a resolved incident sitting on someone's pager for no reason.

There is deliberately no upper bound. A for_runs larger than the check will ever reach is a silent "never alert", and mode: monitor already says that out loud — so if that is what you want, say it that way.

repeat_interval

A Go duration (30m, 2h); unset means never. It re-asserts an episode that is already open and failing, so a failure you have not fixed keeps refreshing instead of going quiet after its one notification: on PagerDuty it re-triggers the same incident, and on chat it lands as an update in the same thread.

Two limits, both worth knowing before you rely on it:

  • The re-assert rides the check's own run — there is no second timer — so it happens on the first run at or after the interval has elapsed. On a regular schedule that means the effective period rounds up to a multiple of the schedule, which is rarely the interval you wrote: repeat_interval: 30m on a check scheduled 0 3 * * * re-asserts once a day, and on */20 * * * * it re-asserts every 40 minutes, because at +20m the interval has not elapsed yet. Only when the interval is an exact multiple of the schedule do you get the number you asked for. On an irregular schedule — including */25, whose runs wrap at the top of the hour — there is no single period at all; the rule still holds run by run. The knob can only make the reminder rarer than the check's own schedule, never more frequent — to hear about a failure more often, run the check more often.
  • Only a fail is re-asserted. Both the open episode and the run doing the re-asserting have to be fail, so a check whose latest run came back error is not re-asserted while it stays that way — an error cannot page, and repeating it would be chat noise with no pager value behind it. A check stuck on error is a fetch or LLM problem, and the error verdict on the dashboard is where that belongs.

Writing good checks

  • Phrase the assertion either way — "Is X healthy?" or "Has X exceeded Y?" — the verdict is always about whether there IS a problem, not a literal yes/no.
  • The LLM sees only the source's configured queries (latest/min/max over the last hour). If your assertion needs a metric, add the query to the source first.
  • One assertion per check. Transitions are per check; a compound check that is half-failing flaps.

Cost

Every check run makes one billable LLM call, counted against llm.max_calls_per_day. A 30-minute schedule is 48 calls/day — and the budget is shared with digests, trends, and webhook summaries: a misconfigured * * * * * check burns 1440 calls/day, exhausts the default cap of 200 by ~03:20 UTC, and every other feature falls back to template output for the rest of the day. (That is the SELF-HOSTED arithmetic; the hosted service refuses that cron outright and reserves a share of the day for the reports — see below.) Size schedules accordingly and use model to point frequent checks at a cheaper model.

When the budget is exhausted, a check does not fall back to a template — a template cannot reason, and letting it vote would flip a genuinely failing check to a spurious pass/RECOVERED. Instead the run is an error ("llm daily budget exhausted"), which notifies once on the transition and shows on the dashboard until the budget resets at midnight UTC.

In the hosted service two rules bound that sharing, and they work as a pair. The 30-minute floor above bounds what a single check can draw. And a fifth of the organization's daily allowance is reserved for reports: checks (and any other call that produces a verdict rather than a summary) are refused once the day's usage reaches four fifths of the cap, while digests and trends keep going to the cap itself. Four fifths is the number to size checks against — 160 calls a day on Solo, 800 on Team — and it is not the tier allowance those pages are usually quoted by, which is 200 and 1000. infrasigns org create prints it for the organization it just made, the Billing page names it in a note under the daily LLM meter, and every refusal it causes increments infrasigns_llm_budget_reserved_denied_total (observability).

So the checks can no longer arrive first in the morning and leave the day's digest on a template — but a check refused this way still reports error and still notifies on the transition, which is deliberate: an operator told their checks paused can act on it, where a silently templated digest looks like a working product. Raise the organization's allowance, or run fewer checks, or run them less often. Self-hosted reserves nothing — one cap, first come first served, exactly as before.

Refused by the service's shared ceiling

Everything above is about an organization's own allowance. The hosted service can carry a second ceiling above it, shared by every organization the operator's key serves — llm.max_fleet_calls_per_day (#472) — and a check refused by that one behaves in the opposite way: no verdict is recorded and no notification is sent. The run is abandoned before it writes anything, so the episode is not advanced, check_state is untouched, nothing transitions and nothing pages; the dashboard goes on showing the previous run's badge. It applies to every call, with no reserve carved out of it — reports and verdicts stop at the same number.

The difference is whose event it is. Your own cap is an allowance you configured and can act on, so an error verdict is information. A shared ceiling is spent by whichever organizations called first, cannot be raised by anyone but the operator, and clears itself at midnight UTC — so an error there would page every check of every organization at once, which is precisely the failure #472 was filed to prevent.

Because nothing is written, the pages have to say it instead, and four do: the Settings page's Health checks section and card, the Notifications page's Health checks feed row, the Billing page's LLM line, and a paused marker on each row of the dashboard's health card. Each names the shared service limit as the cause and offers no remedy, because no tenant control reaches one. Operators have infrasigns_checks_skipped_total{reason="fleet_llm_ceiling"} (observability), which is where a skipped run is counted — no infrasigns_llm_requests_total sample is recorded for it, since the ceiling refuses the call before any request leaves the process. A skipped run is lost, not deferred: the next fire is the ordinary next cadence, because a check is a probe of now and replaying a paused day at midnight would be a burst.

Metrics

infrasigns_checks_evaluated_total{check,result} counts runs by outcome; infrasigns_check_duration_seconds times the full fetch→verdict→notify run.