Kubernetes

InfraSigns ships a Helm 3 chart for production Kubernetes deployments.

Prerequisites

Requirement	Notes
Helm 3.x	brew install helm or helm.sh
PostgreSQL 13+	Accessible from inside the cluster (13+ is required — the schema uses gen_random_uuid() and declarative hash partitioning)
Prometheus	Accessible from inside the cluster
Kubernetes Secret	Containing sensitive env vars (see below)

Create the secret

All sensitive values are injected at runtime from a Kubernetes Secret referenced by existingSecret. Create it before installing the chart:

kubectl create secret generic infrasigns-secrets \
  --from-literal=DATABASE_DSN='postgres://user:password@postgres:5432/infrasigns?sslmode=require' \
  --from-literal=LLM_API_KEY='sk-...' \
  --from-literal=TELEGRAM_TOKEN='123456:ABC-...' \
  --from-literal=TELEGRAM_CHAT_ID='-100123456789' \
  --from-literal=SLACK_WEBHOOK_URL='' \
  --from-literal=SLACK_TOKEN='' \
  --from-literal=SMTP_USER='' \
  --from-literal=SMTP_PASSWORD='' \
  --from-literal=PAGERDUTY_ROUTING_KEY='' \
  --from-literal=HEARTBEAT_URL='' \
  --from-literal=API_TOKEN="$(openssl rand -hex 16)" \
  --from-literal=WEBHOOK_TOKEN="$(openssl rand -hex 16)" \
  --from-literal=BOT_TELEGRAM_TOKEN='' \
  --from-literal=BOT_SLACK_APP_TOKEN='' \
  --from-literal=BOT_SLACK_TOKEN='' \
  --from-literal=DO_TOKEN='' \
  --from-literal=HETZNER_TOKEN='' \
  --from-literal=PROM_TOKEN=''

Keys SLACK_WEBHOOK_URL, SLACK_TOKEN, PAGERDUTY_ROUTING_KEY, and HEARTBEAT_URL can be empty strings — empty values silently disable those channels. PAGERDUTY_ROUTING_KEY supplies the PagerDuty Events API v2 integration key; empty disables the channel, and because unset keys expand to empty, a secret that omits it simply leaves PagerDuty off. PagerDuty is incident-only — it pages on incident firing/resolution (an inbound alert group, or a health check that fails and recovers) and never receives digests. Slack additionally requires choosing a transport in values via config.notify.slack.mode: set it to webhook and provide SLACK_WEBHOOK_URL, or to api and provide SLACK_TOKEN (a chat:write bot token) plus config.notify.slack.channel — the api transport posts incident updates as thread replies. With mode unset the whole Slack block is omitted and neither key is read. SMTP_USER/SMTP_PASSWORD supply email credentials for the SMTP channel (enabled via config.notify.email.smtp_host); set both to authenticate, or leave both empty for an unauthenticated relay. Setting only one is rejected at config validation — and because unset keys expand to empty, an email channel that needs auth but whose secret omits these keys silently sends unauthenticated. LLM_API_KEY can also be empty when config.llm.provider is none (the deterministic analysis engine needs no key). An empty or absent API_TOKEN disables the manual digest trigger endpoint (POST /api/digest/trigger) and the MCP server (/mcp); in cloud sign-in mode both are suppressed regardless of API_TOKEN (they would act on the operator's data, not a tenant's). BOT_TELEGRAM_TOKEN can be empty: the Q&A bot's Telegram transport (enabled via config.bot.telegram.allowed_chat_ids) falls back to TELEGRAM_TOKEN; set it only to run the bot as a separate Telegram identity. An enabled Telegram bot with BOTH keys empty fails config validation — unset keys expand to empty, so this catches a secret that forgot the token instead of silently polling with a dead credential. The Slack transport (enabled via config.bot.slack.allowed_channel_ids) needs BOT_SLACK_APP_TOKEN (the Socket Mode xapp-… app-level token — required, no fallback, so a secret that omits it fails validation) and BOT_SLACK_TOKEN (the xoxb-… reply token, which can be empty to fall back to SLACK_TOKEN).

DO_TOKEN is the DigitalOcean personal access token (read scope) referenced as ${DO_TOKEN} by a config.sources entry of type: digitalocean — see DigitalOcean. Unlike the notify keys, it can NOT be usefully empty: an omitted or empty DO_TOKEN expands to an empty token and that source fails to load (a required-credential fail-loud, not a silent disable). Omit the key entirely if you configure no DigitalOcean source.

HETZNER_TOKEN is the Hetzner Cloud API token (read scope) referenced as ${HETZNER_TOKEN} by a config.sources entry of type: hetzner — see Hetzner Cloud. Like DO_TOKEN, it can NOT be usefully empty: an omitted or empty HETZNER_TOKEN expands to an empty token and that source fails to load (a required-credential fail-loud, not a silent disable). Omit the key entirely if you configure no Hetzner source.

PROM_TOKEN is the bearer token for an authenticated Prometheus-compatible backend (Grafana Cloud, Dash0, Mimir, VictoriaMetrics), referenced as ${PROM_TOKEN} by a config.sources entry of type: prometheus (or loki). Unlike the DigitalOcean/Hetzner tokens it is OPTIONAL — an omitted or empty PROM_TOKEN expands to an empty token, and the source simply queries unauthenticated (the backend returns 401 if it requires auth). The env-var name is a convention: any name works as long as the source's token: '${…}' reference matches. Omit the key entirely if all your Prometheus/Loki sources are unauthenticated.

WEBHOOK_TOKEN enables the inbound alert webhook receiver (POST /webhook/alerts) and post-deploy verification (POST /webhook/deploys). Empty or absent, the receiver is disabled — the endpoint is never mounted unauthenticated — and a config with config.deploys.enabled: true fails validation. Non-secret webhook settings (dedupe_window, model) live under config.webhook in values; the token itself is accepted only via the secret (the chart schema rejects config.webhook.token).

DATABASE_DSN must reference a role that owns the schema (both the chart's migration job and serve's startup migration run on it). The tenant tables carry forced row-level security as defense-in-depth — see the database notes for the backup implication (pg_dump must run as a superuser or BYPASSRLS role) and the least-privilege groundwork.

External Secrets Operator

If you use AWS Secrets Manager, HashiCorp Vault, or similar, create an ExternalSecret that produces the same key set and reference it with existingSecret.

Install

helm install infrasigns oci://ghcr.io/infrasigns/charts/infrasigns \
  --namespace infrasigns --create-namespace \
  --set existingSecret=infrasigns-secrets \
  --set config.sources[0].name=production \
  --set config.sources[0].url=http://prometheus.monitoring.svc:9090 \
  --set config.llm.provider=openai \
  --set config.llm.model=gpt-4o-mini

Or with a values file:

# my-values.yaml
existingSecret: infrasigns-secrets

config:
  sources:
    - name: production
      url: http://prometheus.monitoring.svc:9090
    # No self-hosted Prometheus? Pull AWS CloudWatch (type: cloudwatch, region,
    # IRSA/IAM role), DigitalOcean (type: digitalocean, token: ${DO_TOKEN}), or
    # Hetzner Cloud (type: hetzner, token: ${HETZNER_TOKEN}); probe any HTTP
    # URL (type: healthcheck, endpoints — no secret needed); or run LogQL metric
    # queries against Loki (type: loki, url) instead — see
    # docs/configuration.md#sources.
  llm:
    provider: openai
    model: gpt-4o-mini
  reports:
    digest:
      schedule: "0 8 * * *"
      # stale_episode_after: "24h"   # optional; see the values table below

helm install infrasigns oci://ghcr.io/infrasigns/charts/infrasigns \
  --namespace infrasigns --create-namespace \
  -f my-values.yaml

What the chart creates

Resource	Notes
ConfigMap	Non-sensitive config rendered from config.* values; created as a pre-install/pre-upgrade hook (weight -1) so it exists before the migration Job runs
Job (migrate)	Runs infrasigns migrate up before each install/upgrade (hook weight 0); can be disabled with migration.enabled: false
Deployment	Single replica — see Replica count
Service	ClusterIP on service.port (default 8080); routes to the container via the http named port
ServiceAccount	Created by default; set serviceAccount.create: false to bring your own
Ingress	Disabled by default
ServiceMonitor	Disabled by default; requires prometheus-operator

Upgrade

helm upgrade infrasigns oci://ghcr.io/infrasigns/charts/infrasigns \
  --namespace infrasigns \
  -f my-values.yaml

The migration Job runs automatically before the new Deployment starts. If the Job fails, the upgrade stops and the previous Deployment version keeps running.

The Deployment uses the Recreate strategy: the old pod stops before the new one starts, so expect a brief monitoring gap during upgrades. This is deliberate — the MVP runs a single scheduler worker, and Recreate avoids briefly running two.

Uninstall

helm uninstall infrasigns --namespace infrasigns

Hook resources (ConfigMap, migration Job) are not deleted by helm uninstall — remove them manually if needed:

# If you used a different namespace (-n flag) or release name (instance= value), update both below
kubectl delete configmap,job -n infrasigns -l app.kubernetes.io/instance=infrasigns

Replica count

replicaCount must be 0 or 1. InfraSigns schedules digest, trends, and health-check cycles from a single worker process. The chart schema enforces this — helm install fails immediately if replicaCount > 1. (The scheduler claims each cycle through a database lease that single-flights it, so multi-replica scale-out is a planned flag flip rather than a rewrite; today the chart caps at one worker.)

Use replicaCount: 0 to pause the service without uninstalling.

Enabling ServiceMonitor

If you run prometheus-operator or kube-prometheus-stack:

serviceMonitor:
  enabled: true
  interval: 60s
  labels:
    release: kube-prometheus-stack  # match your Prometheus CR's serviceMonitorSelector

Metrics reference

All application metrics are exported on /metrics under the infrasigns_ prefix (plus the standard Go runtime and process collectors). The full catalog — every metric name, type, label, and label value — lives on the Observability page, along with the /healthz and /readyz probe contracts.

The ServiceMonitor above is the scrape mechanism for /metrics on a prometheus-operator cluster; on a plain Prometheus, add a static scrape config pointing at the Service.

Enabling Ingress

ingress:
  enabled: true
  className: nginx
  hosts:
    - host: infrasigns.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: infrasigns-tls
      hosts:
        - infrasigns.example.com

Key values reference

Value	Default	Description
existingSecret	""	Required. Name of the Kubernetes Secret with sensitive env vars
config.sources	[]	Required. List of {name, url} Prometheus sources
service.port	8080	Kubernetes Service port; routes to the container via the http named port
config.llm.provider	openai	openai, anthropic, or none (deterministic engine, no API key)
config.llm.model	gpt-4o-mini	Model name passed to the LLM API; ignored with none
config.reports.digest.schedule	"0 8 * * *"	Cron expression (UTC) for the daily digest
config.reports.digest.stale_episode_after	unset (24h)	Go duration. An open incident episode nothing has re-asserted for this long still appears on the digest but stops standing its verdict up. Applies to senders that repeat (Alertmanager, Grafana), never to CloudWatch alarms, which only notify on a state change; must be longer than config.webhook.dedupe_window. No "off" value — 0s/negatives are rejected; set something very large ("87600h") to keep the pre-#419 behaviour
config.reports.trends.enabled	false	Enable the periodic trends report
config.reports.trends.schedule	"0 9 * * 1"	Cron expression for the trends report
config.reports.trends.window	"168h"	Analysis look-back (Go duration); defaults to 7d
config.reports.trends.timezone	"UTC"	IANA timezone name for the trends schedule
config.reports.retention_days	unset	Prune archived reports after N days; 0/unset keeps everything forever
config.incidents.retention_days	unset	Prune RESOLVED incident episodes after N days; open episodes are never pruned; 0/unset keeps forever
config.timeline.retention_days	unset	Prune source timeline events (fetch failures/recoveries, check-verdict changes) after N days; 0/unset keeps forever
config.source_health.retention_days	unset	Prune a removed source's frozen per-source rows after N days — its health row and its severity reading (#358) alike (live sources keep updating both, so only removed ones are reaped); 0/unset keeps forever
config.webhook.dedupe_window	unset	Suppress duplicate inbound alerts within this window (Go duration; app default 5m)
config.webhook.model	unset	llm.model override for inbound-alert incident summaries
config.heartbeat.enabled	false	Set to true and provide HEARTBEAT_URL in the secret
migration.enabled	true	Run migrate up as a pre-install/pre-upgrade hook
replicaCount	1	Must be 0 or 1
serviceMonitor.enabled	false	Create a ServiceMonitor for prometheus-operator
image.tag	""	Defaults to chart appVersion; override to pin a specific release