Observability: OTel decision metrics

raxit can export a low-cardinality metrics projection of every decision to an OpenTelemetry collector: a per-decision counter and a decision-latency histogram. This is a dashboard/alerting aid, not evidence — the signed write-ahead log (WAL) remains the only evidence channel.

The enable toggle

The exporter is off by default and turned on with one authoritative security.yaml key: otlp.metrics. Adding it to the otlp: block in your policy file is the only way to enable decision-metrics export:

otlp:
    metrics: true            # THE enable — reviewable, auditable, shows up in `raxit plan`
    endpoint: localhost:4317
    protocol: grpc
    insecure: true

otlp.metrics is a distinct sub-key from otlp.enabled (which turns on decision traces), so metrics and tracing toggle independently. The enable lives in the policy file — not an environment variable — so flipping it is a reviewable change that appears in raxit plan diffs like any other policy edit.

The metrics exporter's endpoint/protocol/insecure/service_name fall back to whatever otlp: block you already configured for decision-trace export — one endpoint, one place to point it. Because otlp.metrics is independent of otlp.enabled, you can run metrics-only export (otlp.metrics: true with otlp.enabled off) or traces-only, or both. Setting either one requires otlp.endpoint and a valid otlp.protocol, validated at load time (fail closed).

The toggle is loud: raxit serve and raxit ext-authz both log a single-line, human-readable summary at startup, every time, whether enabled or not:

raxit serve: otel decision-metrics export: DISABLED (set `otlp.metrics: true` in security.yaml to enable)

raxit serve: otel decision-metrics export: ENABLED -> grpc localhost:4317 (insecure=true)

If you set the env var but forgot the policy key, the startup line names the mismatch instead of silently doing nothing:

raxit serve: otel decision-metrics export: DISABLED — ignoring RAXIT_OTEL_METRICS because security.yaml does not enable it; set `otlp.metrics: true` to enable (env vars only override where metrics point, not whether they export)

What is exported — and what never is

Two instruments, both intentionally low-cardinality:

reason_code is the same stable, closed vocabulary documented in Reason codes — an opaque string like deny.taint_trifecta or permit.warrant, never a free-text message.

Never exported as an attribute or value, on either instrument: raw tool arguments, PII or any classified/tainted value, the resource (tool/endpoint name) the decision was about, the action id, the call_id, or the agent/principal identity. The metrics projection is deliberately more redacted than the trace/span projection (which may carry a hashed args_hash) — it carries nothing beyond the 5-value decision-type enum, the bounded reason-code string, and a timing number.

What this is not

• Not the evidence channel. The hash-chained, optionally Ed25519-signed Decision Provenance Record written to the WAL is the only durable, tamper-evident record of what happened. OTel metrics are a downstream aggregate projection of that same decision stream — useful for a dashboard or an alert, not for compliance evidence or after-the-fact dispute resolution.
• Cannot block or change a decision. The metrics hook is invoked from the governor after the verdict is already final, wrapped in its own recover(), and is non-blocking by construction (in-memory counter/histogram updates only — no network I/O on the decision path; export happens off-thread on a periodic timer). A panicking, slow, or entirely unreachable collector adds zero latency to, and can never alter, a permit/deny/defer/modify verdict. This is the same decision-neutral contract the trace exporter already has, just applied to a Track-B metrics hook instead of a Track-A span.

Local quickstart

1. Start a local collector with a minimal config that just logs what it receives (otelcol-config.yaml):

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

exporters:
  debug:
    verbosity: detailed

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [debug]

docker run --rm -p 4317:4317 \
  -v "$PWD/otelcol-config.yaml:/etc/otelcol/config.yaml" \
  otel/opentelemetry-collector:latest --config /etc/otelcol/config.yaml

(A native otelcol/otelcol-contrib binary with the same config works identically — the docker run form above needs no local install.)

2. Enable metrics in your .raxit/security.yaml's otlp: block (traces can stay off; otlp.metrics is independent of otlp.enabled):

otlp:
    metrics: true       # THE enable — authoritative, reviewable in `raxit plan`
    enabled: false      # trace export stays off; metrics is independent
    endpoint: localhost:4317
    protocol: grpc
    insecure: true

Run raxit plan first if you want to see the toggle as a reviewable change before it takes effect.

3. Start raxit serve:

RAXIT_APPROVALS_TOKEN=devtoken raxit serve --source .raxit/security.yaml

raxit serve: otel decision-metrics export: ENABLED -> grpc localhost:4317 (insecure=true)

4. Drive a couple of decisions, then wait for the exporter's periodic flush (10s default interval, or shut the daemon down to force one):

curl -s -X POST http://127.0.0.1:8080/v1/decide -H "Content-Type: application/json" \
  -d '{"schema":"car/1","agent_id":"default-agent","action":"tool.call","resource":"calc","resource_type":"tool"}'

The collector's debug exporter logs data points like:

Metric #0
Descriptor:
     -> Name: raxit.decisions
NumberDataPoints #0
Data point attributes:
     -> decision_type: Str(allow)
     -> reason_code: Str(permit)
Value: 1
Metric #1
Descriptor:
     -> Name: raxit.decision.duration
     -> Unit: ms
HistogramDataPoints #0
Data point attributes:
     -> decision_type: Str(allow)
Count: 1

If you'd rather see this exercised as an automated test instead of a manual walkthrough, the same shape (a real in-process OTLP/gRPC collector, real exporter, real assertions on the exported data points) is covered by an automated test in CI.

Related

• Audit & the DPR chain — the actual evidence channel.
• Reason codes — the closed reason_code vocabulary.
• Budgets, tenants, egress & sandbox — the otlp: block metrics reuses.