The cost ledger had been writing a row for every provider call since the gateway landed. Sessions and memory made long conversations durable and the model could act and remember. Bedrock gave a first-party path on AWS credits. What was still missing was any way to see the accumulating spend, any way to change routing or enable a new provider without touching the database by hand, and any way for the assistant to learn something that happened after its training data.
Why a control plane
Three concrete problems:
- Costs were real but invisible. A short turn might be a few cents; a long research session or a compaction pass was not.
- Configuration lived in SQL. Enabling a provider, reordering a chain for “mini”, or flipping tools off for debugging required direct writes and a restart.
- The model had no eyes on the present. Every answer was bounded by the knowledge cutoff of whichever provider happened to answer.
The same rule that shaped the rest of the system applied here: deterministic code owns the surfaces and the safety. The web never talks to the credential holder. Every read and every mutation goes through the public brain API.
Aggregates that cannot lie
Five internal endpoints answer the dashboard questions. All of them take a time range (from and to, RFC3339, to exclusive) and all of them add purpose IS DISTINCT FROM 'test'.
- summary returns the header totals: cost_usd, input/output/cache tokens, request count, error count.
- series returns bucketed points (hour/day/week) grouped by provider, model or task category, the shape a stacked chart consumes.
- sessions returns the top N sessions by spend for a “most expensive conversations” table.
- latency returns p50/p95/p99 per provider, computed only over non-error rows (fast failures would otherwise pull the numbers down).
- cache returns cache_read vs fresh input per provider plus the derived hit ratio.
The implementation lives in one place (internal/gateway/ledger/aggregate.go). Bucket and group identifiers are chosen from small Go maps before any SQL is built; unknown values are rejected early. The composite index on (ts, provider, model, task_category) was added so the range scans do not fall back to a ts-only index. Unknown prices stay NULL in the ledger and are only COALESCEd to zero in the sums that need a number.
Test-connection traffic is the canonical example: it is recorded with purpose=‘test’ and task_category=‘admin’ so it appears in the audit trail and in provider health last-success timestamps, but never in any user-facing usage number.
The admin surface and hot reload
Providers and task_routes are now ordinary rows that can be created, patched and deleted through the API.
- PATCH on a provider is partial; only the fields present in the request body are changed. credential_ref is still validated as a name or path, never a secret blob.
- PATCH on a route category accepts a new chain; for every entry the server does an existence check against the providers table before accepting the write.
- A dedicated test endpoint runs a minimal completion (or models list) against the provider using its current config and always books the ledger row with the test purpose.
- Health combines the live snapshot (which providers are enabled and have a resolvable credential_ref) with the most recent success and error timestamps from the ledger.
- Every mutation appends a row to admin_audit with the action, the entity, the id, and before/after JSON.
After a successful write the gateway calls store.Load. The load runs inside a single REPEATABLE READ transaction so a route update cannot observe a provider row that the subsequent snapshot build never saw. The resulting immutable snapshot is swapped in with an atomic pointer. The previous snapshot stays in service if the load fails; the poll loop will try again.
The web never reaches the gateway directly. The brain registers an exhaustive list of patterns (/v1/admin/usage/{rest...}, the provider and route operations) and proxies them, after bearer auth, to the gateway’s internal surface. The list is the source of truth; anything not on it is unreachable. Settings live in the brain itself and are never proxied.
Feature switches that degrade safely
Four global booleans live in a brain-owned settings table. An absent row means enabled: old databases behave exactly as they did before the table existed.
Reads are served from a short mutex-protected cache. On any database error the code returns the enabled defaults. This is intentional: if the config store is unreachable it is better for tools, extraction and compaction to continue than for the assistant to suddenly lose capabilities because a settings row could not be read.
Enforcement is wired at the call sites:
- The turn router checks the tools flag (and whether the purpose is chat) before deciding whether to run the agent loop or fall straight through to a plain gateway completion.
- The two extraction hooks (the fire-and-forget turn-end one and the pre-compaction one) return early when extraction is off.
- The compactor wrapper returns immediately when compaction is disabled.
The remember tool and web_search are just entries in the registry; they are reached only when the code paths that register and invoke tools are live.
Web search, the first outbound tool
web_search lets the model fetch current information. It is backed by a SearXNG instance that is deliberately not reachable from outside the compose network:
- No published ports.
- Mounted a minimal read-only configuration (metrics off, json format enabled, a static secret key that only signs cookies nothing external can reach).
- The brain receives the address as
SEARXNG_URL=http://searxng:8080and passes it only to the tool constructor.
The tool is added to the registry only when the environment variable is present. It is marked exempt from the permission chain (pure read, same category as current_time). The schema is a single required string query. The implementation:
- 15-second timeout
- caps the response body at 2 MiB
- returns at most ten lines of the form
N. title\nurl\nsnippet - turns an empty result set or an upstream error into a plain message rather than a turn-killing exception
The tool description tells the model to follow promising results with web_fetch for full page content. The internal base URL and any implementation detail never leave the tool.
This is the first time the assistant can learn something that happened after the knowledge cut-off of the model that happens to be answering, without ever giving that model (or the user) a third-party search credential or an open channel to arbitrary hosts.
What the surfaces give us
Costs are now queryable. Routing and feature flags can be changed without a restart or a direct database edit. The model can reach the live web through a narrow, auditable, internal-only path. All of it is built on the same primitives (ledger, snapshot reload, bearer proxy, code-enforced flags) that the rest of the system already used.
Next: the assistant reaches my real accounts, Gmail, Calendar, and arbitrary MCP servers, behind the same permission chain.
(The implementation is in internal/gateway/ledger/aggregate.go, internal/gateway/admin/, the admin proxy in internal/brain/api/, the settings store and its enforcement points, internal/brain/tools/builtin/websearch.go, and the SearXNG service definition in compose.)