$ ls articles | grep * # 33 found
Timothy Part 20: Ears and Eyes, Voice Input and Image Attachments
Timothy can now hear and see: a microphone button transcribes speech entirely on-box through a self-hosted Whisper sidecar, and the composer accepts image attachments that are stored content-addressed on the brain and routed to a dedicated vision chain. Both are input-only for now; the model still cannot speak back or generate an image.
Timothy Part 19: A Chat That Survives Reloads and Races
The event log from parts 6 and 7 made turns durable on disk, but the live chat above it still lied under ordinary failure: a reload mid-answer looked stuck, a dead turn looked like a stuck spinner, and parallel tool calls could stack duplicate prompts. This post covers the milestone that made the live experience as honest as the log underneath it: reattach across reloads, one turn per session, persisted permission asks, and turns that say plainly when they produced nothing.
Timothy Part 18: Sensitive Sessions Stay Home
Some content should never reach a cloud provider, no matter which model is smartest. I added a sensitive-tool route that pins a session to a local model the moment it touches something like Gmail, made the pin sticky across every later turn and side-call, and moved local inference off a containerized Ollama onto a native host runner so it's actually fast enough to use.
Timothy Part 17: Mission Schedules, Notifications, and the Canary
A mission that never gets scheduled again is barely different from a chat message. This post covers the cron scheduler that fires mission templates, transition notifications pushed over SSE, per-mission spend read honestly off the cost ledger, an opt-in escalation route for stuck missions, and the golden-mission canary that gates every change to the harness.
Timothy Part 16: Mission Verification, Review, and the Sandbox
A mission that reports itself done is not the same thing as a mission that is done. This part covers the harness-owned verification that never trusts a model's word for it, an antagonistic reviewer that has to be talked out of rejecting, and the move to run every model-authored command in a disposable sandbox instead of the brain's own process.
Timothy Part 15: Missions, a State Machine for Long-Running Agent Work
Chat turns cap out at a few minutes of a model's attention, but real work runs for hours. This post covers the mission engine: a pure Go state machine, an append-only event log, fresh-context workers seeded from persisted artifacts, and the sentinel protocol that catches a worker trying to quit early.
Timothy Part 14: Agents, Not Modes
The chat composer had grown a task-category picker, a skill chip, and a model hint, and none of them agreed with each other. This post covers the agents-first redesign: one profile that bundles a prompt, a route, tool and skill allowlists, and a memory switch, plus the scored routing underneath it that replaced fixed provider chains with live price, latency, and uptime data.
Timothy Part 13: Connectors for Gmail, Calendar, and MCP Tools
The assistant could search the web and run shell commands in its own workspace, but it had no way to touch my inbox, my calendar, or any third-party API. This post covers the connectors framework: a generic MCP client for the escape hatch, a first-party Google kind for OAuth-backed Gmail and Calendar tools, and the two bugs that shipped after the demo worked.
Timothy Part 12: The Control Plane and Web Search
With sessions, memory and the first-party models in place, the ledger was full of numbers no one could see and the model still lived inside its training cut-off. I added usage aggregates, a full admin surface for providers and routes that hot-reloads, feature switches enforced in code, and a web_search tool backed by a self-hosted SearXNG instance that the model can never reach directly.
Timothy Part 10: Remembering on Demand, Three Retrieval Legs, One Token Budget, and a Fence
Per-turn recall runs vector k-NN, full-text, and entity match in parallel over one Postgres table. Results are fused with reciprocal rank fusion plus recency decay, packed under a real-tokenizer budget, and injected as fenced DATA at the tail of the system prompt so provider caches keep hitting. Content can never close the fence.
# ---