Two related gaps closed this milestone. Typing was the only way in, and text was the only thing a message could carry. Now there’s a microphone button that turns speech into a draft, and a paperclip that lets an image ride along with a message so a vision-capable model can actually look at it.
Why local transcription, not a cloud speech API
The easy path for voice input is an API call: ship the audio to a cloud speech-to-text service, get text back. I didn’t take it, for the same reason Timothy runs its own gateway instead of calling providers straight from the browser: every voice message would otherwise leave the box and acquire a per-minute bill and a third party who now has a copy of whatever I said out loud.
So voice input runs through a whisper sidecar, a faster-whisper CPU container that mirrors the shape of the existing markitdown-svc (the file-to-markdown converter I already had). It has no published ports and no auth. It’s reachable only from the brain, on the compose-internal network, the same trust boundary as searxng. Audio in, transcript text out, nothing else touches it.
The model itself is baked into the image at build time:
# whisper-svc/Dockerfile
ENV HF_HOME=/models
ARG WHISPER_MODEL=small
RUN python -c "from faster_whisper import WhisperModel; WhisperModel('${WHISPER_MODEL}', device='cpu', compute_type='int8')" \
&& chmod -R a+rX /models
That matters for the “never leaves the box” claim to actually hold. If the model downloaded lazily on first use, the sidecar would need outbound internet access at least once, and a fresh deploy in an offline environment would fail on the very first recording. Baking it in means the running container never phones out for weights, and the guarantee is airtight from the first boot, not just after a successful first request.
How the transcription flow actually works
Click the mic button in the composer, and the browser’s MediaRecorder starts capturing audio (audio/webm;codecs=opus when supported, falling back otherwise). Click it again and the recorder stops, hands its blob to a transcribeBlob call, and that POSTs the raw bytes to the brain’s /v1/transcribe endpoint.
The brain doesn’t do anything to the audio itself. It caps the body at 25MB, checks it’s non-empty, and forwards the raw bytes to the whisper sidecar’s /transcribe endpoint with a 120-second timeout (CPU transcription of a full minute of audio on the small model can take a while). The sidecar decodes whatever container format the browser handed it, via PyAV’s bundled ffmpeg libraries, so there’s no separate ffmpeg binary needed in the image and no format negotiation between browser and server.
The transcript comes back as plain text and lands in the draft box, not in the message that gets sent. That’s deliberate: nothing about a microphone claim should be taken on faith. The whole point is a human reads what was actually heard before any of it goes anywhere, exactly the same “the user is always in the loop before an irreversible action” instinct that shapes the tool permission system elsewhere in the brain.
The route is /v1/transcribe, mounted only when WHISPER_URL is set, same nil-gating pattern as every other optional surface in the brain (attachments, sandboxd, markitdown).
Storing images without putting binaries in Postgres
Image attachments follow a rule that’s been true since the memory schema went in: binaries never enter the database. An uploaded image lands content-addressed on a volume, keyed by its own sha256 hash, with only metadata (id, mime, size) in a new attachments table.
// internal/brain/attachments/attachments.go
id := fmt.Sprintf("%x", h.Sum(nil))
finalPath := filepath.Join(s.dir, id+ext)
The MIME type is never trusted from the client. The store sniffs it from the first 512 bytes with http.DetectContentType and rejects anything outside a four-item allowlist (PNG, JPEG, WebP, GIF). Content-addressing means uploading the same image twice is a no-op: same hash, same path, an ON CONFLICT DO NOTHING insert.
Session events and the model’s context carry attachment ids, never image bytes. A user message that references an image stores an ImageRef (id plus MIME type), and that’s what rides through the event log, through compaction, through the turn-memory distillation that summarizes a conversation for long-term storage. None of those code paths ever see actual pixels; a short “[image attached]” placeholder stands in wherever summarization needs something to point at.
The one place bytes exist in memory at all is immediately before the request goes to the gateway:
// internal/brain/chat/chat.go, resolveImages
// Resolve image refs into base64 ONLY here, right before the
// gateway call: this is the sole point in the whole turn where
// attachment bytes exist in memory at all (D-045).
That’s a real constraint, not a comment for its own sake: it means an image can’t accidentally get dragged into a compaction summary, a token count, or a memory extraction call, because by the time any of those run, the message only ever held a reference.
Serving an attachment back out is authenticated the same way every other brain endpoint is: GET /v1/attachments/{id} sits behind the same bearer auth as the rest of the API. That has one visible consequence in the UI. A plain <img src="..."> tag can’t carry an Authorization header, so the composer fetches thumbnails through the same authenticated client the rest of the app uses and builds an object URL from the response, rather than pointing the tag straight at the endpoint.
Vision as a route, not a special case
Adding image support to the gateway didn’t mean adding an “if image, do X” branch somewhere. It meant extending the same primitive that already exists: provider.Message gained an Images field, and a new CapVision capability joined the driver capability set from the model-level routing work in Part 11. The gateway derives whether a request needs vision from the message content itself, not from a new flag the caller has to remember to set, and threads that requirement through route resolution as an extra required capability. A chain with no vision-capable entry now fails with a NoRouteError that names “vision” specifically, instead of a generic unhelpful failure.
That capability gate alone would have been enough to make image messages work. It wasn’t enough to make them work well. Before the routing change, an image message rode whatever route the agent or the default chat category happened to use, and the vision capability check would just walk that chain until it found something that could see, which might be the most expensive model configured rather than the cheapest one that can actually do the job.
So image messages now auto-flip to a dedicated vision route, seeded cheapest-first (a Nova Lite entry, then a GLM vision model, then Nova Pro as the expensive fallback). The flip happens in the brain, unconditionally, because the brain has no cheap way to check whether a vision route even exists or is enabled on a given install without adding new polling machinery just for this one check. The gateway is where the safety net lives: if vision resolves to a missing, disabled, or empty chain, it falls back to default automatically, so an install that never configured the route doesn’t break. That fallback only fires when nothing was even tried against vision. If the route exists but genuinely has no vision-capable entries, that’s a real configuration problem and it surfaces as an error rather than silently serving the image to a model that can’t see it.
The unglamorous bug that self-hosting always finds
Every one of these milestones seems to produce at least one deployment-layer bug that has nothing to do with the feature’s logic and everything to do with how containers actually run. This time it was the attachments volume.
Brain runs as an unprivileged nobody user inside a distroless-style runtime image, same as it always has for /workspace. A fresh named Docker volume mounts owned by root by default. /workspace had already been fixed for this in the Dockerfile: the directory is created and chown’d before the volume ever attaches, so the named volume inherits the right ownership on first mount instead of getting root’s default. The attachments feature added a second directory, /attachments, without applying the same fix, and every upload failed with a plain permission-denied error the moment the feature shipped.
The fix was a one-line addition to an existing RUN step:
# deploy/Dockerfile
RUN mkdir -p /workspace /attachments && chown nobody:nobody /workspace /attachments
Nothing subtle about the fix. What’s worth noting is that the bug only exists because of a Docker volume lifecycle detail: ownership set inside the image at build time only takes effect on a fresh named volume’s first mount, so the mistake is invisible until someone actually deploys clean rather than reusing an existing volume from before the feature landed.
The whisper model had a smaller version of the same lesson. The Dockerfile originally cached the downloaded model under /root/.cache, exactly where pip and huggingface_hub put things by default, and exactly where the nobody runtime user has no read access. Combined with an unrelated dependency break (huggingface_hub 1.x dropped its dependency on requests, which the model download path imported), the model either failed to bake in at build time at all, or baked in somewhere the running container couldn’t read, meaning it would try to re-download at boot and fail because the container has no network. Pinning requests explicitly and pointing HF_HOME at a world-readable /models fixed both halves in one small commit.
What still doesn’t work
Input only, both directions. The mic transcribes speech to text; there’s no text-to-speech, so Timothy has no voice of its own yet. Images can be attached and seen; there’s no image generation. Both are real gaps, not oversights, and both are explicitly the next milestone.
Next up: media generation through the gateway, and finally deploying Timothy into the homelab for real.