Timothy Part 4: Pass-Through Chat, the Brain's First API

In Part 3 the gateway learned how to talk to real providers, normalize their streams, pick the right one, and write a cost ledger entry for every call.

The next layer up is the brain. It owns the public surface and the orchestration. For the first slice I kept it deliberately small: a single authenticated endpoint that turns a user message into a gateway request and streams the answer back.

One Public API, One Token

The only thing the outside world sees is the brain.

# from the host (brain published on 8300 by default)
curl -N -X POST http://localhost:8300/v1/chat \
  -H 'content-type: application/json' \
  -H "Authorization: Bearer $TIMOTHY_API_TOKEN" \
  -d '{"message": "Summarize the last three things I told you about Postgres."}'

Everything except /health and /metrics requires the bearer token. If TIMOTHY_API_TOKEN is not set in the environment, the brain fails closed with 503. No token, no service.

Inside the auth middleware I use constant-time comparison so timing attacks are not a thing.

The Chat Service

The brain’s chat handler is small on purpose.

// internal/brain/chat/chat.go
func (s *Service) Chat(ctx context.Context, req Request) (string, <-chan stream.StreamEvent, error) {
    ...
    sessionID := req.SessionID
    if sessionID == "" {
        sessionID = s.createSession(ctx)  // just inserts a row for now
    }

    messages := append(s.history(sessionID), userMsg)

    upstream, err := s.gw.Stream(ctx, gwclient.StreamRequest{ ... })
    ...
    // relay every event, accumulate the reply for the temporary buffer
    for ev := range upstream { ... }
    ...
}

A few deliberate simplifications, all of which the next milestone (real event-sourced sessions) replaces:

• Only an in-memory map of the last 20 messages per session. Enough to give the gateway context for the next turn, not enough to survive a restart.
• A bare sessions table row (id + created_at) so the cost ledger has something to attach to.
• No event log yet. The full append-only session events come later.

The important part is that the brain never sees credentials. It only knows task_category and model_hint. It hands the assembled prompt to the gateway and gets a normalized stream back.

The Terminal Meta Event

The gateway’s last frame on a successful stream is a done with a meta object (provider, model, ledger_id).

The brain does one more thing: after the gateway channel closes, it emits its own terminal event:

{"type":"meta","session_id":"...","provider":"anthropic","model":"claude-sonnet-5","usage":{...},"ledger_id":"..."}

Clients are required to read until they see the meta. That is the contract.

Surviving Mid-Stream Cuts

One nasty edge case: the client opens the connection, the gateway starts streaming, then the network drops before the final meta arrives. The session row was already created. If the client doesn’t know the id, the next message would start a brand new conversation and orphan the previous row.

The fix was two small changes that together close the gap:

Why put the session ID in a response header that arrives before any body bytes?

1. On every 200 response the brain now sets a response header before any body bytes:

   w.Header().Set("X-Session-Id", sessionID)
   

2. Callers read the header as soon as the Response is available, before consuming the SSE body.

Headers arrive early. Even if the body is cut off immediately after, the client already knows which session it is talking to.

On the error path (gateway unreachable before streaming starts) the session id travels in the JSON error body instead.

Talking to the Gateway

The brain talks to the gateway through a tiny client:

// internal/brain/gwclient/gwclient.go
resp, err := c.http.Do(...)
...
err := sse.Read(resp.Body, func(ev sse.Event) bool {
    var se stream.StreamEvent
    json.Unmarshal([]byte(ev.Data), &se)
    ...
})

It uses the shared platform/sse parser so the wire format stays consistent. The client itself stays thin: no routing, no retries, no ledger. Those all live in the gateway where they belong.

A Real End-to-End Call

With the compose stack up and a token configured:

curl -N -H "Authorization: Bearer $TIMOTHY_API_TOKEN" \
  -d '{"message":"Write a haiku about pgvector indexes.","task_category":"mini"}' \
  http://localhost:8300/v1/chat

You see the usual gateway frames (chunks, usage, done) followed by the brain’s meta that includes the session id and the exact provider/model/ledger that served it.

A row appears in the cost ledger with the session id attached.

Wrong token → 401. No token configured on the server → 503. Gateway down → 502 with the session id in the body when one was created.

Why This Shape for Now

I wanted a working chat loop as quickly as possible. The temporary buffer is explicitly documented as a stopgap. Full event-sourced sessions, compaction, and resume will replace the in-memory history next.

The contract that the stream always ends with exactly one meta event turned out to be the right one for clients as well. It keeps the streaming path simple while giving everything needed to render the turn and continue the conversation.

In the next post the first client speaks this protocol, renders the stream, and survives the failure modes the backend just became robust against.