I pay for five AI subscriptions. Claude, Cursor, Kiro, Grok, and GLM, each one excellent at something, each one living in its own tab with its own history, its own memory, and its own idea of who I am.
Every morning I decide which one to open. Coding question? One tab. Market news? Another. Travel research for the next trip? A third, which has forgotten everything I told it last month. None of them talk to each other, none of them show me what a conversation actually cost, and none of them will do anything unless I am sitting there typing.
So I am building my own. Its name is Timothy.
What Timothy Is
Timothy is a self-hosted personal AI assistant: one chat interface across every provider I use, a usage and cost dashboard, long-running automated tasks, web search, and persistent memory that belongs to me, not to whichever vendor is winning this quarter.
The goals are concrete:
- One interface, many models. Each task type routes to the cheapest model that can handle it: cheap models for summaries, strong models for planning and code, realtime models for news.
- Configurable at runtime. Providers are rows in a settings panel: add one, disable one, reorder the routing, no code changes, no restarts. Both API keys and CLI subscriptions work as credentials.
- My state, my database. Conversations, memory, configuration, and costs live in my Postgres. Switching providers never costs me my history.
- Unattended work. Scheduled research briefs, long coding tasks, recurring digests: Timothy initiates, works, and only interrupts me when something genuinely needs a decision.
- Every token accounted for. A cost ledger records every single request, and a dashboard shows me spend per provider, per model, per day.
- More than text. Image, video, and speech generation flow through the same routing and the same ledger as chat.
The Architecture at a Glance
Timothy is three small Go services plus a React web app, all state in one Postgres with pgvector, everything under Docker Compose:
Browser
│
├── web (chat, dashboard, tasks, settings)
│
└── brain public API + agent orchestration + scheduler
├── gateway sole holder of provider credentials,
│ routing, streaming, cost ledger
│ ├── Provider APIs (Anthropic, Z.AI, xAI, ...)
│ └── Provider CLIs (subscriptions as credentials)
└── memoryd long-term memory: typed storage, hybrid retrieval
Three design rules shape everything:
- Deterministic code orchestrates; each LLM session does exactly one thing. Intelligence lives in the models. Memory, state, routing, and safety live in Timothy.
- Only the gateway holds credentials. The brain orchestrates, memoryd stores, the web app renders, none of them ever see an API key.
- Safety lives in code, not prompts. Path allowlists, step ceilings, and permission gates are Go middleware. A prompt can be persuaded; conditional logic cannot.
Why Build Instead of Buy?
Fair question: the market is full of AI chat frontends and agent frameworks.
Memory portability. Every provider now offers some form of memory, and every one of them is a lock-in mechanism. I want facts about my life in a Postgres table I can query, back up, and take with me.
Subscriptions as credentials. Most of my AI spend is subscriptions that do not include API access. Timothy’s gateway treats provider CLIs as first-class drivers alongside APIs: the same routing table can send a coding task to a CLI running on my subscription and a summary to a cheap API model.
Cost visibility. No provider dashboard tells me what a specific conversation cost across providers. A self-owned ledger does.
And honestly, this is the fun part. I have a homelab with Proxmox, Ansible, Vault, Caddy, and Grafana already humming. A personal AI assistant is exactly the kind of workload it was built for.
Where It Will Run
Regular readers know my €210 homelab. Timothy slots straight into it:
| Location | Role |
|---|---|
| Homelab LXC | The four-service stack, behind Caddy at timothy.mol.la |
| Oracle Cloud VM (24GB ARM) | Postgres + pgvector, heavy task execution |
| VM in Bangladesh | Encrypted offsite backups, external uptime probe |
Tailscale connects all three. Vault supplies the secrets. Prometheus and Grafana, already running, get a few new scrape targets and a spend-per-day alert.
The Stack
| Layer | Choice |
|---|---|
| Services | Go, standard library HTTP, pgx |
| Web | streaming client + dashboard + settings |
| Data | PostgreSQL + pgvector (the only datastore) |
| Runtime | Docker Compose, dev and prod |
| Observability | slog JSON, OpenTelemetry, Prometheus, Grafana |
No agent framework, no ORM, no message broker. A single-user workload does not need them, and every dependency I skip is a dependency that cannot obscure a prompt or eat a debugging session.
Posts in This Series
I will document the build as it happens; the plan below will grow and shift as the project does.
| Part | Topic |
|---|---|
| 1 | Why I am building my own AI assistant (this post) |
| 2 | The architecture: three Go services and a rule |
| 3 | The LLM gateway: providers as data, streaming, cost ledger |
| 4 | Pass-through chat: the brain’s first API |
| 5 | The first web chat interface |
| 6 | Sessions that survive: event sourcing the backend |
| 7 | Sessions that survive: resumable UI and replay |
| 8 | Letting the model act: tools, permissions, skills |
| 9 | Memory that belongs to me, not the model vendor: staged facts that only supersede |
| 10 | Remembering on demand: three retrieval legs, one token budget, and a fence |
| 11 | First-party models on AWS: a Bedrock driver and capability-aware routing |
| 12 | The control plane and web search: dashboard aggregates, admin surfaces, feature switches, and a self-hosted search tool |
| 13 | Missions: long-running agent tasks that survive restarts |
| 14 | Generating images, video, and speech through one gateway |
| 15 | Deploying Timothy into the homelab |
In the next post, I walk through the full architecture: why exactly three services, how a request flows from the browser to a provider and back, and the design decisions (D-001 onward) I committed to before writing a single line of code.