brain-rag

For a while my proudest build was a RAG stack over 784GB of the world's knowledge. 341 Kiwix ZIMs on a USB drive: Wikipedia in full, medical field manuals, engineering references, Stack Exchange dumps, textbooks, survival guides. All of it chunked, embedded, and queryable offline, every citation resolving to a locally hosted article over HTTP. No data leaving the room. I called it paxwiki-rag and I was very pleased with myself.

It was a genuinely cool build. It also turned out not to be the search I reach for every day.

In a normal week I do not need offline Wikipedia. What I need constantly is the answer to "where did I write down how this works" and "what did I decide about that." Years of my own notes: a private wiki of every system I run, plus a long memory of preferences and project state. The most valuable corpus I own, and the one I could not actually search, because it was a pile of markdown I had to grep and half-remember.

So I took the same machinery and pointed it at myself. That one stuck. It is called brain-rag, and three AI agents now read from it on every turn.

the brain, and the muscle

The brain is deliberately dumb: plain markdown under git. A wiki of my infrastructure, my agents' memories, my source notes. No database, no lock-in, greppable in a pinch and diffable forever. If brain-rag vanished tomorrow the knowledge would still be sitting there in text.

brain-rag is the muscle bolted on top. A searchable index that answers "where did I write about X" by meaning and by keyword at once, ranked. The brain stays text. The index makes it fast.

the stack

Retrieval is hybrid, two stages.

First, recall: blend 70 percent semantic (vector KNN over sqlite-vec) with 30 percent keyword (FTS5 bm25), each normalized per query. A search for "how the backups work" finds the page that never says the word backup, and a search for an exact error string still lands the literal match. Frontmatter tags and status narrow it further.

Second, an optional cross-encoder rerank for when precision is worth the half second it costs.

Ask a real question instead of a keyword and it synthesizes an answer with citations, every claim linked back to the note it came from. The engine runs in a container on my homelab server, so my laptop spends nothing on embeddings, and the whole round trip is about 90 milliseconds over the tailnet.

The part I am most smug about: it reindexes itself on every commit. I write a note, I commit, and a hook on the bare git repo reindexes the changed files. The index is never stale, because staleness stopped being a maintenance task and became a side effect of saving.

what it looks like

Search in about 90 milliseconds. Rerank adds roughly half a second when I ask for it. Every hit carries its source: the exact file, the heading, and which corpus it came from. Ask for a synthesized answer and every sentence links back to the note that justifies it, so I can check the machine's homework in one click.

It also renders as a graph. Every note is a node, every wikilink an edge, colored by which corpus it belongs to: the wiki in one color, my memory in another, and one color for each agent's private notes. You can see a topic's whole neighborhood at a glance, and see what nothing links to.

it keeps the agents honest

I have three agents living on top of this. A critic that audits the brain for contradictions, an executor that fixes them, and a specialist pointed at my repair bench. The single most-used tool any of them has is brain-search.

That is what makes them usable instead of dangerous. An agent asked about my infrastructure does not get to improvise from a plausible-sounding memory. It searches the brain first, gets the actual documented answer with a citation, and reasons from that. Before I even finish typing, a hook runs my prompt through the index and drops the genuinely relevant notes into context, so recall never depends on a model remembering to look. The brain is the ground truth the whole fleet is chained to.

it audits itself

An index over your own notes rots the same way the notes do: dead links, orphaned pages, claims that drift out of sync with reality. So brain-rag checks its own health. It flags broken wikilinks, pages nothing points to, and frontmatter that has wandered from the source. The critic agent reads those reports and hunts the contradictions down. The knowledge base has stopped being a thing I maintain and started being a thing that tells me where it needs maintaining.

what is next

The direction is an index that improves itself without me babysitting it. Retrieval that tunes its own weights to the shape of the question. A graph that proposes the links I forgot to draw. Health checks that widen from "is this link broken" to "does this page still agree with the machine it claims to describe."

paxwiki-rag wanted to hold the whole world offline. brain-rag wants something narrower and, for me, far more useful: to know everything I have ever written down, to keep it honest, and to put it in front of the agents before they open their mouths. The world's knowledge I can always look up. My own was the part I kept losing.