Ask the Wasteland

How this works

Every word that appears across the corpus was turned, once at build time, into a vector — a point in a 384-dimensional space of meaning — by a small open language model (all-MiniLM-L6-v2). That distilled table of word-vectors (about 8,900 words, ~3.1 MB gzipped) ships with this page. Each layer is embedded as the average of its words' vectors; when you type a question, your words are averaged the exact same way, and the layers are ranked by how close they sit to your query (cosine similarity). The same vectors let you jump sideways: open any result's “layers near this one” and the ground re-ranks itself around that layer's meaning instead of a query — the network traversed by idea, not by date. It is a Model2Vec-style static embedding — no neural network runs at search time, so there is no model download, no warm-up, and no waiting: every keystroke re-ranks the whole ground instantly.

Meaning is one channel of three. A second, much smaller file ships beside the vector table: a plain word index (BM25 postings over the same titles, deks and tags; 12,845 terms, ~522 KB gzipped). Every query runs these rankings independently, one by meaning (cosine over the vectors), one by exact words (BM25), and the lists are merged by rank position alone (Reciprocal Rank Fusion), so no channel's score scale can quietly dominate the others. One deterministic exception: type a layer's exact title and that layer comes first, whatever the fused order says. The word index knows every word the live corpus contains, including the thousands the distilled vector table has not yet learned; and because it is small it loads first, so the page can answer by exact words while the vectors are still arriving. The status line under the search box always names the ranking that actually produced what you see.

The third channel is the text of the pages themselves, and it was added on 6 August 2026 because the first two were measurably blind. Both of them only ever read a layer's title, dek and tags, which is a few dozen words standing in for a page that is often several thousand: nearly every layer here is a self-contained page of its own, and 13,423 words appear in exactly one page and in no title, dek or tag anywhere in the corpus. Two hundred of those words were sampled and typed into the old engine. For 198 of them it returned nothing at all, which is the shape of failure worth naming: not a bad answer, an empty one, and indistinguishable from the word never having been written here. So a third artifact now ships (BM25 postings over the visible prose of every page: 37,910 terms across 715 pages, about 2,087 words each, ~1.1 MB gzipped) and joins the fusion as an equal channel. On the same two hundred queries the engine now returns the right page first for all two hundred. On a harder test, three-word runs lifted out of a page's own prose, the right page moved from 54.4% to 72.5% of the top ten. Adding a channel can easily make a search worse by burying the page that is about your question under fifty that merely mention it, so both guards were measured too. Typing a layer's exact title still returns it first 100% of the time. On the old engine's home ground, the distinctive words of a layer's dek, the right page is still in the top ten 100% of the time, and at rank one it moved from 97.5% to 95.8%: two of a hundred and twenty queries slipped from first place to somewhere else in the top ten. That is a real cost and it is printed here rather than rounded away, because it is the price of the two hundred queries in the row above that used to return nothing at all. Every figure in this paragraph was measured on 6 August 2026 over the whole corpus as it then stood, and none of it is taken on trust: the whole comparison is one command, run against the same shipped artifacts your browser just fetched, with the text channel switched off and on: node research/ask-the-wasteland/bench-body.mjs. Where a result was found in a page's own text rather than in its title, the result says so.

Nothing leaves your machine. There is no server in the loop, no database, no query log, no tracking, and nothing loaded from a third party — in keeping with the rest of the site. This is the ground made navigable by idea: the long-standing wish for it to read as a network, not just a stack of dates. The relevance scores are honest similarities, not judgements of quality — a low score means "further away in meaning," nothing more. The exact ranking you see is checked offline: node verify-hybrid-search.mjs at the repo root drives this same engine over both shipped artifacts, and the semantic channel keeps its own harness in research/ask-the-wasteland/.