Centinel

A civic transparency toolkit — built on the warnings of a Pennsylvania watchman.

Centinel collects the public record of a city — website maps, documents, transcripts, and the changes to all of them over time — and keeps it in a form nobody can quietly edit.

Everything runs on your machine. No document, no transcript, no page leaves it.

Who this book is for

You want to search a corpus somebody else collected. Read What it does, then Searching and Reading a result. Two pages.

You want to collect a city. Read Install, Your first corpus, then the Operate it part. That is the operator’s path: name a source, run it, keep it running, know what broke.

You want to change the code, or trust it. The How it works part walks the pipeline one stage at a time, from an address to a cited passage. Each page says what the stage does, what it refuses to do, and why.

You want the settled reasoning. This book is a guide. The specifications it was written from are in docs/ and go much deeper.

The three principles

Documents over promises. Every byte is content-addressed. The hash covers the raw bytes as served — not a summary, not a re-render, not a cleaned-up copy. Reading a document back verifies that hash, so an edit in place is an error rather than a silent success.

Never trust memory. Files on disk are the only truth. Every index, every database, every embedding is derived and rebuildable. Nothing in this system answers from recall, because there is nothing to recall from. There is only the record, read again.

Notice what disappears. Every version is kept, and every collection run is a full snapshot — so a page that vanishes is a fact the archive holds, not a gap it forgets. A page that starts refusing you is a different fact, recorded differently.

“The federal government will… necessarily absorb the state legislatures.” — Centinel, 1787

Samuel Bryan wrote twenty-four essays under that name, warning that distance from the people is itself a form of tyranny. He was right. He had the wrong scale. The government that answers to no one is the one across the street, because no one is watching.

You are the freeman now. The watchman’s seat is empty. This is what fills it.

What it does

The short version, in one page.

You name a website or a YouTube channel. Centinel finds every address it declares, fetches each one, pulls readable text out of whatever came back — HTML, PDF, spreadsheet, Word document, captions, audio — and makes all of it searchable. It keeps the original bytes forever, and it keeps every version.

centinel source add tampa --site https://www.tampa.gov
centinel run
centinel search "stormwater drainage fee"

That is the whole product. Everything else is a detail of one of those three lines.

The pipeline

One run, six stages, in this order:

Every stage skips work it has already done, and none of them keeps a checkpoint file. The work list is always a subtraction — what the source declares, minus what the log already records. So a second run does nothing, a killed run resumes, and centinel run in cron is the intended use.

What it keeps

~/.centinel/
  blobs/          TRUTH     immutable, content-addressed, pooled across sources
  log/            TRUTH     append-only: observations, discovery runs, status, derivations
  current/        derived   a tree that mirrors the URLs
  centinel.db     derived   SQLite metadata + FTS5   — the keyword arm
  vectors.lance/  derived   LanceDB chunk vectors    — the semantic arm

Only the first two are evidence. Delete everything else and you lose time, not facts. The corpus is one directory. You can hand it to somebody with rsync.

What a search does

Two retrievers run against the same corpus and neither is a warm-up.

query
  ├─ BM25   (SQLite FTS5)          → top 100    instant, no model
  └─ vector (Qwen3-Embedding-4B)   → top 100    one embed call
        └─ RRF fuse (k=60)         → top 40
              └─ Qwen3-Reranker-0.6B → top n    always on

BM25 catches the exact token — a name, a motion number, a dollar figure. The vector arm closes the vocabulary gap: a water quality report says PWSName and Analyte, and no keyword search for drinking water sampling results will ever reach it. The reranker then reads each candidate against the question and reorders. That last step is worth more than either retriever — reranked BM25 measures more than twice as good as raw BM25 — which is why there is no flag to turn it off.

Every result carries a handle: the short hash of the bytes it came from. Anything Centinel prints, Centinel takes back. centinel read <hash> and centinel open <hash> accept it by prefix, git-style.

The models

All local, all Apache-2.0, all fetched by centinel models pull.

The embedder is big and the reranker is small on purpose. The embedder is paid once per corpus, in hours. The reranker is paid per query, in milliseconds. So the budget goes into the embedder once and into the reranker freely.

Inference runs in-process through llama.cpp and whisper.cpp. There is no server, no sidecar, and no second language runtime.

Three surfaces, one definition

Centinel is a library first. The CLI, the HTTP server and the MCP server are thin consumers of it.

$ centinel search "lobbyist meeting log"          # CLI
$ curl -X POST localhost:8787/ops/search -d …     # HTTP: JSON in, JSON out
{"jsonrpc":"2.0","method":"tools/list"}           # MCP: over stdio or over HTTP

Each verb is one annotated Rust function. Adding one puts it on all three surfaces with no central list to update. Agents are clients of the record, never its author — what gets collected does not depend on what any model thought that day.

What it refuses to do

These are the load-bearing refusals. Each one exists because the alternative silently records something false.

• A blocked page is not a deleted page. A WAF 403 and a 404 are the same Err and completely different facts.
• A Resource is an address, not a thing in the world. The same meeting reachable four ways is four Resources. Four honest rows beat one confident wrong one.
• A strategy keys on a product, never on a city. Recognising Hyland OnBase collects every city running OnBase. Teaching it Tampa collects Tampa.
• A count that hit a ceiling says so. An enumeration that stopped early is printed as at least n, because a truncated snapshot looks exactly like a source that shrank.
• An over-long chunk is refused, not truncated. A shortened chunk stored under a hash covering text that was never embedded makes the record lie about what it holds.

Next: Install.

Install

You need Rust 1.91+ and a C++ toolchain. Two dependencies compile llama.cpp and whisper.cpp from source, so expect a long first build.

git clone https://github.com/bennyhodl/centinel
cd centinel
./install.sh

The script checks the host before it builds anything, and names the command for whatever is missing rather than installing a toolchain behind your back.

What the script does that cargo install cannot

It selects the GPU backend. Metal on macOS, CUDA or ROCm on Linux when their toolchains are present.

It tunes for the CPU it is building on. This matters more than it sounds like. llama-cpp-sys-2 reads target-cpu back out of the Rust flags, and when it is not native it sets GGML_NATIVE=OFF and derives ggml’s instruction-set flags from the baseline target features instead. On x86_64-unknown-linux-gnu those are fxsr,sse,sse2,x87 — so a plain cargo install compiles llama.cpp’s CPU kernels with no AVX, no AVX2 and no FMA, and nothing recovers it at runtime. whisper.cpp has no such handling and builds native already. So the untuned case is not “both a little slow”. It is the transcriber tuned and the embedder not, and embedding is the stage measured in days.

The cost is that the binary is then built for the CPU that built it. --portable is the way out, and it leaves RUSTFLAGS alone, so a middle tier that still carries AVX2 and FMA is:

RUSTFLAGS="-C target-cpu=x86-64-v3" ./install.sh --portable

It installs both binaries into one directory. Which is the thing transcription does not work without.

Why two binaries

whisper.cpp and llama.cpp each vendor their own copy of ggml, and both export the same ~534 ggml_* symbols. Linked into one binary, the linker keeps one copy and silently resolves the other library’s calls to it. The two versions are not the same.

Measured on identical audio and model, the linked crates the only variable:

It links without a warning, runs without a crash, and transcribes nothing. There is no error to catch. So centinel links llama.cpp, centinel-whisper links whisper.cpp, and the two meet over a pipe.

centinel finds the worker beside itself first, then $CENTINEL_WHISPER_BIN, then PATH. Installing both with the same command puts them in the same directory, which is all it needs.

By hand

cargo cannot take two --path arguments, and the workspace root is a virtual manifest, so from a clone it is two commands:

cargo install --path crates/centinel
cargo install --path crates/centinel-whisper

Without a clone, one command does both:

cargo install --git https://github.com/bennyhodl/centinel centinel centinel-whisper

Any GPU backend other than Metal is a --features flag on each of the two commands. Passing --features cuda to one and not the other leaves half the pipeline on the CPU.

External tools

Centinel shells out rather than running a second language runtime.

brew install yt-dlp ffmpeg          # macOS
sudo apt install yt-dlp ffmpeg      # Debian/Ubuntu

Keep yt-dlp current. It ships releases in emergency clusters when YouTube changes, and centinel doctor warns once yours is past ninety days.

Then

centinel doctor         # what this machine is missing, and the command for each gap
centinel models pull    # weights for search and transcription

Run doctor first and after every step. It names the fix beside each gap it finds. See Models for what models pull fetches and how much disk it wants.

Next: Your first corpus.

Your first corpus

Ten minutes, one city, and a search that returns something.

1. Check the machine

centinel doctor

It prints the store root it opened, the config file that named it, which binaries are present, and which model weights each pipeline stage is waiting on. Fix what it names before going further. A missing model does not stop collection — the stage is skipped and resumes on a later run — but a missing yt-dlp stops a channel dead.

2. Look before you collect

Point investigate at a host and it will tell you whether anything recognises it, and on what evidence.

$ centinel investigate https://www.hillsclerk.com/

  seed        https://www.hillsclerk.com/  →  200, 212 KB, html
  recognised  sitemap (standard)
              robots.txt allows everything and names a <sitemapindex>
              182 child sitemaps
  crumbs      hover.hillsclerk.com          8 links
              publicrec.hillsclerk.com      2 links

  centinel source add hillsclerk --site https://www.hillsclerk.com/

Nothing is stored. It is a question, and it costs a couple of dozen requests.

Three answers are possible: a strategy with its evidence; a set of crumbs, meaning the system you want is on another host; or nothing, said plainly. All three are useful. See Strategies for what recognition is and why the evidence is printed rather than the verdict alone.

To ask a narrower question — what would extraction make of this one document — use centinel check <url>. It also stores nothing.

3. Name a source

centinel source add tampa --site https://www.tampa.gov

That writes a [[source]] block into your config file. A YouTube channel is the same command with a different key:

centinel source add tampa-council --channel https://www.youtube.com/@CityofTampa

site versus channel is the whole of the website/YouTube difference. The two kinds are peers that differ only in how they are acquired, so there is no centinel youtube verb and adding a third kind would add no verb either.

4. Try it small

centinel run --limit 50

--limit bounds collection, not discovery. The sitemap walk still runs in full, because a truncated snapshot of a source’s address set looks exactly like a source that shrank — that is a fact the archive must not record falsely. Fifty documents is enough to see whether the text coming out is the page’s content or the page’s navigation menu.

Look at what came back:

centinel list
centinel search "budget"

If the extracted text is a cookie banner and a menu, stop and read Reading a document. Collecting ten thousand copies of a navigation bar is the failure this project has spent the most time on.

5. Commit to it

centinel run

Every source, every stage, resumable. Interrupt it and re-run; it starts where it stopped. Run it a second time on an unchanged corpus and it says nothing new in one line.

embed is the one stage that takes real time — on a 400,000-chunk corpus, about a day, once. centinel run --skip embed stops before it, and centinel embed picks it up later. The corpus is keyword-searchable long before it is embedded.

6. Ask it something

$ centinel search "stormwater drainage fee"

Each result carries the passage, the address it came from, when it was observed, which tool derived the text, and a handle — the short hash of the original bytes.

centinel read 3f9a2c1          # the extracted text
centinel open 3f9a2c1          # the original document, in an application

Both take the handle by prefix. See Reading a result.

7. Keep it running

centinel schedule set tampa --cron "0 3 * * *"
centinel serve

serve runs the HTTP and MCP surfaces and fires the configured schedules. Or skip the scheduler entirely and put centinel run in cron — the incremental behaviour is the same either way, because it comes from the store rather than from the runner.

Next: Searching for the user’s path, or Sources for the operator’s.

Searching

centinel search "stormwater drainage fee"
centinel search "budget" --source tampa -n 20
centinel search "lobbyist" --snippet-chars 0        # whole chunk, not an excerpt

What comes back

One ranked passage per result, with everything needed to cite it:

also_at is not decoration. The same paragraph appears on fifty pages of a council site, and each of those addresses is its own document with its own bytes and its own history. Each entry carries its own hash, so you can open any of them.

Two arms, and why the second one matters

BM25 catches exact tokens. Names, motions, ordinance numbers, dollar figures — most of what people actually search meeting records for.

The vector arm closes the vocabulary gap. Measured on a real corpus: "drinking water sampling results" returns nothing from keyword search, because the water report says PWSName, Analyte and UCMR 5, and the only chunk containing the word “drinking” is a tax table about Drinking Places (Alcoholic Beverages). BM25 is behaving correctly and is still useless.

Both run, both return their top 100, the two rankings are fused, and a cross-encoder reranks the survivors. There is no flag to skip the reranker, because the measured gap is too large to make it an option. Search has the mechanism.

Read the header line

stormwater drainage fee    2 results · bm25→rerank · 397,830 chunks indexed
! keyword search only — no vectors at ~/.centinel/vectors.lance — run `centinel embed` first

The method field names which stages actually ran, and it is assembled from what ran rather than written out per call site. Four values are possible:

This is the field to read first. A rank is a position inside a set and says nothing about the size of that set. The fusion weights by rank alone, so the vector arm’s rank 1 counts exactly the same whether it was drawn from 397,830 vectors or from 2,309. A partly embedded corpus therefore does not degrade gently — it promotes confident results from a tiny pool and looks identical to a complete one.

So the report always carries total_chunks_indexed beside vectors_indexed, and the terminal prints the share whenever it is not 100%. no_vectors and no_rerank carry why a stage did not run. An absent stage is a different answer, not a slower one.

What a search cannot tell you

A chunk’s absence from an arm is a fact about what has been processed, never about whether it answers your question. The same holds one stage earlier: a PDF that failed extraction is not in the index at all, and no search will report its absence. centinel list and the run report are where coverage lives.

If you searched for something you are confident is in the corpus and got nothing, the order to check is:

1. Was it collected? centinel list shows resource counts and liveness per source.
2. Was text derived from it? The run report counts unreadable documents per stage.
3. Was it indexed? total_chunks_indexed in the search report.
4. Was it embedded? vectors_indexed in the same report.

Cost

A warm process answers in about a second — the reranker dominates. A cold CLI invocation pays the model load every time: 11 seconds measured on a corpus with no vector table, so with only the 0.6B reranker loaded. A query that also builds the 4B embedder pays more.

centinel serve and centinel mcp load both once and keep them. If you are going to ask more than a couple of questions, ask them through one of those.

Next: Reading a result.

Reading a result

The rule: anything Centinel prints, Centinel takes back.

A citation is only useful if the form on the screen is the form you can type. Printing an identifier the tool then refuses is worse than printing nothing, because it looks like it worked. So search, read and open all lead their provenance line with a handle — the short blob hash — and all three accept it back by prefix, git-style.

centinel read 3f9a2c1          # the extracted text
centinel open 3f9a2c1          # the original document, in an application

read

Prints the derived text: what an extractor made of the bytes. This is the text that was chunked, indexed and embedded, so it is the text a search actually matched against.

Reading it back is the fastest way to answer why did this result look like that. If the extracted text is a navigation menu, you are looking at a page whose content lives somewhere the reader did not reach — see Reading a document.

open

Hands the original file to an application. Configure which one per content kind:

[open]
# Either an application name, or a command template containing {path}.
#   pdf      = "Adobe Acrobat"
#   markdown = "Obsidian"
#   html     = "Safari"
#   text     = "nvim {path}"
#
# "system" hands the file to the OS default handler.
default = "system"

open’s launcher is the one child process Centinel does not kill when its caller exits — it may be somebody’s editor, so it takes the terminal and waits. Every other external program is bounded by a deadline and dies with the process that started it.

Original versus derived

Both are addressable, and they are not the same thing.

A search result carries both hashes, because they answer different questions. blob_sha is what the server gave us and what an archive is for. derived_sha is what the character span indexes into — without it the span is uninterpretable, because it is an offset into one particular extraction and nothing else in the result says which.

Both are valid targets for read and open. Resolving a derived hash means finding the Observation it was derived from and saying so.

Reading verifies

read and open fetch the whole blob and check that it still hashes to its address. An edit in place is an error, not a silent success. That is the point of content-addressing, and it is why classification — which only ever looks at the first few kilobytes — uses a different, unverified read path. A partial read cannot be checked against a whole-file digest, so anything shown to a person or written back into the record uses the whole one.

Where the files are

The store mirrors the URLs under current/<source>/, so a corpus is browsable with ordinary tools. That tree is derived — it is rebuildable from blobs/ and log/, and deleting it costs minutes. The evidence is the blob pool and the log.

Next: From an agent.

From an agent

Centinel is a library, a CLI, an HTTP server and an MCP endpoint. Every verb is reachable from all four, because all four are generated from the same function definition.

Agents sit on top. They are clients of the record, never its author. What gets collected does not depend on what any model happened to think that day.

MCP over stdio

centinel mcp

Point any MCP client at that command. Every op appears in tools/list with a JSON Schema derived from its argument struct — the same struct that produces the CLI’s flags and help text, so the two can never drift.

MCP tool calls wait and return once. Base MCP has no streaming channel for tool results, so a long-running op holds the call open. For a multi-hour crawl, prefer the HTTP streaming route or the CLI.

HTTP

centinel serve --bind 127.0.0.1:8787

curl -X POST localhost:8787/ops/search \
  -H 'content-type: application/json' \
  -d '{"query":"lobbyist meeting log","limit":5}'

Op failures are 400, not 500. Nearly every failure reachable here is a bad argument or an unreachable upstream, which is caller-actionable.

There is no access control, which is why the default bind is loopback. Binding to a non-loopback address logs a warning rather than silently exposing the store. A scheme for this is deliberately unspecified — inventing one here would foreclose the decision.

serve also fires the configured schedules. --no-schedule serves the read API without them, which is what a machine that serves a corpus somebody else collects wants.

Why serve rather than shell out

Both serve and mcp load the embedder and the reranker once and keep them. A short CLI invocation pays the model load on every query — 11 seconds measured with only the 0.6B reranker resident, more once the 4B embedder is also being built. An agent asking a sequence of questions should hold one process open.

Reach

Every op declares a reach — who may cause it to run — and the remote surfaces honour it:

Operator ops change the corpus, so letting a remote caller add a source is letting it choose the corpus one step earlier. Host ops act on the machine — open launches a configured command, models pulls gigabytes — and not even the scheduler may fire them, because a multi-gigabyte download must never ambush a 3am run.

A non-Public op is invisible and also unreachable: the listings filter it out, and the HTTP handler refuses it on call. Hiding alone is not access control.

The JSON is the same JSON

The CLI renders reports for a terminal, but it renders the same erased JSON the HTTP route returns. A terminal can never be shown a field HTTP would not return. So a pipeline can develop against centinel search x --json and move to the HTTP route without re-reading anything.

centinel list                    # a terminal → prose
centinel list | jq '.sources'    # a pipe → JSON, exactly as before
centinel list --json             # force JSON on a terminal

The destination decides the default. --json and --pretty override the format; --color=auto|always|never overrides colour independently, and NO_COLOR is honoured.

Next: Sources.

Sources

A Source is one thing you collect: a website, or a YouTube channel. It is a trait in the code, not an entity with a kind field, and that shows up here as one key in the config.

[[source]]
id   = "tampa"
site = "https://www.tampa.gov"

[[source]]
id                   = "tampa-council"
channel              = "https://www.youtube.com/@CityofTampa"
audio_if_no_captions = true

site versus channel is the whole difference. Everything downstream — extraction, chunking, indexing, search — is a shared model, because acquisition is the only place the two genuinely differ.

For a website, site is any URL on it; only the origin is used.

Adding one

centinel source add tampa --site https://www.tampa.gov
centinel source add tampa-council --channel https://www.youtube.com/@CityofTampa
centinel source list
centinel source remove tampa

source add writes into whichever config file was found, and into ~/.centinel/centinel.toml when none was — beside the store the same command collects into.

Before adding a host you have not collected before, run centinel investigate <url>. It reports what recognises the host, on what evidence, and roughly how much is behind it. See Strategies.

Where the config lives

Nearest answer wins:

1. $CENTINEL_CONFIG
2. ./centinel.toml
3. ~/.centinel/centinel.toml
4. ~/.config/centinel/config.toml

A per-project centinel.toml still wins, so a checkout travels with its own sources. centinel doctor prints which file was used and which store root it named.

There is a starting point at contrib/centinel.toml.example.

Defaults

[defaults]
rps = 1.0                                  # requests per second, per host
embed_model = "qwen3-embedding-4b"
transcribe_model = "whisper-large-v3-turbo"
lang = "en"

rps is deliberately slow. Politeness is per host, which is also why acquisition runs per source rather than corpus-wide.

Where the store lives

~/.centinel, unless something says otherwise. Nearest answer wins:

It is in $HOME because a store is a corpus you keep, not an artefact of the directory you were standing in. This defaulted to .centinel in the working directory once, and the result was that every shell got its own corpus: a separate blob pool, a separate log, a separate index, none of them answering a search against the others, and none of it visible until a search from one directory up came back empty.

The config is intent; the store is fact

They can disagree, and that disagreement is a real state you will hit.

Running centinel discover --source hillsborough --site … by hand collects a source the config never named. run then ignores it — correctly, because nothing declared it. Left alone, that is an invisible corpus: collected, indexed, searchable, and never refreshed.

So source list reports the union and marks what the config does not name:

$ centinel source list
   source        kind  resources             target
✓  tampa         site      1,847             https://www.tampa.gov
   hillsborough  site        412  untracked  https://www.hillsboroughcounty.org

1 source is in the store but not in the config — `centinel run` skips it.
  centinel source adopt

Those addresses are read back out of the log, not guessed. A DiscoveryRun records its method — sitemap, playlist — and the resources say where from. A channel is the interesting case: the log records the videos, never the channel they were listed from, but the archived yt-dlp -J document beside each recording carries uploader_url.

centinel source adopt          # write every recoverable one into the config
centinel source add hillsborough    # the same, for one, with no --site needed

A source whose address cannot be recovered is named and skipped, rather than written as a block that would fail on the next run.

One-off addresses

For URLs outside any discovery run:

centinel ingest https://example.gov/some/document.pdf

It fetches into the content-addressed store like any other acquisition. The document then extracts, indexes and embeds with everything else.

Next: The run.

The run

centinel run                      # every source: discover → collect → extract → index → embed
centinel run --source tampa       # one of them
centinel run --limit 50           # bound collection, to try a site before committing an hour
centinel run --skip embed         # stop before the hours-long stage

Typing six stages in the right order is a chore that also has to be got right — index before extract silently indexes nothing — so the order is written down once, and run is the command you actually use.

Two phases

  per source   discover → collect                      network-bound, per-host paced
  then once    extract → transcribe → index → embed    CPU-bound, model-backed

Acquisition is per source because politeness is per host, and because a 403 on one site must not stop the next.

Derivation is corpus-wide because transcribe and embed each build a multi-gigabyte model. With twenty sources, naive per-source chaining spends more time loading weights than embedding. It also fixes an ordering hazard for free: index runs after every source has extracted, so a chunk that appears in two sources is placed against both.

Incremental is inherited, not implemented

Nothing in run diffs anything. Every stage already skips work it has done, and none of them keeps a checkpoint file. The work list is always a subtraction:

So a second run does nothing, at every stage, for the same structural reason the first one was resumable. Kill it at chunk 40,000 and re-run; it starts at 40,001.

That is what makes this the cron command. Twice a day costs one sitemap walk per source plus whatever actually changed, and a run that found nothing says nothing new in one line.

Because a re-crawled site is about 95% identical text, identical chunks hash identically and never reach the embedding model twice.

--limit bounds collection, not discovery

Nothing may silently cap a discovery run. A truncated snapshot of a source’s address set looks exactly like a source that shrank, and the archive would record that as a fact. So --limit applies to how much gets fetched, never to how much gets enumerated.

Where an enumeration does stop on a ceiling, it says so, and the count is printed as at least n. dunedin.gov once printed a checkmark beside 500 addresses against a real 1,625 because that caveat was inferred rather than reported.

Failure is partial, and it is reported

A source that fails is isolated: its remaining stages are skipped, every other source still runs, and the report names which broke.

A stage whose model is not installed is skipped, not failed — an hour of crawling must not be thrown away over a download that was never started, and the stage resumes on the next run once the weights are there.

A corpus-wide stage where some targets failed and others did not is still a failure, and it keeps the numbers of the calls that worked. Half a corpus extracted is still half a corpus extracted.

The report carries both a summary and an error. The summary is the line a person reads — 1 of 19 failed. The error is every failure joined, for a machine. Rendering the second in place of the first shows one source’s error as though it were the whole story.

Reading the numbers

Two kinds of figure, and confusing them records something false:

total_chunks is the size of the whole index. Summing a three-source run’s three answers would report the index as three times its size.

The stages, individually

run performs these in order; each is also its own command, for when you want one:

centinel discover --source tampa --site https://www.tampa.gov --rps 3
centinel collect  --source tampa --limit 50 --rps 5
centinel extract
centinel transcribe
centinel index
centinel embed

collect also takes --match, a coarse substring filter for exploration — --match /assets/ pulls just the documents.

embed has two flags worth knowing before you commit hours:

centinel embed --dry-run       # what would be embedded, without loading a model
centinel embed --limit 100     # sample before committing

--dry-run creates no table. A plan must leave nothing behind.

Next: Schedules.

Schedules

A schedule is a saved run and a cadence — nothing more. It has no arguments of its own that run does not have. If you can type it, you can schedule it.

centinel schedule set tampa-daily --cron "0 3 * * *" --source tampa
centinel schedules
centinel history
centinel serve

centinel serve is what fires them. --no-schedule serves the read API without firing anything, which is what a machine that serves a corpus somebody else collects wants.

Writing one

centinel schedule set <id> [flags]
centinel schedule rm  <id>

On a terminal with no arguments, the CLI asks. Everywhere else the id and cadence are required — the op itself never prompts, because an op that blocks on input cannot be called over HTTP.

Whichever way you write one, schedule set prints the next few fire times in the schedule’s own zone. Three dates settle whether 0 3 * * 1 meant Mondays or the 1st, and that is the last cheap moment to notice it did not.

Why cron, and why a zone

Cron, because a civic record has a shape a fixed interval cannot express: before the Tuesday meeting, overnight, not during business hours.

The zone is part of the schedule rather than the host, because a corpus outlives the machine that collects it, and “3am” means a different instant after a daylight-saving change. Storing the zone keeps the intent rather than the offset.

Jitter exists because forks are the point. If a hundred cities each run the same default cadence, they all hit their upstreams on the same minute.

Schedules live in the config

centinel.toml, as [[schedule]] blocks, beside the [[source]] blocks they name. Not in the store, because the store is fact — what was collected — and a cadence is intent. The same distinction that makes source list report a union.

Creation is a command rather than hand-edited TOML because a hand-written block invites two silent mistakes: a cron expression that parses and means something else, and a source id that does not exist. centinel schedules --check runs exactly the validation serve performs before binding, so you can find out after editing rather than at the next restart.

An invalid schedule refuses to start the server. A scheduler that silently drops the one broken entry is a scheduler you cannot trust the other twenty of.

One lane

Runs do not overlap. One at a time, per store, with the lock on disk — because the CLI is a second process, and centinel run typed by hand while the server is mid-run is the ordinary case, not an edge case.

The queue is depth one per schedule, FIFO, with no merging. A fire that arrives while something is running waits; a second one that arrives while one is already waiting is dropped, because two identical pending runs do the work of one.

Catch-up fires once on startup when a schedule is overdue, never a backlog. A machine that was off for a week does not wake up owing seven runs.

Reading what happened

centinel schedules                  # what is configured, when each next fires, how the last went
centinel history                    # every attempt, newest first
centinel history --failed
centinel history --schedule tampa-daily
centinel history --source tampa
centinel history --since 2026-08-01T00:00:00Z
centinel history --run 8f3c         # one run, by id or unambiguous prefix — the whole report

history covers manual runs too, not only scheduled ones. It is a record of attempts.

One record per attempt, not per success. A crash leaves evidence: the record is written when the run starts, so a run that died is a row that never completed rather than a gap you have to infer.

The run id is its start instant, which is why a prefix is enough to name one.

Additions and subtractions

The history record carries the arithmetic of what each run changed — bytes that entered the corpus, and what the discovery delta was. Nothing is ever removed: a Resource that vanished from a sitemap is a subtraction from the snapshot, not a deletion from the archive. The bytes stay, and the liveness changes.

The counts are in the record; the addresses are in the log. That split is deliberate — a run record that listed every address would be a second copy of the log, and the second copy is the one that goes stale.

Next: Models.

Models

Everything runs locally. Four roles, all Apache-2.0, all pinned to a repository, a commit revision and a SHA-256 per file.

centinel models              # what is installed, what is missing
centinel models pull         # fetch what the pipeline needs
centinel models verify       # re-check digests on disk
centinel models prune        # remove files the registry no longer names
centinel models rm <id>      # remove one

centinel models pull is the fix named by every error that reports a missing weight, and it is spelled in exactly one place in the code. It used to be written out at seven call sites, so a rename would have left six of them naming a command that does not exist.

The roles

Readiness is rolled up per role, not per model, because the registry carries alternates and any one installed model fills its role. See Models in the registry for every entry and every quantization.

Why these sizes

The embedder is big and the reranker is small on purpose, and the reason is where the cost lands.

The embedder is paid once per corpus, in hours. The reranker is paid per query, in milliseconds. On MTEB English Retrieval, Qwen3-Embedding scores 61.83 at 0.6B, 68.46 at 4B and 69.44 at 8B. Nearly the whole gain is 0.6B→4B, and 8B buys about a point for roughly double the embedding time. So the budget goes into the embedder once and into the reranker freely.

Quantization follows the same logic. Q8_0 over Q4_K_M for the embedder, because quantization there is amortised over hours of work rather than paid per query.

Licence decided the family. Centinel auto-downloads weights and forks redistribute them, which rules out EmbeddingGemma (Gemma licence) and Jina’s reranker (CC-BY-NC).

Changing the embedder is a rebuild, not a config edit

The vector table records which model wrote it. A query vector from any other model is refused at open, naming the fix.

This matters more than it sounds like. Vectors from two models live in different spaces and still return a confident ranked list. There is no symptom — no error, no warning, no empty result. Just a worse ordering, forever.

Width is guarded by the schema itself: the column is a fixed-size list of exactly dims floats, so a wrong width cannot be written at all. The two registry embedders have deliberately different widths (2,560 and 1,024) so a swap is loud rather than subtle.

A consequence worth stating plainly: search is never told which embedder to use. It asks the table. A reader configured differently would otherwise have its query refused and quietly fall back to one arm.

Missing weights degrade; they do not refuse

Reranking is always on — meaning there is no flag that silently returns worse results. That is not a promise that a machine with no reranker weights refuses to search.

Missing weights degrade the ordering and say so, in the no_rerank field and in the terminal header. They never turn a query into an error a reader cannot act on. The same holds for an unbuilt vector table: a corpus is keyword-searchable long before it is embedded.

In the pipeline, a stage whose model is missing is skipped, not failed. An hour of crawling must not be thrown away over a download that was never started, and the stage resumes on the next run once the weights are there.

Where they live, and what they cost

Under a host cache, laid out as <root>/<repo>/<revision>/ — the on-disk tree mirrors the Hugging Face repository, so a path on disk is a path in the repo.

A partial download is a .part file, counted toward bytes present and reported as resumable. centinel models is a report and leaves nothing behind: it resolves without creating.

Disk for the vectors themselves, which is separate from the weights: 397,830 chunks × 2,560 dimensions × 4 bytes = 3.79 GiB.

models is a Host op. Not even the scheduler may fire it, because a multi-gigabyte download must never ambush a 3am run.

The runtime

GGUF through llama.cpp for the embedder and the reranker, in-process. GGUF through whisper.cpp for transcription, in a separate binary — see Install for why the two cannot be linked together.

Not ONNX. The onnx-community exports are decoder graphs carrying a KV cache, and CoreML refuses tensors with zero elements — which is exactly what an empty cache is. That makes ONNX permanently CPU-only on Apple Silicon. Measured on the same model both ways: ONNX on CPU gives 5.5 chunks/sec, llama.cpp on Metal gives 18.5.

Because everything runs locally, output quality varies by machine. So the model tier that produced an artifact is part of its provenance — every derivation records the tool, the version and the tier that made it.

Next: When something is wrong.

When something is wrong

Start with centinel doctor. It prints the store root it opened, the config file that named it, the corpus size, which binaries are present, and which pipeline gates are blocked by missing weights. It names the fix beside each gap.

Read the readiness report correctly

A missing binary carries a need, and the three are not the same:

pdftoppm and tesseract are planned. They were once reported as required with zero call sites between them, so a correctly installed machine was told it was not ready. A readiness check that is wrong pessimistically is the kind people learn to ignore.

yt-dlp is the one dependency that reports staleness, because its breakage is predictable rather than surprising — YouTube changes and it ships releases in emergency clusters. doctor warns at ninety days.

The corpus looks collected and holds nothing

This is the failure mode to watch for, and it is silent. Every symptom looks like success: resources found, acquisitions succeeded, liveness live on all of them, every address indexed. The corpus gains hundreds of copies of a navigation menu.

Three real shapes of it:

The page is a wrapper. On tampa.gov, 915 of 1005 pages held their text in a JavaScript var pdfURL, and the HTML we kept was a print notice. The document was at an address nothing had fetched. This is what enclosure scanning exists for — see Reading a document.

The reader took the whole page. hillsclerk.com enumerates 177 addresses without a mistake and hands back 23,213 characters of navigation for a page whose content is one sentence. The fix is the page’s own marked region — <main>, <article> — read before anything guesses.

The strategy was wrong and confident. 75 Resources, 75 successful acquisitions, 75 copies of a menu reading “Preview link expired”, and not one budget figure. This is why investigate prints the evidence for a recognition rather than the verdict alone.

The check, before you commit an hour:

centinel investigate https://host/       # who recognises this, and on what evidence
centinel check https://host/some/page    # what would extraction make of this one document
centinel run --limit 50                  # then look at what came back
centinel read <handle>

investigate and check both store nothing.

A search returns nothing you expected

Work backwards through the pipeline. Each stage can be the answer, and each one reports its own coverage:

1. Collected? centinel list — resource counts and liveness per source.
2. Text derived? The extract report counts unreadable documents and names them.
3. Indexed? total_chunks_indexed in the search report.
4. Embedded? vectors_indexed in the same report, beside it.

Step 4 is the one people miss. RRF weights by rank alone, so a corpus with 2,309 vectors out of 397,830 chunks does not degrade gently — it promotes confident results from a tiny pool and looks identical to a complete one. The terminal prints the share whenever it is not 100%.

A source stopped returning anything

Check liveness. A refusal is recorded as one of four states, and the distinction is load-bearing:

A CloudFront or Akamai 403 would otherwise be indistinguishable from “the site didn’t change”. Recording it as Gone would log a live page as deleted.

If a whole source turns Blocked, slow down. rps in [defaults] is per host and is deliberately low. A descriptive --user-agent measurably reduces WAF 403s.

A count looks too round

An enumeration that stopped on a ceiling reports truncated, and a truncated count is printed as at least n. If you see a suspiciously round number without that caveat, check the version — this was once inferred three different ways and none of them worked.

Extraction found nothing in a PDF

pdf-inspector flagging pages_needing_ocr is a claim about what the reader could decode, not about what the page holds. Reading the first as the second once wrote off 168 of 490 PDFs that had a text layer all along.

There is a fallback — pdftotext — and its job is not to guess again at the same question. It is the admission that the first tool’s silence was never evidence.

A verdict of “nothing could be derived from this” is recorded as an Underivable, carrying the pipeline version that reached it. Bumping that version is how a better extractor gets another go at what an older one gave up on. --refresh re-derives everything, which is expensive and deliberate.

The store is in two places

If searches come back empty from one directory and full from another, you have two stores. The root defaults to ~/.centinel for exactly this reason. centinel doctor prints which root it opened and which config file named it — compare those two lines between the directories.

Things that are safe to delete

Only blobs/ and log/ are truth. Everything else rebuilds.

Derived is not the same as cheap. Backing up the vectors is cp -R; a .lance dataset is an ordinary directory and the copy opens and queries.

Next: The shape.

The shape

A library first. The CLI, the HTTP server and the MCP server are thin consumers of it. Agents are clients, not the engine.

crates/
  centinel-core/    domain model · store · config · op registry · ops · rendering
  centinel-macros/  the #[op] attribute
  centinel/         the binary: CLI, HTTP, MCP
  centinel-whisper/ the transcription worker — links whisper.cpp, and nothing else may
docs/
  SPEC.md           the settled specification
  research/         ~3,850 lines, ~450 primary-source citations
centinel.toml       what to collect, and where to keep it
~/.centinel/        the store, by default

One process, six stages

  per source   discover → collect                      network-bound, per-host paced
  then once    extract → transcribe → index → embed    CPU-bound, model-backed

Each stage is a chapter in this part of the book, in that order.

Where the variation is quarantined

Two ideas carry most of the weight, and both are about keeping a difference in one place.

Source is a trait, not an entity with a kind field. A crawled website and a YouTube channel differ in enumerate, acquire and change_signal. Everything downstream is one shared model. The alternative is a kind field and a match on it at every stage, which is what the codebase actually had before the trait had implementations — nine sites a third kind would have had to find.

A Resource is an address, not a thing in the world. The January 14 council meeting reachable as a Granicus RSS item, an HTML page, a Legistar Matter and a YouTube video is four Resources, and the model makes no claim they are related. Identity resolution across access paths is fuzzy, and a wrong merge silently corrupts the record. Four honest rows beat one confident wrong one.

The half that does not vary lives in acquire: one loop that derives the work list from the log, turns refusals into a status, and keeps the counters — for any Source. So discover and collect are single verbs that name what happens rather than how. There is no centinel youtube, and a third Source kind adds no verb either.

The same shape, four times

The codebase reaches one conclusion repeatedly, and it is worth naming once because everything in this part is an instance of it:

A registry is a list whose elements answer for themselves. It holds no match.

The failure mode is identical every time it is got wrong: adding a case means edits in several places, and the compiler asks for none of them. One missing arm meant every caption track landed on disk as .bin. Another meant a fallback reader was unreachable for the 168 documents it was written for.

Depth, and where the seams are

The interface is the test surface. acquire’s loop is tested through the Source trait by a scripted adapter, which is how resumption, liveness-on-refusal and multi-artifact addresses became testable without standing up HTTP or yt-dlp.

One adapter is a hypothetical seam; two is a real one. The Source trait was drawn before the second adapter existed, and its first shape was wrong: fetch(&Resource) -> Fetched had no possible implementation for a video, which is one address holding metadata, captions and audio. So acquire returns a list.

Next: The store.

The store

<root>/
  blobs/ab/cd/abcd1234…          TRUTH    immutable, content-addressed, pooled across sources
  log/<source>/YYYY-MM.jsonl     TRUTH    append-only: observations, discovery runs, status, derivations
  current/<source>/…             DERIVED  a tree that mirrors the URLs
  centinel.db                    DERIVED  SQLite metadata + FTS5   — the BM25 arm
  vectors.lance/                 DERIVED  LanceDB chunk vectors    — the vector arm

Only the first two are truth. That is what makes the index disposable and the corpus something you can hand to somebody with rsync.

Blobs are pooled across sources — the same PDF on two .gov sites stores once. Logs and trees are per source, so a single city’s corpus stays separable for handoff.

Derived is not the same as cheap

Everything derived is rebuildable; only some of it is rebuildable over a coffee.

Both are safe to delete in the sense that nothing evidentiary is lost, and one of them is a very expensive thing to delete by accident. Backing up the vectors is cp -R.

This distinction cost a whole architecture. The specification originally called for a separate durable embedding cache — a portable append-only file of vectors, on the argument that swapping vector backends should be a re-import rather than a re-embed. That was reversed after measurement: a .lance dataset is already an ordinary directory, cp -R copies it, the copy opens and queries, and a plain scan reads every vector back out. Extracting vectors from Lance is the re-import.

What the cache would have cost is a second write path and a pipeline stage with its own skip predicate — and a wrong skip predicate is the defect this project has paid the most for. So embed writes vectors where search reads them.

Two hashes, because they answer different questions

A page whose only variation is a rotated CSRF token yields a new blob_sha and an unchanged fingerprint — archived faithfully, no change event. Raw-only would produce a new version every recrawl forever. Normalized-only would destroy the ability to prove what the server actually served.

The normalization rules are currently a deliberately naive whitespace collapse, marked as a placeholder.

Head read versus whole read

get_blob reads the whole file and verifies it against its address, because this is an evidentiary archive. blob_head reads the first few kilobytes and verifies nothing, because a partial read cannot be checked against a whole-file digest.

Classification uses the second. Anything shown to a person or written back into the record uses the first.

Replay

One Source’s log, read once and answerable many times. Every derived view — liveness, the latest Observation per Resource, what was derived from what — is an in-memory scan over the records that one disk read produced.

A Replay is a snapshot. It answers what the log said when it was read, so a caller that appends and wants to see the append takes a new one.

The layout is named once

Where each thing lives under the store root is named in the store module and nowhere else. A path spelled out by a caller is a second, unenforced copy of that module’s header.

The root is the identity of the corpus

store does not answer which store. It is handed a root, and config decides it. The default lives in $HOME because it once lived in the working directory, and centinel run from two directories built two corpora that shared no blobs, answered no search against each other, and looked identical from the inside.

Next: The record.

The record

The domain model. Nine types, and most of the design is in what each one refuses to represent.

Source  (trait — acquisition varies, nothing downstream does)
  ├─ SiteSource      enumerate: sitemap    id: URL         signal: content hash    (computed)
  ├─ ChannelSource   enumerate: playlist   id: video id    signal: metadata revision
  └─ ApiClient       enumerate: paged query  id: vendor GUID  signal: LastModifiedUtc (asserted)
                     — not implemented; the shape the first two left room for

DiscoveryRun    full snapshot of the Resource set a run observed
Resource        (source, natural_key) — an ADDRESS
ResourceStatus  Live | Gone | Blocked | Error, + since, consecutive_failures, last_checked
Observation     one successful fetch — ALWAYS backed by a Blob
Blob            content-addressed bytes
Derivation      Blob → Blob edge, carrying tool + version + model tier + anchors
Underivable     a derivation attempted that produced nothing — tool + version + reason
ChangeEvent     materialized index, rebuildable from Observations

An Observation always has bytes

There is no failure variant, by construction. A failed fetch appends nothing — it mutates ResourceStatus in place instead.

Blocked is the load-bearing one. A CloudFront or Akamai 403 would otherwise be indistinguishable from “the page didn’t change” — measured live against real .gov hosts — and recording it as Gone would log a live page as deleted.

The same distinction repeats one level down, in external tools. yt-dlp reporting “video unavailable” is evidence about the video. A missing binary, or a hang that had to be killed, is evidence about this machine and says nothing about the video.

A Derivation always has bytes too, so Underivable exists

Underivable is the peer of ResourceStatus on the derivation side. Without it, “we tried and there was nothing to get” is unrecordable.

That matters because every stage computes its work list by subtraction. If the only recordable outcome were a Derivation, the extract predicate could only ever be a derivation exists — which is never true for an audio file, so every one of them would be read, hashed and re-attempted on every run for the life of the corpus.

The empty blob recorded as derived text is that invariant broken, not a thing that exists. It matters because of which record it is: the pipeline version is carried on the Underivable, so a verdict mis-filed as a Derivation is beyond the reach of the one mechanism for revisiting it. The empty blob is therefore excluded from the extract predicate, and “no bytes” is turned into an Underivable at the write site rather than in whichever reader happened to get it wrong — because every reader can get it wrong the same way.

Pipeline version

Carried on every Underivable. A verdict belongs to one pipeline at one version and says nothing about the next. Bumping it is how a better extractor gets another go at what an older one gave up on.

An append-only log cannot un-write what a past run recorded, so this is the only cheap way back. --refresh over the whole corpus is the expensive one.

What is deliberately not an entity

Document, Transcript and Sitemap are not types. Derived artifacts are Blobs linked by a Derivation carrying tool, version and model tier — so “the source changed” stays mechanically distinguishable from “tesseract was upgraded”. A sitemap is a DiscoveryRun snapshot.

DiscoveryRun is a full snapshot

Not a delta. Resources appearing and vanishing between runs is the discovery delta, computed from two snapshots.

Which is why nothing may silently cap one. A truncated snapshot looks exactly like a source that shrank, and the archive would record that as a fact. run --limit applies to collection and not to discovery for this reason alone.

Where an enumeration genuinely stops on a ceiling, it says so. Truncated is a field on the enumeration, answered by every Source, and it is the one caveat that changes what the count means rather than qualifying it — so a count is printed as at least n wherever it is true.

That field was inferred three ways before it was reported, and none of them worked. A shrinking delta cannot fire on a first run, or on a source that genuinely grew past the cap. A stopped at substring in the warning list starts lying the day the wording changes. The strategy that stopped early is the only thing that ever knew.

Notes

A Note is a line of provenance a Source wants shown, and how it should read. It lets a report print which sitemaps were walked, or which channel tabs returned nothing, without the renderer learning what a sitemap or a tab is.

A new adapter explains itself through this and edits no renderer. The same mechanism carries a strategy’s recognition evidence and its warnings.

Next: Strategies.

Strategies

A strategy answers one question: where are the addresses?

It is a sitemap index, a directory listing, a vendor product that serves its records a particular way. Recognition and enumeration are the same object, and that pairing is the design rather than a convenience.

pub trait Strategy: Send + Sync {
    /// The name recorded on the DiscoveryRun.
    fn name(&self) -> &'static str;

    /// What did you see, and how sure are you? `None` is a valid, common answer.
    fn recognise(&self, seed: &Fetched) -> Option<Recognition>;

    /// Produce the complete address set.
    fn enumerate(&self, seed: &Fetched) -> anyhow::Result<Enumerated>;

    /// Did this strategy name **pages**, or did it name **documents**?
    fn addresses_are(&self) -> Addresses { Addresses::Pages }
}

• You cannot add a strategy without teaching it to recognise itself.
• You cannot recognise a shape you cannot then handle.

Without that invariant the registry rots into a set of confident half-answers, and a confident half-answer is the most expensive thing in this system.

The unit of contribution is a strategy, never a site

A strategy keys on a product, a framework, a server default, or a standard. Never on a jurisdiction.

pub enum Keyed {
    Product(&'static str),        // "Hyland OnBase Agenda Online"
    Framework(&'static str),      // "ASP.NET WebForms + Telerik RadGrid"
    ServerDefault(&'static str),  // "IIS directory index"
    Standard(&'static str),       // "sitemap.xml"
}

Every one of those ships to many cities, which is what makes the work amortise. Recognising Hyland OnBase collects every city running OnBase; teaching it Tampa collects Tampa. Keyed has no Jurisdiction variant, so the rule is enforced by the type rather than by review.

A strategy that could key on a city is a fork with extra steps.

Two sightings, not one

A strategy merges when it recognises two deployments. A reviewer can ask a pull request one question — which two hosts does this recognise? — and the answer is checkable. OnBase at one city is a note. OnBase at two cities is a strategy.

More specific wins

pub fn specificity(self) -> u8 {
    match self {
        Self::Product(_) => 0,
        Self::Framework(_) => 1,
        Self::ServerDefault(_) => 2,
        Self::Standard(_) => 3,
    }
}

Lower is more specific, and this is not a tiebreak detail. It is the difference between collecting a site and collecting its front door.

A real host running OnBase also serves a robots.txt, so both the product strategy and the sitemap standard answer for it. The sitemap answer is true — there is a sitemap and it enumerates cleanly. It is also nearly worthless, because the meetings are in a JSON literal that no sitemap names.

A recogniser that keyed on the vendor saw more than one that keyed on a standard every server can satisfy, so it ranks ahead of it.

Pages or documents

addresses_are is the third method and it earns its place.

sitemap names pages, so a page can still hide a PDF inside a viewer and the enclosure scan must run. A product strategy that names documents leaves nothing to find, and running the scan there is exactly what once invented URLs like /251agendaonline/.pdf?documentType=.

Without this method, acquire would have to test the strategy’s name — the match the registry exists to prevent.

The registry holds no match

The registry is a list. Each element tests itself. Nothing dispatches on a kind.

This is content-kind detection one level up: magic bytes are a list of (signature → kind) where each element answers for itself, and a strategy registry is magic bytes for websites. Registration is link-time, so there is nowhere to forget to add one.

Two strategies are registered today — sitemap and listing — with listing covering both IIS and Apache/nginx directory indexes.

Recognition must carry evidence

A wrong recognition is silent, and it produces a run that looks perfect:

75 Resources, 75 successful acquisitions, 75 Observations, liveness all live. Each address is its own placement, so all 75 index. The corpus gains 75 copies of a navigation menu that says “Preview link expired”, and not one budget figure.

So a Recognition carries what was seen, not only a verdict. Five rules follow, and each one comes from a real failure rather than from caution.

1. Evidence, not a verdict. The operator accepts or rejects on what was seen.

2. None is a first-class answer. A registry that always answers is lying.

3. “No collection strategy” is a valid, final finding. A query box behind a server-side CAPTCHA has no Resource set behind it, and generating case numbers would invent a corpus rather than observe one. The registry must be able to recognise a query box and refuse it — a strategy that enumerates nothing on purpose.

4. A recogniser may answer “wrong address”. The visible URL is not always the fetchable one. A third valid output is a crumb: the system you want is on another host, recorded and not followed. One Source per exact host; the operator promotes crumbs, and that is what bounds the recursion.

5. What was recognised is written down, and later disagreement warns. Sites change. When a vendor ships a new version and its fingerprint stops matching, the operator gets a warning — not a silent switch to a weaker strategy and an empty corpus. Same shape as the pipeline version on an Underivable: a verdict belongs to one version and says nothing about the next.

A Recognition also carries warnings, which are carried forward into every run rather than printed once. A host that answers HTTP 200 on its error page is a fact about every future acquisition, not about the moment somebody noticed.

What the operator types

investigate is not a separate feature. It is the registry’s front door — one module, two callers:

$ centinel investigate https://www.hillsclerk.com/

  seed        https://www.hillsclerk.com/  →  200, 212 KB, html
  recognised  sitemap (standard)
              robots.txt allows everything and names a <sitemapindex>
              182 child sitemaps
  crumbs      hover.hillsclerk.com          8 links
              publicrec.hillsclerk.com      2 links

  centinel source add hillsclerk --site https://www.hillsclerk.com/

Three outputs: a strategy with its evidence, a set of crumbs, or nothing said plainly. Promotion needs no new command — source add already does it.

A probe is deliberately small: 25 requests, 500 addresses kept. This is a question asked while deciding whether a host is worth collecting, often about ten hosts in a row, and a walk that takes minutes is one nobody runs twice. A probe that fills its ceiling reports truncated, so a floor is never printed as a total.

Recognition is not reading

The registry was briefly a second answer to how do we get text out of this, and that was a mistake worth recording. hillsclerk.com is recognised by sitemap, enumerates 177 addresses without a mistake, and hands back 23,213 characters of navigation for a page whose content is one sentence. True — and it does not follow that the fix belongs beside a crawl strategy.

Reading belongs to extraction, which already dispatches to an ordered list of readers. A second registry in front of that list opted out of every invariant the list exists to hold.

Next: Acquisition.

Acquisition

Two verbs, one loop.

enumerate produces the complete Resource set a Source declares. A sitemap walk, a playlist listing, a paged query. Always a full snapshot, never a delta.

acquire retrieves everything one address holds. It returns a list, because a video is one address holding metadata, captions and audio. An earlier interface returned one blob, and no adapter could implement it — that mismatch is why the trait sat unimplemented while its job was done twice by hand.

Each returned artifact becomes its own Observation with its own history.

The loop does not vary

discover and collect name what happens, not how. One loop derives the work list from the log, turns refusals into a ResourceStatus, and keeps the counters — for any Source. sources::from_config is the only code that picks an adapter.

So there is no centinel youtube, and adding a third Source kind adds no verb.

The marker

The address whose presence in the log proves a Resource was acquired. The single line on which resumption varies:

Keying resumption on captions would re-fetch a whole catalogue every run, because about 7% of a real council channel has none and never will. An enclosure is the same case one level down: a page whose attachment 404s is still a page we have, and keying on the attachment would re-fetch the page forever.

Refusals

An acquisition that failed carries a Liveness rather than an error type, because the caller’s job is to record what kind of failure this was, not to propagate it. A WAF 403 and a 404 are the same Err and completely different facts. One type covers HTTP and yt-dlp alike.

Enclosures

A page can carry a document at its own address rather than containing it: the PDF a CMS renders in a viewer, an RFQ’s attached drawings. Those are found in the page’s HTML, fetched during acquire, and stored as their own artifacts with their own Observations and histories.

Without this, the page enters the corpus looking collected and carrying nothing. On tampa.gov, 915 of 1005 pages extracted to a date and a print notice, with the proclamation itself at an address nothing had fetched.

One level, same host. The page’s own HTML is scanned; what comes back is not. A second level makes acquisition a recursive crawler with no snapshot to bound it — and that is enumerate’s job, which is where a complete address set comes from.

A strategy that names documents rather than pages skips the scan entirely. See addresses_are.

Politeness

Per host, and deliberately slow. rps = 1.0 by default. Acquisition runs per source for this reason, and because a 403 on one site must not stop the next.

robots.txt is honoured, and a robots denial is recorded as Blocked — refused, not absent. A descriptive User-Agent measurably reduces WAF 403s.

External programs

Every child process goes through one module. That is what makes these true of all of them at once:

• it dies with its caller,
• it carries a deadline,
• it never reads our stdin.

Seven call sites used to make those choices separately, and all seven made none of them.

Deadline versus stall timeout. A deadline bounds total time and suits a call with a known shape — a version probe, a metadata fetch. A stall timeout bounds silence, and is the only workable guard on a job whose honest duration is hours. A transcription still reporting progress after four hours is working; one that has said nothing for ten minutes is wedged.

Heartbeat. Output that proves a child is alive. The whisper worker’s stderr is both its diagnostics and its heartbeat, which is why the stall timer resets on any line rather than only on a progress report.

The one exception is open’s launcher. It may be somebody’s editor, so it takes the terminal and waits.

Content kinds

One word for what a blob is, and deliberately coarser than its format. document covers Word, PowerPoint, OpenDocument, RTF and EPUB, because extraction asks all five the same question.

It is decided from a 4 KB head, so it can only ever answer what the first bytes prove. A .docx and a .pptx are both zip-container until something reads the ZIP central directory at the end of the file. Sharpening the kind — making it say docx — would put a guess in the record at the one point where nothing has read enough of the file to know, and every stage downstream would carry the guess. The precise format is a different question, answered later, by a reader holding the whole verified blob.

The evidence order

classify asks, in order:

1. the declared type — a content-type a server actually sent;
2. the magic bytes;
3. only if both came back empty, the extension off the served address.

A name is the last evidence consulted, never the first. Step 3 is reached only when there is no header worth the word — application/octet-stream is IIS’s default for an extension missing from its MIME map, not a claim about the content — and no evidence in the bytes, because the formats it rescues are the ones whose first bytes are ordinary text.

Without step 3, 2.2 GB of .csv on one Florida clerk’s file server was collected, classified other, claimed by no reader, and recorded underivable in silence. What stays forbidden is a supplied filename outranking a server that declared something real.

A file read off disk has no headers at all, so check infers a type from the extension and says that it did. Presenting a guess as a header would put a filename’s opinion where the archive expects a server’s.

One table, not five

The four questions a content kind answers — what is this, what would a server have called it, what should the file be named, is it worth fetching on its own — are all projections of one table.

They were five tables in three modules that no compiler related to each other, so adding a kind meant ten edits and the compiler asked for none. The arm one of them was missing is why every caption track landed on disk as .bin; the one another was missing would mean the document at the end of a link is never fetched at all. Both failures are silent, and both look like a site that had nothing.

The word in the record stays a string, because the log is append-only and a store written by a newer build holds kinds this one has never heard of.

Next: Reading a document.

Reading a document

Extraction turns collected bytes into searchable text. It dispatches on content kind to an ordered list of readers, tried in order.

pub fn readers_for(kind: ContentKind) -> &'static [Reader] {
    match kind {
        Html                 => &[Reader::Marked, Reader::Readability, Reader::WholePage],
        Pdf                  => &[Reader::PdfInspector, Reader::Poppler],
        Spreadsheet          => &[Reader::Spreadsheet],
        Document | ZipContainer => &[Reader::AnyDoc],
        Captions             => &[Reader::Captions],
        Text | Csv | Json | Xml => &[Reader::Passthrough],
        Markdown | Audio | Other => &[],
    }
}

The empty rows are deliberate. audio goes to transcription; markdown is already derived text; other is bytes nothing here claims.

The order is data

There is one definition of produced nothing, shared by every pair:

fn produced_text(outcome: &Extracted) -> bool {
    outcome.text().is_some_and(|t| !t.trim().is_empty())
}

That used to be three mechanisms — a bool for PDF, a free-text note for HTML, a re-route for documents — which meant each pair decided for itself what failure meant. One of them decided wrong: the code returned before the fallback whenever the primary said Unextractable, which is exactly the verdict the PDF reader files for a PDF whose text layer it cannot see. So the fallback that entry existed for was unreachable by the 168 documents it was written for.

Written once, the predicate can be wrong once. Written per pair, it is wrong per pair.

A fallback is not a second guess

pdf-inspector is primary because it produces markdown, and headings become the chunk heading path. pdftotext is the fallback because flat text beats none.

The reason there is a fallback at all: a page flagged pages_needing_ocr is a claim about what the reader could decode, not about what the page holds. Reading the first as the second wrote off 168 of 490 PDFs that had a text layer all along — an executive order, signed minutes, a 315,000-character action plan.

A fallback is not a second guess at the same question. It is the admission that the first tool’s silence was never evidence.

recovered_by_fallback counts how often the second reader spoke. It counts for every kind with a fallback, not only PDF, which is what makes the HTML pair’s rate visible at all — and it is the number to watch after any change to a primary reader.

The marked region

The part of a page the page itself declares to be its content: <main>, [role=main], #main-content, .main-content, <article> — widest first.

It is the first reader for HTML, ahead of readability, because readability is a guess about where the content is and this is the page’s own answer. 298 of 300 measured documents carry a marker.

That is a rule about HTML, not about a vendor, so it is a Reader in the list and not a registry in front of it. It had its own registry for one commit, and that registry opted out of every invariant the list holds: recovered_by_fallback was hardcoded false for the reader that handles 99% of HTML, a recognised-but-empty read left no note, and two code paths returned different text for the same bytes because only one of them consulted it.

A reader that answers nothing falls through — the same contract every other reader is held to, and it needed no new mechanism to state.

The title

A document’s own name is in <title>, og:title and <h1>, and nowhere in the body.

So it is written into the extracted text as an # H1, not merely recorded beside it. Only the text is searched, and as a heading it enters every chunk’s heading path.

The caption extractor already followed the same rule for the same reason: a recording titled “Mayor Castor 2026 Budget” never says “Castor” aloud, and a proclamation page never says what it proclaims.

Enclosures

A page that carries its document at a separate address is handled during acquisition, not here. By the time extraction runs, the enclosure is its own artifact with its own Observation, and it reads like any other PDF.

What “nothing” is recorded as

An extraction that was attempted and produced nothing is an Underivable, carrying the tool, the version, the reason and the pipeline version.

It is not the empty blob recorded as derived text. That would file the verdict as a Derivation — beyond the reach of the version mechanism that exists to revisit it — and an append-only log cannot un-write the 490 a past run already recorded. So “no bytes” is turned into an Underivable at the write site, because every reader can get it wrong the same way.

Checking one document

centinel check https://host/some/document.pdf
centinel check ./local-file.docx

It runs the same dispatch and prints what came out. Nothing is stored.

This is the fastest way to answer why did this page index as a navigation menu. A file read off disk has no content-type, so check infers one from the extension and says that it did.

Next: Transcription.

Transcription

Speech to text, for recordings with no captions. Local Whisper, in a separate process.

centinel transcribe

The work list is the usual subtraction: blobs derived by the transcriber, subtracted from the audio blobs.

Captions first, audio second

A YouTube video is one address holding up to three artifacts: metadata, captions and audio. Captions are free, already text, and usually good enough. So audio is only fetched when there are none, which is what audio_if_no_captions in a [[source]] block controls.

About 7% of a real council channel has no captions and never will. That is also why resumption keys on the metadata artifact rather than on captions — keying on captions would re-fetch a whole catalogue every run.

Why it is a separate binary

whisper.cpp and llama.cpp each vendor their own copy of ggml, and both export the same ~534 ggml_* symbols. Linked into one binary, the linker keeps one copy and silently resolves the other library’s calls to it. The two versions are not the same.

Measured on identical audio and model, the linked crates the only variable:

It links without a warning, runs without a crash, and transcribes nothing. There is no error to catch.

So centinel links llama.cpp, centinel-whisper links whisper.cpp, and the two meet over a pipe. centinel finds the worker beside itself first, then $CENTINEL_WHISPER_BIN, then PATH.

This is not a preference. It is the reason there are two binaries at all, and installing only one leaves the pipeline silently broken in a way that produces no error.

The models

whisper-tiny is also in the registry, at 39M parameters. It is a smoke test for the pipeline, not an archive.

Q8_0 over f16 here for a different reason than the embedder’s: near-lossless at half the download.

Voice activity detection is not optional polish. Whisper hallucinates confidently over silence, and a hallucinated sentence in a meeting transcript is exactly the kind of false record this whole system is built to refuse.

Audio handling

ffmpeg decodes to 16 kHz mono PCM, which is what Whisper wants. yt-dlp fetches. Both are external programs, so both go through the module that owns child processes — each carries a bound, dies with its caller, and never reads our stdin.

Transcription is the job that needs a stall timeout rather than a deadline. Its honest duration is hours, so bounding total time would kill working runs. Bounding silence works: a run still reporting progress after four hours is fine; one that has said nothing for ten minutes is wedged.

The worker’s stderr is both its diagnostics and its heartbeat, which is why the stall timer resets on any line rather than only on a progress report.

What the transcript carries

A derived blob, linked to the audio blob by a Derivation that names the tool, its version and the model tier. Because everything runs locally, output quality varies by machine — so the tier that produced an artifact is part of its provenance, not an implementation detail.

The video’s title is written into the transcript text as a heading, for the same reason a page’s title is: a recording called “Mayor Castor 2026 Budget” never says “Castor” aloud, and only the text is searched.

Not built yet

Transcript-aware chunking. Agenda-aligned spans and per-chunk timestamps are what turn a search hit into a watch?v=X&t=4271s citation. Today a transcript is chunked like any other text.

Next: Chunking and the index.

Chunking and the index

centinel index

Cuts derived text into chunks and writes them to centinel.db — SQLite metadata plus FTS5, which is the BM25 arm of search.

The unit is a chunk, and its identity is its text

A chunk is a passage of derived text. Its id is chunk_hash: the SHA-256 of the text itself. Not of the document it came from. Not of the address it was found at.

Everything downstream falls out of that one choice.

The same passage on fifty pages is one row in chunk and fifty rows in placement. A council’s standard notice paragraph is embedded once, not fifty times. And a monthly recrawl of a site that is about 95% unchanged produces about 95% identical hashes, so embed only ever sees what genuinely changed.

Chunk geometry is load-bearing far outside chunking

chunk_hash hashes the chunk’s text, and the geometry decides the text.

Change the target or the overlap and you get a wholly different set of hashes. The old chunks stay in the index and every vector in the table is orphaned — a corpus that took a day to embed, silently detached from the text it describes.

So the index records the geometry its hashes were built with, and refuses a change that is not a rebuild.

Placement

Where a chunk sits: which source, which address, which derived blob, which character span, plus the heading trail, the observation time, and the tool that derived it. That is what makes a result citable.

The address is part of a placement’s identity, and the derived blob is not enough on its own. Two pages can extract to byte-identical text — two proclamations issued the same day, once the template is stripped. Every rule that treated one derived blob as one document lost the addresses after the first: the index key collapsed them, and the resume predicate called them done.

That was 285 of 1005 pages collected, extracted, and absent from every search, each citing another page’s URL.

A chunk with several placements is why a search result carries also_at — and why each entry there carries its own hash. A different address is a different document, with its own bytes and its own history, so the handle cannot be inferred from the one above it.

The heading path

Chunks carry the markdown heading trail they sit under. This is why the PDF reader that produces markdown is primary over the one that produces flat text, and why a document’s title is written into the text as an # H1 rather than recorded beside it.

Only the text is searched. A heading that is not in the text is a heading no query can reach.

The write batch

The rows index commits as a unit, and it is one document — because that is the unit the skip predicate subtracts.

A batch is chosen by the skip predicate, not by what makes the writer fastest. Widen it to span documents and a crash mid-batch leaves placements for a document the predicate will nonetheless call done: a page collected, extracted, and absent from every search. That is the exact defect the per address rule exists to prevent.

Narrowing it is merely slow. A commit is a WAL checkpoint and an FTS5 flush, and one per row paid both 450,000 times on a corpus of this size.

The skip predicate

Placements already written, per address. Not derivations, not documents — placements, per address, for the reason above.

Why FTS5, and why it stays

The BM25 arm remains on SQLite FTS5 rather than moving to LanceDB’s own Tantivy index, even though LanceDB ships one. The two arms are deliberately independent stores: either rebuilds without touching the other.

The same reasoning is why LanceDB’s built-in RRF reranker goes unused. It can only fuse arms Lance owns.

Not built yet

Transcript-aware chunking — agenda-aligned spans and per-chunk timestamps, which is what turns a hit into a timestamped citation into a recording.

Next: Embeddings.

Embeddings

The expensive stage. About a day, once, on a 400,000-chunk corpus.

centinel embed --dry-run          # what would be embedded, without loading a model
centinel embed --limit 100        # sample before committing hours
centinel embed                    # the rest; re-run to resume

The model

Qwen3-Embedding-4B, Q8_0 GGUF, 2,560 dimensions, 32K context, Apache-2.0, and published by Qwen themselves. Run in-process through llama-cpp-2 — no server, no sidecar, no second language runtime.

Licence decided the family. Centinel auto-downloads weights and forks redistribute them, which rules out EmbeddingGemma (Gemma licence) and Jina’s reranker (CC-BY-NC).

Size is decided by where the cost lands. The embedder is paid once per corpus in hours; the reranker is paid per query in milliseconds. On MTEB English Retrieval, 0.6B scores 61.83, 4B scores 68.46 and 8B scores 69.44 — nearly the whole gain is 0.6B→4B, and 8B buys about a point for roughly double the embedding time. So the budget goes into the embedder once and into the reranker freely.

Q8_0 over Q4_K_M for the same reason: quantization here is amortised over hours of work rather than paid per query.

The recipe, and why it is written down

Three things a generic embedding wrapper would get wrong, and none of them errors:

1. Last-token pooling, not mean pooling.
2. An instruction prefix on queries only. A query is wrapped as Instruct: {task}\nQuery:{q}; a document is embedded bare. The asymmetry is the model’s — it was trained to treat the relationship as directional.
3. L2 normalization, so cosine similarity is a dot product.

Each one produces plausible vectors when wrong: slightly worse retrieval, no error anywhere, no symptom. So the code carries a test that asserts on semantics rather than on shapes.

Batching is not optional

A llama.cpp context and its KV cache are built per call, not per text.

So the batch is the unit of work, not the chunk. A batch that fails as a unit — usually one over-long chunk — is retried individually, so one bad chunk cannot cost the other 31.

An over-long text is refused, not truncated. A silently shortened chunk would be stored under a chunk_hash covering text that was never embedded, which makes the record lie about what it holds. (The reranker does the opposite, for a reason that is explained there.)

The whole run goes into a single spawn_blocking. Inference would otherwise stall the async runtime, which matters because an HTTP caller’s connection has to survive a multi-hour run.

Resumability is a consequence, not a feature

No checkpoint file. The work list is:

index chunk hashes  −  stored chunk hashes

Kill it at chunk 40,000 and re-run; it starts at 40,001. Lance commits a version per append, so what landed before the kill is there.

--dry-run creates no table. A plan must leave nothing behind.

The table

vectors.lance/    one table, two columns
                    chunk_hash  Utf8
                    vector      FixedSizeList<Float32, dims>

No text, no placements, no source. centinel.db holds those, and a second copy goes out of date the first time the corpus changes. chunk_hash is the join both stores already use.

The model is a property of the table. Its id lives in the schema metadata, and a query vector from any other model is refused at open, naming the fix. Vectors from two models are in different spaces and still return a confident ranked list — there is no symptom but a worse ordering.

Width is guarded by the schema itself: the column is a fixed-size list of exactly dims floats, so a wrong width cannot be written at all. The two registry embedders have different widths (2,560 and 1,024) on purpose.

This also means search is never told which embedder to use. It asks the table. A reader configured differently would otherwise have its query refused and quietly fall back to one arm.

One table, not one per model. Changing the embedder is already a full re-embed rather than a config edit, so a second model is a rebuild.

There is no embedding cache

The specification originally called for one: a durable, portable, append-only file of vectors beside the static files, on the argument that “swapping vector backends is a re-import, not a re-embed.”

That was reversed after measurement. A .lance dataset is an ordinary directory — a manifest, a transaction log, data files. cp -R copies it, the copy opens and queries, and a plain scan reads every vector back out. Extracting vectors from Lance is the re-import. Publishing is a directory copy, and so is backup. Lance’s transaction log handles an interrupted run better than truncating a torn record did.

What the cache cost was a second write path and a pipeline stage with its own skip predicate — and a wrong skip predicate is the defect that has cost this project the most. So embed writes vectors where search reads them.

Cost

A full corpus is 397,830 × 2,560 × 4 bytes.

Not built yet

ANN indexing. With no index, Lance scans flat — exact, and the right answer while the table is small. An IVF_PQ index starts to earn its cost somewhere above roughly 100,000 rows, which is a threshold to measure rather than a number to trust.

MRL truncation. Qwen3-Embedding is Matryoshka, so a narrower index is a prefix slice of a stored vector rather than a re-embed. A reversible decision, deferred until something measures a need for it.

Next: Search.

Search

How a question becomes a cited passage.

query
  ├─ BM25   (SQLite FTS5)          → top 100    instant, no model
  └─ vector (Qwen3-Embedding-4B)   → top 100    one embed call
        └─ RRF fuse (k=60)         → top 40
              └─ Qwen3-Reranker-0.6B → top n    always on

Everything here runs on the machine in front of you. Two model files and two files on disk.

Why both arms

Neither is a warm-up.

BM25 catches exact tokens. Names, motions, ordinance numbers, dollar figures — what people actually search meeting records for. On the BRIGHT benchmark BM25 scores 13.7 against BGE-large’s 13.8. Vector-only search fails hardest on precisely these.

The vector arm closes the vocabulary gap. Measured on the real corpus: "drinking water sampling results" returns nothing from FTS5, because the water report says PWSName, Analyte and UCMR 5, and the only chunk containing “drinking” is a tax table about Drinking Places (Alcoholic Beverages). BM25 is behaving correctly and is still useless. That case is asserted as a test, not described in a comment.

RRF

score(chunk) = Σ  1 / (60 + rank_in_arm)

Top 100 from each arm, fused on chunk_hash, top 40 kept.

Rank-based on purpose. The two arms produce scores on incomparable scales — FTS5’s negated BM25 against a cosine similarity — and normalising them into one number is a hidden weighting. Ranks are what the arms genuinely agree on.

k = 60 keeps the gap between rank 1 and rank 2 small, so agreement between the arms matters more than either arm’s confidence, which is the whole reason to fuse rather than pick.

Ties break on chunk_hash, so the same query twice returns the same order. A HashMap iterates arbitrarily, and two equal-scoring chunks swapping places between runs reads as the corpus having changed.

Reranking, always on

Qwen3-Reranker-0.6B, Q8_0 GGUF, 32K context, Apache-2.0. The weights are a community conversion by ggml-org — the llama.cpp organisation — because Qwen publish GGUF for the embedder only. Digests pin exactly what is fetched.

One command, one answer, and no fast path that silently returns worse results. The measured gap is why: BM25 goes from 14.8 to 33.4 nDCG@10 when reranked, and reranked BM25 beats an expensively-trained reasoning-tuned dense retriever used alone at 29.1. A default that returned the 14.8 would be a footgun.

The architectural consequence is the reason the first stage can be cheap:

A cheap stage that over-fetches, plus a good reranker, beats an expensive retriever alone.

The first stage only has to get the right passage into the top 40. It does not have to rank it. That is why the window is wider than any --limit anyone types.

It is not an embedding model

That is the whole difficulty. Qwen3-Reranker is a causal language model. It emits no vector. It is asked a yes/no question, and the answer is read from the logits at the final position:

score = softmax([logit(no), logit(yes)])[1]     →  P(yes), in [0, 1]

So it runs pooling None and reads logits, where the embedder runs pooling Last and reads embeddings. They share a runtime and nothing else.

Three things must be exactly right, and none of them errors when wrong:

1. The chat template, verbatim — the system line included. A hand-rolled prompt gets a fluent answer to a different question.
2. yes and no as single tokens. More than one piece and the logit being read belongs to a fragment. Checked at load, because a model that fails this cannot be scored at all.
3. Softmax over exactly those two logits, not over the vocabulary. The absolute logits drift with document length; their difference does not.

The softmax is written as a difference, 1 / (1 + exp(no - yes)), rather than two exp calls over a shared denominator. Raw logits can be large enough to overflow to infinity, which yields NaN and an order that depends on the sort’s tie-breaking.

Refused versus truncated

Unlike embed, an over-long document is truncated, not refused.

A passage is one candidate among many, and a shortened judgement is still a judgement, where refusing would drop a result the first stage chose. Nothing here is stored, so nothing can lie about what it covers. That is the entire difference: embed writes a record, and a record must not claim to cover text that was never read.

--source

The BM25 arm filters in SQL. The vector arm cannot: Lance carries no source column, and a chunk has many placements across sources.

So it over-fetches 5×, then post-filters in one query for the whole candidate set. It can still under-fill on a corpus one source dominates — a known limit of the post-filter, not a bug in it.

What a result tells you about itself

A rank is a position inside a set. It says nothing about the size of that set.

RRF weights by rank alone, so the vector arm’s rank 1 counts exactly the same whether it was drawn from 397,830 vectors or from 2,309. A partly embedded corpus therefore does not degrade gently — it promotes confident results from a tiny pool and looks identical to a complete one.

This is the same error shape as pages_needing_ocr: a chunk’s absence from an arm is a fact about what has been processed, never about whether it answers the question.

So every report says what actually happened:

method is assembled from what ran, never hard-coded — it is the one field a reader trusts to know what they are looking at, and a stale literal there is worse than no field. The terminal prints the coverage share whenever it is not 100%.

stormwater drainage fee    2 results · bm25→rerank · 397,830 chunks indexed
! keyword search only — no vectors at ~/.centinel/vectors.lance — run `centinel embed` first

Always on is not the same as always available. The rule forbids a flag that silently returns worse results. It does not promise that a machine with no reranker weights refuses to search. Missing weights degrade the answer and say so; they never turn a query into an error a reader cannot act on. The same holds for an unbuilt vector table: a corpus is keyword-searchable long before it is embedded.

The handle

Every result leads its provenance line with the short blob hash, because anything Centinel prints, Centinel takes back. centinel read <hash> and centinel open <hash> accept it by prefix.

What it uses

lancedb’s default features are the S3, GCS, Azure and OSS object stores — every one a network path out of a machine that nothing is supposed to leave.

Cost

The last row is worth knowing, and the measurement is honest about what it covers: 11.35 s on the Tampa corpus with no vector table, so only the 0.6B reranker was loaded. A query that also builds the 4B embedder pays more. centinel serve and centinel mcp load both once; a short CLI invocation pays on every query.

Next: Ops.

Ops

An op is an ordinary async function. Annotating it puts it on the CLI, in the MCP tool list, and at an HTTP route — with no central registration list to update.

/// List sources in the store with resource counts and liveness.
#[op(group = "corpus")]
pub async fn list(ctx: &Ctx, args: ListArgs) -> anyhow::Result<ListReport> { … }

$ centinel list --max-problems 5              # CLI: flags and help from the same struct
$ curl -X POST localhost:8787/ops/list        # HTTP: JSON in, JSON out
{"jsonrpc":"2.0","method":"tools/list"}       # MCP: JSON Schema from the same struct

The registry

  #[op] async fn search(&Ctx, SearchArgs) -> Result<SearchReport>
        │
        ├── augment_clap ─────────► CLI flags + help text
        ├── schema ───────────────► MCP tool JSON Schema / HTTP request body
        ├── invoke ───────────────► one type-erased call path for all three
        └── render ───────────────► the report, in a terminal's idiom (CLI only)

Registration is link-time, via inventory. There is nowhere to forget to add an op. The binary names no individual op; it iterates the registry — including for --help, which is why an op cannot exist and be invisible.

Why a proc macro rather than build-time codegen or a runtime registry: codegen puts generated source in the tree and makes the definition site not the source of truth; a runtime registry needs an explicit register(…) call per op — exactly the central list this avoids, and exactly the thing people forget. Accepted cost: proc macros degrade error messages, mitigated by keeping the expansion thin.

Ops are thin

Argument validation, a call into the store or into acquisition against a Source, and a serializable result. Behaviour that deserves tests belongs in the library, not in an op body.

No op knows a site from a channel. discover and collect name what happens, not how. Which adapter you get is decided once, from the [[source]] block.

Three axes on every op

Group — pipeline, stage, corpus or host. It decides only the heading the op lists under in centinel --help. Sixteen verbs in one alphabetical column make collect, embed and doctor look like peer choices, when the first two are steps of what run does for you and the third is a health check.

Reach — who may cause it to run:

Two independent booleans would describe four states, and only three exist. The fourth — “the scheduler may fire it and so may any HTTP caller” — is the exact defect this enum exists to prevent, and a pair of booleans leaves it one typo away.

There is a registry-wide invariant rather than a list of names: every op in pipeline or stage must have Reach::Operator. A test over the whole registry covers the op somebody adds next year, not the ones somebody remembered.

Both enforcement points matter: the listings filter non-Public ops out, and the HTTP handler refuses them on call. Hiding alone is not access control.

Long-running — whether the op emits progress.

All three live on the op rather than in the CLI crate, for the same reason registration does: there is nowhere to forget them.

Reports are rendered, not printed

A report is the right shape for HTTP and MCP — a model reads JSON better than it reads a table — and the wrong shape for a person, who gets forty lines of quoted keys where four lines would do.

$ centinel list                    # a terminal → prose
$ centinel list | jq '.sources'    # a pipe → JSON, exactly as before
$ centinel list --json             # force JSON on a terminal
$ centinel search x --pretty | less -R    # force prose into a pager

The destination decides the default. --json / --pretty override the format and --color=auto|always|never overrides the colour, independently. NO_COLOR is honoured and loses only to an explicit --color always.

Rendering reads the same erased JSON invoke produced, so a terminal can never be shown a field HTTP would not return — and a report that skip_serializing_if hides from the wire is equally invisible here. That round-trip means every report type must deserialize from its own serialized form, which is a property any Rust consumer of the HTTP API needs anyway.

Each report implements Render beside its own definition, and there is no structural fallback — a new op will not compile until its report says how it reads. That is the opposite of a central list: forgetting is impossible because the compiler asks at the definition site, in the one place that knows what the numbers mean.

Long-running operations

The hardest case. Ops emit progress one way and never learn who called them.

A ProgressEvent carries an optional id and a unit. Events sharing an id are one unit of work, so a renderer can keep a bar per file plus an aggregate beside it rather than one bar whose meaning shifts underneath the operator. unit: bytes is what turns 312000000/613527539 into 297 MiB / 585 MiB at 18.4 MiB/s.

Both are presentation hints. The op emits them and never learns whether anything drew a bar.

/stream holds the connection open rather than returning a job id — honest for the spine, and the durable job store belongs with scheduling.

Adding one

1. Write the async function in crates/centinel-core/src/ops/.
2. Annotate it with #[op(...)], giving it a group and a reach.
3. Give its args struct clap::Args, Serialize, Deserialize, JsonSchema.
4. Implement Render for its report, beside the report.

There is no step 5. It is now a CLI subcommand, an MCP tool and an HTTP route.

Next: Commands.

Commands

Every verb below is one annotated Rust function, and each is reachable from the CLI, from HTTP at POST /ops/<name>, and from MCP — except where reach forbids it. See Ops.

centinel --help builds this list by iterating the registry, so it is never out of date.

Pipeline

Stages

Each is what run does for you, available on its own.

Corpus

Host

Servers

serve --no-schedule serves the read API without firing any [[schedule]].

Global flags

Output format defaults to the destination: prose to a terminal, JSON to a pipe.

Flags worth knowing

centinel run --source tampa --limit 50 --skip embed
centinel collect --source tampa --match /assets/ --rps 5
centinel embed --dry-run
centinel embed --limit 100
centinel search "budget" --source tampa -n 20 --snippet-chars 0
centinel schedules --check
centinel history --failed --since 2026-08-01T00:00:00Z
centinel history --run 8f3c

--limit on run and collect bounds collection, never discovery — a truncated snapshot of a source’s address set would look exactly like a source that shrank.

--user-agent and --timeout-secs are available on the ops that fetch on your behalf (check, investigate). A descriptive User-Agent measurably reduces WAF 403s.

Handles

search, read and open all print a short blob hash, and all three accept one back by prefix, git-style.

centinel read 3f9a2c1
centinel open 3f9a2c1

Both the original blob_sha and the derived_sha are valid targets.

Models in the registry

Every entry pins a Hugging Face repository, a commit revision, and a SHA-256 per file. The on-disk tree mirrors the repo: <cache>/<repo>/<revision>/<path>.

Readiness is rolled up per role, not per model — the registry carries alternates, and any one installed model fills its role.

Embedding — gates search’s vector arm

qwen3-embedding-4b (default)

Dense retrieval. 2,560-dim Matryoshka, 32K context. Qwen/Qwen3-Embedding-4B-GGUF, first-party, Apache-2.0.

qwen3-embedding-0.6b

Dense retrieval, 1,024-dim. Faster, materially weaker, and a different vector space. Qwen/Qwen3-Embedding-0.6B-GGUF.

The two embedders have deliberately different widths. Changing embedder is a full re-embed rather than a config edit, and distinct dimensions are what make that failure loud instead of silent — the vector column is a fixed-size list of exactly dims floats, so a wrong width cannot be written at all.

Reranking — gates search’s final ordering

qwen3-reranker-0.6b

Second-stage reranking. 32K context. ggml-org/Qwen3-Reranker-0.6B-Q8_0-GGUF, Apache-2.0.

A community conversion by the llama.cpp organisation, because Qwen publish GGUF for the embedder only. Digests pin exactly what is fetched.

It emits no vector — it is a causal LM scored from two logits. See Search.

Transcription — gates the transcribe stage

whisper-large-v3-turbo (default)

Speech to text. Near-large accuracy at about 8× the speed. ggerganov/whisper.cpp.

whisper-tiny

39M parameters. A smoke test for the pipeline, not an archive.

Voice activity — gates the transcribe stage

silero-vad

Voice activity detection. Keeps Whisper from hallucinating over dead air. ggml-org/whisper-vad.

Which runtime loads which

The file extension is what tells the two runtimes apart, and it is checked:

The two cannot be linked into one binary — see Transcription.

Managing them

centinel models              # what is installed, what is missing
centinel models pull         # fetch what the pipeline needs
centinel models verify       # re-check digests on disk
centinel models prune        # remove files the registry no longer names
centinel models rm <id>

centinel models pull is the fix named by every error that reports a missing weight, and it is spelled in exactly one place in the code.

models is a Host op. Not even the scheduler may fire it — a multi-gigabyte download must never ambush a 3am run.

Glossary

The words this codebase uses, and what each one is load-bearing for. A term earns a place here when getting it wrong would produce working code that records something false.

The authoritative version is CONTEXT.md in the repository root, which goes deeper on each entry.

Acquisition

Source — a trait, not an entity with a kind field. The one thing that varies between a crawled website and a YouTube channel, quarantined behind enumerate and acquire.

Adapter — a concrete Source. SiteSource and ChannelSource today.

Enumerate — produce the complete Resource set a Source declares. Always a full snapshot, never a delta.

Acquire — retrieve everything one address holds. Returns a list, because a video is one address holding metadata, captions and audio.

Artifact — one thing retrieved, at its own address. A page is one; a video is up to three.

Enclosure — a document a page carries at its own address rather than contains. One level, same host.

Marker — the address whose presence in the log proves a Resource was acquired. The page itself for a site; the metadata sub-resource for a video.

Refusal — an acquisition that failed, carrying a Liveness rather than an error type.

Content kind — one word for what a blob is, deliberately coarser than its format. Decided from a 4 KB head.

Declared vs inferred type — a content-type a server sent, against one read off a filename. Both feed classification and only the first is evidence.

Note — a line of provenance a Source wants shown, and how it should read.

Crumb — an off-host link recorded, not followed. One Source per exact host; the operator promotes crumbs, and that is what bounds the recursion.

The store

Truth vs derived — only blobs/ and log/ are truth. current/, centinel.db and vectors.lance/ are derived and rebuildable.

Derived is not the same as cheap — centinel.db is minutes; vectors.lance/ is about a day on a 400,000-chunk corpus.

Replay — one Source’s log, read once and answerable many times. A snapshot: it answers what the log said when it was read.

Store root — which store. The identity of the corpus. Defaults to ~/.centinel.

Head read vs whole read — blob_head reads the first few kilobytes and verifies nothing; get_blob reads the whole file and verifies it against its address.

BlobSha vs Fingerprint — the hash of the bytes as served (evidentiary) versus the hash of normalized content (the change signal).

The record

Resource — an address, not a thing in the world. The same meeting reachable four ways is four Resources.

Observation — one successful acquisition, always backed by bytes. There is no failure variant.

Liveness — Live, Gone, Blocked, Error.

Blocked — refused in a way that is not evidence of absence: WAF 403, 429, robots denial, YouTube’s bot wall.

DiscoveryRun — a full snapshot of the Resource set one enumeration observed.

Truncated — an enumeration that stopped on a ceiling rather than on the end of the source. A count is printed as at least n wherever it is true.

Underivable — a derivation that was attempted and produced nothing. The peer of ResourceStatus on the derivation side.

Pipeline version — carried on every Underivable. A verdict belongs to one pipeline at one version and says nothing about the next.

Strategies

Strategy — recognition and enumeration as one object. Answers where are the addresses.

Recognition — what a strategy saw, and on what evidence. The operator accepts or rejects on the evidence.

Keyed — what a strategy keys on: Product, Framework, ServerDefault, Standard. There is no Jurisdiction variant and there will not be one.

Specificity — lower is more specific, and more specific wins. The difference between collecting a site and collecting its front door.

Pass — one enumeration in progress: the queue, the ceilings, and what has been kept. A strategy must not decide what a ceiling means.

Lead — a host nothing recognised, and what was measured about it.

Extraction

Primary and fallback reader — two tools for one kind, tried in order, and the record names whichever one spoke. A fallback is not a second guess at the same question; it is the admission that the first tool’s silence was never evidence.

The order is data — readers_for is a list per content kind, with one shared definition of produced nothing.

Marked region — the part of a page the page itself declares to be its content: <main>, [role=main], #main-content, .main-content, <article>, widest first.

Title — the document’s own name, written into the extracted text as an # H1 rather than recorded beside it, because only the text is searched.

Retrieval

Chunk — a passage of derived text. Its id is the SHA-256 of the text itself.

Chunk geometry — the target and overlap sizes. Load-bearing far outside chunking, because the geometry decides the text and the text decides the hash.

Placement — where a chunk sits: which address, which derived blob, which character span. The address is part of its identity.

Write batch — the rows index commits as a unit. One document, because that is the unit the skip predicate subtracts.

Arm — one retriever feeding the fusion. There are two: BM25 over FTS5, and cosine over LanceDB.

Rank vs pool — a rank is a position inside a set and says nothing about the size of that set.

Method — the name of the pipeline that produced this ordering, assembled from what actually ran.

Always on vs always available — there is no flag that silently returns worse results. That is not a promise that a machine with no weights refuses to search.

Handle — a hash that identifies one blob and that the tool will accept back, by prefix.

Original vs derived blob — the bytes as served versus what an extraction or a transcription produced from them. Both addressable; only the first is an Observation.

The run report

Tally — the numbers one stage produced, folded across however many calls it took.

count vs total — work this run did (two calls add) versus what the store now holds (the last answer wins).

Partial failure — a corpus-wide stage where some targets failed and others did not. Still a failure, and it keeps the numbers of the calls that worked.

Summary vs error — the line a person reads, against every failure joined for a machine.

Host

Need — what a missing binary costs: required, optional, or planned.

Gate — a pipeline stage that a set of weights blocks: search, or transcription. Rolled up per role.

Stale — a binary that is present and working but old enough that breakage is expected. Only yt-dlp answers.

Tool — one invocation of an external program. The only way this codebase starts a child process.

Deadline vs stall timeout — a deadline bounds total time; a stall timeout bounds silence, and is the only workable guard on a job whose honest duration is hours.

Heartbeat — output that proves a child is alive.

Reach — who may cause an op to run: Public, Operator, Host.

The deeper docs

This book is a guide. It is written from the specifications below, which go considerably deeper and are the authority wherever the two disagree.

They live in the repository, not in the book, because they are working documents that change with the code.

In the repository

The research files

Each one is a survey of primary sources behind a decision this book states as settled.

• semantic-search.md — embedders, rerankers, hybrid retrieval, the benchmark numbers quoted in Search.
• pdf-and-ocr.md — what actually reads a .gov PDF, and what a “needs OCR” flag means.
• youtube-and-transcription.md — captions, yt-dlp, Whisper, and voice activity detection.
• crawling-and-sitemaps.md — sitemaps, robots, politeness, and the shapes municipal sites come in.

Reading order

To use it: you are already done. This book covers it.

To operate a corpus you care about: docs/FIELD-NOTES.md, because it is a catalogue of the ways a collection looks successful and holds nothing.

To change the code: CONTEXT.md first, then docs/SPEC.md. The vocabulary is load-bearing, and most of the specification is a series of arguments about what a word is allowed to mean.

To trust it: docs/research/, and the tests. Several of the claims in this book — the vocabulary-gap example, the embedding recipe, the fallback reader — are asserted as tests rather than described in comments, precisely because they fail silently when wrong.

Contributing a strategy

The highest-leverage contribution is a strategy, because it keys on a product, a framework, a server default or a standard, and every one of those ships to many cities.

The bar is two sightings. A reviewer asks one question — which two hosts does this recognise? — and the answer is checkable. See Strategies and docs/STRATEGIES.md §16 for the one-file-per-strategy layout.

MIT licensed. Fork this for your city.