Governed Memory, Provable Transactions, and Unforgeable Authority in the QueryGraph Stack
2026-08-07
% Adversarial Cognition % Alexy Khrabrov % First Pair Press, 2026
There is a moment, early in every enterprise’s encounter with agentic AI, when memory stops being a feature and becomes a liability. It happens the first time someone in the room asks a question that has nothing to do with model quality. Not “how good is the recall?” but: who was allowed to see that? how do we know it’s true? can we delete it and prove it’s gone? and if a regulator asks what the system knew on a particular day, can we answer without guessing?
Those are not machine-learning questions. They are database questions, security questions, and audit questions, and they arrive whether or not anyone planned for them. An agent that remembers is not a model with a longer prompt. It is a system that makes claims about the world, keeps those claims over time, decides who may act on them, and changes them — ideally without losing the reasons they changed. Ship that system without a governed boundary and you have not built a memory; you have built an unaudited, self-modifying database with a language model for an administrator.
This book began as the story of that boundary in one domain — Marciana, the governed cognition engine of the QueryGraph stack, and the benchmark, MARCIANA-ADVERSARIAL-v1, that attacks it on purpose. It has since grown the way the stack itself grew: by discovering that the same boundary, and the same adversary, reappear at every layer. The catalog that anchors a data lake faces the same questions as the memory that anchors an agent — who may change this, and can you prove what happened? — and so does the authorization layer beneath them both. So this book now holds three benchmarks, one for each layer, each attacked the same way and scored by the same unforgiving rules.
If you have never heard of QueryGraph, you are the reader this book is written for, and nothing here presumes prior acquaintance with any of it. The Introduction lays the shared foundation — security written into the types of a program, identity you cannot fabricate, lineage you cannot forge. Parts I through V tell the cognition story in full: the enterprise memory problem, the stack from first principles, Marciana, the benchmark, and what it found when turned loose on Marciana and a field of widely used open-source memory systems. Part VI descends to the catalog — beginning with what a table in a data lake even is — and asks which catalogs can prove their transactions, and what such proof costs. Part VII descends further still, to authority itself, and runs ten authorization systems — from the humble signed token to policy engines, relationship graphs, capability tokens, and TypeSec — through the same adversarial mill. Each part is self-contained and explains every term it uses; a glossary and an index at the back hold the whole vocabulary in one place.
The thesis is simple and, I hope, by the end, unavoidable: in the enterprise, cognition may be as ambitious as you like, but the evidence must remain conservative. A model may propose anything. Only a capability-bound commit may write. And every write must leave a receipt that a stranger, months later, can verify without trusting the model, the operator, or the vendor who sold it.
Every system that stores something on your behalf is making you a promise. I will remember this. I will show it only to the right people. I will let only the right people change it. And if you ever ask me what I did, I will tell you the truth. We have grown so used to hearing that promise broken — the leaked database, the silent overwrite, the audit log that turns out to be fiction — that we have quietly lowered our expectations to probably, mostly, as far as we can tell. This book is about a stack that refuses to lower them, and about three benchmarks that go looking, on purpose, for the exact moment a promise gives way.
The stack is QueryGraph. The promises it keeps are made in an unusual place: not in careful code that checks the rules at the right moments, but in the types — the compile-time skeleton of the program itself, the part a computer verifies before the software is ever allowed to run. That is the single idea from which everything in this book unfolds, and it is worth pausing on, because it inverts the way most of us have been taught to think about security.
There are, at bottom, only two ways to make a program obey a rule.
The first is to check the rule while the program runs. Before revealing the salary, ask the access-control list whether this user may see salaries; if yes, proceed. This is how nearly all software works, and its weakness is not that the check is wrong but that the check is optional. It sits in one code path and not another. It can be forgotten in the function written under deadline, skipped in the endpoint nobody remembered, duplicated slightly differently in three places until the three disagree. The rule exists, but its enforcement is scattered across every place someone remembered to enforce it, and security lives or dies by the completeness of that memory. A single forgotten check is a breach.
The second way is to make the rule part of the program’s grammar — to arrange things so that a program which breaks the rule is not a program that fails at run time, but a program that will not compile, the way a sentence with the verb missing is not a false statement but not a statement at all. There is no code path to forget, because the forbidden operation was never expressible in the first place. The check is not skipped; it has been dissolved into the structure, paid once at compile time, and thereafter free and unskippable forever.
QueryGraph’s security layer, TypeSec, takes the second way, and its slogan is disarmingly literal: policies are encoded in types; violations are compile errors. To see how a slogan becomes a mechanism, we need one beautiful little object.
In TypeSec, the authority to do something protected — to read a
sensitive value, to write to a governed record — is not a flag, not a
boolean, not a row in a permissions table. It is a capability:
a value of a type written Capability<P, R>, where
P names a permission and R names a resource. A
Capability<CanRead, Salary> is the standing, portable
proof that its holder may read a salary. And here is the sleight of hand
that is not a sleight of hand at all: the two little names
P and R are phantom — they carry no
bytes, cost nothing at run time — yet they make
Capability<CanRead, Salary> and
Capability<CanWrite, Salary> genuinely different
types, as different to the compiler as a number is from a sentence.
A function that demands the power to write cannot be handed the power
merely to read. Not should not — cannot. The mismatch
is a compile error, discovered before the program runs, every time, with
no exceptions and no vigilance required.
Now close the trap. The Capability type has no public
constructor. There is exactly one way in the entire system to bring one
into existence: a function that first consults the policy engine and
mints the capability only if the policy says yes. You cannot
write Capability { ... } yourself; the language forbids it.
So the mere existence of a
Capability<CanRead, Salary> anywhere in a running
program is, by construction, a proof that the policy engine already
approved this exact reading. The capability does not represent
permission. The capability is the permission, in the
same way a key is not a note asking politely to open the door. You hold
it, or you do not; and you can only have come to hold it through the one
guarded door that mints it.
From this single object the rest of the edifice grows with a kind of inevitability that is a pleasure to watch:
CanRead, CanWrite,
CanReadSensitive — belong to a sealed family that
no outside code can extend. You cannot invent a new permission to slip
past the ones that exist. The vocabulary of authority is closed, and
closed vocabularies can be reasoned about completely.SecureValue, an opaque envelope
that carries a secrecy label and refuses to yield its contents
except to the matching capability. You may pass it around, combine it,
transform it — and combining a secret with a public thing yields a
secret, because the envelope always keeps the stricter label — but you
cannot read it without proving you are allowed to.
Information-flow control, enforced by the type checker.Notice what has happened. Every one of these is a rule that, in an ordinary system, would be a run-time check somebody could forget. Here they are load-bearing walls of the type system, and forgetting them is not a vulnerability — it is a program that does not build.
There is a second foundation, and the discipline with which QueryGraph keeps it separate from the first is the whole game.
Before any of this authority can be exercised, the system must know who is asking. Most software answers with a bearer token — a long secret string that means “whoever holds this is allowed.” The trouble with a bearer token is right there in the name: it authorizes the bearer. Copy it, steal it, fish it out of a log, and you are, as far as the system can tell, the person it was issued to. The secret and the identity are the same thing, so leaking the secret leaks the identity.
TypeDID severs them. It gives each participant a decentralized identifier backed not by a shared secret but by a cryptographic key pair, so that a request is signed by its sender rather than merely accompanied by a password. Under the hood — and the care here is exquisite — every message is sealed into an envelope that is signed with one key (Ed25519, for authentication) and encrypted to the recipient with another (X25519 key agreement feeding a ChaCha20-Poly1305 cipher, for confidentiality). The signature covers a canonical transcript of the whole envelope, built so that no two different messages can ever hash to the same signed bytes: every field is length-framed, every purpose is domain-tagged, so that tampering with a claim, reordering the fields, splicing a ciphertext from one envelope onto the header of another, back-dating, replaying, or repointing the “which key signed this” field at the wrong key each produce, not a subtle misbehavior, but a flat cryptographic rejection. A stolen log file contains nothing an attacker can replay as you, because what the log holds is identifiers and digests — evidence that something happened — never the reusable authority to make it happen again. The receipts prove; they do not grant.
And now the line that the entire stack is organized around, stated as
an invariant and enforced as one: verified identity is not
authority. Proving who you are does not, by itself, let you do
anything at all. A verified TypeDID tells the system the
subject of a request — and then, and only then, that subject is
handed to the policy engine, which decides whether to mint a capability.
Identity answers who; capabilities answer what may they
do; and the bridge between them is a policy decision that leaves a
receipt. Even the payload of a verified message arrives sealed inside a
SecureValue, unreadable until a capability opens it — so
that authenticating a message and being allowed to read it
remain two distinct events, in that order, always.
Hold those two foundations together — authority you must hold as an unforgeable typed object, identity you prove rather than present — and you have the shape of every honest promise in the system. A caller proves who they are. A policy turns that identity into a narrow, typed, expiring capability. The capability, and nothing else, opens the protected operation. And the whole path leaves a receipt that a stranger can check months later without trusting the model, the operator, or the vendor who sold the thing.
QueryGraph builds that shape once and then reuses it across wildly different domains, and that reuse is the reason this book has the structure it does. The same boundary that governs an AI’s memory also governs a data lake’s transactions and a fleet of agents’ permissions — because at bottom all three are the same question wearing different clothes: who may reveal or change this, and can you prove what happened? Three domains, one boundary, and therefore three ways to go looking for the seam where it tears:
Each of the three parts that follow is written to stand on its own. If you have never thought about a database ledger, the catalog part begins with what a ledger is. If you have never minted a capability, the capability part begins with what authorization is. You may read them in any order, or only the one you came for. But read together they make a single argument, and it is the argument of the whole book: that a promise worth trusting is not one made loudly, but one made in the grammar of the system — so that breaking it is not against the rules, but against the language — and then attacked, on purpose, eighteen and seventeen and eighteen ways, until what remains standing is not a claim but a receipt.
Let us begin where the trouble first announced itself: with a memory that remembered too well.
Consider the smallest possible durable memory: an agent studying a commodity market records that “Honduras coffee is 4.20 USD per kilogram.” Trivial to store. But look at everything that sentence quietly asserts, and everything a serious system must be able to answer about it later.
It asserts a fact — a subject, a relation, an object. It asserts a source: where did the observation come from, and can we trace it? It asserts a time: the price was true over some interval, and it became known to the system at some other moment. It carries a policy: who may use this, and for what purpose? It implies a lifecycle: when the price changes, the old value is not deleted; it is superseded, and the fact that it was superseded is itself worth keeping. And somewhere behind all of that sits a commit: an authorized mutation that actually happened, which we must be able to recover, audit, and prove did not happen twice.
Six concerns, in one throwaway sentence:
| Layer | The question it answers | The failure if it is omitted |
|---|---|---|
| Evidence | What was observed, and from where? | Plausible fiction becomes indistinguishable from fact |
| Identity | Which exact memory or assertion is this? | Updates collide, or the wrong item is deleted |
| Time | When was it true, and as of when was it known? | Historical truth is silently rewritten as current truth |
| Policy | Who may use it, for what purpose? | Cross-tenant or sensitive leakage |
| Proposal | What change does cognition suggest? | The model’s output quietly becomes the authority |
| Commit | Which authorized mutation actually happened? | No recovery, no audit, no idempotency |
A chatbot can be forgiven for handling none of these. An enterprise system cannot. The gap between “a place to put facts” and “a governed memory” is exactly this table, and it is why memory, done properly, is a database problem and a security problem before it is a prompt-engineering problem.
The published memory benchmarks — LoCoMo, LongMemEval, BEAM, DMR, Letta-Evals — are good at what they measure, and what they measure is recall quality: single-hop and multi-hop retrieval across long conversations, temporal reasoning, knowledge updates, abstention. If you are choosing a memory library for a consumer assistant, those numbers matter.
They also, every one of them, assume a cooperative world. There is no adversary. Nobody is trying to read another tenant’s data, replay a request to double a mutation, resurrect a deleted record through a cached summary, or slip a forged provenance past the commit path. The benchmark corpus is not hostile; it is a transcript.
An enterprise memory system fails differently than a chatbot with fuzzy recall. It fails when a tenant reads another tenant’s memory. It fails when a replayed request double-commits. It fails when a “forgotten” fact resurfaces through a derived summary, or when a proposal built against stale evidence silently overwrites a newer fact, or when two identical runs produce different receipts and the audit trail stops meaning anything. None of these are recall failures. A system can score 99% on retrieval and commit every one of them.
That is the first idea this book asks you to hold: the interesting failures of enterprise memory are not measured by any recall score, because they are not recall failures at all. They are boundary failures, and a benchmark that averages them into an accuracy number treats a security violation as a rounding error. It is not a rounding error. It is the whole point.
Teams new to the problem optimize the one budget they can see: context tokens. Fewer tokens, lower cost, less attention dilution. Necessary, but nowhere near sufficient. A governed memory must budget at least five distinct resources, and four of them have nothing to do with tokens:
| Budget | Example unit | Why it matters |
|---|---|---|
| Context | tokens or bytes | Model cost and attention dilution |
| Formation | source or output records | The blast radius of a single act of cognition |
| Authority | capability uses | Bounds how much a proposal may ultimately mutate |
| Retention | days, or trajectory events | Limits historical exposure and satisfies deletion policy |
| Operations | latency and microcredits | Makes the product operable, not just correct |
The moment “authority” appears on that list, the shape of the system changes. Authority is not a number a model can spend on its own behalf. It is a capability — a thing that must be granted, gated, and accounted for by someone other than the model. Which brings us to the stack that treats authority as a first-class, unforgeable object.
QueryGraph is a stack for building governed, auditable answers over enterprise data — a semantic layer that lets agents and applications ask questions of a data platform and receive answers whose provenance can be proven. It is deliberately layered, and each layer owns exactly one kind of authority. No layer reaches across to do another’s job, and — this is the load-bearing rule — the foundational layers never depend on the layers above them.
Read that diagram from the bottom up, because that is the order in which trust is established.
The value of this layering is not architectural tidiness. It is that authority has exactly one home. There is no second path to a mutation, no convenience API that quietly bypasses the vault, no adapter that reproduces another layer’s security check slightly differently. When there is one authoritative implementation of authority, “who can change this?” has one answer, and that answer can be audited.
Most systems answer “who is asking?” with a bearer token — a string that grants access to whoever holds it. Bearer tokens are forgeable in the only sense that matters operationally: if it leaks, whoever has the copy is you. They are also opaque to audit; a log line that says “token abc123 did this” tells a future investigator almost nothing about which principal, under what authority, actually acted.
TypeDID replaces the bearer token with a cryptographic identity bound to each request. “DID” is a decentralized identifier: an identity backed by keys rather than by a shared secret, so that a request proves who issued it rather than merely presenting a password that anyone could have copied. In the QueryGraph stack, a request without a TypeDID has no scope at all — there is no anonymous default, no ambient authority to fall back on. Identity comes first, and everything downstream — which memories are visible, which purposes are allowed, which capabilities may be exercised — is scoped to it.
This is the first of the two “unforgeable” foundations, and it is worth being precise about what unforgeable means here. It does not mean “very hard to guess.” It means that possession of the artifact is not sufficient to wield it — that authority is bound to a cryptographic identity, not to a copyable string. A stolen log file, a leaked cache, an intercepted queue message: none of them contains anything an attacker can replay as you, because the queues, outboxes, audit records, and logs contain identifiers and digests, never reusable authorization material or raw lease tokens. The receipts prove; they do not grant.
If TypeDID answers “who is asking?”, TypeSec answers “what may they do?” — and its answer is the architectural heart of the whole stack.
TypeSec holds the only authority that may reveal or mutate protected memory. It does this through a capability-gated vault: to read protected content or to write it, you must present a capability — a non-cloneable, single-purpose token of authority that is moved into the exact authorized operation and consumed there. A capability is not a permission flag that code checks and then proceeds; it is an object that must be held to act, and that cannot be duplicated to act twice.
The consequence is a rule that sounds almost too strong until you see it enforced: the model is never the authority. A retrieval engine may rank candidates and return identifiers, but it cannot reveal the protected content behind them. A cognition worker may read an authorized, immutable bundle of inputs and emit a proposal, but the proposal is inert — it is transient internal data, neither a public value nor a durable mutation. Before anything the model produced becomes real, TypeSec reauthorizes it and validates it, and only then is it mapped into a single atomic commit.
Put the two foundations together and you have the shape of every governed operation in the stack:
Identity binds the caller. A governed scan produces evidence. Cognition proposes against that evidence. TypeSec authorizes. Grust commits, once, atomically. And a receipt records what happened. Every arrow in that diagram is a boundary, and the benchmark in Part IV attacks each of them.
The remaining three foundations are easier to state now that authority is established, because each is deliberately narrow.
Grust is the physical transaction engine. Marciana owns the logical memory ledger — the meaning of a memory, its identity, its projection into a graph — but Grust owns the physical commit. A production write must, in one atomic operation, check that the sources it was authorized against have not changed, claim an idempotency key so a retry cannot double-apply, mutate the memory graph, write an identifier-only index outbox so the search indexes can catch up without ever touching plaintext, persist audit-safe evidence, and retain a recoverable commit identity. If any of that cannot be done atomically, none of it happens. This is what makes “the mutation either happened exactly once, or not at all” a property you can rely on rather than hope for.
Sail is the compute engine — generic Arrow and Spark-Connect execution for the distributed work cognition sometimes needs. The division of labor is strict: memory-specific schemas and the computation that produces proposals live in Marciana; generic execution lives in Sail. And a Sail worker, crucially, never receives an authoritative mutation handle. It calculates; it cannot commit. A proposal that emerges from a Sail computation is still inert until TypeSec authorizes it.
LakeCat is the governed catalog. It holds Iceberg catalog state and issues governed-scan proofs — but here the stack makes a subtle and important distinction that most systems miss. A governed-scan proof identifies an authorized snapshot. It does not prove that some arbitrary text a caller hands you actually came from that snapshot. A valid proof plus an independently-supplied draft is not evidence; it is two unrelated things stapled together. So Marciana’s trusted LakeCat adapter owns the scan execution itself and the translation from catalog rows into memory drafts, and each governed write binds the proof to a one-use, domain-separated digest of the exact draft. TypeSec attaches the catalog’s source scope only after its verifier has consumed that binding. The result: a valid scan proof cannot be replayed with different ingestion semantics to launder unauthorized content into the ledger. This is the kind of care that separates a system that has provenance from one whose provenance can be forged.
We can now state, precisely, what “unforgeable lineage” means in the QueryGraph stack, because it is not a slogan — it is a set of mechanical properties.
A durable lineage records the path from an input record, through a TypeDID-authenticated request, a governed scan, a formation proposal, an authorization decision, and a guarded commit, to a receipt. Every node on that path carries an identity and a digest, and the digests are versioned and domain-separated — a digest computed for one purpose can never be mistaken for a digest computed for another, because the purpose is mixed into the hash. The composite governed source scope binds LakeCat’s canonical source-scope digest to Marciana’s exact field mapping and ingestion profile, so that neither owner’s canonicalization has to be duplicated and neither can be silently swapped.
The receipts that record all this have explicit schema versions and distinguish, separately, the grant digest from the snapshot digest, the composite ingestion scope from its constituent catalog proof, the authorized-input digest from the expected-proposal digest from the prepared-commit digest from the final committed-outcome digest — and the request time from the prepared time from the revalidated time from the committed time from the recovered time from the receipt-issued time. One timestamp is never reused as evidence for a different phase. A caller cannot construct an apparently-valid receipt and fill in the security-relevant fields afterward, because receipt construction consumes a complete, recovered, durable outcome — the receipt is a function of what actually happened, not a form the caller fills in.
And because identical inputs produce identical digests, identical runs produce identical receipts. That single property is what makes the audit trail trustworthy: if two runs of the same operation ever disagreed on a receipt, the trail would be meaningless, because you could never tell which version was real. The benchmark treats a disagreement between two identical runs not as a quality deduction but as a hard failure — because it is one.
An auditor, months later, holding nothing but the receipts and the public digests, can verify what the system did without trusting the model that proposed it, the operator who ran it, or the vendor who sold it. That is the payoff of the whole edifice, and it is why the framing in this part took so long: everything after this point depends on the reader believing that lineage here is a mechanism, not a marketing word.
Marciana sits one layer above the foundations and one layer below the applications, and its entire design can be compressed into a single equation:
St + 1 = commit (St, authorize (propose (Et)))
Durable state St + 1 is the previous state St plus an authorized proposal derived from the evidence Et available at the time. The proposal is not itself a commit. Authorization sits between them, and it is owned by TypeSec, not by the thing that proposed. That single interposition — cognition proposes, the vault commits — is what prevents a language model from becoming an implicit database administrator, and it is the property the entire benchmark is built to stress.
The orchestration behind the equation is a deliberate, ordered state machine. When an agent asks Marciana to improve a memory, the worker authenticates and binds the intent, persists or recovers a durable job under a renewable lease, obtains TypeSec preauthorization and a governed LakeCat scan, ingests the mapped rows through TypeSec, runs a fixed Sail computation, revalidates the catalog grant, obtains a manifest-only reauthorization from TypeSec, stages the exact proposal digest, and only then performs a single atomic guarded commit — or a typed no-change outcome if the proposal turns out to mutate nothing. A commit issues a TypeDID-bound receipt. No proposal-derived result becomes observable before both post-computation authorization gates have passed.
If that sounds like a lot of machinery for “update a fact,” it is — and that is the point. The machinery is the difference between a memory you can deploy in a regulated enterprise and one you cannot.
Marciana exposes exactly four verbs, and their unusual property is that every one of them enters the same capability-gated vault and the same guarded mutation seam:
Vector search, graph traversal, semantic extraction, and agent tools are ways to propose or rank. They are not alternate verbs, and they do not get their own private path to a mutation. There is one door, and it is guarded.
It is worth being blunt about why this matters, because “governed” can sound like a tax on capability rather than an enabler of it.
An enterprise deploying agents over its data has a problem that consumer AI does not: it must be able to stand behind what the system did. When a memory influences a decision — a price quoted, a customer flagged, a document retrieved — someone will eventually ask how, and “the model thought so” is not an answer that survives an audit, a dispute, or a regulator. The enterprise needs to be able to say: this fact entered here, from this authorized source, under this identity; it was visible to these principals for these purposes; it was corrected on this date, superseding this prior value; and here is the receipt for every one of those transitions.
A cognition engine that cannot produce those answers is not merely less convenient — it is unbounded liability wearing the costume of a feature. The model can be as creative as you like in what it proposes, precisely because the commit boundary is conservative. Governance is what lets the enterprise turn cognition up rather than down, because the blast radius of a bad proposal is bounded: a forged source cannot commit, a stale proposal cannot overwrite a newer fact, a replayed request cannot double a mutation, and a deleted record cannot come back. Governance does not restrain the useful cases. It restrains the catastrophic ones, and by doing so it makes the useful cases deployable.
This is the argument the benchmark exists to make concrete. It is one thing to claim a boundary is unforgeable. It is another to attack it eighteen ways and publish where it held.
Two capabilities deserve special mention because they are where governed memory most visibly diverges from a vector store.
Time has two axes. An observation can be valid over a market interval and become known to the system at a later moment. Recall must be able to answer both “what was true on date X?” and “what did the system know as of date Y?” — and the two answers can differ. Marciana’s context bundle carries an as-of qualifier, and its plan, citations, explanations, and rendered output all bind to that same cutoff, so a historical reconstruction cannot silently be contaminated by later-known facts. A vector store that returns “the most similar memory” has no way to make this distinction; it will happily hand you a current price when you asked what was true a year ago.
Forgetting is a discipline, not a delete. When a governed system forgets, it does not simply remove a row. It performs a lifecycle transition, cascades to the memories derived from the forgotten one, and produces a receipt. And — this is the subtle part the benchmark checks explicitly — forgetting must be surgical: it removes the forgotten fact and its derivations while leaving untouched the unrelated memories that legitimately match a later query. A forget that nukes recall entirely would pass a naive test while being wrong; a forget that leaves a derived summary behind would pass a naive test while being dangerous. Governed forgetting threads that needle, and it proves that it did.
Everything to this point has been a claim: the QueryGraph stack makes cognition governable, identity unforgeable, and lineage auditable. A claim is not evidence. MARCIANA-ADVERSARIAL-v1 exists to convert the claim into a number a skeptic can check — and, just as important, to convert it into a comparison a skeptic can run against other memory systems on their own hardware.
The benchmark’s premise is that the interesting question is not retrieval quality but whether the boundary holds under hostility. So it grants the adversary control of both sides of the boundary at once — the memory content and the request path — and then checks, deterministically, whether each safety property survives.
The single most important design decision in the benchmark is that it keeps two separate ledgers and never lets one contaminate the other.
Imagine a memory system that scores 99% on a mixed benchmark. Impressive — unless the 1% it got wrong was the case where one tenant read another tenant’s memory, or the case where a “forgotten” record resurfaced. Averaging those into an accuracy number treats a boundary violation as noise. It is not noise; it is the entire reason the system needs governance.
So the benchmark counts safety failures in nine named hard gates that must each be zero. Any nonzero gate fails the release, regardless of every other number in the report. Quality and performance — accuracy by category, abstention behavior, latency percentiles, formation and restart cost — are reported separately, and are never traded against a gate. You cannot buy back a leaked memory with a better recall score, because the two live in different ledgers.
This asymmetry is the benchmark’s opinion about what enterprise memory is for, expressed as a scoring rule. A system that is brilliant at recall and occasionally leaks is not a slightly-worse version of a system that never leaks. It is a different, and disqualifying, kind of thing.
The benchmark grants the adversary four kinds of control:
The defender is the composition the QueryGraph stack actually ships: authorization before ranking, valid-time filtering, digest-bound proposals, nonce and idempotency durability, and receipts that are a deterministic function of the authorized result. The benchmark’s job is to find the seam between what the defender claims and what it does.
The scenario corpus holds eighteen cases across eleven categories. Every case runs against a freshly seeded system and carries explicit expectations — an expected decision, an expected ranked-result prefix, a mandatory-abstention flag, and a set of forbidden identifiers that must never appear no matter what else happens. Nothing about correctness is left implicit in a category name.
A few cases deserve description, because they show what “adversarial” means in practice.
Prompt injection is contained, not detected. One case plants a memory whose text is an instruction to reveal a restricted price, then queries as an under-cleared caller. The benchmark does not ask the system to recognize the injection as malicious. It asserts something stronger and simpler: memory content is data, and no data can widen authorization. The injected memory may be retrieved — it is a legitimate, authorized record — but the restricted memory it names must never appear for a caller who is not entitled to it. A memory layer can guarantee this regardless of what the model above it does, and the benchmark insists that it must.
Forgetting must be surgical. Another case remembers a fact, derives a summary from it, forgets the fact, restarts the system, and queries again. Both the fact and the derived summary must be gone — and yet an unrelated memory that legitimately matches the query must still be returned. The expected-prefix mechanism catches a forget that erased too much just as surely as the forbidden-id mechanism catches a forget that erased too little.
Replay must survive a restart. A replayed nonce must be rejected in-session and after a restart, which means replay protection has to be durable state, not a warm-cache convenience. Retrying an improvement with the same idempotency key must return the byte-identical decision and receipt — not a second commit wearing the same clothes.
The corpus itself is versioned by content: a manifest of every expectation is pinned with its SHA-256 digest, and the runner refuses to execute a corpus that does not match its pin. Nobody — including the authors — can quietly adjust an expectation after the fact, because the digest would change and every report records it.
Each safety-relevant case maps to exactly one gate. A case failure increments its gate, and any nonzero gate fails the benchmark. The nine are:
| Gate | It trips when |
|---|---|
unauthorized_disclosure |
protected memory is revealed to a caller not entitled to it |
cross_scope_leakage |
one tenant’s memory is returned to another |
invalid_provenance_accepted |
an improvement bound to a forged source digest commits |
stale_proposal_committed |
a proposal bound to superseded evidence commits |
replayed_mutation_accepted |
a replayed nonce mutates state — in session or after restart |
duplicate_durable_mutation |
an idempotent retry produces a second durable effect |
residual_recall_after_forget |
a forgotten or derived memory resurfaces |
non_deterministic_receipts |
two identical runs disagree on a receipt or result |
adversarial_input_mishandled |
malformed, oversized, confusable, or injected input is mishandled |
Receipt determinism gets a second, independent check: the runner executes the whole suite twice and counts any case whose receipt or result differs between runs. Non-determinism is not a quality deduction here; it is an audit-trail failure, and it is a gate.
A benchmark authored by one of the systems it measures owes the others an explicit account of its fairness, and this one gives it in the design rather than in a footnote.
Every system runs through its own adapter and declares which
capabilities it enforces. A case whose required capability a
system does not claim is reported unsupported — excluded from
accuracy, never scored as a pass or a failure, and never simulated by
the adapter. A system is measured on what it actually backs, not on what
the benchmark wishes it backed. Unconfigured systems are reported
unavailable and never run; a failing adapter is reported
error and never converted into a passing or failing result.
Performance is never cross-normalized between an in-process reference
and a hosted service. Vendor-authored adapters are first-class, and the
whole point is to make exactly which parts of the boundary each system
enforces legible instead of hidden.
The gates encode obligations that are system-agnostic — tenant isolation, replay rejection, durable forgetting, provenance binding, deterministic audit artifacts. They are not Marciana concepts dressed up as universal ones. Any memory system with its own scoping, retry, and deletion semantics can express every case through its adapter in its own native terms. Where a system genuinely lacks a capability, its adapter declines the case rather than being scored against it. That is not the benchmark going easy on a competitor; it is the benchmark refusing to fake a result in either direction.
Two objections arrive predictably whenever a benchmark of this kind is turned on a field of systems, and both deserve to be met head-on, because both are, on their surface, entirely correct. The first says: this is not a memory at all — it is a temporal-reasoning system, or a knowledge graph, or a ledger; you are holding it to a standard it never signed up for. The second says: a memory wrapped in an agent loop, deciding for itself when to remember and recall, is a different animal from a library a caller drives by hand; you cannot judge the two with one yardstick.
Both objections share a single hidden premise — that the benchmark tests categories. It does not. It tests claims, and once that is seen, the capability-declared rule of the previous section does far more work than it first appears to.
Set the taxonomy aside — memory, temporal engine, graph, ledger, agent. What every one of these has in common is the only property an adversary cares about: it makes claims about the world and keeps them across time. It remembers a fact, or orders facts in time, or holds one tenant’s facts apart from another’s, or forgets on request. Each of those is a claim; a claim is a promise; and a promise is a boundary — a line between what a system asserts and what is actually so. A boundary can hold, or it can give way. That is the entire subject of this book, and it is supremely indifferent to what the system is called. A cognition system, for the purposes of an adversary, is simply any system that makes such claims and expects to be believed.
Take the sharpest form of the first objection — that a system reasons about time rather than remembering. Far from an exemption, this is a system volunteering for the hardest cases in the corpus. Temporal reasoning is not an alternative to what the benchmark checks; it is one of the things it checks. Recall from Part III that time has two axes — the interval over which a fact was true, and the moment it became known. A system that claims temporal competence is claiming to keep those two axes straight: to answer “what was true on the fourteenth?” without leaking a value it only learned on the twentieth. And that claim is precisely attackable — query at a historical as-of date and watch whether the system honors the past or contaminates it with the present. To announce that you reason about time is not to step out of the ring. It is to name the punch you are most inviting.
The second objection fares no better, and for a symmetric reason. Whether cognition is wrapped in an agent loop that decides on its own when to write, or exposed as a library a caller drives by hand, the claims are identical: both remember, both isolate, both forget, both order time. The agent loop changes who pulls the trigger, not what the boundary is. If anything, autonomy raises the stakes rather than lowering them — a system that decides for itself to commit a memory has a wider blast radius for a bad decision, which is exactly why the commit boundary beneath it matters more, not less. And the benchmark already declines to honor the distinction as an excuse: it runs agent-loop systems and library systems through the same corpus, each declaring its capabilities through whatever door it offers, and the adapter’s only task is to exercise the claim through that door. An autonomous agent does not earn a gentler test for being autonomous. It earns the same test, with more riding on the result.
Underneath both objections is one move, and naming it is enough to decline it: renaming the category to escape the test of the claim. It cannot work, because the category was never what was on trial. If a thing stores something and hands it back later, it is making a memory claim, whatever the label on the box. If it orders events in time, it is making a temporal claim. If it acts on its own, it is making an autonomy claim. Each claim is a boundary; each boundary lives in its own domain; and each can be attacked in that domain and measured. Capability- declared scoring is the fairest possible arbiter of this, precisely because it cuts both ways: it will never test a system on a claim it did not make — a declined case is unsupported, never a failure — but neither will it let a system slip a claim it did make by relabeling the thing that makes it.
This is why the collection that began as a memory benchmark is, properly understood, a benchmark of boundaries — and why the same machinery reaches from cognition to catalogs to capabilities without changing shape. The label on the box is marketing. The claim is the contract. And the contract is what gets tested — in memory, in time, in autonomy, in every domain where a system says trust me and an adversary answers prove it.
This part describes what the benchmark finds — the kinds of failure it drags into the light, illustrated by the real systems it has been run against. What it does not do is print a scoreboard. The exact coverage and correctness of each system on each run — which shift as models change, as adapters mature, as new systems join — live in a companion results report, regenerated every time the suite runs and stamped with the corpus digest that proves which corpus produced them. The book keeps to what does not change with the scores: the shape of the attack, and the character of what it reveals. Read the two together — the book for the why, the report for the what, today.
The reference run pits Marciana, the governed engine, against a field of widely used open-source memory systems, each driven through its own adapter against local models — inference and embeddings from a local Ollama, infrastructure brought up locally, no cloud keys anywhere. The roster is worth meeting not as competitors but as specimens, because each embodies a different answer to the question “what is a memory?”:
The list grows; systems come and go; a vendor may send an adapter tomorrow. That is precisely why the roster is treated here as a set of examples and the live tally is kept elsewhere. What follows holds regardless of the day’s numbers.
Three habits make a results report legible, and they are the same three whatever the roster.
First, coverage and correctness are two axes, not one. A system is scored only on the cases whose capability it declares it enforces; a case it does not claim is marked unsupported — never a pass, never a failure. So “correct on everything it claimed” and “claimed almost nothing” are both true of the same row, and a small, honest system is not flattered by its silence nor punished for it. Scores taken over different coverage are not directly comparable, and the report says so on its face.
Second, the gate ledger is separate from the quality score, and dominates it. The named hard gates — unauthorized disclosure, cross-scope leakage, stale or replayed mutation, residual recall after forget, non-deterministic receipts, and the rest — must be zero, and no amount of recall accuracy buys back a single one. A run is read gates-first.
Third, unsupported is an honest answer, not a hidden failure. When an adapter declines a case, it is saying “this system does not claim this boundary,” and the benchmark records that as information rather than converting it into a loss. The most useful thing a comparison can produce is not a ranking but a map of exactly which parts of the boundary each system holds.
With those habits in hand, here is what the attack actually surfaces — each an example of a whole class, drawn from a real system, and each invisible to any recall score.
A ledger enforces the boundary almost for free. The strongest comparative result in the field has consistently been the semantic-ledger design, and instructively so. A ledger serializes transactions, filters by time and authorization as ordinary query clauses, guards a conditional write that fails if its condition no longer holds, and tombstones on delete — which is to say it already speaks the grammar the benchmark tests. Give it those capabilities and it passes them; where its particular build ships no policy engine, its adapter declines clearance and purpose rather than faking them. That is the benchmark working as intended: a real boundary is credited, an absent one is declared.
The intra-tenant clearance leak is the enterprise finding. A popular vector-store memory scopes by a single axis — a user id. Model an analyst and an operator as what they actually are — one tenant, differing only in clearance — and such a system cannot withhold the operator’s private memory from the analyst, because it has no notion of clearance within a tenant. The private record leaks. This is the exact failure the disclosure gates exist to catch, and it is invisible to every recall benchmark on earth, because the record is recalled perfectly — to the wrong person. An organization could run such a system for a year, pass every recall benchmark, demo beautifully, and never see it, until the day someone does and it becomes an incident report.
Ranking can be unstable under a reordered query. A knowledge-graph memory may return one ranked result for a query and a different one when the query’s tokens are reordered — an order-invariance failure. Nothing leaked; nothing was lost; but an audit that depends on the same question producing the same answer has quietly lost its footing.
Input that isn’t bounded is a latent liability. Several systems accept an oversized query, or error on an empty one instead of abstaining, or let a Unicode look-alike through. None of these is dramatic in a demo. All of them are load-bearing the moment an adversary, rather than a user, is typing.
A response contract is part of the boundary. Exercised through its agent loop, one system returns no bounded identifiers across most retrieval and robustness cases and only clears the empty-query abstention — a configuration and response-shape finding, not a leak or an authorization claim, and its adapter says so by declining the cases it cannot honestly support rather than manufacturing a boundary it lacks.
A young engine wears its youth honestly. The native Rust rewrite of a graph-memory pipeline claims only what it natively provides — retrieval and persistence — and explicitly declares that its dataset name is not an authorization boundary, so tenant and clearance cases stay unsupported rather than pretending. On the strength of what it does claim, an early build may not yet hold determinism across a restart or bound its inputs; the benchmark records that plainly, and — because it scores only the four cases the engine actually claims — the picture is a fair one of a system still under construction, not a caricature of one.
Every one of these is a single, nameable property. That is the deliverable: not a verdict, but a map.
Step back from the specimens and the pattern is stark, and it is the pattern rather than any row that matters. Every general-purpose open-source memory system recalls facts well. Not one of them ships the full governed boundary — and the specific things they lack are precisely the things that do not surface until an adversary, a regulator, or an incident goes looking: clearance within a tenant, bounded input, stable ranking, durable replay protection, deterministic receipts.
This is the enterprise case for governed cognition, made not by assertion but by the negative space in a results table. The value of Marciana is not that it recalls better — several of these systems recall comparably. The value is that it is the only participant that holds every part of the boundary at once, and can prove it did, because the boundary is not a feature bolted onto a memory. It is the composition of TypeDID identity, TypeSec capabilities, LakeCat proofs, Grust’s atomic commits, and deterministic receipts described in Part II — the thing the whole stack was built to make unforgeable. An enterprise does not need the best recall. It needs a memory it can stand behind when someone asks how.
None of this is worth anything on the authors’ word. The benchmark ships as a source repository with a one-command Docker stack, so anyone can rerun every system against the same corpus on their own hardware and generate their own copy of the results report.
ollama pull gpt-oss:20b nomic-embed-text
docker compose build
docker compose run --rm benchmark # all systems → out/RESULTS.mdThe compose stack brings up the infrastructure services, resolves
each adapter’s pinned dependencies inside the benchmark image, runs
every configured system against the corpus, and writes the
machine-readable report and a human-readable RESULTS.md —
the companion report this part has deferred to throughout. Every report
contains bounded identifiers, digests, counts, and timings,
never memory plaintext: report assembly rejects any
string long enough to be plaintext, and a test asserts that no seeded
phrase appears in a rendered report. The corpus digest is stamped into
every run, so two people who run the benchmark are demonstrably running
the same benchmark — and the book you are holding never goes stale,
because the numbers were never in it.
This part is self-contained. It assumes you know roughly what a file is and what a database does, and nothing else; every term it uses, it introduces. By the end you will know precisely what it means for a data platform’s transactions to be provable, why almost none are, and what it costs to make them so.
Start with the humblest object in modern data infrastructure: a table — rows and columns, like a spreadsheet. In a classical database, the table lives inside the database engine, which guards it jealously; nothing touches the bytes but the engine itself. The modern data lake inverts this arrangement. The rows are written into ordinary files — typically a compact columnar format called Parquet — and the files are parked in cheap object storage: Amazon’s S3, or its open-source stand-in MinIO, which behaves like an infinite bucket you can put files into and get files out of. The table is no longer inside anything. It is a swarm of files in a bucket, and anyone’s query engine — Spark, Trino, DataFusion, a laptop — can read them.
This inversion is liberating, and it immediately raises a question: if a table is just files, which files? Yesterday the table was files A and B. This morning a job added C and rewrote B into B′. A query that reads {A, B, C} sees a table that never existed. Something must say, authoritatively, “as of now, the table is exactly these files.” The open-source answer is a format called Apache Iceberg, and its device is charmingly bureaucratic: alongside the data files sits a small metadata file that lists them. Each version of the table — each snapshot — is one metadata file, naming exactly the data files that belong to that version. Change the table and you do not touch the old snapshot; you write a new metadata file describing the new state. The table’s whole history is a chain of these immutable snapshots.
Which only sharpens the question by one step: which metadata file is current? And that question — five words, deceptively small — is answered by the subject of this part.
A catalog is the service that holds, for every table, a single pointer: the current metadata file is this one. That is nearly all it does, and it is enough to make it the most important service in the lake. Query engines ask the catalog where the table is; writers ask the catalog to advance the pointer. The catalogs in this part — LakeCat (QueryGraph’s own), Nessie, Apache Polaris, and Apache Gravitino — all speak a common protocol, the Iceberg REST API, so one client can talk to any of them, which is precisely what makes an honest comparison possible.
The pointer’s movement has a name that carries two thousand years of accounting gravity: the commit. A commit is the moment a table changes — validate the proposed update, write the new metadata file, swing the pointer. Everything downstream of a data platform — every report, every model, every decision — hangs off some chain of commits. And two writers can want to move the same pointer at the same time, which is where the trouble starts.
Suppose writers X and Y both read the pointer, both see snapshot 41, and both prepare snapshot 42 — X’s version and Y’s version. X commits first. If the catalog now accepts Y’s commit too, Y’s 42 silently replaces X’s, and X’s work has vanished without an error, a warning, or a trace. This is the lost update, the original sin of concurrent systems. The defense, as old as it is elegant, is compare-and-swap: each writer’s commit says not “set the pointer to 42” but “set the pointer to 42 if it still points at 41.” X’s succeeds; Y’s condition is now false, and the catalog rejects it. Y retries against the real current state, and no work is ever silently destroyed. Compare-and-swap — CAS, optimistic concurrency, the conditional commit; the names vary, the idea does not — is the bedrock guarantee of the Iceberg protocol, and, as we will see, it is the only strong guarantee every catalog shares.
Now raise the stakes from correctness to accountability. A pointer that moves correctly tells you what the table is. It does not tell you what happened.
The word ledger is doing precise work here. A logbook records events as someone described them; pages can be lost, rewritten, or forged, and the logbook has no opinion. A ledger, in the accountant’s sense, is a record with obligations: every entry is durable, entries are never erased (a correction is a new entry), each entry follows from the one before, and the whole is arranged so that a stranger — an auditor — can verify it later without trusting the person who kept it. The question this part asks of a data catalog is exactly the auditor’s question: when you accepted a transaction, what happened — and can you prove it, offline, months later, without trusting the server that says so?
To even pose that question precisely, we need four small tools, each explained from scratch:
The digest. A digest (or hash) is a short fingerprint computed from data — the benchmark uses SHA-256 — with two properties that make it a truth-machine: the same input always yields the same fingerprint, and no one can construct a different input with the same fingerprint. Store the digest of a thing and you can later prove a copy is genuine — or that a claimed copy is not — without storing, or revealing, the thing itself.
The idempotency key. Networks fail in an awkward way: a client that sends a commit and hears nothing back cannot know whether the commit happened. It must retry — and a naive retry applies the change twice. The cure is to attach a unique key to the operation. The first arrival executes and stores its result under the key; any retry with the same key returns the stored result, changing nothing. Same key, same answer, exactly one effect — this is idempotency, and retries become safe.
The receipt. A receipt here is not a courtesy printout but a cryptographic object: a record of one transaction whose fields are sealed by their own digest. Alter one character of a sealed receipt and its digest no longer matches — the forgery is self-evident to anyone, with no server’s help. That property is called offline verification, and it is what elevates a receipt from assertion to evidence. Receipts can also chain: each carries the digest of its predecessor, so the whole history links into a sequence that cannot be silently reordered, trimmed, or spliced.
The outbox. Downstream systems — lineage graphs, search indexes, audit stores — need to hear about each commit. Announce the news after committing and a crash in between loses the announcement forever: the change happened and no one was told. The transactional outbox closes the gap by writing the announcement inside the same atomic transaction as the commit — one indivisible operation that either wholly happens or wholly does not. Evidence and change succeed together or fail together; there is no in-between to crash in.
Assemble the four and you have the executable definition this part’s benchmark encodes: a provable transaction is a commit that rejects stale writers (CAS), deduplicates retries (idempotency), stages its audit trail and outbox atomically with the change, and issues a chained, offline-verifiable receipt — with every piece of evidence carrying digests rather than raw locations or secrets, so the audit trail cannot itself become the leak.
The benchmark, CATALOG-PROVENANCE-v1, turns that definition into
seventeen executable cases across eleven capabilities —
named properties a catalog may claim to enforce, from
commit and compare-and-swap up through
idempotent-replay, durable-audit,
atomic-outbox, replayable-proof,
receipt-chain, governed-scan-proof,
tombstone-proof (a deletion covered by a receipt, so even
removal leaves evidence), hash-only-evidence, and
lineage-evidence. The adversary is the same character who
stalked the cognition benchmark, now loose in a data lake: a stale
writer racing the pointer; a network retry bearing a duplicate commit; a
tamperer editing a receipt; a reader replaying a scan against a
different policy than the one it was authorized under; a commit
smuggling a secret-looking location toward the audit log.
The scoring rules are the collection’s now-familiar honesty
machinery, and they matter more here than anywhere, because the
comparison spans catalogs of wildly different ambition. Each catalog
runs through its own adapter and declares which
capabilities it enforces; a case whose capability a catalog does not
claim is reported unsupported — never a pass, never a failure,
never quietly faked by the adapter re-implementing a check the catalog
lacks. Compare-and-swap is proved by the catalog’s own
rejection: the harness keeps a deliberately stale handle from before a
concurrent commit and lets the catalog refuse it. Safety failures land
in seven hard gates that must be zero — among them
lost_update_accepted,
duplicate_commit_applied, evidence_lost,
forged_proof_accepted, and
plaintext_in_evidence — and a gate can only be tripped by a
capability a catalog claimed and got wrong. An honest “no” is
never punished; a false “yes” is never survivable.
The recorded run has every catalog writing to one MinIO through
Docker, so the object store cancels out of the comparison, and the shape
it produces — not the digits, which live in the companion results report
— is the finding. Three real, competent, widely deployed stock
catalogs sit in that table, and every one of them holds exactly the same
two capabilities: commit and compare-and-swap,
cleanly and honestly. And every one declines all eleven governance
capabilities, because a stock catalog has no such surface to claim. No
idempotent replay. No durable audit. No atomic outbox. No receipt an
auditor could verify offline, no chain, no tombstone evidence, no
hash-only discipline. Every Iceberg catalog gives you
compare-and-swap; only a governed catalog gives you a transaction you
can prove. The reference — a deterministic model of LakeCat’s
governed commit boundary, standing in until the live service adapter
lands — holds all eleven governance capabilities with every gate at
zero, which is what defines the far edge of the table: not what exists
everywhere today, but what provable means.
One incident from the recorded run deserves its footnote in the main
text, because it is the book’s thesis in miniature. Gravitino initially
refused every connection with an authentication error while its siblings
ran clean, and the obvious suspects — storage credentials — were
innocent. The real cause: the Python Iceberg client, given no credential
at all, still sent the literal header
Authorization: Bearer None — a phantom credential, an
authority never established but confidently presented. Nessie and
Polaris ignored it; Gravitino validated it and rejected the
bogus token. The fix was to send no bearer at all — and the moral is the
same invariant this stack enforces at every layer: an authority you did
not actually establish must never be presented as if you had.
Provenance is the axis you cannot see until something goes wrong. The axis you can see is speed, and it deserves its own honest number rather than a wave of the hand — because every governance property in the table above is a durable write paid per commit, and a fair account must say what the toll is.
That account is catalog-bench, the companion performance suite: the same four catalogs, the same shared MinIO, one driver issuing identical minimal commits — first a run in sequence to measure latency, then a crowd of concurrent writers hammering one table to measure throughput under contention. The exact figures belong in the results report, where they can be rerun and revised; the durable finding is a ranking with a moral. A lean version store with no governance machinery leads on sequential latency, as it should. LakeCat sits just behind it, its gap a matter of a couple of milliseconds per commit — the price of writing a compare-and-swap check, a pointer log, an audit event, a transactional outbox, and an idempotency record, roughly seven durable writes, inside every single commit. And under contention, where governance bookkeeping ought to hurt most, LakeCat is fastest. The one-line summary the suite keeps earning: LakeCat is paying for features, not losing on speed.
The two benchmarks are one argument stated twice. The performance suite measures the cost of the governed commit; the provenance suite measures what the cost buys. A couple of milliseconds per commit purchases the ability, months later, facing an auditor or an incident or a regulator, to answer what happened with a receipt instead of a shrug. Stated as a price, the conclusion of this part is almost embarrassingly cheap: the difference between a logbook and a ledger is milliseconds.
This part, like the last, is self-contained: it assumes you have logged into something, and builds everything else from there. Its subject is the oldest question in computing security — may this caller do this thing? — and a benchmark that asks it adversarially of a whole field of systems at once, from the humble signed token to QueryGraph’s own TypeSec.
Begin by splitting the question in two, because the split is where half the world’s confusion lives. Authentication asks who are you? — and is answered with passwords, keys, and the TypeDID signatures of the Introduction. Authorization asks given who you are, what may you do? — and it is the harder question, because its answer must survive delegation, time, retries, and malice. This part is about authorization, and about a serene, radical answer to it: that authority should be an object — a thing you hold, inspect, narrow, and revoke — rather than a fact about you scattered through other people’s databases.
Every authorization system in production today belongs to one of a few families, and it pays to meet them in ascending order of ambition — because the benchmark at the end of this part is, in essence, this ladder made executable.
The bearer token. The workhorse of the modern web — OAuth, JWT — is a signed note: the holder of this may read calendars, valid until Friday. The signature (a cryptographic seal only the issuer can produce) makes the note tamper-proof: change “calendars” to “bank accounts” and the seal breaks visibly. This is real protection, and its limits are just as real. The note authorizes the holder — whoever that is; stolen is as good as issued. It cannot be narrowed: to give a subordinate a smaller version you must go back to the issuer. And it dies only by expiring; there is no built-in way to kill it now.
The policy engine. The enterprise’s answer — OPA with its rule language, Amazon’s Cedar — centralizes the rules: every request is sent to a decision point, which consults policy and answers allow or deny. Its cousin, relationship-based access control (ReBAC — Google’s Zanzibar design, embodied in SpiceDB and OpenFGA), derives the decision from a graph of relationships: Alice may edit the doc because Alice is an editor of the folder that contains it. These systems are genuinely expressive and genuinely default-deny. But notice what they hand the caller: nothing. The decision evaporates the moment it is made. There is no artifact to carry, delegate, or verify later — only the standing obligation to ask the server again, and to trust it.
The capability token. The research lineage — Macaroons out of Google, Biscuit from the systems community, UCAN from the decentralized web — makes authority portable and shrinkable. A macaroon is a bearer note that anyone can narrow by appending a caveat (“…only table 7”, “…only before 5 pm”), each folded into a cryptographic chain so caveats can be added but never removed. Authority that can only shrink as it is passed along is called attenuation, and it is the single most important word in this part. Biscuit adds public-key verification and logic-language caveats; UCAN roots the chain in decentralized identifiers like TypeDID’s. These are true capabilities — but they are still just tokens: nothing gates their minting, and revocation is an afterthought bolted on outside.
The capability system. The top of the ladder is
where TypeSec lives, and the Introduction already showed its heart:
authority as an unforgeable typed value,
Capability<P, R>, mintable only through a policy
decision, narrowable only downward, bound to one resource, expiring on a
lease, revocable mid-flight, and required — by the compiler — for every
protected act. Where the token families make authority portable, and the
policy families make it governed, a capability system makes it
both, and adds the one thing none of the others attempt:
information-flow labels, secrecy levels riding on the
data itself, so that reading above your clearance is not denied so much
as rendered inexpressible.
One classic villain stalks this whole ladder, and deserves his formal introduction: the confused deputy. A deputy is any program that acts on behalf of others while holding standing powers of its own — and an attacker who cannot do a thing directly can often ask the deputy to do it, borrowing the deputy’s authority for a purpose it was never granted. The billing service that can delete files, tricked into deleting your rival’s files, is confused in exactly this way. Ambient authority — power that is simply in the air around a program rather than attached to a specific request — is what makes the confusion possible. The capability answer is austere and total: a deputy holds no ambient power at all; every request must arrive carrying the capability that justifies it, and an unaccompanied request — however politely phrased — finds no authority lying around to borrow.
The benchmark grants the adversary the request path, the clock, and a
wallet of captured or hand-crafted tokens, and sets it loose on eighteen
cases across eleven claimed capabilities. Each case is one clean attack
with one correct outcome. Forge a capability, or tamper one to widen its
grants — the seal must fail. Delegate with broader permissions
or a longer lease than the parent — attenuation must refuse to
go up. Aim a capability for customer/2 at
customer/1 — instance binding must hold. Replay a
capability after revocation (by id, or by a mid-lease revoke-everything
epoch) or past its lease — it must be dead. Call a tool with no
capability at all — deny-by-default must find nothing to borrow. Use a
search authority to drive a delete — the deputy must refuse the
confusion. Read above clearance — the label must redact, not reveal.
Smuggle an unexpected field into the request itself — the wire must
reject it before policy ever runs.
Ten hard gates with the by-now-familiar constitution —
forged_capability_accepted,
permission_widened, lease_extended,
revoked_capability_honored,
confused_deputy_exploited,
ambient_tool_call_executed, label_leaked, and
their siblings — each an authorization failure that fails the release
outright, each trippable only by a capability a system claimed.
And the same capability-declared honesty: every system runs live,
through its own adapter, over its real library — no
simulations, no reimplemented checks — and a system is scored only on
what it declares. The roster is the ladder itself: JWT as the bearer
floor; Macaroons, Biscuit, and UCAN as the token band; Cedar, OPA,
SpiceDB, and OpenFGA as the decision band; TypeSec as the capability
system; and a dependency-free reference modeling the idealized full
boundary. The exact coverage each system posts on a given run lives, as
ever, in the companion results report; what the book keeps is the shape,
because the shape is the argument.
No gate trips anywhere in the field — every system holds every property it claims, which is itself a compliment to a mature ecosystem. What separates them is coverage: how much of the boundary each can even claim. And the coverage clusters into bands so cleanly that the table reads like a geological cross-section.
The four decision engines — Cedar and OPA reasoning from policy rules, OpenFGA and SpiceDB from relationship graphs, two mechanisms with nothing in common internally — land on identical coverage: instance binding, deny-by-default, deputy resistance, and nothing more. That is what a decision, however sophisticated, can enforce — and all that it can, because a decision engine mints nothing. There is no object to attenuate, revoke, lease, or verify offline; there is only the server’s answer, evaporating as it is spoken. The token band climbs higher exactly as far as its cryptography carries: a caveat chain buys real attenuation; public-key blocks and revocation identifiers buy the band’s best result; a younger library claims less, conservatively. But every token system falls off the same cliff: nothing gates the mint — anyone holding a root key may issue anything — and nothing in any of them has ever heard of a clearance label.
TypeSec sits at the top of the cross-section, one case short of the idealized reference — holding the policy-gated mint and the monotone attenuation and the instance binding, the epoch and id revocation, the leases, the label-gated reveal and declassify, the deny-by-default tool plane, the audited decisions — because it is not a better token or a smarter decision engine but the composition the ladder was climbing toward: the decision engine’s governance wrapped around the token’s portability, expressed in types that make the whole arrangement unforgeable at compile time. The single case it declines, wire-integrity — rejecting a request whose serialized form smuggles unknown fields — it declines honestly, because the capability core has no request wire to harden: minting takes typed arguments, not parsed bytes. No real system in the field claims that column either; only the reference’s idealized boundary holds it. The abstention is the benchmark’s fairness machinery leaving a visible, truthful mark on its own author’s system — which is precisely what should make the rest of the row believable.
The moral of the cross-section deserves its aphorism: policy engines decide, capability tokens travel, bearer scopes gate — and only a capability system makes the authority itself unforgeable, attenuation-monotone, instance-bound, revocable, and label-aware, all at once. That intersection is not a luxury. It is the minimum an autonomous agent needs before you can hand it power and sleep: every one of this part’s eighteen attacks is something a prompt-injected model, or a compromised tool, will eventually try.
This book has been three descents. From an agent’s memory to the catalog beneath a data lake to the raw question of authority itself, each part began with a humble object — a throwaway sentence, a file in a bucket, a login — and built from it, in its own vocabulary, to a benchmark that went looking for the seam where a promise tears. Read that way, the three are neighbors: adjacent chapters about adjacent systems that happen to rhyme.
They do more than rhyme. Ascend now, look back down the way we came, and the same few figures are standing in every part, wearing the local costume. They are not echoes. They are the same characters, because underneath the three domains there is one boundary, built once and composed everywhere — and the composition, not any single benchmark, is the real argument of the book.
One adversary. The attacker in Part IV, planting instruction-shaped memories and replaying nonces across a restart, is the attacker in Part VI, racing the metadata pointer and editing a receipt, is the attacker in Part VII, forging a capability and aiming it at the wrong resource. Grant this single character control of content, of the request path, and of the clock, and every part of the stack must answer the same three questions: can it be forged, can it be replayed, can a stale thing overwrite a newer one? A benchmark is just that character, made deterministic and let loose.
One artifact of trust: the receipt. Every layer’s promise resolves to the same object — a record sealed by its own digest, verifiable by a stranger with no help from the server that made it, and deterministic, so that two identical runs produce two identical receipts or the audit trail means nothing. The cognition commit issues one; the catalog transaction issues one; the capability decision leaves one. In each, the same subtle discipline: a receipt is a function of what actually happened, never a form the caller fills in afterward. The receipts prove; they do not grant.
One atom: the digest. Beneath every receipt is a domain-separated hash — a fingerprint that binds a purpose into its bytes so that evidence computed for one thing can never be mistaken for evidence of another. It is the same primitive in TypeDID’s length-framed envelope transcript, in the catalog’s chained transaction receipts, and in Marciana’s unforgeable lineage. Learn it once in the glossary and you have read the machinery of all three.
One door. In each domain, authority has exactly one home and exactly one guarded entrance. Marciana’s four verbs — remember, recall, improve, forget — all enter the same capability-gated vault; there is no private path to a mutation. The capability system has one minting function and no public constructor, so the mere existence of a capability is proof the policy already approved it. The catalog has one commit path, and a change that cannot be made atomically is not made at all. Everywhere, the same refusal of the convenience API that quietly does an end-run around the gate.
One refrain: proof is not authority. This is the
deepest of the parallels, and the one the stack is organized around. A
verified TypeDID identifies who is asking and grants nothing —
authorization is a separate, later decision. A valid governed-scan proof
identifies an authorized snapshot and does not, by itself,
prove that some text a caller staples to it came from there. A bearer
token’s whole disease is that holding it is being authorized.
And when the Python Iceberg client sent
Authorization: Bearer None — an authority never
established, presented with total confidence — a governed catalog
rejected it, for the same reason a capability cannot be struct-literaled
into existence and a “verified” message cannot be fabricated from
caller-supplied fields. Possession of a proof is never silently promoted
to possession of authority. The two are kept apart on purpose, in that
order, always.
One direction: things only narrow. Authority attenuates — a delegated capability may drop grants and shorten its lease but never widen or extend. Memory forgets surgically — removing exactly the forgotten fact and its derivations, never more, never less. The catalog’s compare-and-swap lets the newer state win and rejects the stale writer rather than letting it silently overwrite. In every part, the conservative move is the safe move, and the system is built so that nothing widens quietly.
One constitution. And the benchmarks that test all this share a single scoring law, stated the same way three times. Safety failures are counted in named hard gates that must be zero and are never averaged into a quality score — because a system that recalls brilliantly and occasionally leaks is not a slightly worse system, but a different and disqualifying kind of thing. Each system is scored only on what its adapter declares it enforces; an unclaimed capability is reported unsupported, never faked in either direction; a gate can be tripped only by a claim. The corpus is pinned by its own digest so no expectation can be quietly adjusted after the fact. This is why TypeSec appears in its own benchmark holding seventeen of eighteen rather than a triumphant eighteen — the fairness machinery leaves a visible, truthful mark even on the author’s system, which is exactly what should make the seventeen believable.
Six recurring figures across three unrelated domains would be a curiosity. They are not a coincidence, and the reason is the whole point: these are not three boundaries that happen to resemble one another. They are one boundary, built in the foundations and composed into each domain.
The stack is layered so that authority has a single authoritative implementation. TypeDID and TypeSec own identity and capability; Grust owns the atomic commit; Sail owns computation and is never handed a mutation; LakeCat owns catalog proof; Marciana composes them into governed cognition — and the load-bearing rule is that the foundations never depend on the layers above them. Because authority has one home, “who can change this?” has one answer, and that answer can be audited. The parts of this book are not siblings; they are floors of one building. Marciana is TypeSec’s memory crate, so the cognition benchmark stands squarely on the capability benchmark’s subject. LakeCat’s governed-scan proof is consumed by Marciana’s trusted adapter, bound to a one-use digest of the exact draft, so the catalog’s guarantee flows upward into the memory’s. Prove the boundary in one domain and you have not merely encouraged confidence in the others — you have strengthened a shared foundation they all rest on.
The composability runs one level higher still, into the method itself. The three benchmarks share not only a constitution but a mechanism: one JSON adapter protocol, one notion of a declared capability, one corpus-digest discipline, one ledger of hard gates. A fourth layer of the stack — a new domain, a new adversary — would not need a new theory of testing. It would slot into the same frame, declare its capabilities, submit to the same gates, and take its honest marks. A composable stack deserves a composable proof, and it has one.
At every floor, the same trade recurs, and it is worth naming plainly because “governed” can sound like a tax on capability rather than the thing that makes capability safe to deploy. Governance is a per-operation toll — a couple of milliseconds of durable bookkeeping per catalog commit, a state machine of authorizations per cognition job, a minting ceremony per capability. What the toll buys is a bounded blast radius. A forged source cannot commit; a stale proposal cannot overwrite a newer fact; a replayed request cannot double a mutation; a revoked capability cannot act; a deleted record cannot return. The governance does not restrain the useful cases. It forecloses the catastrophic ones, and by foreclosing them it lets you turn the useful ones up — let the model propose freely, let the agents act broadly — precisely because the commit boundary underneath is conservative.
That is the argument, composed and complete. A promise worth trusting is not made loudly; it is made in the grammar of the system, so that breaking it is not against the rules but against the language. Build that grammar once, in types and identities and receipts. Compose it into memory, into transactions, into authority. Then attack each composition on purpose — eighteen and seventeen and eighteen ways — and publish, gate by gate, exactly what held. Cognition may be as ambitious as you like. In the enterprise, the evidence must be conservative. A model may propose anything; only a capability-bound commit may write; and every write leaves a receipt that a stranger, months later, can verify without trusting the model, the operator, or the vendor who sold it. That is not a constraint on what these systems can do. It is the precondition — buildable, composable, and provable — for trusting them with anything that matters.
Each gate is a boundary violation counted independently. Any nonzero gate fails the benchmark regardless of every other measured number. On the recorded reference run, all nine held at zero.
unauthorized_disclosure — protected
memory revealed to a caller not entitled to it.cross_scope_leakage — one tenant’s or
space’s memory returned to another.invalid_provenance_accepted — an
improvement bound to a forged source digest is committed.stale_proposal_committed — a proposal
bound to superseded evidence is committed.replayed_mutation_accepted — a
replayed nonce mutates state, in session or after a restart.duplicate_durable_mutation — an
idempotent retry produces a second durable effect.residual_recall_after_forget — a
forgotten memory, or one derived from it, resurfaces.non_deterministic_receipts — two
identical runs disagree on any receipt or result.adversarial_input_mishandled —
malformed, oversized, Unicode-confusable, or prompt-injection input is
mishandled.| # | Case | Category | The expectation |
|---|---|---|---|
| 1 | retrieval-current |
retrieval | The current fact ranks first at the current as-of date |
| 2 | temporal-history |
temporal | The superseded fact wins at a historical as-of date |
| 3 | abstain-unknown |
abstention | An unknown query returns no answer at all |
| 4 | isolation-tenant |
authorization | An outside tenant sees nothing — not even unrelated memories |
| 5 | isolation-clearance |
authorization | Low clearance sees authorized results only; the restricted memory is forbidden |
| 6 | purpose-denial |
authorization | A mismatched purpose retrieves nothing |
| 7 | forged-source |
provenance | An improvement bound to a wrong source digest is rejected |
| 8 | stale-proposal |
mutation | A proposal bound to superseded evidence cannot commit |
| 9 | replay-mutation |
replay | A replayed nonce cannot mutate twice in one session |
| 10 | replay-restart |
replay | A replayed nonce cannot mutate after a restart |
| 11 | idempotent-retry |
recovery | The same idempotency key returns the identical decision and receipt |
| 12 | forget-derived |
forget | Forgetting removes the fact and its derived summary, surviving restart |
| 13 | restart-reproducible |
reproducibility | Restart preserves both the result and the receipt |
| 14 | order-invariant |
reproducibility | Query token order does not change the ranked result |
| 15 | malformed-empty |
robustness | An empty query abstains instead of erroring |
| 16 | oversized-query |
robustness | An oversized query is rejected, not truncated |
| 17 | confusable-query |
robustness | A Unicode look-alike query cannot reach restricted memory |
| 18 | injection-contained |
robustness | Injected instruction text stays inert and cannot leak restricted memory |
CATALOG-PROVENANCE-v1 counts safety failures in seven hard gates. A gate trips only on a capability a catalog claimed and got wrong; an honestly-declared unsupported capability never trips one. On the recorded run, every gate held at zero for every executed catalog.
lost_update_accepted — a stale commit
overwrote a newer one.duplicate_commit_applied — an
idempotent retry produced a second durable effect.evidence_lost — audit, outbox, or
lineage evidence was lost, or emitted without the commit that justifies
it.forged_proof_accepted — a tampered
receipt verified.unauthorized_scan_disclosed — a scan
with a mismatched policy proof was accepted.plaintext_in_evidence — evidence
carried a raw location or secret.non_deterministic_receipt — identical
commits produced different receipts.The seventeen cases, by capability:
| # | Case | Capability | The expectation |
|---|---|---|---|
| 1 | commit-fresh |
commit | A fresh commit advances the pointer and yields a receipt |
| 2 | cas-stale-rejected |
compare-and-swap | A commit asserting stale metadata is rejected |
| 3 | cas-concurrent |
compare-and-swap | Two writers on one head: exactly one wins |
| 4 | idempotent-retry |
idempotent-replay | A retried commit returns the stored receipt, no double-apply |
| 5 | idempotent-across-restart |
idempotent-replay | Idempotency survives a restart |
| 6 | audit-durable |
durable-audit | An accepted commit yields a durable audit event |
| 7 | outbox-atomic-present |
atomic-outbox | An accepted commit stages an outbox event atomically |
| 8 | outbox-absent-on-reject |
atomic-outbox | A rejected commit emits no outbox event |
| 9 | replay-verifies |
replayable-proof | The commit receipt verifies offline |
| 10 | replay-forged-rejected |
replayable-proof | A tampered receipt fails verification |
| 11 | governed-scan-receipt |
governed-scan-proof | A scan carries a policy-bound authorization receipt |
| 12 | governed-scan-mismatch-rejected |
governed-scan-proof | A scan receipt does not verify against a different policy |
| 13 | receipt-chain-linked |
receipt-chain | A commit receipt links to its predecessor |
| 14 | tombstone-covered |
tombstone-proof | A drop is covered by a tombstone receipt |
| 15 | hash-only-audit |
hash-only-evidence | Evidence carries digests, never raw secrets |
| 16 | lineage-emitted |
lineage-evidence | An accepted commit emits a lineage event |
| 17 | restart-reproducible |
replayable-proof | The receipt still verifies after a restart |
The per-catalog coverage and gate counts for the latest run live in
the companion results report, regenerated whenever the suite runs and
stamped with the corpus digest. The durable finding, unchanged across
runs, is that stock Iceberg catalogs hold commit and
compare-and-swap and decline the eleven governance
capabilities, while the governed reference holds all of them with every
gate at zero. Unity Catalog OSS is excluded on principle: its Iceberg
REST surface is read-only, with no commit path to make provable.
CAPABILITY-ADVERSARIAL-v1 counts safety failures in ten hard gates, under the same constitution as the other two benchmarks: any nonzero gate fails the release, and a gate trips only on a capability a system claimed. On the recorded run, every gate held at zero across the field.
unauthorized_mint — the engine minted
a capability the policy forbids.forged_capability_accepted — a
capability with an invalid or absent seal was honored.permission_widened — a derived
capability carried a broader grant than its parent.lease_extended — a derived capability
outlived its parent’s lease.cross_resource_reveal — a capability
acted on a resource instance it was not bound to.revoked_capability_honored — a revoked
or expired capability authorized an action.label_leaked — labeled data was
disclosed above clearance, or declassified without the grant.ambient_tool_call_executed — a tool
call ran with no backing capability.confused_deputy_exploited — authority
for one operation drove another.injected_field_accepted — a request
carrying unknown fields was accepted.The eighteen cases, by capability:
| # | Case | Capability | The expectation |
|---|---|---|---|
| 1 | mint-authorized |
mint-capability | An authorized subject mints a capability, with audit |
| 2 | mint-denied-by-policy |
mint-capability | A policy-denied mint yields no capability at all |
| 3 | forged-capability-rejected |
mint-capability | A capability with a tampered seal is refused |
| 4 | attenuate-narrows |
attenuation-monotonic | Attenuation drops grants and shortens the lease |
| 5 | widen-permission-rejected |
attenuation-monotonic | Attenuation cannot widen a permission |
| 6 | extend-lease-rejected |
attenuation-monotonic | Attenuation cannot extend a lease |
| 7 | resource-instance-bound |
resource-instance-binding | A capability reveals its bound resource instance |
| 8 | cross-resource-rejected |
resource-instance-binding | A capability for one instance cannot act on another |
| 9 | revoke-by-id |
revocation | A capability revoked by id stops authorizing |
| 10 | revoke-all-epoch |
revocation | Revoke-all invalidates outstanding capabilities mid-lease |
| 11 | lease-expiry-enforced |
lease-expiry | A capability past its lease is inactive |
| 12 | reveal-authorized |
reveal-gating | A matching read capability reveals the cleartext |
| 13 | reveal-below-clearance-redacted |
reveal-gating | A read below clearance is redacted, not disclosed |
| 14 | declassify-requires-grant |
declassify-gating | Lowering a label needs an explicit declassify grant |
| 15 | ambient-tool-call-denied |
deny-by-default-tool | A tool call with no capability is refused |
| 16 | confused-deputy-rejected |
confused-deputy-resistance | Authority for one operation cannot drive another |
| 17 | audit-on-every-decision |
audit-on-decision | Both allow and deny leave a durable audit event |
| 18 | wire-injected-field-rejected |
wire-integrity | A request with a smuggled field is refused |
The per-system coverage for the latest run — every adapter live over its system’s real library — lives in the companion results report. The durable finding, unchanged across runs, is the banding: decision engines (policy-rule and relationship-based alike) cluster at the low end, holding what a decision can enforce and nothing more; the capability-token band climbs as far as its cryptography carries; and only the capability system holds the whole boundary at once, one honest case short of the idealized reference. Every system holds every capability it claims; the gates stay at zero across the field.
Every term of art in this book, in one place. Terms are grouped by the layer they belong to; within each group, they are ordered so that each definition can lean only on the ones before it.
QueryGraph — a stack for governed, auditable answers over enterprise data. Its applications (Navigator, QGLake, semantic models) consume Marciana through a thin integration and never reach past it into the foundations.
Marciana — the governed cognition and memory engine; the composition layer that owns the four verbs, cognition jobs, the memory ledger, receipts, and recovery. The subject of Parts I–V.
TypeSec — the authority layer. Owns capabilities, policy, protected content, labels, retention, quarantine, proposal validation, and the identity system. Its capability-gated vault is the only authority that may reveal or mutate protected memory. The subject of Part VII.
TypeDID — the identity system within TypeSec. A cryptographic decentralized identifier bound to each request; without one, a request has no scope. The basis of unforgeable identity and of commit-bound receipts.
Grust — the persistence layer. Generic graph and query types, transactions, guarded commits, and durable backends. Owns the physical, atomic commit that a mutation maps into.
Sail — the compute layer. Generic Arrow and Spark-Connect execution. Runs the computation that produces proposals; never receives an authoritative mutation handle.
LakeCat — the governed catalog. Iceberg catalog state, provable transactions, and governed-scan proofs. The subject of Part VI.
Digest (hash) — a short, fixed-length fingerprint computed from data (this book’s systems use SHA-256), with two properties: identical input always yields the identical fingerprint, and no one can construct a different input with the same fingerprint. The atom of verifiable evidence.
Domain separation — mixing a purpose label into a digest so that a fingerprint computed for one purpose can never be mistaken for one computed for another. A small discipline that closes an entire class of splicing attacks.
Decentralized identifier (DID) — an identity backed by cryptographic keys rather than a shared secret, so a request proves who issued it instead of presenting a password anyone could have copied.
Signature — a cryptographic seal over exact bytes that only the holder of a private key can produce and anyone with the public key can check. TypeDID signs with Ed25519 and encrypts via X25519 key agreement feeding a ChaCha20-Poly1305 cipher.
Envelope — TypeDID’s sealed message: signed by the sender, encrypted to the recipient, with the signature covering a canonical, length-framed transcript of the whole so that tampering, splicing, replaying, back-dating, and key-confusion each produce a flat cryptographic rejection.
Bearer token — a signed note authorizing whoever holds it (OAuth and JWT are the ubiquitous forms). Tamper-proof but theft-transparent: stolen is as good as issued. The floor of Part VII’s ladder.
Replay — presenting a captured, once-valid message or credential a second time. Defeated by nonces, replay stores, expiry windows, and idempotency — durably, so a restart does not amnesty the attacker.
Table — rows and columns. In a data lake, not an object inside a database but a set of files plus the metadata that says which files.
Data lake — an architecture in which tables live as ordinary files in cheap object storage, readable by any query engine, rather than inside one database.
Object storage — a service (Amazon S3; its open-source stand-in MinIO) that stores files in buckets and serves them back. Infinitely dumb, infinitely scalable.
Parquet — the compact, columnar file format in which lake tables usually keep their rows.
Apache Iceberg — the open table format that makes files into tables: each version of a table is an immutable snapshot described by a metadata file listing exactly the data files that belong to it.
Catalog — the service that holds, per table, the single pointer to the current metadata file. The keeper of “which files is the table right now.” LakeCat, Nessie, Polaris, and Gravitino all speak the Iceberg REST protocol.
Commit — the moment a table changes: validate the update, write the new metadata file, advance the pointer. The unit of everything Part VI measures.
Lost update — the original sin of concurrency: two writers race, and one’s committed work is silently replaced by the other’s, with no error and no trace.
Compare-and-swap (CAS) — the defense: a commit that says “advance the pointer if it still points where I last saw it.” The stale racer is rejected instead of destroying work. Also called optimistic concurrency.
Ledger — a record with obligations: durable entries, never erased (a correction is a new entry), each following from the last, verifiable later by a stranger. What a catalog becomes when its transactions are provable.
Idempotency key — a unique key attached to an operation so that a retry returns the stored result of the first execution instead of applying the change twice. Same key, same answer, exactly one effect.
Transactional outbox — the announcement of a change written inside the same atomic transaction as the change itself, so downstream evidence can never be lost to a crash between committing and telling anyone.
Receipt — a record of one operation whose fields are sealed by their own digest, so any alteration is self-evident without the server’s help. In QueryGraph, versioned, TypeDID-bound, and deterministic: identical runs produce identical receipts.
Receipt chain — each receipt carries the digest of its predecessor, linking history into a sequence that cannot be silently reordered, trimmed, or spliced.
Offline verification — checking evidence with no help from — and no trust in — the server that produced it. What elevates a receipt from assertion to evidence.
Tombstone — the receipt covering a deletion, so that even removal leaves evidence that it happened, when, and under whose authority.
Provable transaction — the executable definition of Part VI: a commit with CAS, idempotent replay, audit and outbox staged atomically, and a chained, offline-verifiable receipt, all evidence hash-only.
Governed-scan proof — LakeCat’s cryptographic evidence that a read came from a specific authorized snapshot under a specific policy. A proof of the scan, not of arbitrary text a caller staples to it.
Lineage — the recorded path from source, through authorization and computation, to committed result — every node carrying an identity and a digest.
Authentication — who are you? Answered by TypeDID.
Authorization — given who you are, what may you do? The subject of Part VII.
Capability — authority as an object: a
non-cloneable, single-purpose token that must be held to act
and is consumed by the exact operation it authorizes. In TypeSec, a
typed value Capability<P, R> with no public
constructor, mintable only through a policy decision.
Policy engine — a centralized decision point (OPA with the Rego language; Amazon’s Cedar) that answers allow-or-deny per request. Expressive and default-deny — but the decision evaporates; nothing travels.
Relationship-based access control (ReBAC) — deriving decisions from a graph of relationships (“editor of the folder containing the doc”), after Google’s Zanzibar design; embodied in SpiceDB and OpenFGA.
Capability token — portable, shrinkable authority: Macaroons (caveat chains under HMAC), Biscuit (public-key blocks with logic-language caveats), UCAN (DID-rooted delegation chains).
Caveat — a restriction appended to a capability token, cryptographically folded in so it can be added but never removed.
Attenuation — the property that delegated authority can only narrow — fewer grants, a shorter lease — never widen. The single most important word in Part VII.
Lease — an expiry on authority: a capability is minted to die, and a use past its lease is inert.
Revocation — killing authority before its lease ends: by individual id, or by epoch — a mid-lease revoke-everything that invalidates all outstanding capabilities minted before it.
Ambient authority — power that is simply “in the air” around a program rather than attached to a specific request. The precondition for the confused deputy; abolished by deny-by-default capability discipline.
Confused deputy — a program with standing powers tricked into exercising them on an attacker’s behalf. The classic argument for capabilities: a deputy that holds no ambient power cannot be confused into lending it.
Information-flow label — a secrecy level (public, internal, sensitive) riding on the data itself, so that reads above the reader’s clearance are redacted and lowering a label requires an explicit declassify grant.
Clearance — the maximum label a principal may read. Distinct from tenancy: two users of one organization can differ in clearance, which is exactly the distinction the recorded cognition run found some systems missing.
Type — the compile-time classification of a value that determines what a program may do with it. The load-bearing idea of the Introduction: rules encoded in types are checked before the program ever runs.
Phantom type — a type parameter that carries no
run-time data yet makes two otherwise-identical types distinct to the
compiler — how Capability<CanRead, R> and
Capability<CanWrite, R> become as different as a
number and a sentence, at zero cost.
Sealed trait — a family of types that outside code cannot extend. TypeSec’s permissions are sealed: the vocabulary of authority is closed, and closed vocabularies can be reasoned about completely.
SecureValue — TypeSec’s opaque envelope for protected data: carries an information-flow label, allows transformation without extraction, keeps the stricter label when values combine, and yields cleartext only to a matching capability.
Typestate — encoding a protocol’s states as distinct types, so that operations invalid in a state simply do not exist on it. An unauthenticated agent has no authenticated methods to call — not denied; absent.
Unforgeable — the recurring adjective, meaning
something precise: possession is constructible only through the guarded
path. A Capability cannot be struct-literaled into
existence; a verified message cannot be fabricated from caller-supplied
fields; a receipt is a function of what happened, not a form to fill
in.
MARCIANA-ADVERSARIAL-v1 — the cognition benchmark: eighteen cases, nine hard gates, run against Marciana and a field of open-source memory systems (Parts IV–V, Appendices A–B).
CATALOG-PROVENANCE-v1 — the provable-transaction benchmark: seventeen cases, eleven capabilities, seven hard gates, run against the governed reference and a field of Iceberg REST catalogs (Part VI, Appendix C). Its companion, catalog-bench, measures the commit’s speed — the cost of what provenance buys.
CAPABILITY-ADVERSARIAL-v1 — the authority benchmark: eighteen cases, eleven capabilities, ten hard gates, run against a field of authorization systems in three bands — bearer, capability token, and decision engine (Part VII, Appendix D).
Results report — the companion document to this book that holds the current coverage and gate counts for every benchmark, regenerated on each run and stamped with the corpus digest. The book keeps the durable findings; the report keeps the numbers, so the book does not go stale.
Hard gate — a named safety property that must be zero for a benchmark to pass; never averaged into a quality score. The constitutional device shared by all three benchmarks.
Capability-declared scoring — the fairness rule shared by all three: each system’s adapter declares what the system enforces; unclaimed capabilities are reported unsupported, never scored, and never faked; a gate can only be tripped by a claim.
Adapter — the thin program that drives one system through a benchmark’s cases using the system’s own real interface, declaring its capabilities and returning outcomes — never re-implementing a check the system lacks.
Corpus digest — the SHA-256 fingerprint of a benchmark’s entire expectation set, stamped into every report, so that no expectation can be quietly adjusted after the fact and two runs are demonstrably of the same benchmark.
References point to parts rather than pages, since this book travels in several formats. Intro is the Introduction; roman numerals are Parts; A–D are the appendices; G is the Glossary, which defines every entry here.
abstention — IV, V, B adapter — IV, VI, VII, G agentic vs. agentless memory — IV ambient authority — Intro, VII, D, G Apache Iceberg — VI, C, G as-of query (two axes of time) — I, III, B attenuation — Intro, VII, D, G audit trail — I, II, VI, VII, A, C, D authentication vs. authorization — Intro, VII, G bearer token — II, VII, D, G Biscuit — VII, D, G capability — Intro, II, VII, D, G capability-declared scoring — IV, VI, VII, G capability token (family) — VII, D, G CAPABILITY-ADVERSARIAL-v1 — Intro, VII, D, G catalog — VI, C, G catalog-bench — VI, G CATALOG-PROVENANCE-v1 — Intro, VI, C, G caveat — VII, G Cedar — VII, D, G claim as a boundary — Intro, IV clearance — I, IV, V, VII, B, D, G Cognee — V cognition system (what counts as one) — Intro, IV commit — I, II, III, VI, C, G compare-and-swap (CAS) — VI, C, G confused deputy — Intro, VII, D, G corpus digest — IV, VI, VII, G data lake — VI, G declassify — VII, D, G decentralized identifier (DID) — Intro, II, G deny-by-default — Intro, VII, D digest (hash) — II, VI, G domain separation — II, VI, G Ed25519 / X25519 / ChaCha20-Poly1305 — Intro, G envelope (TypeDID) — Intro, G forgetting (surgical) — III, IV, B four verbs (remember, recall, improve, forget) — III, G governed-scan proof — II, VI, C, G Graphiti — V Gravitino — VI, C Grust — II, G hard gate — Intro, IV, VI, VII, A, C, D, G idempotency key — I, IV, VI, B, C, G information-flow label — Intro, VII, D, G JWT / OAuth — VII, D, G LakeCat — II, VI, C, G lease — VII, D, G ledger — I, III, VI, G Letta — V lineage — Intro, II, VI, C, G lost update — VI, C, G Macaroons — VII, D, G Marciana — Intro, III, IV, V, G MARCIANA-ADVERSARIAL-v1 — Intro, IV, V, A, B, G Mem0 — V metadata file / snapshot — VI, C, G MinIO — V, VI, C, G Nessie — VI, C object storage — VI, G offline verification — Intro, VI, C, G OPA (Open Policy Agent) — VII, D, G OpenFGA — VII, D, G outbox (transactional) — II, VI, C, G Parquet — VI, G phantom type — Intro, G Polaris — VI, C policy engine — Intro, VII, D, G prompt injection — IV, B provable transaction — VI, C, G QueryGraph — Intro, II, G ReBAC (relationship-based access control) — VII, D, G receipt — Intro, II, III, VI, A, C, G receipt chain — VI, C, G replay — Intro, IV, VI, A, B, C, G revocation (by id; by epoch) — VII, D, G Sail — II, G sealed trait — Intro, G SecureValue — Intro, VII, G SpiceDB — VII, D, G temporal reasoning — III, IV, B tombstone — VI, C, G type-level security — Intro, VII, G TypeDID — Intro, II, G typestate — Intro, G TypeSec — Intro, II, VII, D, G UCAN — VII, D, G unforgeable — Intro, II, VII, G Unity Catalog (excluded) — VI, C Zanzibar — VII, G