The adversarial benchmark is now its own public project at querygraph/adversarial-cognition,
extracted from Marciana at commit cbf3592. It pins an
18-case, 11-category corpus and separates hard safety gates from quality
and latency. The corpus digest is
d879b8a53039d84134bf8b35f21a398c497b94605bddf1a4995854aa1cb798b9,
so a future run can prove exactly which questions and policies were
tested.
The reference implementation is deterministic: every supported case is correct, its full-case latency P50 is 36.1 microseconds, and all nine hard gates are zero. The comparative run records unsupported capabilities instead of silently treating them as failures:
| System | Supported | Correct | Accuracy | Unsupported | Result |
|---|---|---|---|---|---|
| Marciana reference | 18 | 18 | 100% | 0 | all nine hard gates zero |
| Akka + Fluree | 16 | 16 | 100% | 2 | no clearance/purpose engine |
| Letta 0.16.8 | 9 | 7 | 78% | 9 | no input-robustness boundary |
| Graphiti (Kuzu) | 8 | 6 | 75% | 10 | retrieval is not token-order stable |
| Mem0 (OSS) | 9 | 6 | 67% | 9 | lower-clearance private-memory leak |
| Cognee (OSS) | 8 | 5 | 63% | 10 | empty-input and input-bound failures |
The hard-gate set covers unauthorized disclosure, cross-scope
leakage, forged or stale proposals, replay and duplicate mutation,
residual recall after forget, nondeterministic receipts, and malformed
or injection-shaped input. LLM-backed systems are naturally model,
embedding, and hardware dependent; their unsupported cases are not
extrapolated into a score. The benchmark is a regression instrument for
responsible cognition, not a marketing leaderboard. Its full design and
run instructions are in MARCIANA-ADVERSARIAL-v1.md.