3 Part I: The Substrate

The four components underneath the semantic layer: the graph and its query language, the security fabric and its identity machinery, the catalog and its handoff, and the compute engine. Each chapter explains its component from scratch and ends with worked examples.

3.1 Chapter 1. Grust: The Property Graph

Grust is a modern property-graph API for Rust with a deliberately plain core model:

Graph = nodes + edges
Node  = id + label + properties
Edge  = optional id + from + to + label + properties

That shape is expressive enough for persistent graph databases but small enough for tests, import/export tools, scrapers, and knowledge-graph pipelines. The design principle is strict separation of concerns: application code builds a grust::Graph; backend crates decide how to persist or query it. Grust is not competing with petgraph for in-memory graph algorithms — it is the persistent property-graph layer: stable application IDs, node and edge labels, typed properties, optional schema metadata, traversal expressed as an IR rather than a query string, and an async GraphStore trait for persistence backends.

The workspace ships a facade crate (grust-graph, imported as grust) over a core (model, builder, schema validation, traversal IR, GraphStore) and a family of backends:

The builder deduplicates nodes by id and, by default, edges by (from, label, to); domains that need multi-edges opt into EdgePolicy::AllowDuplicates. Typed property schemas validate graphs before they reach a store.

QueryGraph uses Grust twice. First, the semantic graph — datasets, fields, ontology terms, agents, policies, as nodes and edges — loads into a Grust store. Second, the Sail fork compiles a Cypher extension into the engine (Chapter 7), reusing Grust’s model, so the same graph is queryable from Spark sessions.

3.1.1 Worked example: build a graph, store it, traverse it

use grust::prelude::*;

let mut builder = GraphBuilder::new();
let dataset = builder
    .node("Dataset", "dataset:county-finance")
    .prop("source", "sail.qg_lakehouse.government_finance__countydata")
    .finish();
let metric = builder
    .node("Metric", "metric:fiscal-capacity")
    .prop("expression", "SUM(total_revenue - mandated_spend)")
    .finish();
builder.edge("MEASURED_OVER", &metric, &dataset).finish();
let graph = builder.build();

// Persist and traverse (memory store; SailGraphStore is the same trait).
let store = MemoryGraphStore::new();
store.put_graph(&graph).await?;
let datasets = store
    .traverse(
        Traversal::from_node("metric:fiscal-capacity")
            .out("MEASURED_OVER")
            .to("Dataset"),
    )
    .await?;
assert_eq!(datasets.len(), 1);

The traversal is an IR, not a string: the same Traversal runs against memory, PostgreSQL, or Sail, each backend lowering it natively.

3.1.2 API reference

3.2 Chapter 2. The Query Language: GQL/Cypher

Crab (0.11) gave the graph a language: a standards-conformant GQL/Cypher layer — lexer, parser, AST, semantic analysis — over the property graph, with read pushdown into Sail and SQLite, first-class Decimal/Duration/temporal values, and catalog procedures such as CALL db.labels().

Lobster (0.12) completes it. The merged Full39075 profile brings:

• CALL { … } subqueries;
• table-valued functions (CALL with correlated arguments);
• shortestPath() / allShortestPaths() on the read reference;
• backend-native query passthrough escape hatches;
• an executable portable read corpus (the conformance tests run as code);
• atomic Cypher transaction batches: CypherTransaction accumulates eagerly-planned write statements between START TRANSACTION/BEGIN and commit, executing them atomically behind the transaction surface.

Reads are portable: grust-cypher::read::run_read_query executes Cypher against any grust::Graph with a reference executor, while stores like SailGraphStore push the identical query down to the backend. Writable Cypher is a strict, backend-neutral mutation plan (cypher_mutation_plan) that stores execute natively.

3.2.1 Worked example: Cypher over the semantic graph

Exactly how qg-rust queries its semantic graph (src/cypher.rs):

use grust_cypher::read::run_read_query;
use grust_cypher::CypherParameters;

let table = run_read_query(
    &graph,
    "MATCH (m:Metric)-[:MEASURED_OVER]->(d:Dataset) \
     RETURN m.expression, d.source",
    &CypherParameters::new(),
)?;

Through the Sail fork’s extension (Chapter 7), the same MATCH also runs from any Spark Connect session — PySpark included — against the graph the lakehouse projects.

3.2.2 API reference

3.3 Chapter 3. TypeSec: Capabilities as Types

TypeSec encodes permissions as types. Most security systems check permissions at runtime — and a guard-based check can be forgotten, skipped, or bypassed:

// ❌ Guard-based — one missed call and the policy is fiction.
if acl.check(user, "write", resource) {
    resource.write(data);
}

TypeSec inverts this. If your code does not hold a Capability<CanWrite, Report>, the write method does not exist for you:

// ✅ Type-level — the capability IS the proof.
fn write(cap: Capability<CanWrite, Report>, report: &Report) {
    // `cap` existing in scope means the policy engine approved this.
}

The Capability<P, R> struct is unforgeable by construction:

• its constructor is pub(crate) — only the policy engine creates one;
• P and R are phantom types — Capability<CanRead, Report> and Capability<CanWrite, Report> are different types;
• the Permission trait is sealed — no new permissions outside typesec-core.

The only production path to a capability runs a policy check and emits an audit event; a denial is a typed error carrying a receipt, never a capability. Policy violations are compile errors, not incident reports.

QueryGraph’s supervised agent story (Part II, Chapter 16) mints capabilities for exactly the actions its topology needs: AiCanInfer, CanReadSensitive, CanDelegate, CanAggregateSummaries, CanReadDatasetCompartment, and CanDeriveRedactedSummary.

3.3.1 Worked example: a capability is the proof

use typesec::prelude::*;

// Without a Capability<CanRead, Report> in hand, this function
// cannot be called — the check is the type system's, not yours.
fn read_report(cap: Capability<CanRead, Report>, report: &Report) -> Summary {
    summarize(report) // `cap` in scope == the policy engine said yes
}

let decision = engine.check(&subject, Action::Read, &report);
match decision {
    Ok(cap) => read_report(cap, &report),      // audit event already emitted
    Err(denied) => return Err(denied.receipt()) // a denial is data, not panic
};

3.3.2 API reference

3.4 Chapter 4. TypeDID: Identity, Envelopes, and the Torcello Platform

On top of the capability core, TypeSec provides the machinery QueryGraph uses for agent identity and messaging:

• Deterministic Ed25519 keys: Ed25519DidKey::from_seed derives a keypair from a seed (SHA-256 of the seed as the private key), so an agent recreated from the same seed signs identically across processes — and, as QueryGraph proves, across languages.
• did:key documents: public keys published as W3C DID identifiers anyone can resolve without a registry.
• Signed, encrypted envelopes under the ed25519-x25519-chacha20 profile, with a request/reply conversation model (the typedid/a2a protocol label — made literal by QueryGraph’s A2A card, Part III).
• Audit-safe attestations: each verified envelope exposes who did what to which resource at which privacy level — without revealing the payload or the signing material. The envelope digest binds the attestation to the exact message.

Torcello (0.12) grows this fabric into a platform other agent stacks plug into: an agent-framework interop plane guarding OpenAI, Anthropic, LangChain, and Pydantic-AI tool calls; an MCP dialect and mcp-gate, a deny-by-default MCP stdio proxy; signed decision receipts with decision logging and replay; JSON-Schema-validated tool bindings and a #[typesec_tool] macro; an OpenTelemetry audit sink; a WASM decision core for JS/TS agents; a PyPI-ready Python package; and an OpenAI/Anthropic-compatible enforcement proxy — the “governed inference proxy” pattern, built at the security layer. QueryGraph already builds against Torcello; adopting these surfaces rather than duplicating them is the next integration step (see Future Work).

3.4.1 Worked example: one seed, one identity, two languages

Python signs:

from querygraph.typedid import TypeDidAgent

supervisor = TypeDidAgent.new("SupervisorAgent")
envelope = supervisor.request(
    TypeDidAgent.new("FinanceAgent"),
    action="summarize", resource="compartment:finance",
    payload={"question": "Where is fiscal stress highest?"},
)
print(envelope.signature[:24])            # ed25519:8a7a231b6f67f4…
print(envelope.verification_method[:32])  # did:key:z6Mkrdhpo…

Rust mints the identical identity from the identical seed:

use querygraph::agent::PyTypeDidEnvelope;

let envelope = PyTypeDidEnvelope::signed(
    "querygraph-agent:SupervisorAgent",   // same seed ⇒ same did:key
    "did:example:recipient",
    "summarize", "compartment:finance",
    serde_json::json!({"question": "Where is fiscal stress highest?"}),
);
assert!(envelope.verify().signature_valid);

A fixture test pins the shared did:key, so the derivations can never drift.

3.4.2 API reference

3.5 Chapter 5. LakeCat: The Iceberg REST Catalog

LakeCat is a Rust-native Apache Iceberg REST catalog and QueryGraph foundation. Standard Iceberg clients speak to it on ordinary REST catalog paths (/catalog/v1); underneath, it binds catalog state, governed Sail planning, TypeSec receipts, and Grust projection to the same accepted table transition, so the catalog’s view of a table and the governance evidence about it can never drift apart.

The catalog spine runs on Turso MVCC: commits to different tables run truly concurrently, and a same-table race converges to exactly one winner through a pointer compare-and-swap — no global write lock. The audit event, the lineage outbox row, and the idempotency record are written in the same transaction as the table change, which is what lets a downstream consumer accept catalog state as proof rather than as a best-effort side effect.

Scan planning routes through the Sail-facing engine: point-in-time scans produce opaque Iceberg REST plan-task tokens from stable Sail metadata; append-only incremental scans over a snapshot chain plan only the manifests added in the requested range, expanding delete manifests through Sail’s delete-file index. REST scan filters are validated against Sail’s generated Iceberg expression models before planning.

Ocelot (0.3.0) proves the boundary from the stock client’s side: Iceberg REST conformance demonstrated by a PyIceberg round-trip — spec-correct error types (403 on authorization denial, 409 on a duplicate namespace, 404 on a missing one), listTables, honest update rejection on the default build, and fail-closed commit-requirement validation — over dependencies moved to Grust “Lobster” and TypeSec “Torcello”. The release rests on a recorded release-candidate proof harness (Chapter 6’s handoff, verified end to end, 40/40 QueryGraph tests green).

3.5.1 API reference

3.6 Chapter 6. The Bootstrap Handoff

LakeCat’s signature surface for QueryGraph is the bootstrap bundle at /querygraph/v1/bootstrap: a projection of live catalog tables into Croissant, CDIF, OSI, ODRL, OpenLineage, and a Grust-ready graph envelope.

The wire format and its verification live in a small shared crate, qglake-bundle, used by both producer and consumer. qg-rust does not keep a hand-written copy of those types: it deserializes the canonical QueryGraphBootstrap, runs LakeCat’s own verify_manifest, and layers its Cypher import plan on top. The bundle cannot mean two slightly different things on the two sides of the boundary.

3.6.1 Worked example: from catalog to governed import

# Terminal 1: the catalog, with Sail planning, TypeSec receipts,
# and Grust projection bound to table transitions.
LAKECAT_BIND_ADDR=127.0.0.1:8181 \
LAKECAT_TURSO_PATH=target/local/catalog.turso \
LAKECAT_GRUST_TURSO_PATH=target/local/catalog-graph.turso \
cargo run -p lakecat-service \
  --features sail-local,typesec-local,grust-turso-local,turso-local

# Terminal 2: standard Iceberg clients see an ordinary REST catalog…
cargo run -p lakecat-cli -- config --catalog http://127.0.0.1:8181

# …and QueryGraph sees live tables projected into Croissant, CDIF,
# OSI, ODRL, OpenLineage, and a Grust-ready graph envelope.
curl -s http://127.0.0.1:8181/querygraph/v1/bootstrap > bootstrap.json

# qg-rust verifies the bundle with LakeCat's own shared wire crate
# (qglake-bundle) and emits its import plan:
cargo run -- lakecat-import --bundle bootstrap.json --output import-plan.json

The import command prints the bundle’s verification report; QueryGraph accepts catalog state as proof.

3.6.2 API reference

3.7 Chapter 7. Sail and the Cypher Extension

Sail is the Spark-compatible lakehouse engine (from lakehq) that QueryGraph uses as its compute and storage substrate: typed Parquet tables registered in a Spark Connect-compatible server, queryable from PySpark, with the QueryGraph audit trail (qg_audit) living alongside the data it audits.

The stack’s fork (branch grust) adds a Cypher graph-query extension compiled into Sail itself — roughly 5,600 lines across twenty files: a Cypher AST in sail-sql-parser, graph analysis in sail-sql-analyzer, and plan resolution in sail-plan, with Spark Connect integration and a design document (docs/development/graph-extension.md). Crucially, it reuses Grust’s property-graph model and schema validation rather than reimplementing them.

The consequence: the same semantic graph that QueryGraph loads through Grust is MATCH-able from any Spark Connect session — no separate graph database in the deployment, no second copy of the graph to drift.

# From PySpark, against the Sail server the lakehouse already runs:
spark.sql("MATCH (m:Metric)-[:MEASURED_OVER]->(d:Dataset) "
          "RETURN m.expression, d.source").show()