Coppice Rust research implementation

Rust implementation of the group E2EE design in ../hypersnap/FIP-coppice-spec.md. The Python research models remain unchanged.

The library handles receiving inventory, exact covers, compact authenticated directories, encrypted control messages, sender/contribution ratchets, offline entry, optional archives, pending dependencies and immutable retries. Applications supply storage/runtime policy, authenticated private enrollment and owner authorization decisions. Included adapters provide encrypted persistence, platform key storage, libp2p relays and member-held encrypted governance.

Experimental cryptographic software. The wire format is a research candidate; there is no security audit or proof of the complete composition. See the spec review for the implemented scope and remaining limits.

Use

Driver<S, T, A> is the main integration API. It automates the low-level protocol work, including ready-to-send controls, proof retrieval, sender updates and outbox acknowledgements. Client<S> remains available for custom schedulers.

let mut device = Driver::new(client, delivery, governance, DriverOptions::default())?;
device.sync()?;
let (id, _) = device.send(b"hello")?;
let report = device.poll()?;
// Verified incoming plaintext is already committed to Store before Event::Message is returned.

Included adapters:

• RemoteDelivery<T> and RemoteMembership<T> exchange opaque, bounded RPC bytes with canonical 128-byte padding. No network runtime is imposed.
• ReplicatedDelivery<T> scopes polling to authenticated server routes and durable arrival streams, so failover does not skip differently ordered records.
• ReplicaServer<S, P>, ReplicaNetwork and SqliteReplicaStore provide a permissionless server mesh with short gossipsub announcements, chunked libp2p request/response repair, periodic reconciliation and persistent local quotas.
• MemoryDelivery enforces group capabilities, finalized live-capability rotation, immutable records, batched discovery, blinded proof assembly, mailbox cursors and dependency-preserving garbage collection.
• OwnerGovernance<S,P> keeps the signing log on a member-held owner device and encrypts governance RPCs end to end through arbitrary relays. ApprovalPolicy decides mutations; signed member requests protect private rosters even from servers that know delivery tokens.
• Optional SqliteStore<W> encrypts snapshots and messages, advances an independent rollback witness, reuses only unchanged encrypted state blocks, handles interrupted commits and supports message pagination.
• KeyStore/KeyStoreWitness isolate endpoint secrets. Native macOS Keychain, Windows Credential Manager and Linux Secret Service adapters have no file fallback. A disposable cross-process validation example is included.
• ArchiveVault moves retained source capabilities out of live state into encrypted copy-on-write indexes. Handoff is crash-safe; recipient exports build one page at a time. Resumable collection reclaims obsolete paths and failed-write orphans. See source archive storage.
• ContentJobs encrypts resumable 256 KiB attachment chunks in a separate Store. MessageContent embeds the resulting capability inside signed message bytes. Paged history imports verify original authorship and commit plaintext/cursors atomically.
• InventoryPlan assigns each fixed tree audience to one publisher. Shared pool boxes avoid repeating private pools in every recipient package. Private enrollment forwarding state is retained for restart/retry.
• InventoryPolicy monitors usable covers and emits signed private replenishment work. Independent enrollment reserves support queued offline deliveries; readiness still requires recipient import. Signed retirement preserves unread published tickets. Explicit directory growth preserves stable slot assignments.

Follow the integration guide for complete enrollment, transport, storage, finality, retry and backlog contracts. The chat example demonstrates the public driver API. See the replication contract, membership/metadata discussion and paged-history/attachment design for the new boundaries.

Compact entries and bandwidth

A common entry carries committed roster/cover roots, one author credential and proof, and fresh independent era secrets. Each wrapper carries its selected cover proof. The recipient verifies its own membership through a blinded hash directory; the service never receives its credential or directory key. The authority verifies the complete private roster and exact cover before finalization. Changing authors on an unchanged membership does not force unchanged hash nodes to be uploaded again. Membership changes reblind active and empty directory leaves and republish the directory to prevent former members recognizing known slots. Unchanged roster scopes now reuse encrypted roster objects through small per-era capabilities. Membership changes/growth rotate the object and key; the driver handles retrieval and validation. See roster evidence.

Cached latest-entry return uses four requests: fresh head, batched wrapper discovery, common ciphertext and own-membership proof. Full private rosters are fetched only for management and contribution validation. Messaging supplies unknown-sender proofs through encrypted per-era directory records.

The earlier fixed-server 128-device benchmark, with seven spread removals, measured 7,296 bytes downloaded in four requests, including padded RPC framing and no full-roster read. The 1,000-device/two-removal run also measured 7,296 bytes in four requests. Those reports predate D2 metadata controls and directory reblinding. Setup and author traffic are accounted separately in the bandwidth report. Run the complete workloads with:

cargo run --release --example bandwidth -- 128
cargo run --release --example bandwidth -- 1000

Eight past installed sessions are retained by default for delayed unread live messages. Their processing cannot rewind the current era. Resource exhaustion explicitly defers entry; forget_era permanently abandons a chosen past backlog. Latest return skips never-installed intermediate eras without reconstructing them. Archives and history grants remain a separate optional path.

An earlier D2/member-held-owner 128-device run measured 7,424 downloaded bytes in four requests, including sealed governance RPCs and no full roster read. It also accounts for encrypted roster evidence and directory reblinding. See the updated bandwidth report for separate setup/change costs.

Build and verify

Requires Rust 1.89+, CMake 3.15+, a C compiler, and Make or Ninja. Native sources are vendored; builds do not fetch native code or require liboqs/OpenSSL installed. SQLite is optional and builds its bundled C dependency when enabled.

cargo test --all-features --all-targets
cargo clippy --all-features --all-targets -- -D warnings
cargo run --example chat
cargo doc --all-features --no-deps
python3 tools/vendor_liboqs.py --check

Exact cryptographic suite

The algorithms remain NTRU-HPS-2048-677, Falcon-padded-512, SHAKE256 and ChaCha20-Poly1305. NTRU formats are 930/1234/930-byte public/private/ciphertext; Falcon formats are 897/1281/666-byte public/private/padded-signature. There is no ML-KEM, ML-DSA, FN-DSA encoding, algorithm fallback or classical hybrid.

The backend uses an unmodified subset of liboqs 0.16.0, pinned to commit 5a1a854b0dc9f2141bdc771c555ee60c37950183. Build and runtime checks allow only the two specified public-key algorithms. Independent SHAKE vectors and fixtures from both the former backend and liboqs check transcript/encoding compatibility. See backend provenance and updates. Algorithm-specific upstream support is community tier; the backend does not establish security of the overall scheme.

New controls and live frames use CoppR4D2; legacy CoppR4D1 records remain readable. Endpoint snapshots are version 8. Version-2/3/4/5/6/7 snapshots migrate without resetting keys, counters, immutable retries or replay protection. Legacy pools remain readable; new era authoring requires freshly provisioned scoped metadata. Version-1 snapshots fail closed. Upgrade all participants before using new D2/archive-retention formats. The cryptographic suite and exact primitive encodings have not changed.

Deployment limits

Secure endpoint storage must retire superseded state and use an independent rollback witness. FileWitness supplies atomic filesystem operations, not physical erasure or protection against restoring its own directory. Retained application plaintext and explicitly retained unread/archive capabilities remain available on endpoint compromise. Never clone a sending identity or restore it from backup.

The selected governance key is member-held; all servers may be adversarial. Permissionless peer counts confer no membership authority. The optional libp2p feature supplies a reference swarm and coppice-relay binary; applications still choose retention, admission and concealed-origin endpoint routing. Anonymous gossipsub fields alone do not hide the first-hop peer/IP. New scoped controls and random private enrollment/grant selectors revoke future cryptographic metadata access through excluded audiences. Old scopes remain readable to their holders. See owner governance and the privacy discussion.

Directory capacity is a historical-slot/DoS bound, defaulting to 128. The previous 4,096 maximum is now explicitly growable to 65,536. Ordinary transitions preserve removed slots. Explicit compaction provisions fresh inventory before canonically reassigning retained members into a new layout. This is not a claim of 65,536 active-device scalability: active cohort size, authorization frames and storage budgets impose separate bounds.

Legacy grants retain their 2,048-record wire limit. New encrypted paged grants import one bounded leaf at a time. Streaming attachment jobs embed opaque capabilities in signed message content, so history grants include attachment access automatically. Separate job storage keeps file buffers out of live ratchet state. An optional separately witnessed source vault indexes retained capabilities and prepares exports incrementally. Bounded ciphertext collection preserves current and frozen views while reclaiming quota. SQLite supports bounded page compaction, explicit older-file rebuilds and authenticated reachable legacy-node migration. A restart/churn fixture measures settled archive disk sizes; broader group-scale measurements are complemented by a persistent multi-producer lifecycle fixture. Unidentified legacy rows are preserved and charged to quota. Current-owner-authorized handoff, recipient-bound bootstrap, durable exact-intent approvals and a pluggable concealed-origin RPC boundary are implemented; see lifecycle integration. Durable private enrollment, typed pending approvals, protected-provider IPC, operator controls and a runnable two-process relay fixture are covered by deployment composition. Large owner logs use paged handoffs; locally pinned compact verifier checkpoints and retained ancestry pages support owner histories beyond sixteen changes. New archives retain source-era and admission-page cohorts rather than all new group history. Full traffic-analysis defenses and an independent security audit remain outstanding. Native keystore runtime checks still need the appropriate signed macOS host and Linux/Windows sessions. See scale work. See verification and audit scope. Resource limits never trigger weaker encryption or incomplete covers.