Add stable processing profiles and fingerprints

[salmonumbrella]

What changed

Docbank now has canonical processing-profile/v1 policy bytes and separate fingerprints for rendition, evidence, lexical work, embedding inputs, vector spaces, retention, and provider disclosure. Each fingerprint changes only when its own semantic inputs change, so switching an embedding model does not invalidate a rendition and changing retention does not rebuild evidence.

Named TOML profiles cover rendition, embedding, retrieval, and complete processing policy. Credential references stay symbolic as credential:<name>; runtime secrets never enter canonical profile bytes.

Why

One whole-profile hash would rebuild unrelated work, while incomplete component hashes could reuse artifacts across materially different policy or disclosure boundaries. The catalog and workers need stable, narrow cache identities before they can safely share work.

Usage

Resolve and fingerprint a named profile:

profile, err := cfg.ProcessingProfile("archive")
canonical, fingerprints, err := document.CanonicalProfile(profile)

This PR defines and validates policy only. It does not execute providers or publish derivatives.

Part of #176 (F3). Stacks on #182.

[roborev-ci]

roborev: Combined Review (f2dc8bc)

Review found three medium-severity correctness issues in evidence canonicalization and Markdown task-list preservation; no newly exploitable security weakness was identified.

Medium

• document/normalize.go:655 — GFM task-list checkboxes are discarded because input is classified as active HTML before the checkbox-specific branch runs. Checked and unchecked items therefore lose their status in the durable rendition. Handle parser-generated task inputs before active-HTML filtering, preserve the checked state, and test checked/unchecked Markdown round-trips.

• document/evidence_codec.go:64 — Locators are validated independently, allowing complete evidence to contain duplicate or out-of-order page, slide, sheet, or spine locators, or inconsistent zero- and one-based origins. Add document-level validation enforcing consistent origins and strictly ordered, unique singleton locators, including for normalized evidence.

• document/evidence_codec.go:734 — Normalized locator names and artifact pointers are not required to already use NFC/LF canonical form. Canonically equivalent strings, including duplicate logical artifacts represented in NFC/NFD, can therefore produce distinct accepted IDs and checksums. Require every normalized string field to equal canonicalEvidenceString(value) before validating its ID.

---

Reviewers: 2 done | Synthesis: codex, 9s | Total: 16m38s

[wesm]

Working on #180 now

[roborev-ci]

roborev: Combined Review (8e4b1df)

The change has three medium-severity correctness and integrity issues despite no identified security-control regressions.

Medium

• document/normalize.go:795 — Task-list checkbox state is lost. isActiveHTML("input") returns true before the case "input" branch is reached, so GFM task-list checkboxes are always discarded. Preserve parser-generated task-list inputs as [x] or [ ] while continuing to discard raw active inputs.

• document/normalize.go:570 — Active inline HTML bodies are not fully suppressed. Raw inline HTML nodes use isolated writers, so skip state does not span matching tags. For example, before <script>private()</script> after retains private(). Pair well-formed active opening and closing spans and suppress enclosed content while keeping unmatched or malformed tags isolated.

• document/rendition_normalize.go:82 — Rendition checksums do not cover all durable manifest data. Unit checksums omit Locator, and the top-level checksum hashes the constant RenditionContractV1 instead of the manifest’s actual ContractVersion. Distinct manifests can therefore share a recomputed checksum after provenance or contract-version changes. Canonically hash every non-derived manifest field and consistently validate derived IDs and checksums.

---

Reviewers: 2 done | Synthesis: codex, 11s | Total: 15m28s

[roborev-ci]

roborev: Combined Review (2366893)

Code review found three medium-severity issues; no high or critical findings.

Medium

• document/normalize.go:795 — isActiveHTML("input") returns before checkbox handling, so GFM task-list inputs are discarded and checked/unchecked items become indistinguishable in durable renditions. Preserve parser-generated checkbox state before filtering active raw HTML, emitting [x] or [ ].

• document/rendition_normalize.go:82 — Rendition checksums do not cover the complete manifest as documented. Changes to fields such as ContractVersion, unit Locator, and nested IDs may go undetected. Hash canonical representations of all manifest fields except checksum fields, or validate every nested identity and checksum.

• internal/config/config.go:412 — Retrieval profiles allow candidate limits up to 1,000,000, but retrieval APIs reject values above embedding.MaxCandidateLimit (1,000). Validate both retrieval limits against that constant and add coverage for the 1,001 boundary.

---

Reviewers: 2 done | Synthesis: codex, 9s | Total: 18m55s

[roborev-ci]

roborev: Combined Review (7cd16ae)

Review found two medium-severity normalization issues; no critical or high-severity findings.

Medium

• document/normalize.go:564 — Inline active HTML bodies are retained. Goldmark splits before <script>secret</script> after into separate raw tag spans, so the tags are discarded while secret is persisted and indexed. Pair matching active-element spans and suppress their intervening content without letting unmatched tags affect later Markdown. Add coverage for closed inline script, iframe, and textarea elements.

• document/normalize.go:795 — The active-element check handles every input before the checkbox case, causing generated GFM task-list inputs to be discarded and task completion states ([x]/[ ]) to be lost. Recognize renderer-generated task checkboxes first and emit their state from the checked attribute, while continuing to discard provider-supplied raw inputs.

---

Reviewers: 2 done | Synthesis: codex, 9s | Total: 12m40s

[wesm]

rebasing

[roborev-ci]

roborev: Combined Review (328ffb3)

Review identified two medium-severity correctness issues in processing-profile assembly; no high or critical findings.

Medium

• internal/config/config.go:581 — The selected retrieval profile is checked for existence but then discarded. Different lexical/vector limits therefore yield identical assembled profiles and fingerprints, preventing the documented retrieval policy from being applied through this API. Include the resolved retrieval policy in the assembled contract, or return an aggregate containing both the document profile and retrieval policy.

• internal/config/config.go:565 — CanonicalProfile normalizes a copy, but ProcessingProfile returns the original strings. CRLF or decomposed-Unicode values may share a fingerprint with normalized values while adapters receive different model, formatter, or descriptor strings. Return the canonicalized profile, or reject noncanonical input so executed policy matches its fingerprint.

---

Reviewers: 2 done | Synthesis: codex, 7s | Total: 6m40s

[roborev-ci]

roborev: Combined Review (51e2ae5)

Verdict: One medium-severity configuration validation issue should be fixed before merge.

Medium

• internal/config/config.go:412 — Retrieval limits up to 1,000,000 pass configuration validation, but the shared retrieval implementation rejects values above embedding.MaxCandidateLimit (1,000). This permits valid-but-unexecutable configurations. Derive the validation limit from embedding.MaxCandidateLimit or validate by constructing the runtime retrieval policy, and add coverage for a limit of 1,001.

---

Reviewers: 2 done | Synthesis: codex, 7s | Total: 9m5s

[roborev-ci]

roborev: Combined Review (dfef185)

Code review found one medium-severity validation gap.

Medium

• internal/config/config.go:564 — ProcessingProfile does not enforce embedding.MaxCandidateLimit. Because Load does not validate automatically, profiles with limits from 1,001 through 1,000,000 can be assembled successfully but are later rejected by shared retrieval. Validate the selected retrieval binding in ProcessingProfile or assembleProcessingProfile, and add a direct assembler test covering 1,001.

---

Reviewers: 2 done | Synthesis: codex, 6s | Total: 7m53s

[roborev-ci]

roborev: Combined Review (b68b71b)

One medium-severity validation inconsistency should be fixed before merging.

Medium

• internal/config/config.go:631 — ProcessingProfile skips validateEmbeddingProfileConfig, while embeddingDocumentBinding silently discards a populated chunk policy for original_file inputs. Direct callers can receive a successful profile whose fingerprint does not represent the declared configuration, even though Config.Validate rejects the same input.
  • Fix: Validate each selected embedding profile before conversion and add coverage for calling ProcessingProfile with an original_file binding that defines chunk settings.

---

Reviewers: 2 done | Synthesis: codex, 26s | Total: 8m19s

[roborev-ci]

roborev: Combined Review (4085981)

The change needs one medium-severity fix: profile validation accepts retrieval limits that shared retrieval cannot execute.

Medium

• document/profile.go:585 — Public profile validation permits retrieval limits up to 1,000,000, but shared retrieval operations reject limits above 1,000. Direct CanonicalProfile callers can therefore create durable profiles that validate successfully but cannot execute. Share the 1,000-candidate ceiling across packages and add a test confirming that CanonicalProfile rejects 1,001.

---

Reviewers: 2 done | Synthesis: codex, 7s | Total: 6m10s

[roborev-ci]

roborev: Combined Review (1c2c31e)

No issues found.

---

Reviewers: 2 done | Synthesis: codex | Total: 12m43s