Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8f3979a
feat: upgrade source anchors to SHA-256 provenance
drdeepaksatheesh Sep 15, 2026
9ede12c
feat: add versioned local document sidecar model
drdeepaksatheesh Sep 15, 2026
8ad28dd
test: cover SHA-256 anchors and range recovery
drdeepaksatheesh Sep 15, 2026
1644622
test: add sidecar provenance regression coverage
drdeepaksatheesh Sep 15, 2026
570a411
feat: restore anchors by source text as fallback
drdeepaksatheesh Sep 15, 2026
025efd3
feat: persist SHA-256 sidecars and add dark mode
drdeepaksatheesh Sep 15, 2026
8137aaa
feat: add persistent dark theme styling
drdeepaksatheesh Sep 15, 2026
20ae3c8
docs: define document sidecar JSON schema
drdeepaksatheesh Sep 15, 2026
56ed927
build: pin frontend dependencies and bump v0.0.2
drdeepaksatheesh Sep 15, 2026
fceb128
build: add cross-file version consistency check
drdeepaksatheesh Sep 15, 2026
672b941
build: bump Tauri bundle version and identifier
drdeepaksatheesh Sep 15, 2026
a726a82
build: align Rust package version for v0.0.2
drdeepaksatheesh Sep 15, 2026
7ae60c5
ci: harden version and Windows bundle validation
drdeepaksatheesh Sep 15, 2026
65059ec
docs: add evidence-based release validation checklist
drdeepaksatheesh Sep 15, 2026
91d4b49
feat: persist sidecars through Tauri app-data storage
drdeepaksatheesh Sep 15, 2026
adbd446
fix: await durable sidecar load and save
drdeepaksatheesh Sep 15, 2026
d3e136f
feat: add durable app-data sidecar storage commands
drdeepaksatheesh Sep 15, 2026
76adfc0
docs: update roadmap for v0.0.2 hardening scope
drdeepaksatheesh Sep 15, 2026
e222e35
docs: record v0.0.2 provenance storage architecture
drdeepaksatheesh Sep 15, 2026
11dcf1d
docs: update README for v0.0.2 provenance milestone
drdeepaksatheesh Sep 15, 2026
1adb310
fix: hash a concrete ArrayBuffer for WebCrypto typing
drdeepaksatheesh Sep 15, 2026
1070536
security: upgrade Vite and Vitest past audited advisories
drdeepaksatheesh Sep 15, 2026
c6789b5
docs: record source provenance sidecar decision
drdeepaksatheesh Sep 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 22 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,14 @@ jobs:
cache-dependency-path: package.json
- name: Install dependencies
run: npm install --ignore-scripts
- name: Version consistency
run: npm run check:version
- name: Unit tests
run: npm test
- name: Web build
run: npm run build
- name: Dependency audit (critical only)
run: npm audit --audit-level=critical

windows-installer:
name: Windows NSIS installer
Expand All @@ -41,11 +45,27 @@ jobs:
workspaces: src-tauri
- name: Install frontend dependencies
run: npm install --ignore-scripts
- name: Version consistency
run: npm run check:version
- name: Build NSIS installer
run: npm run tauri build -- --bundles nsis
- name: Upload Windows installer
- name: Validate installer and write checksum
shell: pwsh
run: |
$files = @(Get-ChildItem "src-tauri/target/release/bundle/nsis/*.exe")
if ($files.Count -ne 1) { throw "Expected exactly one NSIS installer, found $($files.Count)." }
$installer = $files[0]
if ($installer.Length -lt 500000) { throw "Installer is unexpectedly small: $($installer.Length) bytes." }
$hash = (Get-FileHash $installer.FullName -Algorithm SHA256).Hash.ToLowerInvariant()
"$hash $($installer.Name)" | Set-Content -Encoding ascii "src-tauri/target/release/bundle/nsis/installer-sha256.txt"
Write-Host "Validated installer: $($installer.Name) ($($installer.Length) bytes)"
Write-Host "SHA-256: $hash"
- name: Upload validated Windows installer
uses: actions/upload-artifact@v4
with:
name: open-document-understanding-windows-nsis
path: src-tauri/target/release/bundle/nsis/*.exe
path: |
src-tauri/target/release/bundle/nsis/*.exe
src-tauri/target/release/bundle/nsis/installer-sha256.txt
if-no-files-found: error
retention-days: 90
60 changes: 38 additions & 22 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,17 +25,20 @@ Desktop application
│ └── document retrieval
├── Persistence
│ ├── settings
│ ├── durable per-document sidecars
│ ├── model metadata
│ └── reversible sidecar annotations
│ └── reversible annotations/provenance
└── Optional network verifier
└── explicit outbound requests only
```

## Application shell

The planned desktop shell is Tauri 2 with a TypeScript/React front end. This gives the project a modern document interface while preserving a native desktop packaging path.
The desktop shell is Tauri 2 with a TypeScript/React front end. This gives the project a modern document interface while preserving a native desktop packaging path.

PDF rendering should use PDF.js or an equivalent permissively licensed renderer suitable for selectable-text overlays.
PDF rendering uses PDF.js with a canvas plus selectable text overlay.

The application UI supports persistent Light/Dark mode. The rendered PDF itself is not recolored or rewritten by theme changes; dark mode changes only the reader chrome/background so the document remains visually faithful.

## Local engine

Expand All @@ -53,33 +56,42 @@ This keeps future replacement with ONNX Runtime, C++, Rust, llama.cpp-class runt

## Source anchoring

A source anchor should identify enough information to restore the user to the exact context used for an answer.
Beginning in v0.0.2, source identity uses cryptographic hashes rather than the fast UI fingerprint used by v0.0.1.

Initial anchor fields should include:
A source anchor contains:

- document fingerprint;
- page number;
- selected source text;
- optional text-item/block identifiers;
- bounding box(es) when reliable;
- normalized source hash.
- SHA-256 of the complete source document bytes;
- one-based page number;
- normalized selected source text;
- SHA-256 of the normalized source quote;
- PDF text-item start/end indices;
- start/end character offsets;
- creation time;
- schema/version information.

Anchors should survive ordinary UI actions and be stored separately from the source document.
Restoration first attempts the stored text-item range. If those renderer boundaries have shifted, the page text layer is searched for the same normalized quote and the recovered range is highlighted. The original PDF remains unchanged.

## Document sidecars

The original file should remain byte-for-byte untouched unless the user deliberately exports a derivative document.
The original file remains byte-for-byte untouched unless the user deliberately exports a derivative document.

v0.0.2 introduces a versioned JSON sidecar model. In the installed Tauri application, sidecars are stored under the application's data directory in a dedicated `sidecars` directory, keyed by the source document's SHA-256. The browser-development fallback uses local storage only for development convenience.

Generated material should be stored in a sidecar format that can contain:
A sidecar can contain records for:

- source-document fingerprint;
- anchors;
- source selections;
- translations;
- explanations;
- questions/answers;
- user notes/corrections;
- model/version provenance;
- domain-pack provenance;
- verification citations/status.
- verification citations/status;
- human-review status.

v0.0.2 writes source-selection records only; future AI outputs must attach to the same provenance model rather than inventing a parallel store.

The sidecar JSON format is described by `schemas/document-sidecar.schema.json`.

The sidecar must not be assumed legally safe to redistribute merely because it does not contain the original PDF.

Expand Down Expand Up @@ -143,17 +155,21 @@ Engineering requirements include:
- explicit visible network state;
- deterministic logging of externally transmitted verification queries where reasonable;
- minimal permissions;
- dependency review for unexpected network behavior.
- dependency review for unexpected network behavior;
- document and sidecar size limits at the native boundary;
- SHA-256 source identity for provenance.

## Packaging
## Packaging and release validation

The application should be packaged from early development.
The application is packaged from early development.

Reference distribution path:

`source -> CI build -> platform bundle/installer -> clean-machine smoke test`
`source -> CI checks -> platform bundle/installer -> checksum -> clean-machine smoke test`

Windows is the initial reference platform. CI verifies application-version consistency, runs tests/builds, produces exactly one NSIS installer, applies a minimum-size sanity check, calculates SHA-256 and uploads the installer with its checksum manifest. The clean-machine GUI workflow is documented separately in `RELEASE_CHECKLIST.md`.

Windows is the initial reference platform. Linux and macOS follow once the Windows vertical slice is stable. Android is a first-class future platform, especially for camera/SOP/accessibility use cases.
Linux and macOS follow once the Windows vertical slice is stable. Android is a first-class future platform, especially for camera/SOP/accessibility use cases.

## Architectural decision discipline

Expand Down
29 changes: 17 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,24 @@ A privacy-first, open-source document reader whose goal is simple:

The project started from a medical-education use case but is intentionally generic. Medical science will become an optional domain pack rather than the core application.

## Current milestone: v0.0.1
## Current milestone: v0.0.2

The first milestone is intentionally foundational:
v0.0.1 proved that the application can be built, installed and used on a clean Windows machine. v0.0.2 hardens the provenance layer before any local AI is introduced.

- desktop application shell using Tauri 2;
- selectable-text PDF opening and rendering using PDF.js;
- paragraph/text selection;
- a source-anchored side panel with **Translate / Explain / Ask** modes;
- no AI model yet;
- no OCR yet;
- no document upload at runtime.
Current capabilities include:

The point is to prove the product shell, packaging boundary, source anchoring and privacy model before adding model complexity.
- Tauri 2 + React/TypeScript desktop application;
- local selectable-text PDF opening and PDF.js rendering;
- source passage selection and return-to-source highlighting;
- SHA-256 identity for source documents and normalized source quotes;
- durable per-document JSON sidecars stored in application data;
- fallback source restoration when PDF text-item boundaries shift;
- visible **Translate / Explain / Ask** modes, still deliberately non-generative;
- persistent Light/Dark mode;
- visible **LOCAL ONLY** state;
- Windows NSIS packaging with CI-generated SHA-256 installer manifest.

There is still **no AI model, OCR or runtime internet verification path** in v0.0.2.

## Product principles

Expand All @@ -34,11 +39,11 @@ The point is to prove the product shell, packaging boundary, source anchoring an

## Development policy

`PROJECT_MANDATE.md` is the standing execution mandate for AI-assisted development. `PRODUCT_SPEC.md` and `ARCHITECTURE.md` are the product and engineering sources of truth.
`PROJECT_MANDATE.md` is the standing execution mandate for AI-assisted development. `PRODUCT_SPEC.md`, `ARCHITECTURE.md`, `ROADMAP.md` and `RELEASE_CHECKLIST.md` are project sources of truth.

## Status

This repository is at the product-shell stage. Do not use it for clinical decisions, safety-critical instructions or authoritative translation yet.
This repository is in the provenance-hardening stage. Do not use it for clinical decisions, safety-critical instructions or authoritative translation yet.

## License

Expand Down
43 changes: 43 additions & 0 deletions RELEASE_CHECKLIST.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Release validation checklist

A build is not a release merely because CI compiled it. This checklist defines the evidence required before calling a Windows milestone validated.

## Automated checks

- [ ] application/package/Tauri versions agree (`npm run check:version`)
- [ ] unit tests pass
- [ ] production web build passes
- [ ] dependency audit has no critical npm advisory
- [ ] Windows Tauri build succeeds
- [ ] exactly one NSIS installer is produced
- [ ] installer is larger than the minimum sanity threshold
- [ ] CI emits a SHA-256 checksum manifest beside the installer
- [ ] CI uploads the validated installer + checksum as one artifact

## Clean-machine Windows smoke test

Use a Windows machine/environment that is not the CI workspace and does not rely on developer tooling.

1. Install the generated NSIS `.exe`.
2. Launch Open Document Understanding from the installed application.
3. Confirm the displayed application version matches the release version.
4. Open a selectable-text PDF from local storage.
5. Confirm the document renders and remains selectable.
6. Select a source passage and confirm the right panel shows the exact source.
7. Change page/zoom, then use **Return to source** and confirm the source is restored/highlighted.
8. Close the application, reopen it, reopen the same PDF, and confirm the source record restores from the local sidecar.
9. Toggle dark mode, restart the application, and confirm the theme preference persists.
10. Confirm the application still displays **LOCAL ONLY** and that no online verification feature is present in this milestone.

## Evidence to record

For each validated milestone, record in the issue or release notes:

- CI run identifier;
- installer filename;
- installer SHA-256;
- clean-machine operating system;
- smoke-test result;
- known limitations.

Do not claim a clean-machine validation unless the GUI workflow was actually exercised.
32 changes: 19 additions & 13 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,40 @@

This roadmap is intentionally milestone-based rather than date-based. Each milestone should produce something testable and closer to a distributable application.

## v0.0.1 — Product shell
## v0.0.1 — Product shell ✅

Goal: prove the application boundary before adding AI complexity.

Completed and clean-machine validated on Windows:

- Tauri 2 desktop shell;
- React/TypeScript UI;
- open local selectable-text PDF;
- PDF rendering with text selection;
- source side panel;
- Translate / Explain / Ask controls present but allowed to be stubbed;
- Translate / Explain / Ask controls present but stubbed;
- local/network status indicator;
- basic settings storage;
- CI build workflow;
- Windows packaging path;
- synthetic test documents.

Exit criterion: the application can be installed on a second Windows machine, opens a PDF, and records a stable source selection.
- Windows NSIS packaging;
- synthetic smoke-test document.

## v0.0.2 — Robust source anchoring
## v0.0.2 — Robust source anchoring and release hardening (current)

- SHA-256 document identity;
- SHA-256 normalized quote identity;
- page/selection anchoring;
- normalized source hashes;
- fallback source-text range recovery if PDF text-item boundaries shift;
- jump back to source;
- selection persistence;
- sidecar schema;
- regression tests for source anchoring.

Exit criterion: a stored answer/annotation can reliably return the user to the exact source context after reopening the document.
- durable per-document JSON sidecars in application data storage;
- versioned sidecar schema for future Translate / Explain / Ask provenance;
- regression tests for hash stability, range recovery and sidecar records;
- persistent Light/Dark mode;
- cross-file version consistency checks;
- validated Windows installer checksum manifest;
- release-validation checklist.

Exit criterion: a stored source/provenance record survives application restart, reopens against the same SHA-256-identified PDF, reliably restores/highlights its source context, and the v0.0.2 Windows installer passes the documented clean-machine validation including persistent dark mode.

## v0.1 — First local understanding loop

Expand Down
42 changes: 42 additions & 0 deletions docs/adr/0001-source-provenance-sidecars.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# ADR 0001: SHA-256 source identity and local provenance sidecars

- Status: Accepted
- Milestone: v0.0.2

## Context

Future Translate, Explain and Ask outputs must remain traceable to the exact source passage that produced them. The v0.0.1 reader used a fast non-cryptographic fingerprint suitable only for UI restoration and stored one last anchor in web storage.

That is insufficient for durable provenance because it does not provide a strong document identity, does not support multiple records, and would encourage later AI features to invent their own persistence model.

## Decision

1. Identify the complete source document by SHA-256 of its bytes.
2. Normalize a selected source quote and identify that quote by SHA-256 as well.
3. Store page, PDF text-item range and character offsets in the source anchor.
4. If stored PDF text-item boundaries no longer reproduce the exact quote, search the page text layer for the same normalized quote and recover the highlight range.
5. Store document-associated provenance in a versioned JSON sidecar keyed by document SHA-256.
6. In the installed Tauri application, write sidecars to the application's data directory rather than modifying the PDF or writing beside a potentially read-only/copyrighted source file.
7. Reserve the same sidecar record model for future `translate`, `explain`, `ask` and `verify` operations.
8. Keep the browser-only development fallback in localStorage, but do not treat browser localStorage as the desktop durability mechanism.

## Consequences

### Positive

- the original document remains byte-for-byte untouched;
- provenance is tied to a strong document identity;
- multiple source/AI records can coexist;
- future AI features inherit a provenance structure instead of adding one later;
- application upgrades can evolve the sidecar independently through schema versions.

### Trade-offs

- SHA-256 adds a small local hashing delay when opening very large PDFs;
- sidecars are currently local to the application installation and are not portable/exportable yet;
- exact restoration still depends on PDF text extraction quality;
- scanned/image-only PDFs require the later OCR anchoring design.

## Explicit non-decision

A sidecar containing a translation or explanation is not automatically safe to redistribute merely because the original PDF is absent. Copyright/licensing policy for exported/shared sidecars remains a separate product/legal decision.
29 changes: 15 additions & 14 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,29 +1,30 @@
{
"name": "open-document-understanding",
"private": true,
"version": "0.0.1",
"version": "0.0.2",
"type": "module",
"scripts": {
"dev": "vite --port 1420",
"build": "tsc -b && vite build",
"test": "vitest run",
"check:version": "node scripts/check-version.mjs",
"tauri": "tauri"
},
"dependencies": {
"@tauri-apps/api": "^2.8.0",
"@tauri-apps/plugin-dialog": "^2.4.0",
"pdfjs-dist": "^5.4.149",
"react": "^19.1.1",
"react-dom": "^19.1.1"
"@tauri-apps/api": "2.8.0",
"@tauri-apps/plugin-dialog": "2.4.0",
"pdfjs-dist": "5.4.149",
"react": "19.1.1",
"react-dom": "19.1.1"
},
"devDependencies": {
"@tauri-apps/cli": "^2.8.4",
"@types/react": "^19.1.10",
"@types/react-dom": "^19.1.7",
"@vitejs/plugin-react": "^5.0.4",
"jsdom": "^27.0.0",
"typescript": "^5.9.2",
"vite": "^7.1.7",
"vitest": "^3.2.4"
"@tauri-apps/cli": "2.8.4",
"@types/react": "19.1.10",
"@types/react-dom": "19.1.7",
"@vitejs/plugin-react": "5.0.4",
"jsdom": "27.0.0",
"typescript": "5.9.2",
"vite": "7.3.6",
"vitest": "3.2.7"
}
}
Loading
Loading