v0.1a: first local Hindi Translate/Explain provenance slice

README.md

@@ -8,9 +8,9 @@ A privacy-first, open-source document reader whose goal is simple:
 
 The project started from a medical-education use case but is intentionally generic. Medical science will become an optional domain pack rather than the core application.
 
-## Current milestone: v0.0.2
+## Current milestone: v0.1a
 
-v0.0.1 proved that the application can be built, installed and used on a clean Windows machine. v0.0.2 hardens the provenance layer before any local AI is introduced.
+v0.0.1 proved clean-machine Windows installation. v0.0.2 added durable source provenance and sidecars. v0.1a adds the first end-to-end **local Hindi Translate/Explain path** through that provenance architecture.
 
 Current capabilities include:
 
@@ -20,12 +20,27 @@ Current capabilities include:
 - SHA-256 identity for source documents and normalized source quotes;
 - durable per-document JSON sidecars stored in application data;
 - fallback source restoration when PDF text-item boundaries shift;
-- visible **Translate / Explain / Ask** modes, still deliberately non-generative;
+- separate **Translate / Explain / Ask** modes;
+- a tiny built-in **Hindi reference engine** for deterministic Translate/Explain regression passages;
+- generated-output provenance including language, level, engine id/version/local flag, verification state and review state;
+- restoration of saved Translate/Explain output when the same source is reopened;
 - persistent Light/Dark mode;
 - visible **LOCAL ONLY** state;
 - Windows NSIS packaging with CI-generated SHA-256 installer manifest.
 
-There is still **no AI model, OCR or runtime internet verification path** in v0.0.2.
+### Important v0.1a limitation
+
+The built-in Hindi engine is **not a general translator**. It contains a very small, explicit regression catalog spanning medical, administrative and safety examples. Unknown text is rejected as unsupported rather than fabricated.
+
+This milestone exists to prove:
+
+`source → local engine → generated record → sidecar → close/reopen → provenance restore`
+
+before introducing a heavyweight local model runtime and model-pack distribution system.
+
+See `docs/LOCAL_HINDI_REFERENCE_SLICE.md` and ADR 0002 for the exact scope and rationale.
+
+There is still **no OCR, runtime internet verification path, or general local AI model** in v0.1a.
 
 ## Product principles
 
@@ -36,14 +51,15 @@ There is still **no AI model, OCR or runtime internet verification path** in v0.
 5. **Domain knowledge is modular.** Medical, administrative, technical, safety and education packs extend the generic reader.
 6. **Every explanation remains anchored to its source.** The user can always see the exact page and selected text.
 7. **Installable from the beginning.** Releases should be usable on another computer without Python, Git, Conda or a terminal.
+8. **Unsupported beats fabricated.** A local engine that cannot responsibly handle a passage must say so.
 
 ## Development policy
 
 `PROJECT_MANDATE.md` is the standing execution mandate for AI-assisted development. `PRODUCT_SPEC.md`, `ARCHITECTURE.md`, `ROADMAP.md` and `RELEASE_CHECKLIST.md` are project sources of truth.
 
 ## Status
 
-This repository is in the provenance-hardening stage. Do not use it for clinical decisions, safety-critical instructions or authoritative translation yet.
+This repository is in the local-intelligence architecture stage. Do not use it for clinical decisions, safety-critical instructions or authoritative translation yet.
 
 ## License
 

ROADMAP.md

@@ -20,7 +20,7 @@ Completed and clean-machine validated on Windows:
 - Windows NSIS packaging;
 - synthetic smoke-test document.
 
-## v0.0.2 — Robust source anchoring and release hardening (current)
+## v0.0.2 — Robust source anchoring and release hardening ✅
 
 - SHA-256 document identity;
 - SHA-256 normalized quote identity;
@@ -33,19 +33,35 @@ Completed and clean-machine validated on Windows:
 - persistent Light/Dark mode;
 - cross-file version consistency checks;
 - validated Windows installer checksum manifest;
-- release-validation checklist.
+- clean-machine persistence validation.
 
-Exit criterion: a stored source/provenance record survives application restart, reopens against the same SHA-256-identified PDF, reliably restores/highlights its source context, and the v0.0.2 Windows installer passes the documented clean-machine validation including persistent dark mode.
+## v0.1a — Local Hindi reference intelligence (current)
 
-## v0.1 — First local understanding loop
+Goal: prove the complete local-generation lifecycle without coupling it to a heavyweight model runtime.
 
-- local translation engine;
-- English -> Hindi vertical slice;
-- local explanation engine;
+- deterministic built-in Hindi reference engine;
+- Translate and Simple Explain operations;
+- medical, administrative and safety regression passages;
+- explicit `unsupported` behavior for unknown text;
+- generated-output sidecar records with language, level and engine provenance;
+- externally-verified and human-reviewed states remain explicit and false by default;
+- saved generated outputs restore when the same source is reopened;
+- no runtime network path added;
+- Windows installer and CI validation retained.
+
+Exit criterion: a supported English passage can be translated and explained in Hindi locally, each output is separately persisted with its exact source anchor and engine provenance, and the output survives restart/reopen. Unknown passages must be rejected rather than fabricated.
+
+## v0.1 — First production local understanding loop
+
+- replace the reference engine behind the same contract with a distributable local model/runtime;
+- English -> Hindi general translation vertical slice;
+- local explanation engine beyond the fixed regression catalog;
 - document-grounded Ask mode;
 - clear provenance labels for Translate / Explain / Ask;
 - no document upload required;
-- model pack manifest and installation path.
+- model pack manifest and installation path;
+- CPU-only Windows benchmark and memory/latency targets;
+- model/runtime license documentation.
 
 Exit criterion: a normal Windows user can install the app + required model pack and use Translate, Explain and Ask on a local English PDF without internet access.
 

docs/LOCAL_HINDI_REFERENCE_SLICE.md

@@ -0,0 +1,92 @@
+# Local Hindi Translate/Explain reference slice
+
+## Purpose
+
+v0.1a is the smallest end-to-end proof that a source-anchored passage can be sent to a **local understanding engine**, produce a Hindi output, persist that output in the document sidecar, close/reopen the app, and restore the generated result with provenance intact.
+
+It is deliberately **not** a general-purpose translator yet.
+
+## Why a reference engine first
+
+A production Indic model introduces several independent engineering problems at the same time:
+
+- model-pack download and offline installation;
+- inference runtime and CPU/GPU fallback;
+- RAM/disk requirements;
+- tokenizer/model compatibility;
+- model and dataset licensing;
+- model versioning and upgrades;
+- packaging size and release distribution;
+- quality evaluation for translation and explanation.
+
+The reference engine lets the application prove the architectural path before those concerns are added.
+
+## Supported behavior
+
+The built-in engine is `odu.hindi-reference` version `0.1a.1`.
+
+It supports:
+
+- target language: **Hindi**;
+- operations: **Translate** and **Explain**;
+- explanation level: **Simple**;
+- a tiny deterministic regression catalog containing medical, administrative and safety passages.
+
+Unknown passages are returned as `unsupported`. The engine must never improvise an answer outside the catalog.
+
+## Provenance path
+
+```text
+PDF source selection
+    ↓
+SourceAnchor v2
+(document SHA-256 + quote SHA-256 + page/range)
+    ↓
+local Hindi reference engine
+    ↓
+translate OR explain output
+    ↓
+ProvenanceRecord
+(operation + output + language + level + engine metadata)
+    ↓
+per-document local JSON sidecar
+    ↓
+close/reopen document
+    ↓
+output restored from the same sidecar
+```
+
+Every generated record explicitly stores:
+
+- operation (`translate` or `explain`);
+- exact source anchor;
+- generated output;
+- target language;
+- explanation level where applicable;
+- engine id/version/local flag;
+- `externallyVerified: false`;
+- `humanReviewed: false`.
+
+## Privacy
+
+The reference engine is ordinary TypeScript bundled inside the desktop application. It performs no HTTP requests and adds no network permission or runtime endpoint. The current CSP remains local-only.
+
+## Quality limitation
+
+A successful output means only that the architecture worked for a supported regression passage. It does **not** mean the application can accurately translate arbitrary English text or safely explain arbitrary medical/safety material.
+
+The UI therefore labels generated output as coming from a **LOCAL REFERENCE ENGINE** and displays that it has not been externally verified or human reviewed.
+
+## Next step
+
+Replace the deterministic reference implementation behind the same engine contract with an installable local model pack. The first production candidate should be evaluated separately for:
+
+1. English → Hindi translation quality;
+2. simple-English/Hindi explanation quality;
+3. CPU-only latency and RAM on ordinary Windows laptops;
+4. distributable model license;
+5. fully offline installation;
+6. medical terminology preservation;
+7. deterministic provenance/version identification.
+
+The model-pack runtime must not require changes to the sidecar provenance contract introduced here.

docs/adr/0002-reference-engine-before-model-runtime.md

@@ -0,0 +1,44 @@
+# ADR 0002: Prove local intelligence with a deterministic reference engine before adding a model runtime
+
+- Status: Accepted
+- Date: 2026-09-15
+
+## Context
+
+The product needs real local Hindi translation/explanation, but introducing a production model immediately would couple provenance work to model packaging, runtime selection, hardware fallback, licensing, memory/performance and distribution concerns.
+
+The provenance layer is already stable enough to store generated outputs. We need to prove the entire generated-output lifecycle independently from model quality.
+
+## Decision
+
+Introduce a tiny deterministic in-process Hindi reference engine with an explicit regression catalog. It supports only known passages and returns `unsupported` for everything else.
+
+The engine uses the same interface and provenance metadata expected from future model-backed engines:
+
+- operation;
+- output;
+- target language;
+- explanation level;
+- engine id/version/local flag;
+- exact source anchor;
+- external-verification and human-review flags.
+
+No network capability is added.
+
+## Consequences
+
+Positive:
+
+- UI → local engine → provenance record → durable sidecar → reopen/restore can be tested end-to-end now.
+- The application never pretends that a fixture-backed engine is a general translator.
+- Future model runtimes can replace the implementation without changing the sidecar contract.
+- CI remains fast and deterministic.
+
+Negative:
+
+- v0.1a is not useful for arbitrary documents yet.
+- The reference outputs must be clearly labelled to avoid giving users a false impression of coverage.
+
+## Follow-up
+
+A separate model-pack milestone will select and benchmark a distributable local English↔Indic translation/explanation runtime and preserve this provenance contract.

package.json

@@ -1,7 +1,7 @@
 {
   "name": "open-document-understanding",
   "private": true,
-  "version": "0.0.2",
+  "version": "0.1.0-alpha.1",
   "type": "module",
   "scripts": {
     "dev": "vite --port 1420",

schemas/document-sidecar.schema.json

@@ -27,6 +27,8 @@
           "createdAt": { "type": "string" },
           "output": { "type": "string" },
           "question": { "type": "string" },
+          "targetLanguage": { "type": "string" },
+          "explanationLevel": { "type": "string" },
           "externallyVerified": { "type": "boolean" },
           "humanReviewed": { "type": "boolean" },
           "engine": {

src-tauri/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "open-document-understanding"
-version = "0.0.2"
+version = "0.1.0-alpha.1"
 description = "Privacy-first local document understanding reader"
 authors = ["Open Document Understanding contributors"]
 edition = "2021"

src-tauri/tauri.conf.json

@@ -1,7 +1,7 @@
 {
   "$schema": "https://schema.tauri.app/config/2",
   "productName": "Open Document Understanding",
-  "version": "0.0.2",
+  "version": "0.1.0-alpha.1",
   "identifier": "org.opendocumentunderstanding.desktop",
   "build": {
     "beforeDevCommand": "npm run dev",