A passive scanner that reads your Genshin Impact collection off the screen — and never touches the game.
You click through your own Bag and character screens. It watches the window, reads each panel with a line-recognition model trained on the game's own font, validates every number against the game's tables, and writes a GOOD file that Genshin Optimizer imports — artifacts, weapons and characters.
No input is ever sent to the game. No memory is read. It is a screen reader with a parser attached, the same capture API OBS uses.
| Target | Fields |
|---|---|
| Artifacts | set · slot · rarity · level · main stat · substats (incl. unactivated) · equipped-on · lock |
| Weapons | name · type · rarity · level & ascension · refinement · equipped-on · lock |
| Characters | key · level & ascension · constellation · talent levels (constellation boosts removed) |
Anything the parser cannot verify is rejected, never written — a wrong digit is worse than a missing entry.
Everything it has logged is browsable in the app, with rarity, refinement, constellation and talent levels read at a glance:
Requirements — Windows 10/11 · Genshin Impact at 1920×1080 borderless, English client.
Install — grab the -setup.exe from the
latest release.
It installs per-user, so there is no admin prompt.
The installer is unsigned, so Windows SmartScreen shows "Windows protected your PC". Choose More info → Run anyway. Signing a Windows binary means buying a certificate; this is a free open-source tool, so it ships unsigned.
Scanning
- Open the Models tab and install the Genshin Impact model (2.7 MB) — models are downloaded per game, so you only store what you scan.
- Launch the game (1920×1080 borderless) and the app.
- Pick where the GOOD file goes, press Start.
- Click through Bag → Artifacts, Bag → Weapons, and each character's Attributes / Constellation / Talents tabs. Each panel logs as you land on it.
- Import the file in Genshin Optimizer: Settings → Database → Import → GOOD.
The window can float over the game (On top) or drop to the tray (minimize) — scanning continues either way.
From source — needs Rust and Node 20+:
npm install
npm run tauri dev # develop
npm run tauri build # NSIS installer -> target/release/bundle/nsis/A bare clone builds the app and passes every unit test. The labeled corpus and trained models are bulk data and live in OpenCollectionReader-data — pull a game's worth only when you want the accuracy harness or training:
python scripts/fetch-data.py genshin # corpus + model, SHA-verified
cargo test -p ocr-core --test corpus -- --nocapture # replay all 495 panels
Windows Graphics Capture ── 10 frames/s, cropped per profile rect
│
├─ detect ─────── has the panel changed and settled? (hash + stability gate)
│ character screens: tab glow, element badge, header signature
│ 6 settled frames → per-pixel minimum wipes drifting sparkles
│
└─ queue ───────► reader thread (bounded, 16 deep)
│
├─ normalize per-region: invert, colour-axis projection,
│ contrast stretch → 32px line
├─ recognize CNN + CTC, ~15 regions read in parallel
├─ parse snap names to GOOD keys, verify numbers
│ against main-stat curves, roll values, caps
└─ store dedupe → GOOD v3 file (atomic write)
Capture and detection stay on the capture thread; recognition runs on its own thread behind a bounded queue, so scrolling is never blocked by a read.
| Panel | Read time |
|---|---|
| Artifact | ~115 ms |
| Weapon | ~102 ms |
| Character · Attributes | ~183 ms |
| Character · Constellation | ~3 ms (ring geometry, no text) |
| Character · Talents | ~200 ms (up to 12 candidate lines confirmed) |
Medians measured from the app's own log across 175 reads in the current build. Scroll speed is limited by the game's panel animation, not the reader.
A small CNN + CTC line recognizer, one per game, run in-process by
tract — no Python and no service: reading
happens entirely on your machine. It takes a whole region as one sequence, so
kerned pairs never have to be cut apart.
Models are installed, not bundled. A profile per game means bundling would ship every game's weights to every user; instead the app fetches only the models you ask for, verifies each against the SHA-256 published in the data repo's catalogue, and stores them in its data folder. That is the app's only network access, and it happens only when you press Install.
| Architecture | 5 conv blocks → 1×1 conv → per-column logits, greedy CTC decode |
| Parameters | 687,148 |
| Genshin model | genshin.onnx — 2.7 MB, charset of 75 glyphs, revision 4 |
| Input | one grayscale line, 32 px tall, padded to 512 px |
| Confidence | per-line minimum over value-bearing characters; low-confidence regions are gated by the parser |
Real captures from live play, labeled in the app's own Review tab, plus synthetic lines rendered from the same font to cover rare glyphs.
| Source | Lines |
|---|---|
| Real (labeled panel regions) | 4,306 |
| Synthetic (rendered from the game font) | 5,240 |
| Total training lines | 9,546 |
The labeled panel corpus behind those lines:
| Kind | Panels | Spread |
|---|---|---|
| Artifacts | 370 | 20 sets · 300× 5★, 70× 4★ · 38 different equipped characters |
| Weapons | 34 | all five types · 10× 3★, 23× 4★, 1× 5★ |
| Character tabs | 91 | 31 Attributes, 31 Constellation, 29 Talents |
| Electro 21 · Pyro 15 · Cryo 14 · Anemo 12 · Geo 12 · Dendro 9 · Hydro 8 | ||
| Total | 495 |
24 epochs · batch 32 · AdamW @ 1e-3 · trained on CPU in 79 minutes.
Every run leaves training/runs/<run>/ with config.json, metrics.csv,
summary.json and this graph (training/plot.py).
| Metric | Result |
|---|---|
| Best validation line accuracy | 99.81 % (8 wrong of 4,306 lines) |
| Best character error rate | 0.021 % |
| Final training loss | 0.0005 |
cargo test -p ocr-core --test corpus -- --nocapture replays every labeled
panel through capture-shaped crops, the recognizer and the parsers:
| Corpus | Panels | Perfect | Wrong-but-accepted |
|---|---|---|---|
| Artifacts | 370 | 370 (100 %) | 0 |
| Weapons | 34 | 34 (100 %) | 0 |
| Character tabs | 91 | 91 (100 %) | 0 |
| Total | 495 | 495 (100 %) | 0 |
"Perfect" is strict: every field of the panel must match the label exactly. One wrong digit, one wrong boost flag, or one region the model was unsure about counts as a miss. The number that matters most is the last column — across all 495 panels the pipeline never accepted a wrong value.
Recording, labeling, adopting and training are all first-class: see training/README.md.
# 1. record → Scan tab, tick "Record panels for training"
# 2. label → Review tab (the model pre-fills, you correct)
python scripts/fetch-data.py genshin # 0. pull the existing corpus + model
python scripts/adopt-samples.py # 3. adopt into fixtures/samples
cargo run -p ocr-core --example export_lines # 4. export training lines
python training/train.py --split even # 5. honest number (train even ids, validate odd)
python training/train.py --split all # ship: train on everything
cargo test -p ocr-core --test corpus -- --nocapture # 6. measure the pipeline
python scripts/pack-corpus.py genshin # 7. publish: zip + catalogue snippetGrown corpora and retrained models are published as
OpenCollectionReader-data
releases — the app and fetch-data.py read its catalogue, so shipping a
better model never needs an app release.
crates/ocr-core/ the engine — game-agnostic
capture.rs Windows Graphics Capture → cropped frames
detect.rs "is this a new, settled panel?"
crnn.rs ONNX line recognizer + preprocessing
ocr.rs region → text, fanned across threads
pipeline.rs capture thread → bounded queue → reader thread
store.rs / model.rs GOOD v3 shapes and atomic writes
profile.rs the seam: geometry, model, parsers, per-game predicates
games/genshin/ everything Genshin: vocab, parsers, rarity tables, geometry
src/ React UI — Scan · Collection · Review · Log
src-tauri/ Tauri shell, tray, window chrome
training/ dataset, model, train.py, plot.py, per-run metrics
fixtures/samples/ labels.jsonl + exemplar panels (full corpus: the data repo)
scripts/ adopt samples, generate rarity tables, icon, CDP helpers
A second game is a second profile, not a rewrite. Engine modules never
import games/* outside tests; geometry, colours, vocab and parsers reach the
engine as profile data and function pointers. What generalizes when HSR or ZZZ
arrives — and what deliberately waits until their shapes are known — is written
down in the design spec.
- Passive by design. No synthetic input, no injection, no memory reads — it only looks at pixels the game already drew.
- Rejects are kept. Anything that fails validation is saved to the rejects folder so it can be labeled and folded into the next training run.
- Your data stays local. The GOOD file, samples and logs never leave your machine. The app's only network access is downloading a model you asked for.
Fonts (Chakra Petch, IBM Plex Mono) are bundled under the SIL Open Font License —
see public/fonts/. Genshin Impact is a trademark of HoYoverse; this project is
unaffiliated and reads only what is already on your screen.