Repository navigation
#3750 PR B: the 17 header-only whole-file model reads outside apr qa — magic/header/metadata readers in their format crates #3761
Copy link
Copy link
Open
Labels
P2Medium priorityMedium prioritykind:codeWork is a code change (derived rule, #4159)Work is a code change (derived rule, #4159)owner:la-0.71Owned by the 0.71 look-ahead slotOwned by the 0.71 look-ahead slotsize:MK̂ 3-4 hK̂ 3-4 h
Milestone
Description
Sub-issue of #3750 (split ruled by the cop, 2026-09-21): PR A (the apr qa path, the sweep ledger docs/audits/3750-whole-file-model-reads.md, and the gx10 before/after) lands first. This is PR B, in 0.69.1 because #3745's derived cells run every model-taking verb (lint, rosetta inspect, eval, the converter metadata paths) on every inventory model, including the 30B on gx10, where each of these reads is a 17 GiB allocation.
These 17 sites read the WHOLE model to use its magic, header or metadata. The ledger's reading is quoted per row:
-
apr-cli/src/commands/benchmark.rs:104run_realizar_benchmark — PR-B MAGIC-ONLY: bytes used only for detect_format on the first 8 -
apr-cli/src/commands/benchmark.rs:167run_gguf_benchmark — PR-B HEADER-ONLY: GGUFModel::from_bytes only for the tokenizer; the model is mapped separately -
apr-cli/src/commands/embed_viz.rs:382gguf_vocab — PR-B HEADER-ONLY: LlamaTokenizer::from_gguf_bytes needs only the header vocabulary -
apr-cli/src/commands/eval/mod.rs:746count_safetensors_keys — PR-B HEADER-ONLY: SafeTensors: 8-byte length + JSON header, counts keys -
apr-cli/src/commands/eval/mod.rs:922verify_single_file — PR-B HEADER-ONLY: SafeTensors header checks; the size check needs only the file length -
aprender-core/src/format/converter/apr_export_fn.rs:133detect_apr_architecture_for_completeness — PR-B HEADER-ONLY: APR metadata architecture only -
aprender-core/src/format/converter/tensor.rs:202extract_apr_tokenizer_hint — PR-B HEADER-ONLY: tokenizer hint from the APR metadata section -
aprender-core/src/format/converter/tensor.rs:226read_apr_metadata — PR-B HEADER-ONLY: APR metadata only -
aprender-core/src/format/converter/tensor.rs:382extract_user_metadata — PR-B HEADER-ONLY: user metadata from the APR metadata section -
aprender-core/src/format/converter/tensor.rs:439detect_apr_quantization — PR-B HEADER-ONLY: counts tensor dtypes from the APR tensor index -
aprender-core/src/format/lint/lint.rs:187lint_safetensors_file — PR-B HEADER-ONLY: SafeTensors metadata from the header; tensors come from the existing map -
aprender-core/src/format/lint/lint.rs:324lint_apr_v2_file — PR-B HEADER-ONLY: lints APR metadata fields -
aprender-core/src/format/onnx/reader.rs:467is_onnx_file — PR-B MAGIC-ONLY: checks data[0] == 0x08 only -
aprender-core/src/format/rosetta/validate_inspect.rs:552inspect_apr — PR-B HEADER-ONLY: rosetta inspect: metadata + tensor index entries -
aprender-core/src/format/safetensors.rs:448list_tensors — PR-B CONDITIONAL: GGUF/APR v1 listing: data only with --stats -
aprender-serve/src/apr/helpers.rs:309is_apr_file — PR-B MAGIC-ONLY: data[0..4] == MAGIC only -
aprender-serve/src/apr/helpers.rs:325format_from_magic — PR-B MAGIC-ONLY: format from the first 4 bytes only
done_when
- each site above reads only what it uses (a bounded magic read, or the format's header prefix); the APR v2 prefix reader lives in apr-format and the SafeTensors one in aprender-core, sharing ONE bounded-prefix policy (first read + cap) with apr-cli's model_header.rs
- the
list_tensorsGGUF/APR v1 path reads tensor data only when--statsasks for it - a case row per reader proving it never touches tensor data (a sparse multi-GB fixture + a peak-RSS bound, as PR A's), and a mutant that puts a whole-file read back goes RED
- the ledger's rows are updated to CONVERTED
Refs #3750
Metadata
Metadata
Assignees
Labels
P2Medium priorityMedium prioritykind:codeWork is a code change (derived rule, #4159)Work is a code change (derived rule, #4159)owner:la-0.71Owned by the 0.71 look-ahead slotOwned by the 0.71 look-ahead slotsize:MK̂ 3-4 hK̂ 3-4 h
Activity
PMAT-3714 R1: qa's MoE predicate streams the GGUF header instead of p…
noahgift commentedon Sep 23, 2026
Milestone moved 0.69.1 → 0.70.0. Reason: #3750 PR B, header-only reads outside qa: hygiene (0.70 Fast Train). Source: the #3988 disposition table (#3988 (comment)), row MISLABELED. Approval: operator, 2026-09-23 (relayed by cop aprender-cf: "yes apply both"). Applied by aprender-cb. Nothing was closed.
fix(#3761): header-only GGUF listing leaked the core error prefix (#3661
fix(#3761): apr inspect on a GGUF reads the header, not 16.7 GB — #4520…
fix(#3761): header_from_file grows through the capped prefix policy —…
3 remaining items
noahgift commentedon Oct 14, 2026
merged into #5305 (master ticket for epic #3598; triage rule 7b, 2026-10-14). It is a checklist row there; reopen this issue when work on it starts or its PR opens.
noahgift commentedon Oct 14, 2026
Moved: this sub-ticket is now a row in #5321 (apr memory footprint: serve Qwen3.5 within 8 GB unified-memory ARM64 CUDA (Orin)), split out of #5305 by theme.