Skip to content

Harden TriAttention model calibration preflight - #47

Merged
aivrar merged 3 commits into
mainfrom
codex/triattention-model-aware-calibration
Aug 21, 2026
Merged

Harden TriAttention model calibration preflight#47
aivrar merged 3 commits into
mainfrom
codex/triattention-model-aware-calibration

Conversation

@aivrar

@aivrar aivrar commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Summary

  • validate the matching Hugging Face model context, nested RoPE metadata, and runtime-version difference before downloading weights
  • estimate official one-shot calibration memory and support explicit per-GPU cuda:N selection
  • add a compatibility runner for nested RoPE and selected-device handling, with UI and documentation updates

Mythos-nano-heretic / IQ4_XS findings

  • IQ4_XS describes the inference GGUF; calibration still loads the source checkpoint in BF16
  • the model declares a 131,072-token context, so a 200,000-token one-shot calibration is rejected
  • its nested RoPE theta is 1,000,000, conflicting with the legacy 10,000 fallback exposed by the pinned runtime; the wrapper now selects the nested value
  • a 200,000-token official run has an estimated 36.3 GiB memory floor and cannot fit a 24 GiB GPU

Verification

  • python -m ruff check (changed Python files)
  • python -m compileall -q multi_turboquant run_ui.py
  • python -m pytest -q (443 passed, 14 skipped)

@aivrar
aivrar marked this pull request as ready for review August 21, 2026 03:05
@aivrar
aivrar merged commit cda91a0 into main Aug 21, 2026
4 checks passed
@aivrar
aivrar deleted the codex/triattention-model-aware-calibration branch August 21, 2026 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant