Conversation
Preserve independent Thinker/Talker configs and export the codec embedding plus Talker decoder graphs. Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
|
|
Performance Comparison
|
🏗️ Architecture Diff
No architecture changes detected. ✅ Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed) |
Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
🟡 Changes recommended
Talker weight loading, MRoPE extraction, hidden-state availability, and GQA cache metadata contain blocking correctness issues.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds Qwen2.5-Omni Talker ONNX export while preserving Thinker-only builds.
Changes:
- Adds Talker configuration extraction and dtype propagation.
- Adds codec embedding, Talker decoder, and KV-cache graphs.
- Extends graph and weight-routing tests.
File summaries
| File | Description |
|---|---|
src/mobius/_builder.py |
Resolves Thinker config and propagates dtype to Talker. |
src/mobius/_configs/_base.py |
Adds Talker dimensions and nested configuration. |
src/mobius/models/__init__.py |
Exports Talker model classes. |
src/mobius/models/qwen25_omni.py |
Implements Talker model and weight routing. |
src/mobius/models/qwen25_omni_test.py |
Tests config extraction and weight routing. |
src/mobius/tasks/_qwen25_omni.py |
Builds Talker embedding and decoder graphs. |
tests/build_graph_test.py |
Verifies six-model and Thinker-only packages. |
Review details
- Files reviewed: 7/7 changed files
- Comments generated: 4
- Review effort level: Balanced
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| # The host combines Thinker reply states, text embeddings, and codec | ||
| # embeddings in the shared embedding space before this projection. | ||
| hidden_states = self.thinker_to_talker_proj(op, inputs_embeds) |
| models["talker_embedding"] = self._build_talker_embedding( | ||
| module.talker.model.embed_tokens, config.talker |
| if model_type == "qwen2_5_omni_text" and parent_config is not None: | ||
| talker_config = getattr(parent_config, "talker_config", None) | ||
| if talker_config is not None: | ||
| if isinstance(talker_config, dict): | ||
| talker_config = type("TalkerConfig", (), talker_config)() | ||
| options["talker"] = ArchitectureConfig.from_transformers(talker_config) |
Register the specialized tiny graph config and public checkpoint used by L1 and L2 coverage checks. Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Preserve the reviewed Thinker implementation and migrated test layout while integrating Talker graph export, nested configuration, and coverage metadata. Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Keep the reviewed full-checkpoint Thinker selection while extending its regression coverage to the Talker models. Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Disable the optional Talker graphs in the Thinker-only synthetic parity test so its strict weight mapping checks only the four components represented by the Hugging Face reference model. Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Expect the optional Talker embedding and decoder in the representative six-model speech package across supported graph dtypes. Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Exclude the six-stage multimodal Talker pipeline from the generic L4/L5 golden runner while retaining its dedicated L1 graph and L3 Thinker parity coverage. Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Expose normalized Thinker hidden states for Talker orchestration, align the directly traced codec embedding weight, preserve non-interleaved MRoPE sections, and stamp Talker cache metadata for fused attention exports. Signed-off-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Findings[P2] Honor the checkpoint's audio-output disable flagFile: The Talker is enabled whenever Exporting such a Thinker-only or fine-tuned checkpoint currently creates Gate Talker configuration extraction on [P2] Add numerical parity coverage for the new Talker pathFile: The existing Qwen2.5-Omni synthetic parity test explicitly sets Errors in Talker weight routing, the 3584-to-896 projection, MRoPE application, attention biases, or cached decoding can therefore produce structurally valid ONNX graphs with incorrect codec logits while all current tests pass. Extend synthetic parity coverage to compare SummaryThe PR adds nested Talker configuration extraction, Talker embedding and decoder models, Thinker hidden-state output, six-component Qwen2.5-Omni packaging, Talker weight routing, and graph/configuration tests. It should respect checkpoints that disable audio output and add numerical coverage for the new model path. |
Summary
Validation