Issue Description:
I was reading the Intern-S2-Mobius paper and was very interested in its claimed "Dynamic Latent Reasoning" and "Multi-Token Prediction" capabilities, which are said to contribute to its inference efficiency.
However, when I examined the official inference code in modeling_interns2_mobius.py, I found that the model's forward pass seems to follow a standard layer-by-layer Transformer structure. The generation process also appears to use the conventional autoregressive generate interface, without obvious parallel decoding or multi-token prediction heads (e.g., MTP layers).
My Questions:
More importantly, how is the "Multi-Token Prediction" or "Dynamic Latent Reasoning" mechanism actually implemented during inference?
Is it controlled by an external loop or special sampling strategy in a higher-level script (not yet pushed to the repo)?
Or is it a training-time technique that enables the model's hidden states to encode richer information, which can then be decoded more efficiently with standard autoregressive generation?
Could you provide a code example or point me to the relevant part of the codebase that performs this "few-layer iterative refinement" and "parallel decoding" during inference?
Thank you for your great work and for open-sourcing the model!
Issue Description:
I was reading the Intern-S2-Mobius paper and was very interested in its claimed "Dynamic Latent Reasoning" and "Multi-Token Prediction" capabilities, which are said to contribute to its inference efficiency.
However, when I examined the official inference code in modeling_interns2_mobius.py, I found that the model's forward pass seems to follow a standard layer-by-layer Transformer structure. The generation process also appears to use the conventional autoregressive generate interface, without obvious parallel decoding or multi-token prediction heads (e.g., MTP layers).
My Questions:
More importantly, how is the "Multi-Token Prediction" or "Dynamic Latent Reasoning" mechanism actually implemented during inference?
Is it controlled by an external loop or special sampling strategy in a higher-level script (not yet pushed to the repo)?
Or is it a training-time technique that enables the model's hidden states to encode richer information, which can then be decoded more efficiently with standard autoregressive generation?
Could you provide a code example or point me to the relevant part of the codebase that performs this "few-layer iterative refinement" and "parallel decoding" during inference?
Thank you for your great work and for open-sourcing the model!