Qwen3.6 vLLM support: add save_format="full_wrapper" for quantized model save

[dogwood-flo]

Summary

• Added a save_format parameter ("auto" / "native" / "full_wrapper", default "auto") to Runner.save_quantized_model(). "full_wrapper" remaps a text-only quantized checkpoint (e.g. Qwen3.6) to the composite model.language_model.* namespace that vLLM's composite VLM loader expects (onecomp/runner.py).
• "full_wrapper" is scoped specifically to Qwen3.6 (model_type in qwen3_5_text / qwen3_5_moe_text) — any other model (including other VLMs) raises RuntimeError.

This pull request should be reviewed after merging (#35).

Background

Qwen3.6 quantizes as a text-only checkpoint, but vLLM's composite Qwen3_5ForConditionalGeneration loader expects the model.language_model.* namespace — a mismatch from the plain Hugging Face save layout.

Details

Qwen3.6 vLLM support: full-wrapper quantized model save

• Added a save_format parameter ("auto" / "native" / "full_wrapper", default "auto") to Runner.save_quantized_model(). "full_wrapper" saves a text-only quantized checkpoint (e.g. Qwen3.6) remapped to the composite model.language_model.* state_dict/config namespace that vLLM's full-wrapper VLM loading expects, and additionally saves AutoProcessor/AutoImageProcessor files (onecomp/runner.py)

Bug fix

• Fixed a SyntaxError in onecomp/utils/blockwise.py and broken save logic in Runner.save_quantized_model() (missing save_format parameter, dangling try without except, undefined variable reference), and removed dead duplicate code (onecomp/runner.py)

Test

• Added tests for save_format's namespace-detection/remap helpers and its Qwen3.6-only scoping, plus end-to-end checks that full_wrapper restores model.config after save and that the default save_format remains a no-op for non-Qwen models (tests/onecomp/runner/test_save_format_full_wrapper.py)

Corresponding documents have been modified.

Verified

Verified the implementation with the following script:

import gc

import torch
from vllm import LLM, SamplingParams

from onecomp import GPTQ, CalibrationConfig, ModelConfig, Runner, setup_logger


def main():
    setup_logger()

    # Step 1: Quantize with GPTQ
    save_dir = "./Qwen3.6-27B-gptq-4bit-vllm"

    model_config = ModelConfig(
        model_id="Qwen/Qwen3.6-27B",
        
    )
    quantizer = GPTQ(wbits=4, groupsize=128)
    calibration_config = CalibrationConfig(
        num_calibration_samples=128,
        max_length=512,
    )
    runner = Runner(
        model_config=model_config,
        quantizer=quantizer,
        calibration_config=calibration_config,
        qep=True,
    )
    # NOTE: The calibration settings above are kept compact so the demo
    # runs fast and may be insufficient for real quantisation.  For
    # higher quality, prefer the CalibrationConfig() defaults
    # (max_length=2048, num_calibration_samples=512).
    # For qep=False runs with large calibration data, also pass
    # ``batch_size`` as a CalibrationConfig argument, e.g.
    #   CalibrationConfig(
    #       max_length=2048,
    #       num_calibration_samples=512,
    #       batch_size=128,
    #   )
    # so that Runner.quantize_with_calibration_chunked runs instead of
    # a single all-at-once forward pass.
    runner.run()

    # Step 2: Save the quantized model
    runner.save_quantized_model(save_dir, save_format="full_wrapper")

    # Free GPU memory used by quantization before loading vLLM
    del runner
    gc.collect()
    torch.cuda.empty_cache()

    # Step 3: Load the quantized model with vLLM.
    # gpu_memory_utilization=0.78 leaves headroom for the residual
    # quantizer process (~16 GiB) on a UMA 121.7 GiB device (e.g. DGX
    # Spark / GB200). The vLLM default 0.92 cgroup-OOMs on shared-memory
    # GPUs.
    llm = LLM(
        model=save_dir,
        max_model_len=512,
        dtype="float16",
        enforce_eager=True,
        gpu_memory_utilization=0.78,
    )

    # Step 4: Generate text
    prompts = [
        "Explain what post-training quantization is in one sentence:",
        "The capital of France is",
    ]

    outputs = llm.generate(prompts, SamplingParams(max_tokens=64, temperature=0.0))

    for output in outputs:
        print(f"Prompt:   {output.prompt}")
        print(f"Response: {output.outputs[0].text}")
        print()


if __name__ == "__main__":
    main()