HIGGS: ILP-based mixed-precision quantization

[HDCharles]

Summary

• Adds HIGGS (Heuristic ILP-Guided Grouped Scheme) module for automatic mixed-precision quantization
• Uses per-layer MSE sensitivity analysis and integer linear programming to assign optimal quantization schemes per layer, minimizing expected quality degradation while respecting average bitwidth constraints
• Model-free: operates directly on safetensors checkpoints without loading the full model into GPU memory
• Supports dual weight/activation bitwidth budgets for WNaM scheme selection (W2A4..W8A16)
• Includes depth-based alpha heuristic for layer importance weighting, validated across 5 model architectures
• Fixes transposed MoE expert tensor handling (Llama-4) in split_fused_moe_experts

Architecture

ilp_quantize()
  ├── Phase 1: MSE Collection + ILP Solve (no model load)
  │   ├── HiggsMSECollectorConverter — per-layer MSE under each candidate scheme
  │   ├── compute_heuristic_alphas() — depth-based importance weights
  │   ├── detect_fused_groups() — ensure fused layers get same scheme
  │   └── solve_ilp_mixed_precision() — PuLP ILP with bitwidth constraints
  └── Phase 2: Apply Quantization (optional, quantize=False for config-only)
      └── HiggsQuantizationConverter — apply selected schemes via convert_checkpoint

Key experimental findings

• Alpha heuristic: Layer importance follows a quadratic curve in normalized depth, consistent across Llama, Qwen, Mistral, and Gemma architectures
• Activation quantization: INT8 and INT4 dynamic per-token activation quantization have zero measurable perplexity impact (Δ PPL ≤ ±0.001 across all tested configurations). The activation budget is purely a kernel selection knob, not a quality constraint.
• ILP allocation: With wide WNaM candidate sets (W2A8..W7A8+W8A16), the ILP uses ~4 adjacent bitwidths centered around the target, with architecture-specific spread

Test plan

• Unit tests pass: pytest tests/llmcompressor/transformers/compression/higgs/
• End-to-end: python examples/quantization_higgs/llama3_higgs_example.py
• Config-only mode: ilp_quantize(..., quantize=False) returns config without applying quantization
• Verify fused layer constraints: layers within the same transformer block get the same scheme

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0b31611a-eff0-4f16-ac0c-cacf4f02c7ee

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[mergify]

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews

	Protection	Waiting on
🔴	Require one maintainer review	👀 reviews

🔴 Require one maintainer review

Waiting for any of

• approved-reviews-by=HDCharles
• approved-reviews-by=brian-dellabetta
• approved-reviews-by=dsikka
• approved-reviews-by=kylesayrs
• approved-reviews-by=yiliu30

This rule is failing.

All PRs must have at least one approving review from a maintainer before merging.

• any of:
  • approved-reviews-by=HDCharles
  • approved-reviews-by=brian-dellabetta
  • approved-reviews-by=dsikka
  • approved-reviews-by=kylesayrs
  • approved-reviews-by=yiliu30
• #changes-requested-reviews-by = 0

[mergify]

The quality checks have failed. Please run make style and make quality under
the root directory to adddress the lint failures. You will need to install the
dev optional install to get the required linting packages:
https://github.com/vllm-project/llm-compressor/blob/main/CONTRIBUTING.md

[gemini-code-assist]

Code Review

This pull request introduces HIGGS, an ILP-based mixed-precision quantization framework, along with example scripts for quantization, perplexity measurement, and activation sweeps. It also updates MoE expert splitting to support transposed expert formats. Key feedback includes making the MoE transposition detection robust to sharded checkpoints by checking both gate_up_proj and down_proj shapes, fixing a bug in the perplexity script where prompt_logprobs=0 prevents logprob retrieval, and avoiding numerical instability in the ILP solver by explicitly constraining invalid schemes to zero instead of using a large penalty coefficient.

[gemini-code-assist]

In a sharded checkpoint, gate_up_proj and down_proj tensors for MoE experts are often split across different files (shards). If a shard only contains down_proj and not gate_up_proj, the current detection logic will fail to find gate_up_proj in tensors, defaulting is_transposed to False. This will cause down_proj to be split incorrectly or fail with a shape mismatch. We can make the transposition detection robust to sharding by checking the shapes of both gate_up_proj (where dim2 > dim1 indicates transposition) and down_proj (where dim1 > dim2 indicates transposition).

Suggested change

	is_transposed = False
	for name, tensor in tensors.items():
	if "gate_up_proj" in name and tensor.ndim == 3:
	is_transposed = tensor.shape[2] > tensor.shape[1]
	break
	is_transposed = False
	for name, tensor in tensors.items():
	if tensor.ndim == 3:
	if "gate_up_proj" in name:
	is_transposed = tensor.shape[2] > tensor.shape[1]
	break
	elif "down_proj" in name:
	is_transposed = tensor.shape[1] > tensor.shape[2]
	break

[gemini-code-assist]

Setting prompt_logprobs=0 in SamplingParams prevents vLLM from computing and returning prompt logprobs. As a result, output.prompt_logprobs will be None, causing the perplexity evaluation loop to skip all tokens and fail to compute perplexity (or divide by zero). To enable prompt logprob retrieval, prompt_logprobs must be set to >= 1 (typically 1 is sufficient to get the logprob of the actual token).

Suggested change

	sampling_params = SamplingParams(
	max_tokens=1,
	temperature=0.0,
	prompt_logprobs=0,
	)
	sampling_params = SamplingParams(
	max_tokens=1,
	temperature=0.0,
	prompt_logprobs=1,
	)

[gemini-code-assist]

Using a very large penalty coefficient like 1e10 in the objective function to force variables to zero can cause numerical instability and scaling issues in the ILP solver (PuLP/CBC). A much more robust and numerically stable approach is to add a constraint fixing the variable to 0 (i.e., x[layer][scheme] == 0) for any invalid/inapplicable schemes.

Suggested change

	# Skip if MSE is infinite (scheme not applicable to this layer)
	if mse == float('inf'):
	# Force this variable to 0 by adding very high cost
	objective_terms.append(1e10 * x[layer][scheme])
	else:
	objective_terms.append(mse * alpha * x[layer][scheme])
	# Skip if MSE is infinite (scheme not applicable to this layer)
	if mse == float('inf'):
	prob += (x[layer][scheme] == 0, f"InvalidScheme_{_sanitize_name(layer)}_{scheme}")
	else:
	objective_terms.append(mse * alpha * x[layer][scheme])

[mergify]

The quality checks have failed. Please run make style and make quality under
the root directory to adddress the lint failures. You will need to install the
dev optional install to get the required linting packages:
https://github.com/vllm-project/llm-compressor/blob/main/CONTRIBUTING.md