Perf: Replace full SVD with torch.svd_lowrank for acceleration - #115
Open
lantudou wants to merge 130 commits into
Open
Perf: Replace full SVD with torch.svd_lowrank for acceleration#115lantudou wants to merge 130 commits into
lantudou wants to merge 130 commits into
Conversation
Add PTQ entry scripts, calibration collectors, pipeline config, and example configs for Flux2 Klein and StrongGlassReflectionRemoval models.
The stock qdiff prompts are MSCOCO captions: one clause, no audio, no timeline. H3 packs video, audio and text into a single sequence, so a caption exercises almost none of what it sees in use -- no soundscape, no score, no shot structure. These 32 follow the model's own writing guide: the three core fields, shot numbering with increasing cut times, speaker ids outside the dialogue tags and verbatim text inside, and they span style, shot count, dialogue against silence, camera motion and on-screen text. Every one is exactly 336 tokens, which is what makes sample sharding possible. Measured on the collected caches: the video rows (37296) and audio rows (414) are fixed while the text rows are however long the prompt tokenizes to, with no padding and no attention mask, so unequal prompts are unequal shapes and the sharding fingerprint check rejects them. The length is reached by describing more -- another beat of the action, a sound that belongs to that place, a development in that score -- not by padding. Only the last few tokens are landed with a statement that the framing holds, which goes in the description because the guide caps the soundscape at four sentences and the authored text already uses them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…efault does not
H3 runs one stack over a single packed sequence that already holds the text,
audio and video rows, so a block takes one stream and returns one tensor -- and
three more positional arguments besides:
hidden_states = block(hidden_states, temb, adaln_indices, rotary_emb)
The default layout describes a dual-stream block with two streamed arguments in
and both returned. Left on it, the replay drops two of those four, and the
layout check refuses that outright rather than calibrate a block on a call it
never receives -- so the weight pass would have stopped at the first block,
after the collection had already run.
temb, adaln_indices and rotary_emb are CARRIED: every block in the stack gets
the same three. The token refiner carries no AdaLN and takes the stream alone.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The generic cache builder handles linears and convolutions and refuses everything else, so a module a pass replays whole has to be described by the plugin that declares it. Without that the smoothing pass stopped at the first attention with NotImplementedError -- after the two-hour collection had run. H3 attention is single-stream like the rest of the model: one packed sequence in, one tensor out. The joint layout the dual-stream models use names a second stream that does not exist here. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…beside it Streaming decided by top-level child: one that contains a block is the stack, so it stays on the host with them. That holds while such a child is only a stack, which it is for Flux, Klein and Boogu. H3 token refiner is a stack and a final norm. Left on the host, the norm met device tensors on the first forward -- weight is on cpu, different from other tensors on cuda:0. The placement now descends into a block-holding child and moves the parts that are not blocks; for a container of nothing but blocks the descent finds nothing, so every model that worked is placed as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…forked loader workers num_workers: 0 is load-bearing: a forked loader worker shares the rank's 64GiB weight heap copy-on-write, and the low-rank pass rewrites all of it, privatizing the shared pages one write at a time. 8 workers per rank turned that into ~230GiB of duplicated heap and an OOM kill; inline loading costs far less than the compute between loads. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The checkpoint stays memory-mapped (MAP_PRIVATE, shared page cache across ranks); a mutated layer is written whole to a per-stage scratch overlay and renamed into place, and a restore swaps parameters back to views of the newest version. Nothing writes the original checkpoint, so deleting overlay directories re-runs any suffix of the pipeline. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
What holds a full model in every rank is not loading -- the loaders keep safetensors mapped -- it is mutation: folding a smoothing scale privatizes a block~s pages in every rank, and by the end of the pass each rank owns the model. The layer iterator now closes that loop: after each visit the layer is written once to the pass~s overlay (main process), every rank re-points its parameters at views of that file, and the private copy is freed. The commit is unconditional -- read-only passes write too. The first A/B run proved why: smoothing and the low-rank pass had taken their load-from-cache fast paths, which bypass the iterator, so the overlay was empty and the weight-range pass~s restore reverted every block to the original checkpoint. A restore may only land on a version the same commit just wrote; whether the disk was otherwise current is not knowable from inside one pass. Verified on klein-4B, single-rank and 2-rank sharded: final quantized weights bit-identical with the store on and off (0/238 keys differ), and identical reruns are deterministic. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…autograd Two measured facts. The 44 GiB decode peak (513 GiB on the host) was never the decoder: it was autograd retaining every intermediate because callers ran the decode without inference_mode. And the diffusers H3 VAE chunks time but not space, so a 1344x768 frame cannot fit a 46 GiB card even without the graph. ComfyUI ships the missing half -- 256px spatial tiles with a 64px linear feather (comfy/ldm/minimax/vae.py) -- ported here and installed on the pipeline at build, preserving the decode contract. Measured on a 46 GiB card: 124 frames at 1344x768 decode in 27s at a 15.8 GiB peak, matching the untiled reference at 38 dB (mp4 noise). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The StageCheckpoint machinery (per-block record in all four calibration passes, manifest-after-file ordering, fingerprint guard) was complete but unreachable: ptq() took resume_dirpath and no caller ever passed it. Add a `resume` field to DiffusionPtqRunConfig and feed it through; the H3 run config sets it so long-haul runs resume at the dead block instead of restarting the stage. Verified by kill test on flux2-klein: killed at lowrank 3/25 with three block checkpoints on disk; rerun logged three blocks already calibrated, continued from block 3, and completed through model save. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The function adapts a joint attention's tuple output for evaluation -- a struct-layer concern with no quantization in it -- yet lived in quant/utils, forcing nn/struct to import it lazily up the stack: the one inverted edge in the dependency graph. Move it to nn/struct next to its only consumer, drop the two lazy imports, and keep a re-export in quant/utils so existing imports stay valid. WanAttentionStruct stays in models/generic.py: moving it into nn/struct is blocked by test_struct_module_is_model_agnostic, the repo's own invariant that the struct layer stays model-agnostic. Full suite identical to baseline: 450 passed, 159 subtests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The module was the stage visit driver every calibration pass runs through, but its name said "sharding" and it carried three unrelated jobs. Split by concern, all pure moves: - layer_visit.py: the visit driver (iter_layer_activations) and residency - sample_sharding.py: rank-splitting predicates and the kwargs fingerprint guard - store_setup.py: building/attaching the on-disk weight store and the per-layer commit protocol sharding.py stays as a re-export shim; internal imports point at the real homes, tests keep importing through the shim and double as its regression check. Gates: full suite identical to baseline (450 passed, 159 subtests); cache-warm klein run vs the pre-split off2-model reference is bitwise identical (model 0/238, scale 0/480, wgts 0/160 keys differ). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… the driver ptq.py was the driver plus everything around it (870 lines); it is now the driver (539). Two extractions, both verbatim moves: - pipeline_setup.py: build/target device choice, streaming placement around the blocks, weight-store attach, graph transforms and their pruning, and prepare_denoiser, whose ordering of all of the above is load-bearing. - text_encoder.py: the llm_ptq loop over the pipeline's text encoders. The LLM-stack imports leave the driver's top with it. Two latent bugs guarded while passing, per the architecture review: save_dirpath/save_model are now defined before the nf4/gguf branch that used to own them, so the LoRA and text-encoder regions no longer NameError on an nf4/gguf pipeline; and the per-encoder save path is derived rather than accumulated, so a second text encoder no longer nests under encoder/encoder. ptq.py re-exports the moved names; existing importers (tests, tools, infer_boogu_fakequant) are untouched. Gates: full suite identical to baseline (450 passed, 159 subtests); two cold klein runs, pre- and post-extraction, produce bitwise-identical artifacts (model 0/238, scale 0/480, wgts 0/160) and logs whose full-text diff is two launch timestamps. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…sh paths The four calibration stages each carried an inline copy of the same protocol -- resolve input from the explicit load directory, else the shared cache, else compute; store the computed result into the cache unless read-only; publish into the save directory as a symlink to whatever was loaded or stored, else a copy. stage_io.StageArtifact is that protocol once, with resolve/store/publish; the deliberate per-stage differences (the branch-only environment gate, activation quantization's cache condition, the weight/branch pair computed by one call) stay at the call sites. _link_cache_file moves with it; ptq.py re-exports it. Gates: full suite identical to baseline (450 passed, 159 subtests). Warm klein run vs the step-3 reference: model/scale/wgts bitwise identical, published symlink targets character-identical, absent files absent on both sides. Cold klein run vs the step-3 cold reference: model artifacts bitwise identical, cache tree file lists identical, wgts cache byte-identical, smooth/branch caches tensor-identical (their byte-level drift is pre-existing save nondeterminism, reproduced between two pre-change cold runs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…k checkpoints The per-stage artifacts and the per-pass resume checkpoints share their construction inputs and their lifetime, so one object now owns both: StageStore holds the path decisions ptq makes once (load directory, shared cache, save directory) and hands out StageArtifacts from them, alongside the StageCheckpoint dict. build_stage_checkpoints moves from ptq.py to stage_io.py with it. ptq() shrinks to stage orchestration: every artifact and checkpoint now comes from the store. Gates: full suite identical to baseline (450 passed, 159 subtests); warm klein run vs the step-4a reference bitwise identical (model 0/238, scale 0/480, wgts 0/160) with identical symlink targets. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ch state Calibration folds weights and installs runtime transforms (hooks and cache-key attributes) in one motion; only the weights reach model.pt. Rebuilding a saved model therefore relied on ordering magic: re-run the calibration appliers -- wrong weights, right hooks -- and count on a later load_state_dict to overwrite the weights. Violating that order produces pure noise, which the H3 video debugging hit in practice. The runtime half of each applier now stands alone: - apply_diffusion_smooth_runtime: cache-key attributes and ActivationSmoother hooks from a saved smooth cache, weights untouched. - apply_diffusion_branch_runtime: LowRankBranch hooks from a saved branch state without subtracting the residual (subtract= on apply_low_rank_branch_task), because a saved model.pt already carries the subtracted weights. load_diffusion_weights_state_dict loads weights first and attaches runtime second, and the driver's load-model path applies smooth runtime instead of fold-then-overwrite. Rebuild = load weights + apply smooth runtime + apply branch runtime, in any order. The calibration paths are untouched. Gates: 7 new contract tests (rebuild == calibrated, bit for bit, single and shared-branch; runtime application leaves weights alone); full suite 457 passed; rebuilding klein from a saved model through the driver under old and new code produces identical fingerprints for all 234 parameters, 320 hook sites, and 160 smooth attributes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The one deliberate numeric change of the refactor plan. Cache-cold and cache-warm runs of the same configuration produced deterministically different weights (160/238 keys on klein, single-tensor error up to ~0.6): every stage is written as "search loop when cold, apply loop when warm", and the two loops ran in different places. The search loops hold each block on the device; the apply loops walked the structs with no residency at all, so under offloading the same cached scales were applied on the CPU. bf16 arithmetic is not device-invariant, and GPTQ's re-gridding amplifies the ulp-level fold differences into visible ones. With quant.calib.apply_on_device, the smooth and branch apply loops hold each block on the device exactly as the search loops do (build_apply_residency in layer_visit). Default off in this commit; flipped once validated. Gates: full suite 457 passed. Warm klein run with the flag vs the cold run that wrote the shared cache: bitwise identical (model 0/238, scale 0/480, wgts 0/160) -- the divergence is zero. Warm run without the flag vs the step-4 baseline: bitwise identical -- default behavior unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Flip quant.calib.apply_on_device to True: cache-warm runs now reproduce their cache-cold run bit for bit by default. Validated before flipping -- a warm klein run with the new default is bitwise identical to the cold run that wrote the shared cache (model 0/238, scale 0/480, wgts 0/160 keys differ); full suite 457 passed. Set apply_on_device: false to reproduce artifacts quantized before this change from their caches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Behavior-preserving except where a silence becomes a message: - The GPTQ task-parallel orchestrator carried a row-sharding path that could never run: split_indexes was hardcoded empty, so _gather_row_shards and both split branches were unreachable while reading as live code. Excised. The GPTQ kernel keeps its row_shard capability -- correct and tested on its own -- with the comment explaining why the layer above cannot ask for it yet. - get_needs_outputs_fn (diffusion) always answered False while callers passed it along as a real predicate. Deleted; the call site now passes nothing and states the actual constraint. - The visited-layer total num_blocks + int(...)*3 was hand-repeated in three progress bars; num_visited_layers says it once. - Cache-compatibility checks were bare asserts, which vanish under python -O; the fuse-setting check is now an explicit ValueError naming both settings. - A configured weight store was silently discarded when a pass replays layers after visiting them; it now warns. Gates: full suite 457 passed; warm klein run bitwise identical to the cold reference (model 0/238, scale 0/480, wgts 0/160). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lantudou
force-pushed
the
main
branch
3 times, most recently
from
August 17, 2026 08:52
063c5ca to
091ea87
Compare
The public cut of the branch. Internal project code leaves (the sgfr plugin, its configs, registry entry, tests, and docs example -- the transformer-only guidance stays, written generically); internal filesystem paths become neutral defaults; the scale-sweep baseline tensors move into tests/data so the suite passes on a fresh clone. The README is rewritten for what this repository now is: a post-training quantization framework rebuilt around bringing up new diffusion models -- what the machinery it inherited could not do, what replaced it, how to add a model, and the credit and citation for the upstream quantization science (SVDQuant, mit-han-lab/deepcompressor).
…s found writing them
AGENTS.md is the entry point for whoever lands in this repository first:
repo map, the end-to-end workflow, the layered config system, the
invariants, and the traps that cost hours when hit blind. Three new
guides sit under docs/: architecture.md (layers, the life of a run,
stage IO, memory, the parallel axes), large_models.md (which knob for
which symptom when the model does not fit), troubleshooting.md (symptom
to cause, indexed by the error text the code actually raises).
graph_export.md is rewritten. It was a design memo -- it opened with an
implementation status, spent its first section arguing with an earlier
draft of itself, and ended in a TODO list. It now opens with when you
would want the tool and how to run it, and keeps the reference tables
that were the good part.
Four things the writing turned up, each verified and fixed here:
- The stage-checkpoint fingerprint refusal told the user to "Point
--resume-dir somewhere else". That flag does not exist; it is
--resume. A wrong instruction in an error message is worse than none.
- The per-stage file-cache override built its variable name from the
stage, so the wgts-range stage asked for
DEEPCOMPRESSOR_WGTS-RANGE_FILE_CACHE_DIR -- a name no shell can
export, leaving the override unreachable for the stage that spills
most. Hyphens now become underscores.
- WeightStore defaulted `stages` to ("smooth", "branch", "wgts",
"gptq"), which is not what any pass writes; every caller passed the
real names, so the default was dead and wrong at once. It is required
now.
- onnx and netron were imported by the graph exporter and its viewer but
declared nowhere. They are an optional dependency group: install with
`poetry install --with graph`.
Also: quant.calib.num_workers now defaults to 0. A worker is forked, so
it inherits the rank's weights copy-on-write, and every pass that folds
a scale or subtracts a branch residual privatizes those pages in each
worker, once per rank -- the failure mode that ended a run at 486 GiB of
host memory. Every document here told the reader to override the default
to 0, which is an argument for changing it. Calibration samples are
cached tensors read from disk, so serial loading costs little.
AGENTS.md leaves .gitignore: it is project documentation, not a personal
scratch file.
Full suite: 456 passed, 159 subtests.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… quantize The plugin declared registry and pipeline wiring and nothing else, so struct construction failed on the first block. It now declares what Z-Image is: a single-stream DiT whose noise and context refiners run in parallel before the main stack, a hand-written SwiGLU (w1 gates, w3 lifts, w2 down) the default layout cannot recognize, rotary embeddings applied inside the attention processor, and a block replay of (x streamed; attn_mask, freqs_cis, adaln_input carried). The struct vocabulary holds two block stacks and Z-Image has three parallel ones, so the main stack is quantized and the two refiners are declared as extras: named, visible, kept at the role table's embed precision rather than silently skipped. Folding them into calibration is the N-stack struct generalization, not a bigger plugin. Verified over tiny instances of every builtin plugin's model -- flux, sd3, pixart, sana, wan, flux2, qwenimage, zimage, minimax_h3 -- through patch, layout, struct construction, projection grouping and shared-input union: 9/9 pass. zimage joins the struct golden test (456 passed, 167 subtests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The UNet families predate the plugin system: their structure is wired into nn/struct.py directly, and their builds went through the legacy _default_build fallback with two checkpoint paths hardcoded by name. The plugin supplies what was actually missing -- name resolution and per-name default checkpoints (including the maintained SD 1.5 mirror, since runwayml/stable-diffusion-v1-5 was taken down) -- and pointedly declares no model or pipeline types: re-declaring them would route the UNet through the generic DiT factory tables and overwrite routing that already works. Model configs for sd1.5 and sdxl join examples/. Verified: all four names resolve to the plugin and nothing else; struct construction of a UNet still lands on UNetStruct with the plugin registered; the up-block skip convolutions are replaced by ConcatConv2d (4 on the tiny model) and reappear as per-segment key modules (.conv1.convs.0/.1) so each concatenated input gets its own scale; the patched model still runs forward. sd-unet joins the struct golden test (456 passed, 175 subtests). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t first register_plugin_struct_factories assumed every plugin's denoiser is a DiT: it force-registered the declared model and pipeline types into the DiT factory tables with overwrite=True. Top-level construction survived an SD plugin declaring UNet2DConditionModel only because an isinstance check happens to run before the plugin dispatch -- an invariant enforced nowhere -- and DiTStruct's table was polluted regardless, so a direct DiTStruct.construct on a UNet would misdescribe it instead of failing. The sd-unet plugin shipped with its type declarations stripped as a workaround, which cost model-to-plugin resolution. Binding now takes a claim snapshot before registering: a denoiser type already claimed in any of the three family tables (model, DiT, UNet) is being named for pipeline resolution, not re-described, and is left alone. The snapshot is what still lets a plugin's own new types land in both the model table and the DiT table, and it makes binding idempotent -- rebinding after a late registration changes nothing. The sd-unet plugin declares its model and pipeline types again. Pinned by tests/test_plugin_factory_binding.py: a UNet-claiming plugin cannot reroute core UNet construction, the DiT table stays clean of UNet types with the sd-unet builtin loaded, and binding twice leaves the tables byte-equal. Full suite 459 passed, 175 subtests; the plugin smoke over all nine families and the UNet ConcatConv smoke both pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This fork had never quantized a UNet end to end. DreamShaper 7 (SD 1.5) did, and three things fell over that a DiT never reaches: - ptq.main printed model.config after building the struct -- a debug line left by an early commit that only DiT structs happen to survive. UNetStruct has no config; the run died before smoothing. Removed. - The weight-quantizer coverage guard counted every projection in a block as one that must have settings. In a DiT block that is true. A UNet block holds its resnets' time projections and its transformer's proj_in beside the attention, and the role table keeps those in bf16, so their absence from the cache is the configuration's decision. The guard now asks the same question the range calibration asks -- _weight_quantizer_enabled by key -- so the two agree by construction. - model_outline assumed every block name is family.index. A UNet's mid_block has no index and the viewer died on rsplit. Block names are split tolerantly now, and mid_block is a family of one. Also: the fake-quant probe's reference weight on the load-model path. Since step 5, that path applies smoothing as runtime state only, and a probe snapshot taken there sees the unfolded W -- while the probe's reference run keeps the smoothing hook active and needs W*s. Every hooked module read as off by its own scale; the ones with the largest scales (GEGLU down projections, cross-attention keys) topped the worst list at 500%+ output error, cosine 0.92-0.98 -- a scale, not noise. A probe request now folds, and the state-dict load that follows replaces the folded weight as it always did. Median module error on DreamShaper went from a bogus 17% to 6.4%. New tests: the coverage guard's bf16-member case; the probe fold rule. Full suite 461 passed, 175 subtests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The SD model configs quantized every key module: 99.9% of the UNet on DreamShaper 7, including 488M parameters of resnet 3x3 convolutions -- 57% of the model -- with smoothing and the low-rank branch turned off for them because neither fits a conv. Nunchaku's SDXL engine (nunchaku/models/unets/unet_sdxl.py) never runs those convolutions low-bit: its _patch_resnets_convs is committed and never called, the only conv kernel in its tree is an FP16 depthwise helper, and cross-attention keys and values stay in full precision too. Quantizing them cost quality and bought nothing a deployment could use. The three SD configs now carry the engine's recipe as layer rules: role:resblock_conv and role:attn_add at bf16. What remains low-bit is the linears inside each BasicTransformerBlock -- self-attention QKV and out, cross-attention query and out, the feed-forward -- about 26% of SD 1.5's parameters, which is also where the fused low-rank kernel actually applies. docs/rotation_integration.md records the audit of the rotation stage against the machinery this fork added: seven questions, five findings, the severe one being that a rotated checkpoint carries W*H and nothing that reconstructs the Hadamard, so any out-of-process rebuild -- the Nunchaku backend included -- computes a different function without raising. Rotation stays off until the listed fixes land. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The activation cache built its per-layer tables lazily -- an entry appeared on the first module in a layer that needed caching -- and then indexed them by layer name for every layer it visited. A layer with no cached module never got an entry, and the visit died on the lookup. No DiT reaches that: every block has projections to cache. A UNet does the moment its resnet convolutions are kept in full precision, which is the recipe the SDXL engine can run: DownBlock2D is convolutions only, so under that recipe it caches nothing, and the smoothing pass raised KeyError on down_blocks.3. The per-layer tables are now created when the layer is entered, empty or not, like the two that already were; an empty layer is yielded with an empty cache and every pass skips it the way it skips a block whose cache holds nothing it wants. Pinned by a test that drives the real iterator over a stack whose middle layer caches nothing -- passes with the fix, KeyErrors without it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The code used torch.linalg.svd to compute the full singular value decomposition of the weight matrix. This proved to be computationally expensive and memory-intensive, especially since we only utilize the top rank components.
This PR replaces it with torch.svd_lowrank, utilizing a randomized algorithm to approximate the dominant singular values efficiently.
Changes:
Switched to torch.svd_lowrank for faster decomposition.
Set niter=4 and q=10 (oversampling) to achieve an optimal balance between speed and accuracy.
Adjusted tensor transposition logic to align with svd_lowrank's output format (which returns v instead of Vh).
Performance Impact In my local environment, this optimization significantly eliminates a major bottleneck during quantization:
Low-rank creation latency: Dropped from ~5s to ~100ms.
Total Runtime: The overall calibration and quantization process is approximately 6x faster.
This PR supersedes #111. I've resubmitted a clean version to resolve the conflicts caused by recent breaking changes in the main branch. Apologies for the inconvenience.