Summary
exllamav3/modules/attention_fn/bc_dsa.py imports names from dsa_triton at module
scope, unconditionally. dsa_triton.py itself guards the triton import correctly and
defines those names only inside if has_triton:. So on a machine without triton,
import exllamav3 fails — and because architecture/architectures.py imports every
architecture at module level, the failure is not confined to DeepSeek: no model of any
architecture can be loaded.
This matters on Windows in particular, where official triton has no distribution at all.
Reproduction
Windows 11, Python 3.10, torch==2.10.0+cu128, exllamav3==1.4.4+cu128.torch2.10.0,
no triton installed:
>>> from exllamav3 import Config
>>> Config.from_directory("path/to/any-exl3-model")
Import chain:
exllamav3/architecture/architectures.py (imports every architecture at module level)
-> deepseek_v4
-> dsv4
-> modules/attention_fn/bc_dsa.py:9
from .dsa_triton import _dsa_attn_split_kernel, ...
dsa_triton.py does the right thing:
try:
import triton
import triton.language as tl
has_triton = True
except ImportError:
has_triton = False
if has_triton:
@triton.jit
def _dsa_attn_split_kernel(...):
...
...but bc_dsa.py imports those names regardless, so the guard cannot take effect.
Versions affected
Confirmed present in 1.4.0, 1.4.2, 1.4.3 and 1.4.4.
Suggested fix
Guard the import in bc_dsa.py the same way dsa_triton.py guards its own, and defer the
failure to the point of use — so a machine without triton can still load architectures that
do not need it. Something like:
try:
from .dsa_triton import _dsa_attn_split_kernel, ...
_HAS_DSA_TRITON = True
except ImportError:
_HAS_DSA_TRITON = False
...with a clear error raised only when a DSA path is actually taken.
Workaround
Installing triton-windows supplies a triton module and the import succeeds; with that
in place 1.4.4 loads and generates normally here (23 EXL3 model directories, text and
vision, including the vision tower path).
⚠ One caveat worth recording for other Windows users: with triton-windows present,
additionally installing the declared dependency flash-linear-attention makes
import exllamav3 fail again, this time inside fla/ops/simple_gla/parallel.py under
triton's JIT source parser (observed at triton-windows 3.5.1 and 3.7.1). Omitting
flash-linear-attention works because every fla import inside exllamav3 is guarded;
a GatedDeltaNet model (Qwen3.5-9B) loads and generates without it via gdn.cu.
Summary
exllamav3/modules/attention_fn/bc_dsa.pyimports names fromdsa_tritonat modulescope, unconditionally.
dsa_triton.pyitself guards the triton import correctly anddefines those names only inside
if has_triton:. So on a machine without triton,import exllamav3fails — and becausearchitecture/architectures.pyimports everyarchitecture at module level, the failure is not confined to DeepSeek: no model of any
architecture can be loaded.
This matters on Windows in particular, where official
tritonhas no distribution at all.Reproduction
Windows 11, Python 3.10,
torch==2.10.0+cu128,exllamav3==1.4.4+cu128.torch2.10.0,no triton installed:
Import chain:
dsa_triton.pydoes the right thing:...but
bc_dsa.pyimports those names regardless, so the guard cannot take effect.Versions affected
Confirmed present in 1.4.0, 1.4.2, 1.4.3 and 1.4.4.
Suggested fix
Guard the import in
bc_dsa.pythe same waydsa_triton.pyguards its own, and defer thefailure to the point of use — so a machine without triton can still load architectures that
do not need it. Something like:
...with a clear error raised only when a DSA path is actually taken.
Workaround
Installing
triton-windowssupplies atritonmodule and the import succeeds; with thatin place 1.4.4 loads and generates normally here (23 EXL3 model directories, text and
vision, including the vision tower path).
⚠ One caveat worth recording for other Windows users: with
triton-windowspresent,additionally installing the declared dependency
flash-linear-attentionmakesimport exllamav3fail again, this time insidefla/ops/simple_gla/parallel.pyundertriton's JIT source parser (observed at triton-windows 3.5.1 and 3.7.1). Omitting
flash-linear-attentionworks because everyflaimport inside exllamav3 is guarded;a GatedDeltaNet model (Qwen3.5-9B) loads and generates without it via
gdn.cu.