Skip to content

bc_dsa.py imports triton-only kernels unconditionally, so exl3 1.4.x cannot import on a triton-less machine #309

Description

@pb2025ai-creator

Summary

exllamav3/modules/attention_fn/bc_dsa.py imports names from dsa_triton at module
scope, unconditionally. dsa_triton.py itself guards the triton import correctly and
defines those names only inside if has_triton:. So on a machine without triton,
import exllamav3 fails — and because architecture/architectures.py imports every
architecture at module level, the failure is not confined to DeepSeek: no model of any
architecture can be loaded.

This matters on Windows in particular, where official triton has no distribution at all.

Reproduction

Windows 11, Python 3.10, torch==2.10.0+cu128, exllamav3==1.4.4+cu128.torch2.10.0,
no triton installed:

>>> from exllamav3 import Config
>>> Config.from_directory("path/to/any-exl3-model")

Import chain:

exllamav3/architecture/architectures.py   (imports every architecture at module level)
  -> deepseek_v4
    -> dsv4
      -> modules/attention_fn/bc_dsa.py:9
         from .dsa_triton import _dsa_attn_split_kernel, ...

dsa_triton.py does the right thing:

try:
    import triton
    import triton.language as tl
    has_triton = True
except ImportError:
    has_triton = False

if has_triton:
    @triton.jit
    def _dsa_attn_split_kernel(...):
        ...

...but bc_dsa.py imports those names regardless, so the guard cannot take effect.

Versions affected

Confirmed present in 1.4.0, 1.4.2, 1.4.3 and 1.4.4.

Suggested fix

Guard the import in bc_dsa.py the same way dsa_triton.py guards its own, and defer the
failure to the point of use — so a machine without triton can still load architectures that
do not need it. Something like:

try:
    from .dsa_triton import _dsa_attn_split_kernel, ...
    _HAS_DSA_TRITON = True
except ImportError:
    _HAS_DSA_TRITON = False

...with a clear error raised only when a DSA path is actually taken.

Workaround

Installing triton-windows supplies a triton module and the import succeeds; with that
in place 1.4.4 loads and generates normally here (23 EXL3 model directories, text and
vision, including the vision tower path).

⚠ One caveat worth recording for other Windows users: with triton-windows present,
additionally installing the declared dependency flash-linear-attention makes
import exllamav3 fail again, this time inside fla/ops/simple_gla/parallel.py under
triton's JIT source parser (observed at triton-windows 3.5.1 and 3.7.1). Omitting
flash-linear-attention works because every fla import inside exllamav3 is guarded;
a GatedDeltaNet model (Qwen3.5-9B) loads and generates without it via gdn.cu.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions