[SelectionDAG] SplitVecRes_INSERT_SUBVECTOR asserts for irregular i1 vectors

[Opriego]

Current upstream/main hits an assertion in SelectionDAG type legalization when lowering llvm.experimental.cttz.elts for some irregular-width i1 vectors.

Tested at:

LLVM revision: 2f07067caebf
LLVM version: 24.0.0git
Build: optimized with assertions
Target: x86_64-unknown-linux

Minimal reproducer

target triple = "x86_64-unknown-linux"

define i32 @f() {
  %r = call i32 @llvm.experimental.cttz.elts.i32.v17i1(
      <17 x i1> zeroinitializer, i1 false)
  ret i32 %r
}

declare i32 @llvm.experimental.cttz.elts.i32.v17i1(<17 x i1>, i1)

Reproduce with:

llc -mtriple=x86_64-unknown-linux /tmp/cttz17.ll -o -

The default x86-64 configuration is sufficient to reproduce the assertion.

Actual behavior

llc aborts during SelectionDAG type legalization:

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp:12417:
llvm::SDValue llvm::TargetLowering::getVectorSubVecPointer(...):
Assertion `EltSize * 8 == EltVT.getFixedSizeInBits() &&
           "Converting bits to bytes lost precision"' failed.

Relevant stack frames:

TargetLowering::getVectorSubVecPointer
DAGTypeLegalizer::SplitVecRes_INSERT_SUBVECTOR
DAGTypeLegalizer::SplitVectorResult
DAGTypeLegalizer::run
SelectionDAG::LegalizeTypes

The process exits through SIGABRT (exit=134) in an assertions-enabled build.

Expected behavior

The valid LLVM IR should lower successfully without triggering an assertion in SelectionDAG type legalization.

Analysis

The failure occurs while DAGTypeLegalizer::SplitVecRes_INSERT_SUBVECTOR uses its stack-spill fallback and asks TargetLowering::getVectorSubVecPointer for the address of the inserted subvector:

SDValue SubVecPtr =
    TLI.getVectorSubVecPointer(DAG, StackPtr, VecVT, SubVecVT, Idx);

getVectorSubVecPointer derives a byte-sized element offset from the vector element type:

EVT EltVT = VecVT.getVectorElementType();

unsigned EltSize = EltVT.getFixedSizeInBits() / 8; // FIXME: should be ABI size.
assert(EltSize * 8 == EltVT.getFixedSizeInBits() &&
       "Converting bits to bytes lost precision");

For an i1 vector element:

EltVT.getFixedSizeInBits() = 1
EltSize = 1 / 8 = 0
EltSize * 8 = 0

so converting the sub-byte element size to a byte offset loses precision and triggers the assertion.

This suggests that an INSERT_SUBVECTOR legalization path for an i1 vector is reaching a fallback that assumes byte-addressable elements.

Width observations

With -mattr=+avx512f:

v15i1  OK
v16i1  OK
v17i1  ASSERT
v18i1  ASSERT
v19i1  ASSERT
v20i1  ASSERT
v21i1  ASSERT
v22i1  ASSERT
v23i1  ASSERT
v24i1  ASSERT
v25i1  ASSERT
v26i1  ASSERT
v27i1  ASSERT
v28i1  ASSERT
v29i1  ASSERT
v30i1  ASSERT
v31i1  ASSERT
v32i1  OK
v33i1  ASSERT
v47i1  ASSERT
v48i1  ASSERT
v63i1  ASSERT
v64i1  OK
v65i1  ASSERT

The failure therefore is not limited to the original v17i1 case.

X86 feature observations

For the v17i1 reproducer:

baseline              ASSERT
+sse2                 ASSERT
+avx                  OK
+avx2                 OK
+avx512f              ASSERT
+avx512f,+avx512bw    OK

This suggests that llvm.experimental.cttz.elts is exposing a SelectionDAG legalization path whose use depends on X86 vector type legality, rather than the problem being specific to AVX512F itself.