input_projections are not registered as PyTorch modules, causing missing parameters and CUDA device mismatch #49698

System Info

• OS: Linux
• Python: 3.12
• PyTorch: CUDA-enabled
• Transformers: Issue confirmed in the source code of v5.20.0.

Who can help?

No response

Reproduction

In Hugging Face Mask2Former, input_projections is a regular Python list instead of nn.ModuleList. As a result, projection parameters are not registered, excluded from optimizers, and not moved to CUDA.

from transformers import Mask2FormerConfig
from transformers.models.mask2former.modeling_mask2former import (
    Mask2FormerTransformerModule,
)

config = Mask2FormerConfig(
    feature_size=64,
    hidden_dim=64,
    enforce_input_projection=True,
)

module = Mask2FormerTransformerModule(64, config)

registered = {id(p) for p in module.parameters()}

missing = sum(
    p.numel()
    for layer in module.input_projections
    for p in layer.parameters()
    if id(p) not in registered
)

print("Unregistered parameters:", missing)
# Unregistered parameters: 12480

When running on CUDA, this can also produce:

RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same

Interestingly, the original Facebook Mask2Former implementation correctly uses nn.ModuleList():

https://github.com/facebookresearch/Mask2Former/blob/main/mask2former/modeling/transformer_decoder/mask2former_transformer_decoder.py#L323

Hugging Face implementation:

https://github.com/huggingface/transformers/blob/v5.20.0/src/transformers/models/mask2former/modeling_mask2former.py#L2079

Expected behavior

The input_projections layers should be registered using nn.ModuleList, as in the original Facebook implementation.

This ensures that their parameters are included in model.parameters(), optimizers, and state_dict(), and are correctly moved to the target device using model.to(device).

Suggested fix:

Replace self.input_projections = [] with self.input_projections = nn.ModuleList().