Skip to content

fix(broadcast_tensors): handle zero-size dims in broadcast shape computation - #6024

Merged
huangyiqun merged 2 commits into
flagos-ai:masterfrom
cccxxttt:fix/broadcast-zero-size-dim
Sep 8, 2026
Merged

fix(broadcast_tensors): handle zero-size dims in broadcast shape computation#6024
huangyiqun merged 2 commits into
flagos-ai:masterfrom
cccxxttt:fix/broadcast-zero-size-dim

Conversation

@cccxxttt

@cccxxttt cccxxttt commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Summary

_compute_broadcast_shape in broadcast_tensors.py computed each output dimension as max(dim_size, shape[i]). This is wrong for the broadcasting rule "a size of 1 is broadcastable": max() treats 0 < 1, so a zero-size dimension paired with a singleton (1) collapsed to 1 instead of staying 0. This violates PyTorch broadcasting semantics, where any non-1 size (including 0) wins over a 1, and only two distinct non-1 sizes are incompatible.

Concretely, torch.broadcast_shapes([0], [1]) should give (0,), but the old code returned (1,). This broke the shape=(0,) (empty-tensor) cases in test_smooth_l1_loss, where the loss inputs must broadcast to a zero-size output.

Fix

Resolve each dimension by taking the unique non-1 size:

  • skip size-1 inputs (they are broadcastable and never determine the result),
  • the first non-1 size wins,
  • a second, different non-1 size raises a RuntimeError (matching torch.broadcast_shapes's "The size of tensor a (...) must match the size of tensor b (...) at non-singleton dimension" message),
  • if every input is 1, the result is 1.

This now matches torch.broadcast_shapes behavior, including for zero-size dims:

# before
broadcast_shapes([0], [1])  -> (1,)   # wrong
# after
broadcast_shapes([0], [1])  -> (0,)   # correct
broadcast_shapes([0,3], [1,3]) -> (0,3)
broadcast_shapes([2], [3]) -> RuntimeError

Test

The shape=(0,) cases in tests/test_smooth_l1_loss.py (test_smooth_l1_loss and test_smooth_l1_loss_backward) were previously failing because of this; they now pass.

CUDA_VISIBLE_DEVICES=<n> python3 -m pytest tests/test_smooth_l1_loss.py -v
# 523 passed

🤖 Generated with Claude Code

…utation

_compute_broadcast_shape used max(dim_size, shape[i]) to resolve each
broadcast dimension. A size of 1 is broadcastable, but max() treats 0 as
smaller than 1, so a zero-size dimension paired with a singleton (1)
collapsed to 1 instead of 0. This violates PyTorch broadcasting semantics,
where a non-1 size (including 0) wins over a 1, and only two distinct
non-1 sizes are incompatible.

The shape0 (empty-tensor) cases in test_smooth_l1_loss relied on the
inputs being broadcast to a zero-size output and failed because the
result wrongly became size 1. Resolve each dim by taking the unique
non-1 size (or 1 if every input is 1), and raise a RuntimeError on two
distinct non-1 sizes, matching torch.broadcast_shapes behavior.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@CLAassistant

CLAassistant commented Sep 5, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@huangyiqun
huangyiqun merged commit 3857304 into flagos-ai:master Sep 8, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants