transformPacked: interpolate a whole pixel at a time, and skip the bound check inside the frame

[georgmartius]

Why

8da5ae2 fixed a heap overflow in interpolateN() by replacing the unchecked PIXN() with the range checked PIXELN() for all four bilinear samples. That sits in the innermost loop of the packed (RGB24/BGR24/RGBA) transform, so the question was what the check costs — and whether we can get back to PIXN.

We can, but it turned out not to be where the time went.

Measurements

New bench/bench_transform.c, 1080p, best of five, Ryzen 9 9900X, GCC 15.2 -O3, ms/frame:

	before	PIXN only	this PR
packed RGB24 fixedpoint	17.93	17.28	11.72
packed RGBA fixedpoint	23.05	21.88	13.84
packed RGB24 float	38.86	33.27	—
planar YUV420 (control)	9.08	9.08	9.08

The bound check was worth ~4%. What cost 35% was next to it: transformPacked() called interpolateN() once per channel, and each call recomputed the source indices and all four interpolation weights for the same pixel — 3x over for RGB24, 4x for RGBA.

What changed

• interpolateN() uses the unchecked accessor again when ix_f+1 < width && iy_f+1 < height. The outer test already establishes ix_f/iy_f are in range, so only the "ceil" neighbours can leave the frame; on the last row/column the checked accessor still supplies def. One predictable branch replaces four bound checks.
• New interpolateNall() does all N channels of one destination pixel — indices and weights once, then four loads and a blend per channel. transformPacked() calls it once per pixel. The border case delegates to interpolateN() unchanged.
• The blend moved into blendBiLinN(), shared by both, so the fixed point arithmetic exists once.
• transformfloat.c gets the same interior fast path. It is a test-only reference (not in the library sources), so it keeps the per-channel structure.

The check is still load-bearing

A variant with the check simply deleted diverges on 18 of the 60 verification cases below — it reads past the buffer exactly as before 8da5ae2. Only interior pixels can skip it.

Verification

• bench_transform verify hashes every byte of the result frame over 3 packed formats x crop on/off x 5 transforms chosen to hit interior pixels, the last row/column, and source coordinates far outside the frame: 60/60 bit-identical to master.
• ./tests --all — 38/38.
• Benchmark clean under ASan + UBSan.

🤖 Generated with Claude Code