AVI with packed B-frames: presentation timestamps derived from decode order (mpeg4/XviD)

[a1go3]

Summary

For AVI files carrying MPEG-4 ASP (XviD/DivX) with packed B-frames, the engine pairs decoded pictures with timestamps derived from decode order. Playback shows the picture stepping backwards several times a second while audio runs clean.

aetherctl swdecode diagnoses this itself, and its verdict already points at #407.

Environment

• AetherEngine 6.73.0 (SPM)
• Observed on iPhone 13 Pro / iOS 26.5 (host app, AetherPlayerSurface)
• Reproduced headless with aetherctl on macOS 15.7, x86_64

The headless reproduction uses no host code, so this is not an integration artifact on my side.

Reproduction

$ aetherctl swdecode --frames 120 file:///path/to/sample.avi

[SWDecoder] Opened: 720x304, codec=mpeg4, threads=8, 8-bit
[mpeg4 @ 0x…] Video uses a non-standard and wasteful way to store B-frames
('packed B-frames'). Consider using the mpeg4_unpack_bframes bitstream filter
without encoding but stream copy to fix it.

=== SW DECODER RESULT ===
Codec:                mpeg4 (id=12)
Source resolution:    720x304
Decoder open:         OK
Frames decoded:       38

=== FRAME TIMES (presentation order, first 38) ===
0.042  0.083  0.167  0.125  0.209  0.292  0.250  0.334  0.417  0.375  0.459  0.542
Steps backwards:      12 of 37
VERDICT: the pictures are paired with the wrong timestamps. The container withheld
         its PTS and something invented one from decode order (#407).

12 backwards steps out of 37 — roughly a third of frames. On screen this reads as the image jittering forward and back on any motion.

Source characteristics

A 2000s-era XviD AVI rip. Nothing unusual beyond the packed bitstream:

codec_name=mpeg4
profile=Advanced Simple Profile
codec_tag_string=XVID
width=720  height=304
field_order=unknown
r_frame_rate=2500000/104271   (≈23.975)

Per-frame, over a 25-frame window mid-file:

• interlaced_frame=0 and repeat_pict=0 on every frame — the source is progressive
• best_effort_timestamp_time advances evenly at 0.0417 s with no gaps or duplicates
• frame types I B P B P B B P … — B-frames throughout

So the container's timing is not damaged and no telecine or interlacing is involved. The only unusual property is the packed bitstream.

What I ruled out first

In case it saves someone time:

• Interlacing / field order. Toggling deinterlaceMode (.auto / .software) and deinterlaceFieldRate (.field / .frame) changed nothing, which is consistent with the deinterlacer never engaging — correctly so, since no frame carries AV_FRAME_FLAG_INTERLACED.
• Decode path. Forcing preferredDecodePath = .software changed nothing.
• Frame rate / cadence. Timestamps are evenly spaced; there are no duplicated or dropped frames in the source.

Suggested fix

Two options, either of which should resolve it:

1. Apply the mpeg4_unpack_bframes bitstream filter when the AVI demuxer yields an MPEG-4 stream whose extradata/packets indicate a packed bitstream — this is what libavcodec's own warning recommends.
2. Take presentation time from the decoded frame (best_effort_timestamp / AVFrame.pts) rather than deriving it from packet order, for containers that supply no PTS.

For what it's worth, another FFmpeg-based iOS player handles this class of file by taking the timestamp from the decoded AVFrame rather than from the packet, with a source comment noting that AVI packet PTS order is unreliable for exactly these files.

Why it matters

Packed B-frames were the norm for DivX/XviD AVI rips through roughly 2000–2008, so this affects a broad class of legacy files rather than one damaged sample. docs/formats.md documents AVI and MPEG-4 Part 2 as supported, which is what led me to expect these to play.

Happy to run further aetherctl commands against the sample or test a patch.

[superuser404notfound]

The gate change from #517 is on main (1676c40d), with a doc follow-up in b30c7cd0. Reopened here because the PR body carried a Fixes keyword and GitHub acted on it at merge time; a reporter issue stays open until the reporter has seen it on hardware.

Reproduced independently before merging, against the FATE sample mpeg4/packed_bframes.avi looped to 390 packets so the counter carries weight:

main     Steps backwards:  64 of 239
merged   Steps backwards:   0 of 239

and the full session plays through clean on the same file, disp=+30 lost=0 dpts=33.4/33.4 at 29.97 fps.

What the second file added

The report treats the packed bitstream as incidental. It is not: it is what decides whether the invented axis transposes at all. A plain coding-order XviD AVI of the same shape (mpeg4, tag XVID, video_delay = 1, B-frames throughout, built with ffmpeg -c:v mpeg4 -bf 2) already decodes in order on the unfixed engine, 0 backwards steps of 14.

+genpts writes a coding-order axis in both files. On a well-formed AVI that axis is the presentation ladder shifted by exactly one frame, uniformly, so no pair changes places:

pkt   0      1      2      3      4      5        (dts = the AVI frame index)
dts   0.000  0.042  0.083  0.125  0.167  0.208
pts   0.042  0.167  0.083  0.125  0.292  0.208

A packed stream breaks the correspondence that uniform shift rests on. Its N-VOP placeholder chunks are 7 bytes and carry no picture, while the picture they stand for rides inside the previous chunk. Packets and pictures stop matching one to one, and the shift stops being uniform.

None of that changes the fix, and it is the reason the fix generalises: clearing the invented axis covers every container that withholds PTS, where the mpeg4_unpack_bframes filter libavcodec suggests would cover one codec. The suggestion was correctly left out of scope.

Two details checked against the FFmpeg source rather than assumed, since the whole gate now rests on video_delay:

• libavformat/avidec.c assigns pkt->dts and never assigns pkt->pts anywhere, so on an AVI input any PTS present was necessarily invented.
• libavcodec/mpeg4videodec.c:3332 forces low_delay = 1 on exactly this class of file ("looks like this file was encoded with (divx4/(old)xvid/opendivx)"), which would have read as video_delay = 0 and left the gate disarmed. Line 3077 clears it again on the first B picture, well inside the probe window, which is why the field is trustworthy here. Worth knowing if a future AVI ever reports the defect with the gate not arming.

Still open

The device side. This needs a tagged AetherEngine release and a host build before the iPhone 13 Pro run can be repeated, which is not something to promise a date for here. A retest on the original 720x304 rip once one is out would be the thing that settles this, and the offer to run further commands against the sample is appreciated.

Thanks for the report, the headless reproduction, and the patch. The three together made this a measurement rather than an investigation.

[superuser404notfound]

Released in 6.75.0 (f436d89), gate change and doc follow-up both in the tag. Thanks again for the sample and the patch.