VC-1 interlaced/deinterlaced (yadif hardware) software playback drops significantly more frames per seek than progressive VC-1 playback

[classicjazz]

What happened?

Across every interlaced 29.97i file tested (where [Deinterlace] engaged [hardware]: ... yadif_videotoolbox fires), the per-seek frame-drop rate is roughly 5–10x higher than on progressive 23.976fps VC-1 files, even in sessions with no other errors present:

File	Profile	Seeks	Total drops	Notes
FileA	progressive	11	7	clean
FileB	progressive	18	6	clean
FileE	progressive	11	6	clean
FileG	progressive	9	9	clean
FileF (session 1)	interlaced	18	73 (4.9% of 1504 frames)	clean, no errors
FileJ	interlaced	9	49	clean, no errors
FileI	interlaced	10	73	includes one +33 spike from Bug #491

Additionally, dpts (the reported inter-frame interval) occasionally doubles from the expected ~16.5–17ms field interval to ~33–34ms — observed twice in FileF, once each in FileI and FileJ, never in any progressive file — indicating an actual dropped/skipped field presentation, not just a counting quirk.

Steps to reproduce

1. Open any interlaced VC-1 file that engages hardware deinterlacing (reproduces on FileF, FileI, FileJ).
2. Perform a normal sequence of seeks (no special pattern needed — it reproduces on ordinary scrubbing).
3. Compare layerDrop growth per seek against a progressive VC-1 file under the same seek pattern.

AetherEngine version or commit SHA

6.68.0

Host app

AetherPlayer

Platform

macOS

OS version

Mac OS 27

Device / chip

M1 Max Mac Studio

Playback path

Software (dav1d) host

Source media (for playback bugs)

No response

Error codes / log lines

Anything else

This is present even in sessions with zero hard errors (FileJ: 49 drops over 9 seeks, no Picture size 0x0, no rebuffer pause, corrupt=0 throughout), so it's a distinct, broader issue from Bug #490 rather than fully explained by it — though Bug #490 's failures do add to the total when they occur. The consistent 5–10x multiplier across three different interlaced files/sessions makes this look systemic to the deinterlace path rather than file-specific encoding quirks. Worth checking whether the yadif hardware pipeline (format=nv12,hwupload,yadif_videotoolbox=mode=send_field:parity=auto:deint=interlaced) is more sensitive to the same "landed on a frame without full reference chain" condition that produces the routine keyframe warnings, since a field-based GOP structure would double the number of presentation units per seek relative to progressive content.

[superuser404notfound]

Measured the first suspect, and it is smaller than expected, so this needs one more number from a session before it can be called.

The deinterlace graph is torn down at every seek and rebuilt on the next interlaced frame. That is deliberate (yadif is temporal, and stale references must not cross a discontinuity), and it means an interlaced source pays a graph build per seek that a progressive source never pays. That is the right shape for a cost that appears only on one side of the table. Run here with five seeks on a 480i H.264 fixture through the same hardware chain:

[Deinterlace] engaged [hardware]: 64x64   ... built in 35.6ms   <- cold warm-up graph
[Deinterlace] engaged [hardware]: 720x480 ... built in 0.8ms
[Deinterlace] engaged [hardware]: 720x480 ... built in 2.0ms
[Deinterlace] engaged [hardware]: 720x480 ... built in 3.0ms
[Deinterlace] engaged [hardware]: 720x480 ... built in 3.6ms
[Deinterlace] engaged [hardware]: 720x480 ... built in 1.9ms
[Deinterlace] engaged [hardware]: 720x480 ... built in 2.4ms

At 59.94 output frames per second a frame is 16.7 ms, so 0.8 to 3.6 ms does not account for a run of drops on its own. It is not nothing at 1080i, where the hwframe pool is nine times the surface, so the build cost is now printed on that line (4c286881 on main). A session log will answer it directly rather than by inference.

The table is not yet like-for-like, and the correction is large. mode=send_field emits one frame per FIELD, so the interlaced sessions hand the layer 59.94 frames per second against 23.976 for the progressive files. That is 2.5x the frames, each with a 16.7 ms presentation deadline instead of 41.7 ms. layerDrop is AVSampleBufferDisplayLayer's own counter, so it counts deadline misses, and it scales with both of those. Normalised per frame rather than per seek, a 5-10x multiplier lands nearer 2-4x. Still real, but a different size of claim, and FileF is the only row carrying a rate (4.9% of 1504). The progressive rows' percentages would settle it in one line.

The dpts observation is the more useful half. A step from 16.5 to 33 ms says a field-frame did not reach the layer, and that is not a counting artifact. What separates the two candidate causes is the rest of the same sample: enq against disp says whether the frame was produced and not handed over, lost counts what died between the decoder callback and the layer, and layerDrop counts what the layer itself threw away after accepting it. If enq and disp track each other and only layerDrop moves, the loss is on the presentation side and the frame rate is the story. If disp trails enq, it is upstream of the layer and worth chasing separately. A handful of samples around a seek on FileJ, which is the clean session with no other errors, would be the most informative thing here.

Worth an A/B in the meantime: LoadOptions.deinterlaceFieldRate = .frame switches the chain to send_frame and halves the output rate. If the drop count roughly halves with it, the rate is the answer and the remedy is a pacing question rather than a defect. If it does not move, something on the deinterlace path is losing frames for a reason that is not throughput, and that is worth the dig.

[superuser404notfound]

The build-cost number is in 6.68.2 (33c3816), so a session log now answers the graph-rebuild question directly instead of by inference from a 480i measurement. [Deinterlace] engaged … built in N.Nms, one line per seek on an interlaced source.

The seek-reference fix from #491 is in the same release. It is a separate defect, but it did contribute the +33 spike on FileI, so that row's total should come down on its own.

[superuser404notfound]

AetherPlayer 0.17.2 carries the engine that prints the build cost, so the [Deinterlace] engaged … built in N.Nms lines will be in a 1080i session log from it: https://github.com/superuser404notfound/AetherPlayer/releases/tag/v0.17.2

The samples that would settle the rest are enq / disp / lost / layerDrop from the same SWDiag ticks around a seek on FileJ, plus the same run with deinterlaceFieldRate = .frame if that is reachable from the app side.

[classicjazz]

Update from AetherPlayer testing (build 0.17.2 (67), engine 6.68.2):

The new built in X.Xms timing on [Deinterlace] engaged is helpful: across FileI and FileJ every sample reports 0.6–1.3ms, nowhere near the 35.6ms cold-start figure from the fix. That rules the graph-rebuild cost out as the driver of the drop-rate gap — same conclusion as before, now backed by direct measurement instead of inference.

Re-ran the progressive-vs-interlaced comparison, though this round's seek pattern differs from the original test (FileB clustered into a tight backward-scrubbing sequence; FileI/FileJ used larger, more scattered jumps), so raw totals aren't directly comparable to the original numbers. Normalized per seek, the gap still holds and if anything is larger under this pattern: FileI ≈ 11 drops/seek, FileJ ≈ 18 drops/seek, vs. FileB ≈ 2.6 drops/seek.

One caveat: a separate defect (posted to #491) is inflating some of this round's interlaced totals — three-seek bursts tied to that bug account for 113 and 102 of FileI's and FileJ's total drops respectively. A clean re-measurement of the multiplier here should use a seek pattern that avoids triggering that pattern (moderate single-jump seeks, not chained huge-distance jumps).

[superuser404notfound]

Thanks for the 1080i timings. 0.6 to 1.3 ms against the 35.6 ms cold graph answers the rebuild question from the side that matters, on real content instead of by inference from a 480i fixture, and it agrees with what that fixture said. That suspect is closed.

The A/B is done here, because it turned out to be reachable from neither end. LoadOptions.deinterlaceFieldRate had no lever in aetherctl and none in AetherPlayer, so the run asked for in the last comment could not have been produced by anyone. aetherctl play --deinterlace-field-rate field|frame exists now, and the measurement is below.

Three arms on macOS, four runs each, the same eight-seek pattern in every run. Arm A is a progressive 23.976 H.264 file. Arms B and C are the SAME interlaced 29.97i file through the same yadif_videotoolbox chain, differing in nothing but the field rate.

arm	output	drops/s	drops per frame delivered
A: progressive 23.976, no deinterlacer	24.0 fps	6.09	0.254
B: interlaced, send_field	59.9 fps	17.47	0.291
C: interlaced, send_frame	30.0 fps	7.49	0.250

Per second, B is 2.87x A. That is the shape of the 5-10x per seek in the report, arriving from the same place. Between B and C nothing changes but the field rate: the output rate halves and the drop count falls by 2.33x. Per FRAME delivered, the progressive arm and the send_frame interlaced arm agree to 1.6% (0.254 against 0.250), and those are two different code paths, one of them tearing down and rebuilding a hardware deinterlace graph at every seek.

So the count is a function of frames delivered. What is left over is the roughly 15% that arm B carries above the other two, and that is the part attributable to the 16.7 ms presentation deadline rather than to the number of frames. Real, and a different size of claim than 5-10x.

enq and disp tracked each other on every arm and at every landing, always to within the four frames the reorder buffer holds. Nothing is lost between the decoder and the layer on the deinterlace path, so the loss is entirely on the presentation side, which is the fork the last comment named.

Two things this cannot say. Nothing binds a render surface in a CLI run, so the absolute level is the harness's own and not a session's; only the ratios between arms are the measurement, and all three arms carry the same handicap. And the fixture is 480i H.264, so the deinterlace chain is the shared part while the decoder is not.

On the round-2 numbers. A mean per seek and a note that three seeks inside it carry 113 of the drops cannot both feed the same multiplier. Against a quoted mean of 11 those three are roughly 3.4x the mean each, which means the mean is mostly describing them rather than the other seeks. The per-seek list would settle it, or the median with the burst seeks named.

The burst does not need to be steered around, though. Its cause is fixed: the reposition-window half of #491 is in 6.68.3, and that round was measured on 6.68.2. So the clean re-measurement is the same seek pattern on 6.68.3, not a pattern chosen to miss the bug. A pattern that avoids it also avoids whatever else lives there.

Separately, running that matrix turned up a defect of its own, and it is not this one: a seek can leave video frozen while the clock runs, because the parked-packet FIFO keeps feeding the decoder past that seek's own flush. About one seek in twenty-two here, on the progressive arm too. It is #494, with the fix on fix/ae492-parked-feed-outlives-flush. A frozen pipeline enqueues nothing, so it does not move the layer's drop counter while it lasts and it is not an explanation for the rates above, but it is worth knowing about while reading any post-seek session log.