Repository navigation
Fix AMD keyframe recovery and test encoder mode changes - #58
Merged
whoisaldo merged 2 commits intoOct 11, 2026
Merged
Conversation
FFmpeg 7.1.1's AMF wrapper never reads AVFrame::pict_type, so every forced keyframe on an AMD GPU was ignored. After a reconnect, a keyframe request or dropped USB frames, the iPad waited for AMF's own IDR, which comes every 30 encoded frames. On an idle desktop the capture side resends a frame only every 750 ms, so that wait could reach about 22 seconds. The encode loop now reopens the AMF session instead, because a new session always starts with an IDR that carries its parameter sets. These reopens happen at most once per 500 ms, the same limit the host applies to client keyframe requests. A display mode change that resized the capture (Radeon Super Resolution, VSR, GPU scaling, a resized virtual display) left the encoder on its old frame buffers. A larger mode panicked the encode thread on a slice bound and a smaller one sent a garbled picture. The loop now reopens the encoder at the new size, and restarts the pipeline if that reopen fails. host/tests/amf_encoder.rs drives the real encode stage with synthetic frames and decodes the output with a strict FFmpeg decoder. ETERNAL_AMF_TEST=hardware requires a working AMF encoder, for the Radeon reference PC. CI's Windows job sets fallback, which proves AMF fails cleanly without an AMD driver and the stage falls back to libx264. libx264 runs the same keyframe and mode-change checks on every platform. Unit tests pin the h264_amf and hevc_amf options the iPad decoder needs, the reopen rate limit, and the rule that AMF all-intra access units go out as keyframes.
Keep requests pending through the AMF reopen rate limit and failed reopen attempts. Test the limit boundary and both codec paths, strictly validate every output picture, reject broken streams, and cover HEVC resolution changes. Document the remaining driver and iPad hardware checks. Apply the existing rustfmt correction so workspace formatting passes.
whoisaldo
marked this pull request as ready for review
October 11, 2026 22:45
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
On the pinned FFmpeg 7.1.1 build, AMF ignores per-frame picture-type requests, so reconnects and lost-frame recovery could wait for the next GOP keyframe. Reopen H.264 and HEVC AMF sessions to deliver an IDR, limit keyframe reopens to one every 500 ms, and retain requests through that limit or a failed reopen. The wrapper behavior was checked against the pinned FFmpeg source.
A capture resolution change previously left the encoder using its old buffers, causing a panic when the display grew or a damaged picture when it shrank. Reopen at the new size and let the supervisor restart if that reopen fails.
Add synthetic-frame encoder tests with strict decoding of every output picture, fresh-decoder joins at requested keyframes and resolution changes, and negative checks for corrupt slices and joining on predicted frames. Cover both AMF codecs, preserve the AMF intra-frame keyframe flag, and pin the decoder options. Windows CI requires the missing-driver fallback; Radeon runs require actual AMF hardware. Document the remaining capture and iPad checks. One existing line in updates.rs was formatted to make the workspace formatting check pass.
How it was tested
cargo fmt --all --checkcargo clippy --workspace --all-targets --locked -- -D warningscargo test --workspace --locked, including the ten synthetic end-to-end testsLocal validation used macOS ARM64 with FFmpeg 7.1.5. The five AMF-specific integration cases skip on this build because it has no AMF encoder; passing software tests do not validate an AMD driver. Real Radeon hardware validation remains pending because this machine cannot resolve the repository's
windowsSSH host.scripts/e2e_matrix.sh --ciwas attempted but cannot execute simulator scenarios without/Applications/Xcode.app/Contents/Developer.Keep this PR in draft until the Windows fallback job and
ETERNAL_AMF_TEST=hardwarechecks on a Radeon PC are reviewed. Run the realR-amf-h264andR-amf-hevcrows to verify desktop capture and iPad decoding, plus idle reconnects, display resizing, lock/unlock, and sleep/wake. No iPad source or wire layout changed.