Skip to content

feat(sdes): make the offer's video codec preference order settable - #226

Merged
cbrightly merged 1 commit into
mainfrom
feat/express-a-codec-preference-in-the-offer
Aug 9, 2026
Merged

feat(sdes): make the offer's video codec preference order settable#226
cbrightly merged 1 commit into
mainfrom
feat/express-a-codec-preference-in-the-offer

Conversation

@cbrightly

Copy link
Copy Markdown
Owner

What this is

AIDOT_SDES_VIDEO_PT_ORDER - opt-in, off by default, inert unless set - makes
the video codec preference order in the SDES offer settable per run. It ships
off, with the experiment that would confirm or kill it written out below.

First, a correction to what this repo says is established

docs/ROAD-TO-1.0.md, the CHANGELOG, and the ledger all say our offer
"advertises both video codecs and expresses no preference". Checked against the
code, that is not what the SDP says.

RFC 3264 section 5.1 makes the m=video payload-type list a preference list,
most-preferred first
. The offer built in sdes_open.py has always carried:

m=video <port> RTP/SAVPF 96 97

H264 first. It states a preference, and always has.

What is true is weaker and more useful: nothing ever chose that order.
git log -L on the line shows it arrived verbatim in abe6d6a, the commit that
extracted the SDES open path out of client.py, and it has never been varied
since. Incidental, not deliberate.

Why the correction matters more than the knob

It changes the prior. The A001064 answers H264 most sessions and H265
occasionally for an identical request. Read against an offer that already says
"H264 first", that is a camera which honours our stated preference most of the
time and disregards it some of the time.
So reordering is a weaker candidate
than pinning was, not a stronger one - the camera has already demonstrated it
will ignore first-listed.

I am not going to let the PR read as if a preference vacuum was established and
this fills it. It did not, and this does not.

Why it is still worth being able to set

The efficient profile - hevc 2560x1440 at ~1.1 Mbps against h264 1280x720 at
2.5-4.0 Mbps - has only ever appeared when both codecs are on the wire.
AIDOT_SDES_VIDEO_PT=97 narrows the offer to H265 and returns no video at all,
3 of 3 rounds: narrowing removes the option rather than selecting it.

Reordering is the only untried lever that leaves both codecs offered, so the
camera can still fall back to H264. That is the whole argument, and it is enough
for an off-by-default knob.

What changed

  • The offer's m=video payload-type list and its rtpmap/fmtp block now come
    from one helper, so they cannot drift apart: an m-line naming a payload type
    whose rtpmap was left behind is an offer the camera cannot act on.
  • AIDOT_SDES_VIDEO_PT_ORDER takes a comma- or space-separated payload-type
    list (97,96, or just 97). Whatever is named leads, in the order named;
    every advertised codec not named is appended in the default order.
  • So it can express a preference and can never narrow the offer. The result
    is always a permutation of the full advertised set, for any value including
    garbage - a video m-line with no payload type at all leaves the camera nothing
    to send, which is the one outcome worse than an unpinned choice. Narrowing
    already has its own variable, and the one time it was measured it cost the
    picture.
  • Ordering runs before the existing pin, so with both set the pin wins. Asserted,
    not assumed.
  • A status line, SDES: offer video codec order=97 96, prints whenever the order
    differs from the shipped one.
  • README env table gains a row for this and for AIDOT_SDES_VIDEO_PT, which was
    documented only in the CHANGELOG.

The default is not changed. Unset, the offer is byte-identical to 1.0.0b1.
This path is shared by every SDES camera, the CHANGELOG records fleet-wide
blackouts from changes to shared paths, and the hypothesis is untested on
hardware - a release is validating right now and this must not touch it.

Tests

The offer is assembled inside a several-hundred-line async method that cannot be
invoked standalone, so a test against a private copy of the template would pass
with the production change reverted. Instead:

  • The tests call the same helper the offer builder calls and assert the
    actual payload-type order in the generated m-line and codec block.
  • A byte-identity test pins the default output against the exact literal it
    replaced, character for character - the risk in this refactor is a dropped or
    doubled \r\n at the seams, not the codec names.
  • An invariant test sweeps garbage values and asserts the result is always a
    permutation of (96, 97), never narrowed, never empty, every m-line payload
    type carrying its own rtpmap.
  • An AST guard checks the production builder really interpolates the computed
    list and takes its codec block from the helper, so a hard-coded 96 97 cannot
    come back with every other test still green.
  • One test asserts the order survives SDP compression, since the offer also
    travels compressed in wPayload.offer.sdp, which is what newer firmware
    parses. It exercises protocol._compress_sdp_for_camera; the SDES path runs a
    closure of the same shape (_compress_sdp_req, defined inside the open method
    and not importable), whose video branch has the same two properties asserted
    here. That closure is its untested twin and the test says so.

Both guards were verified by mutation, not by assumption:

  • revert the production m-line edit -> test_the_offer_builder_takes_its_video_codec_list_from_the_helper fails
  • flip _SDES_OFFER_VIDEO_PT_ORDER to (97, 96) -> 3 tests fail, including the byte-identity one

uvx ruff check aidot_cameras/ tests/ clean. pytest tests/ -q --ignore=tests/e2e:
1077 passed, 5 skipped. The e2e tier hangs on this machine after the first test -
confirmed pre-existing: a clean origin/main worktree hangs at the same
point, so it is environmental and not this change. CI runs that tier.

The experiment that would confirm or kill this

CLAIM      Offering 97 before 96 - both still advertised - makes the
           A001064 answer H265 (hevc 2560x1440, ~1.1 Mbps) materially more
           often than an unreordered offer does.

CHAIN      (a) the m=video list is a preference list, most-preferred first
               -> RFC 3264 5.1
           (b) our offer has always said 96 first, by accident not decision
               -> git log -L on the line; abe6d6a, never varied
           (c) the camera reads our offer's video codec list and acts on it
               -> ESTABLISHED. Pinning to 96 gave h264 720p 4 of 4;
                  pinning to 97 changed what it did (to sending no video)
           (d) the knob reaches the SDP on the sessions measured
               -> the order receipt, read per session
           (e) the camera treats ORDER, not just membership, as a signal
               <- the open question, and it has a counter-signal already:
                  it answers H265 some sessions against a 96-first offer,
                  so it demonstrably does not always take first-listed

VERIFIED   (a) (b) (c) as above. (d) is a receipt this PR adds. (e) is what
           the run is for. Nothing about (e) is claimed here.

DESIGN     Two arms, INTERLEAVED and alternating - not blocked. This camera
           measured 839-3698 Kbps across sessions with nothing requested, so
           a blocked run measures time of day, not the arm.

               ORD   AIDOT_SDES_VIDEO_PT_ORDER=97,96
               CTL   unset

           ORD, CTL, ORD, CTL, ORD, CTL. AIDOT_SDES_VIDEO_PT must be UNSET
           in both arms - it narrows, and it would decide the outcome.

RECORD     Per session, or the session does not count:
           - the sha under test, per measurement, not per run. A merge that
             moved the tree underneath a run has voided one of these before.
           - the order receipt: "SDES: offer video codec order=97 96" in the
             ORD arm, and ABSENT in the CTL arm. An arm that cannot show the
             knob was applied cannot distinguish an effect from a
             coincidence - the first pin attempt read as a confirmed result
             for two sessions before a missing receipt caught that it had
             never reached the SDP.
           - the readout: "camera <id>: video profile pt=N codec=X", read
             from the RAW log. scripts/avio_probe.py drops any line lacking
             " RX " or "AVIO", and that filter has already voided a whole
             column of a previous run.
           - bitrate and the per-frame dimensions of the recording, so a
             codec/resolution surprise (hevc at 720p, h264 at 1440p) is
             visible rather than assumed away.

READS      Categorical, so n is small but the margin has to be clean.
           3/3 hevc under ORD against 0/3 under CTL is a result.
           2/3 against 1/3 is not, and needs a rerun at larger n rather
           than a conclusion. Baseline for context: unpinned hevc ran about
           2 in 11 one afternoon, then 0 in 12 in the quality sweep - so the
           control arm coming back all-H264 is the expected case and carries
           no information on its own.

KILL       ORD returns h264 at about the control rate. That says the camera
           does not read m-line order as a constraint, and this joins the
           other thirteen. Record it dead in the ledger and stop.

KILL #2    Any ORD session that returns NO VIDEO. That would mean expressing
           a preference triggers the same failure as narrowing did, and the
           knob must then be documented as unsafe rather than merely opt-in
           - or reverted.

NOT AT     Do not change the default on a 3/3 result alone. The default
STAKE      change is a separate decision needing a fleet-wide check across
           both SDES models (the A001513's h264 is 1280x960, not 1280x720,
           so the models do not behave alike) and a consumer that can
           actually decode hevc 2560x1440.

Do not merge. The knob is inert until someone sets it, and what it is for is the
run above.

A correction first, because the docs in this repo have said otherwise and the
correction is most of the value here. The SDES offer does not "express no
preference" between H264 and H265. RFC 3264 section 5.1 makes the m=video
payload-type list a preference list, most-preferred first, and the offer built
in sdes_open.py has always carried "96 97" - H264 first. What is true is
weaker and more useful: nothing ever chose that order. The line arrived
verbatim when the SDES open path was split out of client.py and has never been
varied since.

That reframes the lever rather than strengthening it. On an A001064 the camera
answers H264 most sessions and H265 occasionally for an identical request,
which read against the offer is a camera that honours our stated first choice
most of the time and disregards it some of the time. So expressing a
preference is a weaker candidate than pinning was, not a stronger one.

It is still worth being able to set, because the efficient profile - hevc
2560x1440 at about 1.1 Mbps against h264 1280x720 at 2.5-4.0 Mbps - has only
ever appeared when both codecs are on the wire. AIDOT_SDES_VIDEO_PT=97 narrows
the offer to H265 and returns no video at all, 3 of 3 rounds: narrowing removes
the option rather than selecting it. Reordering is the only untried lever that
leaves both codecs offered, so the camera can still fall back to H264.

AIDOT_SDES_VIDEO_PT_ORDER takes a comma- or space-separated payload-type list.
Whatever is named leads, in the order named, and every advertised codec not
named is appended in the default order - so it can express a preference and can
never narrow the offer. The result is always a permutation of the full
advertised set, whatever the value, because an m=video line with no payload
type leaves the camera nothing to send and that is the one outcome worse than
an unpinned choice. Ordering runs before the existing pin, so with both set the
pin wins.

The default is deliberately unchanged and unset is byte-identical to 1.0.0b1.
This path is shared by every SDES camera, the CHANGELOG records fleet-wide
blackouts from changes to shared paths, and whether the camera acts on m-line
order is exactly the untested question - so it ships off, with a status line
that reports the order whenever it differs from the shipped one. A run that
cannot show the knob reached the SDP cannot tell an effect from a coincidence,
which is how the first attempt at the pin read as a confirmed result for two
sessions before the missing receipt caught it.

The m-line and the rtpmap/fmtp block now come from one helper, so they cannot
drift apart: an m-line naming a payload type whose rtpmap was left behind is an
offer the camera cannot act on. The tests assert the payload-type order in the
generated m-line and codec block via the same helper the offer builder calls,
and an AST guard checks the builder really calls it - the offer is assembled
inside a several-hundred-line async method that cannot be invoked standalone,
so a test against a copy of the template would pass with this change reverted.
Both were verified by reverting the production edit and by flipping the default
order, and each mutation fails the suite.
@cbrightly
cbrightly merged commit 7664121 into main Aug 9, 2026
16 checks passed
@cbrightly
cbrightly deleted the feat/express-a-codec-preference-in-the-offer branch August 9, 2026 01:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant