Conversation
- ListData.__init__ assigned self.self.data_block_nr in both the mapped and the stream branch, so reading any ##LD block raised AttributeError: 'ListData' object has no attribute 'self' - _append_column_oriented read signal.axes / signal.conversions / signal.units, none of which exist on Signal; the three locals were never used, so the lines are removed - add a 4.20 column-storage round-trip regression test covering both paths
…self-typo Fix AttributeError reading LDBLOCK (self.self typo) and column-oriented append crash
Container (multiplexed PDU) frames carry a variable sequence of contained
PDUs, each prefixed by a Header_ID/Header_DLC header, so a PDU's byte offset
depends on the lengths of the PDUs before it. extract_mux() only handled
is_multiplexed frames, so container frames were mis-decoded.
- extract_pdus() in bus_logging_utils.py walks the container headers per
frame (vectorized across frames, one pass per PDU slot), gathers each
contained PDU payload and extracts its signals. Each contained PDU becomes
its own channel group, reusing the existing channel-group machinery; a
PDU's signal start bits are PDU-relative so extract_signal applies
unchanged. canmatrix.Frame.unpack is the reference algorithm.
- route is_pdu_container frames through extract_pdus in _extract_can_logging;
all other frames keep using extract_mux.
- fix extract_signal signedness: a signed signal with a standard bit width
(8/16/32/64) at a non-byte-aligned offset was viewed as i{std_size} on the
padded width instead of being sign-extended from its real bit width (e.g. an
8-bit signed field at bit offset 1 returned 225 instead of -31). This
affected the normal extract_mux path too.
- add offline test/test_CAN_pdu_extraction.py validating extract_pdus against
canmatrix.unpack (big/little-endian headers, 0x00/0xFF padding, unique
multi-PDU frames, signed bit-packed signals) plus a full extract_bus_logging
end-to-end case and a static-container skip.
Static (header-less) AUTOSAR containers have a fixed layout: canmatrix rebases each contained PDU's signal start bits to be frame-relative and Frame.unpack itself refuses them, so they were previously skipped. Handle them by decoding every contained PDU straight from the full frame payload, one channel group each (pdu.id is None -> identity keyed on the PDU name). - factor the per-PDU signal emission out of extract_pdus into _emit_pdu_signals, shared by the dynamic (PDU-relative payload) and static (full-frame payload) paths; _contained_pdu_muxer builds the channel-group identity and tolerates a None header id. - test_extract_pdus_static_container validates static extraction against a flat frame carrying the same frame-relative signals (byte-aligned LE/BE plus a non-byte-aligned signed field). - test_extract_bus_logging_canfd_container_e2e runs the full pipeline on genuine CAN-FD container frames (64-byte payload, EDL flag + DataLength members set) to exercise the real-world container transport.
Validated `extract_pdus` against two real OEM CAN-FD bus logs (5.1 M and 2.3 M CAN frames) and three production ARXML databases, using `canmatrix.Frame.unpack` as the oracle. That uncovered three defects that the in-memory tests could not reach: 1. `OverflowError` aborted the whole extraction. Real containers carry opaque 216/288/400-bit blobs declared *signed*; those are kept as a byte matrix, so `as_non_byte_sized_signed_int` computed `1 << 216` on them. Signals wider than an integer dtype now skip two's complement, like unsigned ones already did. 2. Container padding was reported as measured data. A sender may transmit a contained PDU shorter than its declared size — the header DLC is the authority. Signals reaching past the transmitted length are now flagged through `invalidation_bits` instead of surfacing padding as a value. Real data hits this on 2.4 % of contained-PDU occurrences. 3. The header walk could run backwards. canmatrix's ARXML parser marks the synthetic `Header_ID`/`Header_DLC` signals *signed*, so a 0xFF padding byte decodes as a DLC of -1; the offset then moved backwards over a padded tail, rescanning the frame misaligned and inventing contained PDUs out of padding. Both header fields are now read unsigned. Two regression tests added for 1. and 2.; 3. is covered by the existing oracle test now that absolute values are asserted.
`doc/buslogging.rst` is the user-facing bus logging documentation and said nothing about container I-PDUs. Add a section covering what gets decoded (one channel group per contained PDU, addressed like any other bus logging signal), that both the dynamic and static layouts are handled, that samples whose bytes were not transmitted are marked through `invalidation_bits`, and the two known limitations (multiplexed contained PDUs, LIN). Also point the developer note at it, and note there that it is deliberately not part of the built docs — sphinx is configured for `.rst` only.
kipp-ing
force-pushed
the
dynamic-pdu-container-extraction
branch
from
August 15, 2026 15:18
fa2e1be to
2190907
Compare
test_Tabular_BaseWidget_Shortcuts.py:50 called @skipIf(sys.platform == "linux") without the required reason argument, raising a TypeError at import time that aborted collection for the entire test suite on every platform. Add a reason string. The pyinstaller_build workflow hits the same NMake/CMAKE_C_COMPILER failure on windows-latest that wheels.yml was fixed for: force the Ninja generator (and install Ninja + VS Build Tools) the same way. Verified locally: - full-suite collection (228 items, 0 errors) after the skipIf fix, vs. 1 collection error aborting everything before it - rebuilt the extension with CMAKE_GENERATOR=Ninja forced (mirrors what build.yml now sets); scikit-build-core confirms "Building project with Ninja...", extension imports fine, full non-GUI suite (56 passed, 2 skipped) still green against that rebuild
Use the CMake package and target names the system lz4/zstd export
…and-pyinstaller-windows fix: unblock test collection and Windows pyinstaller build
…into development
…container-extraction
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
extract_pdus()added tosrc/asammdf/blocks/bus_logging_utils.py. It decodesAUTOSAR dynamic PDU-container CAN frames. A container frame carries a variable
sequence of contained PDUs, each prefixed by a
Header_IDandHeader_DLCfield. Each contained PDU's byte offset depends on the lengths of the PDUs
before it, so the payload is walked header by header. The walk keeps a per-frame
byte offset and advances it across all frames at once (one pass per header
slot), so frames with different PDU counts, orders, and lengths are handled. For
each contained PDU id the matching payload slices are gathered into one array
and its signals are extracted with the existing
extract_signal. A containedPDU's signal start bits are relative to the PDU payload, so no offset rebasing is
needed.
_extract_can_logging()insrc/asammdf/blocks/mdf_v4.pyroutes frames wheremessage.is_pdu_containeris true throughextract_pdus(). All other frameskeep using
extract_mux().Output: each contained PDU becomes its own channel group, created by the
existing channel-group code.
extract_pdus()returns the same structure asextract_mux()with the PDU identity in the entry key, so the caller isunchanged apart from the routing branch.
Decoding matches
canmatrix.Frame.unpack()for container frames.Signedness fix in extract_signal
extract_signal()decoded a signed signal of standard bit width (8, 16, 32, 64)at a non-byte-aligned offset by viewing the padded byte block as
i{std_size},which does not sign-extend from the field width. Example: an 8-bit signed field
at bit offset 1 returned 225 instead of -31. The condition now also uses
as_non_byte_sized_signed_intwhen the field is not byte aligned. This path isshared with
extract_mux, so the change also affects normal (non-container)extraction of such signals.
Tests
test/test_CAN_pdu_extraction.py(offline, usescanmatrix.Frame.unpack()asthe reference):
extract_pdus()output compared to the reference for big-endian andlittle-endian headers and for 0x00 and 0xFF padding, over frames with random
PDU count and order.
extract_bus_logging()run on a synthesized CAN-logging MF4, checking onechannel group per contained PDU and matching sample values.
Header_ID/Header_DLCreturns no signals.Existing
test/test_CAN_bus_logging.py(OBD2 and J1939) still passes.Not covered
built at 64-byte width without the FD flag set.