Skip to content

fix: VLSD signal data is unreadable after a composed channel - #1307

Open
kipp-ing wants to merge 2 commits into
danielhrisca:developmentfrom
kipp-ing:fix/vlsd-signal-data-index
Open

kipp-ing wants to merge 2 commits into
danielhrisca:developmentfrom
kipp-ing:fix/vlsd-signal-data-index

Conversation

@kipp-ing

@kipp-ing kipp-ing commented Jul 29, 2026

Copy link
Copy Markdown

Background — what VLSD is

Most MDF4 channels are fixed-width columns inside the record: the channel says
"my value lives at byte offset 12, 2 bytes wide", and every sample is the same
size. Some data cannot work that way — a text channel, or a CAN-FD payload whose
length changes from frame to frame. MDF4 stores those out of line: the record
holds only a 64-bit offset, and the actual bytes sit in separate signal-data
blocks elsewhere in the file. Such a channel is a VLSD channel — variable
length signal data
.

To read one, asammdf has to know where that channel's signal-data blocks are.
It collects them per group in Group.signal_data, a plain list indexed by
channel index
, filled while the channel list is parsed. If that list ever falls
out of step with Group.channels, a VLSD channel looks up somebody else's blocks
— or none at all.

That is what happens here.

Problem

MDF4._read_channels appends an extra grp.signal_data.append(None) for
every channel that has a component_addr (a structure or array composition),
while dependencies gets exactly one entry per channel.

So every composed channel shifts the signal-data entry of all following channels
by one, and _load_signal_data then looks in the wrong slot. Reading such a
channel raises:

MdfException: Wrong signal data block refence (0x4AD5260) for VLSD channel "CAN_DataFrame.DataBytes"

Impact

This affects every CAN-FD bus-logging measurement recorded with variable-length
payloads — which is the normal case, since a CAN-FD frame carries anywhere from
0 to 64 bytes. CAN_DataFrame.DataBytes is a VLSD channel, and it lives inside
the composed CAN_DataFrame structure, so it is always one of the shifted ones.
MDF.extract_bus_logging() cannot read the payload at all and the file is
effectively unusable.

Found on two real OEM CAN-FD bus logs (1.0 GB and 157 MB, 5.1 M and 2.3 M CAN
frames); with the fix both extract normally.

Fix

Drop the stray append. grp.signal_data then stays in step with grp.channels.

Test

test_vlsd_channel_after_structure_composition in test/test_mdf4.py — writes a
VLSD (string) channel after a structure channel in the same group, then reads it
back. Fully offline, no downloads.

Without the fix:

AssertionError: 6 != 5                                     # len(signal_data) vs len(channels)
MdfException: Wrong signal data block refence (0x7D0) ...   # reading the channel

test_mdf, test_mdf4, test_CAN_bus_logging, test_endianess, test_cantp
and test_signal all pass.

@kipp-ing
kipp-ing force-pushed the fix/vlsd-signal-data-index branch from cd8c5c2 to 31fa41c Compare July 29, 2026 19:16
@kipp-ing
kipp-ing marked this pull request as ready for review July 29, 2026 19:24
@kipp-ing
kipp-ing marked this pull request as draft July 29, 2026 19:26
@kipp-ing
kipp-ing force-pushed the fix/vlsd-signal-data-index branch from 31fa41c to 5c8cc75 Compare July 29, 2026 19:31
@kipp-ing
kipp-ing changed the base branch from master to development July 29, 2026 19:31
@kipp-ing
kipp-ing marked this pull request as ready for review July 29, 2026 19:31
`MDF4._read_channels` appends an extra `grp.signal_data.append(None)` for
every channel that carries a `component_addr` (a structure or array
composition), while `dependencies` gets exactly one entry per channel.
`Group.signal_data` is indexed by channel index, so each composed channel
shifts the VLSD block info of every following channel by one and the payload
can no longer be located:

    MdfException: Wrong signal data block refence (0x...) for VLSD channel "..."

This bites every CAN-FD bus-logging file recorded with variable-length
payloads: `CAN_DataFrame.DataBytes` is a VLSD channel inside the composed
`CAN_DataFrame` structure, so `MDF.extract_bus_logging` cannot read it at all.

Drop the stray append and cover it with a regression test that writes a VLSD
channel after a structure channel in the same group.
@kipp-ing
kipp-ing force-pushed the fix/vlsd-signal-data-index branch from 5c8cc75 to 54016ac Compare August 15, 2026 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant