CAN-FD Frame: Signals from second PDU in a specific Frame are missing

[mikesmitti]

Python version

('python=3.9.5 (tags/v3.9.5:0a7dcbd, May 3 2021, 17:27:52) [MSC v.1928 64 bit '
'(AMD64)]')
'os=Windows-10-10.0.19042-SP0'
'numpy=1.20.3'
'asammdf=6.2.0'

# Code 

  
# Description

The fastest way to debug is to have the original file. For data protection you can use the static 
method _scramble_ to scramble all text blocks, and send the scrambled file by e-mail.

_Please describe the issue here._


Hello,

I found a problem with CAN-FD.

I have a frame where there can be 2 PDUs in it at one given time.

When there are 2 PDUs in the Frame at once, I dont find the signals from the second PDU (and thus the second signal-group) in the file and therefore get an MdfException.

The signals are in the file, I checked with CANoe and vSignalizer.

The whole stuff is dynamic. There can be 2 PDUs in one frame and in the next frame, there is only one PDU of the 2 possible PDUs.

I THINK (I can´t proof) that the problem occures, when the very first frame in the file with that specific ID has two PDUs in it. 
In that case I get the MdfException.

In case, the very first frame only contains one PDU, it seems to be ok.
I don´t know, if it is possible to check if the signals are missing if there is a frame with 2 PDUs "later" in the file, after the first frame was ok.

Best regards

[danielhrisca]

Hello Mike,

it's hard to figure out what is going on. Any way you can send a scrambled file?

[mikesmitti]

Hello Daniel,
I will try to gather further information.

Best regards

[mikesmitti]

Here is the traceback leading to the MdFexception (shortened to the important parts):

File "xyz_mf4.py", line 661, in plot_by_parameters
mf4_signals = self._mf4file.select(signal_list)
File "...\site-packages\asammdf\mdf.py", line 2656, in select
virtual_groups = self.included_channels(
File "...\site-packages\asammdf\blocks\mdf_v4.py", line 7542, in included_channels
group, idx = self._validate_channel_selection(name)
File "...\site-packages\asammdf\blocks\mdf_common.py", line 113, in _validate_channel_selection
raise MdfException(f'Channel "{name}" not found')

It seems like the missing signal is not found in mdf_common.channels_db.

However, looking into the next file, which has the completely same setup and dbc/arxml files, all works perfectly well

Best regards

Edit:
it is all recorded with CANoe 12. Even the same measurement start. Just stop and start the logging block once. Nothing else changed.

After further investigation, I think it is a problem with multiplexing. Since there are 2 PDUs mapped to the same frame, the second PDU is not found. The first PDU is always found.

The frame has one outer PDU, which contains 8 inner PDUs, in which there are the different signals. Every inner PDU has its own muxer_id.
Maybe the problem arises when there is more than one inner PDU in the actual frame?
Is it possible for asammdf to read TWO PDUs from the very same frame (e.g. the SAME bus message)?

Some background:
I do the stacking as mentioned here #420
After stacking the original and the extract_bus_logging, i do the select.
I tried the select on just the original data, but that does not work at all.

[danielhrisca]

For now we just handle the signal multiplexing inside a PDU, but there must be some other mechanism for multiplexing the PDU's as you have in your file. I can't make progress on this issue without a sample file

[mikesmitti]

I will try to make a sample configuration, but that may take some days.

Best regards

[jazi007]

Hello,

The current release of canmatrix will not allow multiple PDU decoding as they are considered as multiplexed frame. so only first one will be decoded. so imo, even if Daniel is using his own code to decode CAN frames, he cannot know if frame is a multiplexed frame or a pdu container frame with canmatrix<=0.9

Here's the related issue: ebroecker/canmatrix#575
And merged PR: ebroecker/canmatrix#576

hope this helps :-)

++

[danielhrisca]

@jazi007 thank you for the input. Is there any way you could share such an arxml file via e-mail? I can't figure this out from the description alone

[jazi007]

@danielhrisca
you have an example in canmatrix repo:

• https://raw.githubusercontent.com/ebroecker/canmatrix/development/src/canmatrix/tests/ARXMLContainerTest.arxml
• Frame name: Frame_With_Container

it's only one frame but I think it's enough to understand difference between multiplexed frame and contained PDU ones

[mikesmitti]

Hello,

I upgraded to asammdf 7.0.7 (from 6.2.0) today and did a new try on that topic.
Result is that extract_bus_logging does leave an empty opbject, wheras in 6.2.0 it worked properly.

REsulting on that, select, does raise an exception.

I tried it with a minimal example as well:

`
mdffile = MDF(file)
arxml_object = load_can_database(Path(arxml))
dbc_object = load_can_database(Path(dbc))
dbf = {"CAN" : [(arxml, arxml_object), (dbc, dbc_object)]}
mdftemp = mdffile.extract_bus_logging(database_files=dbf)

vals = mdftemp.select(['xyz'])
`
bus_logging_map and channels_db are empty in mdftemp

This is related to this issue:
#420

[kipp-ing]

This should be addressed by #1305. The root cause matches what you described: the extractor did not walk the per-PDU headers within a container frame, so only the PDU at offset 0 was decoded and any subsequent PDU (your "second PDU") went missing / raised MdfException. #1305 adds a real per-frame header walk (Header_ID/Header_DLC), so a variable number of contained PDUs per frame — fully dynamic, as you noted — now each decode into their own channel group. It is validated against canmatrix.Frame.unpack and includes a 64-byte CAN-FD end-to-end test.

If you are able to, testing against your original file would be very helpful:
pip install "git+https://github.com/kipp-ing/asammdf.git@dynamic-pdu-container-extraction"