Summary
A fault that is already active when the gateway starts never gets an entity
freeze frame. Plugins begin polling - and reporting - while
EntityFreezeFrameCapture is still being wired, so the EVENT_CONFIRMED that
would trigger the capture is published before the subscription exists.
Evidence
Gateway startup, same process, 15 ms apart:
[INFO] [plugin.<protocol>]: Fault active: PLC_MINOR_FAULT on <entity>_device
[INFO] [fault_manager]: New fault reported: PLC_MINOR_FAULT (source=<entity>_device)
[INFO] [ros2_medkit_gateway]: EntityFreezeFrameCapture initialized, subscribed to /fault_manager/events
The fault detail then carries a rosbag but no freeze frame, and never gains one
while it stays raised - the capture only fires on the confirm edge.
Why it matters
This is the normal case, not an edge case: a device that is in fault stays in
fault, so every restart of the gateway loses the freeze frames for everything
standing at that moment. entity_freeze_frame.enabled defaults to true, so the
feature silently under-delivers exactly when a customer restarts to look at a
problem.
Cause
GatewayNode loads and starts plugins during construction, while
init_entity_freeze_frame_capture() runs from main.cpp after the shared
subscription executor exists (the executor is built after the node). The
subscription is QoS(100).reliable() with volatile durability, so nothing that
was published before it joined is delivered.
Fix direction
Frames are process-local (frames_ in the capture), so a catch-up restores them
rather than duplicating anything: once the capture is live, list the confirmed
faults and capture a frame for each that has none yet. Deferring plugin start
until after the capture is wired would also work but reorders the plugin
lifecycle against entity introspection.
A frame captured this way holds the values at catch-up time, not at confirm
time, which is worth stating in the field description.
Summary
A fault that is already active when the gateway starts never gets an entity
freeze frame. Plugins begin polling - and reporting - while
EntityFreezeFrameCaptureis still being wired, so theEVENT_CONFIRMEDthatwould trigger the capture is published before the subscription exists.
Evidence
Gateway startup, same process, 15 ms apart:
The fault detail then carries a rosbag but no freeze frame, and never gains one
while it stays raised - the capture only fires on the confirm edge.
Why it matters
This is the normal case, not an edge case: a device that is in fault stays in
fault, so every restart of the gateway loses the freeze frames for everything
standing at that moment.
entity_freeze_frame.enableddefaults to true, so thefeature silently under-delivers exactly when a customer restarts to look at a
problem.
Cause
GatewayNodeloads and starts plugins during construction, whileinit_entity_freeze_frame_capture()runs frommain.cppafter the sharedsubscription executor exists (the executor is built after the node). The
subscription is
QoS(100).reliable()with volatile durability, so nothing thatwas published before it joined is delivered.
Fix direction
Frames are process-local (
frames_in the capture), so a catch-up restores themrather than duplicating anything: once the capture is live, list the confirmed
faults and capture a frame for each that has none yet. Deferring plugin start
until after the capture is wired would also work but reorders the plugin
lifecycle against entity introspection.
A frame captured this way holds the values at catch-up time, not at confirm
time, which is worth stating in the field description.