Skip to content

fix(gateway): freeze-frame the faults that were already raised at startup#563

Open
mfaferek93 wants to merge 1 commit into
mainfrom
fix/562-standing-fault-freeze-frames
Open

fix(gateway): freeze-frame the faults that were already raised at startup#563
mfaferek93 wants to merge 1 commit into
mainfrom
fix/562-standing-fault-freeze-frames

Conversation

@mfaferek93

Copy link
Copy Markdown
Collaborator

Closes #562.

Plugins start reporting during GatewayNode construction, while the capture is
wired later from main.cpp (it needs the shared subscription executor, which is
built after the node). A device that is already in fault confirms in that window,
its EVENT_CONFIRMED is published to nobody, and since the capture only fires on
the confirm edge the fault stays frameless for as long as it is raised.

capture_standing_fault_frames() runs once the capture is live: it lists the
confirmed faults and enqueues a capture for each fault that has no frames yet.
Frames are process-local, so this restores what the missed edge would have
produced rather than duplicating anything, and capture_missed_fault() is a
no-op when the live event already captured the fault.

It runs detached, after wait_for_services with a bounded timeout, so startup
never blocks on the fault services being up. The synthetic event goes through the
same queue and worker as a real one, so the backlog bound and the plugin-call
threading rules are unchanged.

Worth naming: a frame captured this way holds the values at catch-up time, a
second or so after the confirm, not the values at the confirm itself. For a fault
that is standing at boot that is the only honest option - the earlier values no
longer exist anywhere.

Copilot AI review requested due to automatic review settings July 26, 2026 16:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a startup race in the gateway where faults that are already CONFIRMED at boot can miss their entity freeze-frame capture because the /fault_manager/events subscription is wired after plugins begin reporting faults.

Changes:

  • Add a startup catch-up path (capture_standing_fault_frames()) that lists currently confirmed faults after the capture is initialized and enqueues synthetic confirm events for frameless faults.
  • Add EntityFreezeFrameCapture::capture_missed_fault() to enqueue a synthetic confirmed fault event through the existing bounded queue/worker.
  • Expose the new catch-up helper on GatewayNode and document the intended “missed confirm” behavior in headers.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
src/ros2_medkit_gateway/src/gateway_node.cpp Wires a detached startup catch-up thread to list confirmed faults and enqueue missed captures.
src/ros2_medkit_gateway/src/entity_freeze_frame_capture.cpp Adds capture_missed_fault() to synthesize confirm events into the existing capture queue.
src/ros2_medkit_gateway/include/ros2_medkit_gateway/gateway_node.hpp Declares the new capture_standing_fault_frames() method on GatewayNode.
src/ros2_medkit_gateway/include/ros2_medkit_gateway/entity_freeze_frame_capture.hpp Documents and declares capture_missed_fault() for missed-confirm catch-up.

Comment thread src/ros2_medkit_gateway/src/gateway_node.cpp Outdated
Comment thread src/ros2_medkit_gateway/src/entity_freeze_frame_capture.cpp Outdated
Comment thread src/ros2_medkit_gateway/src/entity_freeze_frame_capture.cpp Outdated
Comment thread src/ros2_medkit_gateway/include/ros2_medkit_gateway/gateway_node.hpp Outdated
@mfaferek93
mfaferek93 force-pushed the fix/562-standing-fault-freeze-frames branch 3 times, most recently from 99ab9c0 to 695f691 Compare July 26, 2026 19:40
…rtup

Plugins start reporting while the capture is still being wired, so a device
that is in fault when the box boots confirms before the subscription exists and
its freeze frame is lost for as long as the fault stays raised. The capture now
asks for the standing faults on its own worker thread - the one its destructor
joins - so the wait for the fault services cannot outlive the node.
@mfaferek93
mfaferek93 force-pushed the fix/562-standing-fault-freeze-frames branch from 695f691 to c146a1c Compare July 26, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gateway: faults already raised at startup never get a freeze frame

2 participants