coll/nbc return correct error code from progress during start of round

[Matthew-Whitlock]

When a fault tolerance error causes NBC_Progress to return an error code, it will return the specific fault tolerance error code. NBC_Start_round is ignoring this and returning OMPI_ERROR instead of the correct code, which causes MPI functions to return MPI_ERR_OTHER instead of the correct fault tolerance error code.

This causes intermittent failures of Fenix's CI testing: sandialabs/Fenix#136

[hppritcha]

Review Summary

The PR fixes a bug where NBC_Start_round was discarding specific fault-tolerance error codes from NBC_Progress and returning the generic OMPI_ERROR instead. The fix changes line 602 from return OMPI_ERROR; to return res;.

Analysis

The change is correct and necessary.

The Problem

Looking at NBC_Progress (lines 321-428), it can return several different error codes:

• NBC_OK (0) when complete
• NBC_CONTINUE (3) when still in progress
• Specific MPI error codes like MPI_ERR_PROC_FAILED, MPI_ERR_PROC_FAILED_PENDING, or MPI_ERR_REVOKED (line 391 returns res = handle->super.super.req_status.MPI_ERROR)
• Error codes from NBC_Start_round itself (line 423)

The code at line 599-604 calls NBC_Progress and checks if the result is something other than NBC_OK or NBC_CONTINUE. When fault-tolerance is enabled (OPAL_ENABLE_FT_MPI), NBC_Progress specifically preserves fault-tolerance error codes (lines 347-358), but the current code throws them away by returning the generic OMPI_ERROR.

The Fix

The fix correctly propagates the specific error code (res) instead of the generic OMPI_ERROR. This is consistent with the error handling pattern used elsewhere in the same function:

• Line 483: return res; (MPI_Isend error)
• Line 517: return res; (MPI_Irecv error)
• Line 562: return res; (NBC_Copy error)
• Line 584: return res; (NBC_Unpack error)

Impact

Without this fix, when a process fails during a non-blocking collective operation, the user gets MPI_ERR_OTHER instead of the correct fault-tolerance error like MPI_ERR_PROC_FAILED. This breaks fault-tolerant MPI applications that need to distinguish between different error types to handle failures appropriately.

Verdict

No problems found. The change is:

• ✅ Minimal and surgical (one line)
• ✅ Consistent with surrounding error-handling patterns
• ✅ Preserves important error code semantics for fault tolerance
• ✅ Does not affect the non-fault-tolerance code path (those errors still propagate correctly)

This PR should be merged.

[hppritcha]

/backport v6.0.x

[hppritcha]

/backport v5.0.x