Add test warning system for known flaky tests

[Krilliac]

Known flaky tests (timing-dependent, thread-sensitive, etc.) now produce [ WARN ] instead of [ FAILED ] and do not cause the test suite to return a non-zero exit code. This prevents intermittent CI failures from blocking PRs while keeping the failures visible via GitHub Actions warning annotations and the CI error report.

Changes:

• Tests/TestWarnings.h: central registry of flaky test patterns with reasons
• Tests/TestFramework.h: add g_testsWarned counter
• Tests/TestMain.cpp: warning detection, output, summary, JUnit XML (skipped), --warn-is-error and --list-warnings CLI flags
• extract-errors.sh: route [ WARN ] lines to warnings (not test failures)
• report-ci-errors.js: separate "Test Warnings (Known Flaky)" section in PR comment, distinct from compiler warnings

https://claude.ai/code/session_01BzgBs1HmgxNvPJDeu1o6yK

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e6c3b97d2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[github-actions]

Code Coverage (GCC + lcov)

Utils/MemoryMonitor.h                          |85.7%    14| 0.0%  12|    -    0
Utils/MultiISA.h                               |42.9%    14| 0.0%   6|    -    0
Utils/NetworkHealthMonitor.cpp                 | 113%    15| 0.0%   2|    -    0
Utils/NetworkHealthMonitor.h                   |    -     0|    -   0|    -    0
Utils/OpaqueHandle.h                           | 282%    11| 0.0%  15|    -    0
Utils/ProcessLinux.cpp                         |16.5%   182| 0.0%  28|    -    0
Utils/Profiler.cpp                             |14.2%   113| 0.0%  15|    -    0
Utils/Profiler.h                               |36.4%    33| 0.0%  12|    -    0
Utils/RandomEngine.h                           |29.3%    41| 0.0%  12|    -    0
Utils/Result.h                                 | 100%    29| 0.0%  28|    -    0
Utils/RingBuffer.h                             | 108%    59| 0.0%  64|    -    0
Utils/ScopeGuard.h                             | 142%    36| 0.0%  51|    -    0
Utils/ScopedTimer.h                            |25.0%    12| 0.0%   3|    -    0
Utils/Serializer.h                             | 100%    48| 0.0%  42|    -    0
Utils/SparkConsole.cpp                         |28.5%   144| 0.0%  17|    -    0
Utils/SparkConsole.h                           | 100%     2| 0.0%   2|    -    0
Utils/SparkError.h                             |10.9%    55| 0.0%   6|    -    0
Utils/SplinePath.h                             |    -     0|    -   0|    -    0
Utils/StackTrace.h                             |10.8%    74| 0.0%   8|    -    0
Utils/StateMachine.h                           |54.0%    63| 0.0%  32|    -    0
Utils/StringUtils.h                            |18.1%   116| 0.0%  21|    -    0
Utils/Telemetry.h                              |20.7%   135| 0.0%  22|    -    0
Utils/ThreadDebugger.h                         |11.6%   199| 0.0%  23|    -    0
Utils/ThreadSafeQueue.h                        |27.5%    40| 0.0%  11|    -    0
Utils/TimerManager.h                           |19.8%    96| 0.0%  19|    -    0
Utils/UUID.h                                   |43.2%    37| 0.0%  16|    -    0
Utils/Validate.h                               |    -     0|    -   0|    -    0

[/home/runner/work/SparkEngine/SparkEngine/SparkSDK/Include/Spark/]
IEngineContext.h                               |7300%     1| 0.0%   1|    -    0
ServiceInterfaces.h                            | 200%     3| 0.0%   3|    -    0
Version.h                                      |    -     0|    -   0|    -    0
================================================================================
                                         Total:|29.2% 23751| 0.0%  4k|    -    0

Per-Subsystem Coverage

Subsystem	Lines	Hit	Coverage	Threshold	Status
AI	2776	694	25%	35%	❌
Animation	160	154	96.2%	35%	✅
Audio	0	0	0%	30%	✅
Camera	0	0	0%	40%	✅
Core	2248	1394	62%	40%	✅
ECS	190	114	60%	40%	✅
Editor	6084	1864	30.6%	25%	✅
GameModules	6244	3144	50.4%	30%	✅
Graphics	7203	3796	52.7%	30%	✅
Networking	3379	2247	66.5%	35%	✅
Physics	0	0	0%	35%	✅
Scripting	0	0	0%	30%	✅
Utils	7687	4865	63.3%	60%	✅

Total: 50.8% (18272/35971 lines)

[github-actions]

❌ CI Error Report

Failed jobs: windows-vs2022-Release
Errors: 0 | Test failures: 3 | Test warnings: 15 | Compiler warnings: 0

Test Failures

Test	Jobs
1/1 Test #1: SparkEngineTests .................***Failed 12.46 sec	windows-vs2022-Release
[ FAILED ] Integration_NetworkingECS_ReplicationLatencyJitterPredictionReconciliation (718us, 8 assertions)	windows-vs2022-Release
0% tests passed, 1 tests failed out of 1	windows-vs2022-Release

⚠️ Test Warnings (Known Flaky)

These tests are registered as known flaky and do not block the build:

Test	Jobs
[03:30:38.488] [TID:1204] [WARN ] [Network ] RCON unknown command: nonexistent_cmd (DedicatedServer.cpp:573)	windows-vs2022-Release
[03:30:38.503] [TID:1204] [WARN ] [Network ] NetBuffer::ReadUint8 — buffer overrun at pos 1 (size=1) (NetworkBuffer.c...	windows-vs2022-Release
[03:30:38.503] [TID:1204] [WARN ] [Network ] NetBuffer::ReadUint32 — buffer overrun at pos 1 (need 4, size=1) (Networ...	windows-vs2022-Release
[03:30:38.522] [TID:1204] [WARN ] [AI ] BuildNavMeshWithRecast: empty geometry (RecastDetourBackend.cpp:35)	windows-vs2022-Release
[03:30:38.522] [TID:1204] [WARN ] [AI ] NavMeshBuilder: Recast build failed, falling back to triangle-soup builder (N...	windows-vs2022-Release
[03:30:38.523] [TID:1204] [WARN ] [AI ] NavMeshBuilder: Recast build failed, falling back to triangle-soup builder (N...	windows-vs2022-Release
[ OK ] Tr[03:30:38.524] [TID:1204] [WARN ] [Graphics ] No graphics backend available — falling back to NullRHIDevice ...	windows-vs2022-Release
[03:30:38.524] [TID:1204] [WARN ] [Graphics ] No graphics backend available — falling back to NullRHIDevice (headless...	windows-vs2022-Release
[03:30:38.528] [TID:1204] [WARN ] [Editor ] Cannot host: userName is empty. (CollaborativeEditSession.cpp:454)	windows-vs2022-Release
[03:30:38.528] [TID:1204] [WARN ] [Editor ] Already connected. (CollaborativeEditSession.cpp:459)	windows-vs2022-Release
[03:30:38.531] [TID:1204] [WARN ] [Editor ] BroadcastEdit rejected: nodeId is empty. (CollaborativeEditSession.cpp:1119)	windows-vs2022-Release
[03:30:38.531] [TID:1204] [WARN ] [Editor ] BroadcastEdit rejected: sourceEditor is not set. (CollaborativeEditSessio...	windows-vs2022-Release
[03:30:38.532] [TID:1204] [WARN ] [Editor ] SetLocalSelection rejected: nodeId exceeds 255 chars (length=300). (Colla...	windows-vs2022-Release
[03:30:44.098] [TID:1204] [WARN ] [Network ] Connection rejected for pending client 5: server full (4/4) (NetworkConn...	windows-vs2022-Release
[03:30:44.099] [TID:1204] [WARN ] [Network ] Connection rejected for pending client 5: server full (4/4) (NetworkConn...	windows-vs2022-Release

Updated: 2026-04-10T03:31:18Z — this comment is updated in-place, not duplicated.