Skip to content

Add differential fuzz coverage for provider-message decoding - #14

Closed
numinousmuses wants to merge 2 commits into
masterfrom
forge/task-291edd0bcf6d6d0efc56257bb6863e68
Closed

numinousmuses wants to merge 2 commits into
masterfrom
forge/task-291edd0bcf6d6d0efc56257bb6863e68

Conversation

@numinousmuses

Copy link
Copy Markdown

What this changes

Adds a differential fuzz target for the provider-message dispatch path in coordinator/protocol. The decoder uses a fast scanner to read type before decoding the concrete message; existing tests cover a fixed corpus but there was no differential check on this path.

The new type_scan_fuzz_test.go adds:

  • referenceDecodeProviderMessage, an independent reimplementation that reads type with a standard JSON envelope decode and then unmarshals the matching concrete struct, never touching the fast scanner.
  • assertProviderMessageScanEquivalent, which holds DecodeProviderMessage to that reference on acceptance and on decoded value. Error wording is intentionally not compared.
  • TestProviderMessageScanEquivalenceSeeds and FuzzProviderMessageScanEquivalence over a seed corpus covering representative types, escaped and case-variant type keys, duplicate keys, nested data, malformed JSON, and unknown types. The fuzz target skips inputs over 8 KiB.

This revision adds a directly escaped type key seed and a mixed-case duplicate-key seed that verifies last-match-wins selection across two different known types. The change is test-only; no wire behavior is modified.

Property under test

For any input, the production scanner path accepts exactly when the envelope reference accepts, and when both accept they produce equal ProviderMessage values. A dispatch bug (a case wired to the wrong struct, a dropped case, a scanner reading the wrong key) surfaces as an accept/reject or value divergence.

Verification

The configured Protocol behavior check (439 tests) and Documentation check passed on this revision. Neither check names this target by name, so these passes confirm the protocol suite and docs are green on this revision but do not by themselves establish the new equivalence property. The agent separately reported local seed runs under the race detector, a deliberate dispatch-regression that the new coverage caught, and a bounded 8 KiB fuzz session (134k execs, no crashes) with production code restored; those are local experiments, not independent checks.

Review focus

  • Whether the reference dispatch faithfully mirrors the production switch arms.
  • Whether the seed corpus covers the intended adversarial shapes.
  • Confirming the diff is test-only.

Before / After

flowchart TD
  subgraph Before
    A1[Fixed corpus tests] --> B1[DecodeProviderMessage]
  end
  subgraph After
    A2[Seed + fuzz inputs] --> B2[DecodeProviderMessage scanner path]
    A2 --> C2[Envelope reference decode]
    B2 --> D2{Agree on accept and value?}
    C2 --> D2
    D2 -->|no| E2[Fail: divergence reported]
    D2 -->|yes| F2[Pass]
  end
Loading

Requested in #11. Follow the issue conversation for findings during implementation.

Independent checks configured: Protocol behavior, Documentation.

Recorded check Result
Protocol behavior Passed
Documentation Passed

Source revision d178a220e28b. Verification status comes from execution records; agent observations are provisional.

The proposed change passed independent verification. PR CI checks its merge with the target branch separately. Maintainers review the behavior and decide what ships.

Verification reference

Source: d178a220e28be48e99946b04d724a677b07f3385
Verified tree: ef7823221f1f55da47089e880d02764afb4e60a2
Task: 291edd0bcf6d6d0efc56257bb6863e68

Verified tree: ef78232
Policy: c0f26d7da586fda2c72702cf736512909371e691eaef0893b0d9eeea404ccd85
@numinousmuses

numinousmuses commented Sep 6, 2026 •

Copy link
Copy Markdown
Author

Numinous Forge · Independent verification passed

All three configured checks passed on this revision: Protocol behavior (439 tests), Documentation, and the Coordinator suite under the race detector (5011 tests), which was queued in the previous update. The change is test-only, adding a differential fuzz target that holds DecodeProviderMessage to an independent envelope-based reference on acceptance and decoded value across seeds for escaped and case-variant keys, duplicate keys, nested data, malformed JSON, and unknown types. None of these checks names the new equivalence target, so their passes confirm the protocol and coordinator suites are green here but do not by themselves establish the new property. Review focus: confirm the reference dispatch mirrors the production switch and that the diff is test-only.

Next: Maintainers review the reference dispatch and seed corpus and decide whether to merge; PR CI checks the merge with the target branch separately.

Checks and recent activity

Independent checks configured: Protocol behavior, Documentation, Coordinator.

Recorded check Result
Protocol behavior Passed
Documentation Passed
Coordinator Passed

Source revision 64187bb27d24. Verification status comes from execution records; agent observations are provisional.

This run verifies the PR and does not modify its code.

Recent activity, newest first:

  • 2026-09-06 14:34 UTC: Independent verification passed. Maintainers review the reference dispatch and seed corpus and decide whether to merge; PR CI checks the merge with the target branch separately.
  • 2026-09-06 14:28 UTC: Checking the change. Waiting on the queued Coordinator race-detector suite before verification completes; maintainers review and decide whether to merge.
  • 2026-09-06 14:25 UTC: Checking the change. Documentation and Coordinator checks are still to complete before maintainers review the behavior.
  • 2026-09-06 14:23 UTC: Checking the change. Await the configured checks on this revision; maintainers then review whether the reference mirrors the production switch and decide on merge.
  • 2026-09-06 14:22 UTC: New revision awaiting verification. Wait for the new revision's checks to start.

@numinousmuses numinousmuses moved this from Verifying to Queued in Darkbloom engineering Sep 6, 2026
@numinousmuses numinousmuses moved this from Queued to Verifying in Darkbloom engineering Sep 6, 2026
@numinousmuses numinousmuses moved this from Verifying to Needs review in Darkbloom engineering Sep 6, 2026
@numinousmuses numinousmuses added the trigger:instructed Work was explicitly requested; generated PRs inherit this trigger. label Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

trigger:instructed Work was explicitly requested; generated PRs inherit this trigger.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

1 participant