fix(intake): make the OIA matrix notification at-most-once - #68
Conversation
Closes a gap I left in agenticsorg#58. An outside contributor was notified twice for one submission because of it. ## What happened On 2026-09-28, after agenticsorg#58/agenticsorg#59/agenticsorg#61 merged, all five open submissions were label-cycled (`unlabeled` then `labeled`, three seconds apart) to backfill the notifications they never received while the label race was live. That worked and was the right call: agenticsorg#44 finally got its case brief after 49 days, and agenticsorg#52's matrix row, lost to the concurrency bug in August, was regenerated. But agenticsorg#51 and agenticsorg#53 each received a **second** "Added to the OIA Application Matrix" comment on top of the one they already had from 2026-08-21. @RobLe3 was notified twice for the same submission, on two separate issues. Cause: agenticsorg#58 added idempotency guards to the welcome comment in `on-submission.yml` and the case brief in `governance-agent.yml`, and `scripts/oia-intake-step.mjs` was the one comment path that did not get one. Its only guard was `OIA_SKIP_COMMENT`, which covers the persist-retry path added in agenticsorg#61 and nothing else. This is not a networking failure. It is a missing decision, and a missing decision belongs in a test rather than in care. ## The fix `alreadyPosted(comments, marker)` is a pure, exported predicate. Before posting, the step reads the issue's comments and skips if its own marker is already present. Two markers, exported as constants so the test pins the exact strings: rewording a heading without updating its marker would silently re-enable duplicates, and now that is a build failure. **Fails closed.** If the comment list cannot be read, the step does not post and says so in the log. A missed notification is recoverable by re-applying the label, which is already a supported remediation. A duplicate sent to an outside contributor is not recoverable. The one-off failure notice ("could not auto-classify") is deliberately left unconditional. If a repository URL is still wrong on a re-run, saying so again is useful rather than noise. ## Also: the script is now import-safe Its dispatch ran at module load, so importing it executed intake. That made the guard untestable and, worse, meant an accidental import would act on a live issue: post comments, mutate `docs/data/oia-matrix.json`, write badges. I confirmed this by running the CLI during review, which classified a repository and dirtied the worktree. Wrapped in `main()` behind an entry-point check comparing `import.meta.url` against `pathToFileURL(process.argv[1]).href`, matching `scripts/classify-oia.mjs` and `scripts/mutate.mjs`. A test asserts the module exposes only `alreadyPosted` and the two markers when imported. ## Tests 7 new, suite 171/171. Red-first: stubbing `alreadyPosted` to always return false, which is the pre-fix behaviour, turns 3 of the 7 red. Covered: the exact duplicate that reached agenticsorg#51 and agenticsorg#53; a first notification still goes out; empty, absent and malformed comment lists; the two markers not colliding with each other; and the markers genuinely appearing in the verbatim comment bodies this repo has posted to live issues. ## Worth knowing, since it bears on the same event The 2026-09-28 label-cycle @-mentioned the whole committee across all five submissions. **It produced zero scores.** No `/score` or `/vote` comment exists on any of agenticsorg#44, agenticsorg#51, agenticsorg#52, agenticsorg#53 or agenticsorg#55. Notification was not the constraint. That matters for how much more intake automation is worth building before the committee scores something.
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
It changes production CI behavior that notifies external contributors on live issues, and the network-dependent fail-closed comment paths are not exercised by the automated tests, so human verification is warranted.
Review effort: Balanced
Findings: 1
Open (1)
What changed in this PR
This PR closes an at-most-once gap in the OIA intake notification path. After #58 added idempotency guards to the welcome comment and case brief, scripts/oia-intake-step.mjs remained the one comment path without such a guard, so a label-cycle backfill on 2026-09-28 sent duplicate "Added to the OIA Application Matrix" comments to an outside contributor on #51 and #53. The fix introduces a pure, exported alreadyPosted predicate plus a fail-closed comment lister, and makes the script import-safe by moving its dispatch behind an entry-point check so it can be unit-tested and won't act on a live issue when imported.
Changes:
- Add exported
PENDING_MARKER/PROMOTED_MARKERconstants and a purealreadyPosted(comments, marker)predicate; gate the pending and promotion comments on it, failing closed (not posting) when the comment list cannot be read. - Wrap the intake dispatch in
main()behind animport.meta.urlvspathToFileURL(process.argv[1]).hrefentry-point check, matchingscripts/classify-oia.mjs, so importing the module no longer runs intake. - Add
test/oia-intake-idempotency.test.jswith 7 tests covering the duplicate scenario, first-notification, empty/absent/malformed comment lists, marker non-collision, verbatim live bodies, and import-safety.
| File | Description |
|---|---|
| scripts/oia-intake-step.mjs | Adds markers, alreadyPosted, fail-closed listComments, marker-gated comment(), and an entry-point-guarded main(). |
| test/oia-intake-idempotency.test.js | New unit tests for the idempotency predicate, marker/body alignment, and import-safety. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| const liveePending = '### Added to the OIA Application Matrix (pending review)\n\n`bar181/aisp-open-core` has been auto-classified into the **pending review** band.'; | ||
| const livePromoted = '✅ **acme/thing** promoted to the curated OIA Application Matrix: https://agenticsorg.github.io/community-projects/oia-matrix.html#oia-acme-thing'; | ||
| assert.ok(liveePending.includes(PENDING_MARKER), 'PENDING_MARKER must appear in the real pending comment'); |
|
@copilot Fix the code for all comments in this review thread. When a review comment includes a suggested change, apply the suggestion exactly. Do not make changes beyond what is described in the linked review thread. |

fix(intake): make the OIA matrix notification at-most-once
Closes a gap I left in #58. An outside contributor was notified twice for one
submission because of it.
What happened
On 2026-09-28, after #58/#59/#61 merged, all five open submissions were
label-cycled (
unlabeledthenlabeled, three seconds apart) to backfill thenotifications they never received while the label race was live. That worked and
was the right call: #44 finally got its case brief after 49 days, and #52's
matrix row, lost to the concurrency bug in August, was regenerated.
But #51 and #53 each received a second "Added to the OIA Application Matrix"
comment on top of the one they already had from 2026-08-21. @RobLe3 was notified
twice for the same submission, on two separate issues.
Cause: #58 added idempotency guards to the welcome comment in
on-submission.ymland the case brief ingovernance-agent.yml, andscripts/oia-intake-step.mjswas the one comment path that did not get one. Itsonly guard was
OIA_SKIP_COMMENT, which covers the persist-retry path added in#61 and nothing else.
This is not a networking failure. It is a missing decision, and a missing
decision belongs in a test rather than in care.
The fix
alreadyPosted(comments, marker)is a pure, exported predicate. Before posting,the step reads the issue's comments and skips if its own marker is already
present. Two markers, exported as constants so the test pins the exact strings:
rewording a heading without updating its marker would silently re-enable
duplicates, and now that is a build failure.
Fails closed. If the comment list cannot be read, the step does not post and
says so in the log. A missed notification is recoverable by re-applying the
label, which is already a supported remediation. A duplicate sent to an outside
contributor is not recoverable.
The one-off failure notice ("could not auto-classify") is deliberately left
unconditional. If a repository URL is still wrong on a re-run, saying so again is
useful rather than noise.
Also: the script is now import-safe
Its dispatch ran at module load, so importing it executed intake. That made the
guard untestable and, worse, meant an accidental import would act on a live
issue: post comments, mutate
docs/data/oia-matrix.json, write badges. Iconfirmed this by running the CLI during review, which classified a repository
and dirtied the worktree.
Wrapped in
main()behind an entry-point check comparingimport.meta.urlagainst
pathToFileURL(process.argv[1]).href, matchingscripts/classify-oia.mjsandscripts/mutate.mjs. A test asserts the moduleexposes only
alreadyPostedand the two markers when imported.Tests
7 new, suite 171/171. Red-first: stubbing
alreadyPostedto always return false,which is the pre-fix behaviour, turns 3 of the 7 red.
Covered: the exact duplicate that reached #51 and #53; a first notification still
goes out; empty, absent and malformed comment lists; the two markers not
colliding with each other; and the markers genuinely appearing in the verbatim
comment bodies this repo has posted to live issues.
Worth knowing, since it bears on the same event
The 2026-09-28 label-cycle @-mentioned the whole committee across all five
submissions. It produced zero scores. No
/scoreor/votecomment existson any of #44, #51, #52, #53 or #55.
Notification was not the constraint. That matters for how much more intake
automation is worth building before the committee scores something.