Skip to content

fix(intake): make the OIA matrix notification at-most-once - #68

Merged
michaeloboyle merged 1 commit into
agenticsorg:mainfrom
michaeloboyle:fix/oia-intake-idempotency
Oct 1, 2026
Merged

michaeloboyle merged 1 commit into
agenticsorg:mainfrom
michaeloboyle:fix/oia-intake-idempotency

Conversation

@michaeloboyle

Copy link
Copy Markdown
Collaborator

fix(intake): make the OIA matrix notification at-most-once

Closes a gap I left in #58. An outside contributor was notified twice for one
submission because of it.

What happened

On 2026-09-28, after #58/#59/#61 merged, all five open submissions were
label-cycled (unlabeled then labeled, three seconds apart) to backfill the
notifications they never received while the label race was live. That worked and
was the right call: #44 finally got its case brief after 49 days, and #52's
matrix row, lost to the concurrency bug in August, was regenerated.

But #51 and #53 each received a second "Added to the OIA Application Matrix"
comment on top of the one they already had from 2026-08-21. @RobLe3 was notified
twice for the same submission, on two separate issues.

Cause: #58 added idempotency guards to the welcome comment in
on-submission.yml and the case brief in governance-agent.yml, and
scripts/oia-intake-step.mjs was the one comment path that did not get one. Its
only guard was OIA_SKIP_COMMENT, which covers the persist-retry path added in
#61 and nothing else.

This is not a networking failure. It is a missing decision, and a missing
decision belongs in a test rather than in care.

The fix

alreadyPosted(comments, marker) is a pure, exported predicate. Before posting,
the step reads the issue's comments and skips if its own marker is already
present. Two markers, exported as constants so the test pins the exact strings:
rewording a heading without updating its marker would silently re-enable
duplicates, and now that is a build failure.

Fails closed. If the comment list cannot be read, the step does not post and
says so in the log. A missed notification is recoverable by re-applying the
label, which is already a supported remediation. A duplicate sent to an outside
contributor is not recoverable.

The one-off failure notice ("could not auto-classify") is deliberately left
unconditional. If a repository URL is still wrong on a re-run, saying so again is
useful rather than noise.

Also: the script is now import-safe

Its dispatch ran at module load, so importing it executed intake. That made the
guard untestable and, worse, meant an accidental import would act on a live
issue: post comments, mutate docs/data/oia-matrix.json, write badges. I
confirmed this by running the CLI during review, which classified a repository
and dirtied the worktree.

Wrapped in main() behind an entry-point check comparing import.meta.url
against pathToFileURL(process.argv[1]).href, matching
scripts/classify-oia.mjs and scripts/mutate.mjs. A test asserts the module
exposes only alreadyPosted and the two markers when imported.

Tests

7 new, suite 171/171. Red-first: stubbing alreadyPosted to always return false,
which is the pre-fix behaviour, turns 3 of the 7 red.

Covered: the exact duplicate that reached #51 and #53; a first notification still
goes out; empty, absent and malformed comment lists; the two markers not
colliding with each other; and the markers genuinely appearing in the verbatim
comment bodies this repo has posted to live issues.

Worth knowing, since it bears on the same event

The 2026-09-28 label-cycle @-mentioned the whole committee across all five
submissions. It produced zero scores. No /score or /vote comment exists
on any of #44, #51, #52, #53 or #55.

Notification was not the constraint. That matters for how much more intake
automation is worth building before the committee scores something.

Closes a gap I left in agenticsorg#58. An outside contributor was notified twice for one
submission because of it.

## What happened

On 2026-09-28, after agenticsorg#58/agenticsorg#59/agenticsorg#61 merged, all five open submissions were
label-cycled (`unlabeled` then `labeled`, three seconds apart) to backfill the
notifications they never received while the label race was live. That worked and
was the right call: agenticsorg#44 finally got its case brief after 49 days, and agenticsorg#52's
matrix row, lost to the concurrency bug in August, was regenerated.

But agenticsorg#51 and agenticsorg#53 each received a **second** "Added to the OIA Application Matrix"
comment on top of the one they already had from 2026-08-21. @RobLe3 was notified
twice for the same submission, on two separate issues.

Cause: agenticsorg#58 added idempotency guards to the welcome comment in
`on-submission.yml` and the case brief in `governance-agent.yml`, and
`scripts/oia-intake-step.mjs` was the one comment path that did not get one. Its
only guard was `OIA_SKIP_COMMENT`, which covers the persist-retry path added in
agenticsorg#61 and nothing else.

This is not a networking failure. It is a missing decision, and a missing
decision belongs in a test rather than in care.

## The fix

`alreadyPosted(comments, marker)` is a pure, exported predicate. Before posting,
the step reads the issue's comments and skips if its own marker is already
present. Two markers, exported as constants so the test pins the exact strings:
rewording a heading without updating its marker would silently re-enable
duplicates, and now that is a build failure.

**Fails closed.** If the comment list cannot be read, the step does not post and
says so in the log. A missed notification is recoverable by re-applying the
label, which is already a supported remediation. A duplicate sent to an outside
contributor is not recoverable.

The one-off failure notice ("could not auto-classify") is deliberately left
unconditional. If a repository URL is still wrong on a re-run, saying so again is
useful rather than noise.

## Also: the script is now import-safe

Its dispatch ran at module load, so importing it executed intake. That made the
guard untestable and, worse, meant an accidental import would act on a live
issue: post comments, mutate `docs/data/oia-matrix.json`, write badges. I
confirmed this by running the CLI during review, which classified a repository
and dirtied the worktree.

Wrapped in `main()` behind an entry-point check comparing `import.meta.url`
against `pathToFileURL(process.argv[1]).href`, matching
`scripts/classify-oia.mjs` and `scripts/mutate.mjs`. A test asserts the module
exposes only `alreadyPosted` and the two markers when imported.

## Tests

7 new, suite 171/171. Red-first: stubbing `alreadyPosted` to always return false,
which is the pre-fix behaviour, turns 3 of the 7 red.

Covered: the exact duplicate that reached agenticsorg#51 and agenticsorg#53; a first notification still
goes out; empty, absent and malformed comment lists; the two markers not
colliding with each other; and the markers genuinely appearing in the verbatim
comment bodies this repo has posted to live issues.

## Worth knowing, since it bears on the same event

The 2026-09-28 label-cycle @-mentioned the whole committee across all five
submissions. **It produced zero scores.** No `/score` or `/vote` comment exists
on any of agenticsorg#44, agenticsorg#51, agenticsorg#52, agenticsorg#53 or agenticsorg#55.

Notification was not the constraint. That matters for how much more intake
automation is worth building before the committee scores something.
Copilot AI balanced review requested due to automatic review settings October 1, 2026 13:17
@michaeloboyle
michaeloboyle merged commit b8656b2 into agenticsorg:main Oct 1, 2026
1 check passed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

It changes production CI behavior that notifies external contributors on live issues, and the network-dependent fail-closed comment paths are not exercised by the automated tests, so human verification is warranted.

Review effort: Balanced
Findings: 1 Low severity

Open (1)
What changed in this PR

This PR closes an at-most-once gap in the OIA intake notification path. After #58 added idempotency guards to the welcome comment and case brief, scripts/oia-intake-step.mjs remained the one comment path without such a guard, so a label-cycle backfill on 2026-09-28 sent duplicate "Added to the OIA Application Matrix" comments to an outside contributor on #51 and #53. The fix introduces a pure, exported alreadyPosted predicate plus a fail-closed comment lister, and makes the script import-safe by moving its dispatch behind an entry-point check so it can be unit-tested and won't act on a live issue when imported.

Changes:

  • Add exported PENDING_MARKER/PROMOTED_MARKER constants and a pure alreadyPosted(comments, marker) predicate; gate the pending and promotion comments on it, failing closed (not posting) when the comment list cannot be read.
  • Wrap the intake dispatch in main() behind an import.meta.url vs pathToFileURL(process.argv[1]).href entry-point check, matching scripts/classify-oia.mjs, so importing the module no longer runs intake.
  • Add test/oia-intake-idempotency.test.js with 7 tests covering the duplicate scenario, first-notification, empty/absent/malformed comment lists, marker non-collision, verbatim live bodies, and import-safety.
File Description
scripts/​oia-intake-step.mjs Adds markers, alreadyPosted, fail-closed listComments, marker-gated comment(), and an entry-point-guarded main().
test/​oia-intake-idempotency.test.js New unit tests for the idempotency predicate, marker/body alignment, and import-safety.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +68 to +70
const liveePending = '### Added to the OIA Application Matrix (pending review)\n\n`bar181/aisp-open-core` has been auto-classified into the **pending review** band.';
const livePromoted = '✅ **acme/thing** promoted to the curated OIA Application Matrix: https://agenticsorg.github.io/community-projects/oia-matrix.html#oia-acme-thing';
assert.ok(liveePending.includes(PENDING_MARKER), 'PENDING_MARKER must appear in the real pending comment');
@michaeloboyle

Copy link
Copy Markdown
Collaborator Author

@copilot Fix the code for all comments in this review thread.

When a review comment includes a suggested change, apply the suggestion exactly.

Do not make changes beyond what is described in the linked review thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants