Repository navigation
Close verification gaps between reviewed source, delivered artifact and native acceptance #76
Description
Reviewed source and passing CI have still left the owner discovering functional failures in delivered plugins. Investigate where current Factory ADLC checks lose the connection between the requested outcome, reviewed source, shipped artifact, active host version and actual user workflow. AgentOps must not report product completion while required acceptance evidence is missing.
Observed evidence
- OPD: accepted source
290ead24ac668af7008896bb173471a46510cd78expected eight editor assets. The installed Mac production executable exposed only six and reported thateditor.jswas absent, despite a successful build and six affected tests. An ordinary owner restart proved that this was the active candidate. An isolated rebuild of the same source produced a complete package; complete production home/WASM/provenance proof is now awaiting exact replacement acceptance. No new product implementation was required. The precise build-cache cause is not established. Native end-to-end acceptance was correctly kept open, but the incomplete artifact should have been caught before the owner lifecycle checkpoint. Related: Open and reopen project canvases from the Codex sidebar openpaperdesign#6 and Polish the one-app Home and paper-rs workspace experience openpaperdesign#10. - AIOS: source review, CI and proxy rendering did not establish native sidebar contrast. Owner tests still found a dark icon in dark theme; an earlier candidate also missed the requested centred geometry. Geometry is now accepted by the owner; a new host-controlled currentColor candidate is undergoing qualified adoption. Native light/dark contrast remains a separate gate. Related: Refresh the AIOS Codex sidebar app icon onlinesourdough/OSM-plugin#56 and Let the host colour AIOS’s centred sidebar icon (#56) onlinesourdough/OSM-plugin#57.
- Bridge: installed RC10 now has owner-reported PASS for the requested pending-time queue visibility and queued-user history when its turn starts; no identical queue test is needed. New native archive feedback supersedes the morning missing-toast observation. Undo restored the chat in the observed flow but intermittently displayed a failure notice. All five captured unarchives completed; exact notice/payload correlation is missing. The same Opus5.5 High/Fast-off writer completed one final bounded diagnostic and found no evidenced patch. Bridge records Accept the native cutover before retiring the legacy Factory installation #9 as a nonblocking known residual for usable private RC10, not fixed or universally harmless; Keep AgentOps in the project and clarify execution-host connections #11 closes for the accepted flow and readme skal generelt skirves om og ikke så meta agtigt ift aios osv #10 remains separate. No custom Undo, upstream-cause claim, new source/review/build or repeated GUI test was introduced. Related: Undo can report failure after restoring a chat gustavonline/codex-claude-bridge#9, RC9: obtain timely browser readback after a queued open gustavonline/codex-claude-bridge#10, Queued messages are invisible while waiting gustavonline/codex-claude-bridge#11 and Add opt-in Claude startup and Home for the main Codex app gustavonline/codex-claude-bridge#16.
These are verification and delivery escapes, not evidence of a security/data leak or proof that every review failed. Plugin lifecycle and native UI limitations explain some incomplete observations, but do not excuse missing production-package checks. Private receipts remain with the existing leads; do not publish host paths, logs, credentials or owner screenshots.
Bounded investigation and correction
- Trace the actual handoffs through specification, implementation, tests/evals, independent review, packaging, installation and native acceptance. Identify which check could have caught each observed failure, its responsible actor and what evidence was missing. Distinguish product defects, test gaps, host limitations and inaccurate completion reporting.
- Define the smallest sufficient acceptance evidence for the owner's promised workflow and the exact delivered artifact. Source/CI, package completeness, active host binding and real end-to-end behaviour are separate claims.
NOT RUN, a blocked native step or an acknowledged handoff must not become a green result. - Make AgentOps reconcile actual child outcomes and outstanding acceptance conditions before reporting done. It should own supported verification work and arrange one concrete owner-only checkpoint only where native access genuinely requires the owner. A source-ready or installed-but-unaccepted result must be labelled accordingly. “All green” means all applicable agreed acceptance conditions have evidence for the delivered candidate, not just a green CI aggregate or universal absence of defects.
- Add the smallest useful regression/eval or review/delivery correction at the point of failure. For OPD, demonstrate that a production artifact missing the required editor payload would fail the check. Use realistic failure cases; do not add a generic checklist or duplicate existing tests without evidence.
- Independently review the correction and observe its effect on the ongoing delivery. Report what is now prevented, what remains unproven, and the next responsible action. No universal reliability claim from one successful run.
Ownership and scope
Existing Factory AgentOps owns the bounded investigation with the existing OPD, AIOS and Bridge leads. Keep current product delivery moving; no competing writers, new platform, custom runtime/approval service or broad backlog implementation is required. Prefer existing native tools, project checks and a small canonical method change only if the evidence warrants it.
This is a new current-method functional acceptance issue. It builds on the completed learning cycle in #8, complements the secret-safe release scope of #44, and must not wait for the future platform verification work in #68.
Activity
gustavonline commentedon Oct 17, 2026
The method correction is delivered in PR77 and Factory 0.1.18.
Two concrete escapes were confirmed:
The existing skills now require applicable proof from the exact production artifact before adoption; missing proof holds installation. Accepted amendments must reach both writer and reviewer, and AgentOps must reconcile actual child results before reporting completion. The retained six-asset output fails for missing
editor.jsandpanel.js; the complete replacement passes membership and receipt checks. The cache cause remains unproven. Independent review caught and closed the adoption-timing gap; CI and the bounded decision exercises passed. Both existing skill copies were verified, with Codex explicit loading checked; automatic invocation and fresh Claude loading are not claimed.Latest product readback:
Method delivery is complete. #76 stays open to reconcile OPD’s actual delivery result through its existing lead; it is not a new platform or parallel product task.
gustavonline commentedon Oct 17, 2026
Bridge's bounded archive readback is complete. The actual app/build and bundled CLI match the retained binding. Two archive operations and a separate-thread unarchive have explicit desktop-to-native trace correlation, successful finishes and semantic lifecycle records. This is protocol completion evidence, not visible-feedback acceptance.
The new owner test's exact thread/action was not supplied, and a same-build, same-action native feedback reference remains unavailable. Missing toast is still an observation, not an established Bridge defect. The earlier failed Undo remains separate and unresolved; no custom or forced Undo is authorized. RC10 queued-user history at turn start remains owner-reported PASS. Pending-time visibility and timely browser feedback remain unqualified.
The method boundary is precise: matching the installed version and completing correlated requests does not prove the owner's visible workflow or create a new UX requirement. The existing readback is fulfilled; no generic repeat is needed. Only an already available exact owner-action binding/native reference, or an evidenced contract divergence, can advance the remaining archive diagnosis. Bridge AgentOps retains delivery and any product correction with its existing writer/reviewer; Factory retains the canonical-method work.
Private evidence stays with Bridge. No product change, build/review, repeated GUI test or host mutation was performed for this reconciliation.
Merge pull request #77: Require production evidence before adoption