Repository navigation
Improve the method from observed session friction and outcomes #8
Description
Outcome
After the first real application and Agent Ops trials, use their evidence to improve the method without turning every inconvenience into a new rule or platform feature. Revisit when those trials supply actual session evidence.
Acceptance
- Identify one concrete recurring friction or a meaningful success, including context, outcome and alternative explanation. Separate environment/tool limitations from method defects.
- Compare a targeted instruction change, a native configuration change and doing nothing. Preserve the project vision and both owner decision points.
- Prefer a small canonical change; prune conflicting/duplicated guidance. Do not copy entire external skill catalogs or accumulate mandatory lifecycle steps.
- Rehearse affected decisions in an independent context, include a realistic failure case and retain adverse results. Then observe a suitable real task before claiming improved reliability.
- Record review, authorized publication and actual installed adoption separately. No self-modifying runtime or automatic expansion of product scope.
Matt Pocock's retrospective work and the earlier Factory audit are research inputs. A source's instructions do not authorize its own installation or adoption. No-change is a valid result.
Prior context
#70. These discussions are retained as evidence; their old runtime/UI implementation scope is superseded.
Activity
gustavonline commentedon Oct 8, 2026
One concrete setup friction was observed in a real native application task: GitHub access worked, but the new execution account lacked Git author identity and could not commit. Configuring the approved account identity fixed that host. AgentOps independently compared stopping there with documenting the missing check and recommended the latter.
PR #13 adds eight lines to the existing GitHub setup reference: inspect attribution, use the approved identity at the selected scope, preserve overrides, and read back the first authorized commit. No new helper, runtime, required file or dummy commit. Independent Opus review accepted the patch; package checks passed; it is merged as
5fd0c26and the updated reference is installed in both selected native skill profiles.Evidence limits: the original failure and successful commit were observed; the missing-identity/override decision case was a review exercise. A future fresh installation has not yet exercised the new wording. This small result does not establish general unattended reliability or justify more method rules. The remaining evaluation outcomes stay open.
gustavonline commentedon Oct 9, 2026
The first bounded method improvement is delivered in PR15 and v0.1.3: a completed first run may still have background work that starts another turn. AgentOps now reconciles those continuations before cleanup, archive, restart or completion claims.
The signal appeared in both concurrent native writers. It was a harness behavior, not evidence that T3 needed a replacement runtime. No change and native configuration were considered; one short note was sufficient. A fresh four-case text rehearsal found a missing pointer to the completion signal, which was added and independently source-reviewed. The full final candidate then passed fresh Codex review and all seven package tests.
Publication and adoption are separate verified steps: main/tag/release match the reviewed content; both selected harness scopes on the host contain the release with backups. Claude explicitly loaded the changed references. A fresh native Codex context listed both skills in its catalog and explicitly read the installed files; hashes and new lines matched. This proves installation and loading, not improved reliability on arbitrary future work.
Keep this issue open for observing the final instruction on a suitable real task and the later application trial. No extra trial or application work was started merely to close it. The rehearsal limitations remain recorded.
gustavonline commentedon Oct 9, 2026
A second concrete method improvement is delivered in #17 / 0.1.4, followed by the owner-requested access-model explanation in #18 / v0.1.5. Repeated setup questions exposed unclear role/access boundaries and current guides mixed with historical migration records. Native copied CLI launchers also prevented update detection; repairing their native installation solved that host issue without a Factory updater.
The method now has one setup route, a clearer Foundation/AgentOps/ADLC division, small revision metadata and genuine harness attribution. Historical migration records remain at v0.1.3. Four fresh-context textual decision cases passed; independent final-candidate review accepted the method after a metadata-delimiter regression was fixed. Ten tests pass locally and in CI. The later documentation review accepted the three-level explanation; its unit-test rerun was unavailable under a read-only temporary-directory boundary, so CI/local results remain the test evidence.
Publication and adoption were read back separately: main/tag/release match the reviewed content; 44 installed skill files match the selected stage, and fresh Codex CLI discovery/readback passed. Claude/T3 reload remains pending at its quota reset. No result here establishes improved reliability on a new application or enables unattended OS updates. This issue stays open for the remaining real application observation.
gustavonline commentedon Oct 9, 2026
PR #19 / v0.1.6 præciserer den accepterede praksis: forbedr eller sammenlæg eksisterende skills ved konkret genbrugsværdi, behold applikationsviden lokalt, og brug eksisterende review/adoption. Ingen automatisk forbedringsissue, wiki eller ny skill pr. opgave.
To friske Astra High-kontekster besvarede de samme fire cases mod baseline og kandidat uden evaluatorens forventede svar. Begge var acceptable. Kandidaten var mere eksplicit om worker-options/readback; dette er ikke bevis for højere kvalitet eller lavere forbrug på rigtigt applikationsarbejde. En faktisk fejl i metadata-checket blev derimod reproduceret i Claude-sandboxen, rettet og verificeret med en regression, direkte native check og uafhængigt review.
Releasen er publiceret og adopteret på Z13 med verificerede filer og native indlæsning i begge harnesses. Dette issue forbliver åbent for den tilsvarende observation på en egnet virkelig applikationsopgave. Kastanje-produktarbejde er fortsat parkeret.
gustavonline commentedon Oct 9, 2026
A concrete environment/method mismatch produced one small, reviewed improvement: PR #20 / v0.1.7.
Observed friction: shell sandbox proof initially looked sufficient, but native Claude Edit/Write still reached outside the worktree. A project-local setting was also excluded by the actual launch. Doing nothing would retain a false qualification; a native read restriction alone left writes unchanged. The selected narrower mode held outside edits for individual decisions, at the measured cost of more ordinary read/check approvals.
The method change therefore requires per-tool qualification, loaded-setting evidence and explicit worker-mode readback. It adds no skill, runtime, automatic issue or approval bridge. Opus authored it; a fresh Astra context accepted the exact final candidate and answered three adverse decision cases. The package remains eight skills, seven selected for this installation; 11 tests and CI pass.
Publication and installation were verified separately. All 44 selected installed files match, with explicit reload in both native providers. This is bounded setup/method evidence. The issue stays open for observation on a suitable actual application task before claiming improved application reliability; Kastanje-product work remains parked.
gustavonline commentedon Oct 10, 2026
The first observed-improvement cycle is delivered and adopted through #24 and #25, now Factory v0.1.10.
Real application work exposed duplicated Foundation responsibilities and an unbound writer launch. The small canonical corrections were compared with native configuration/no-change, exercised in separate contexts, independently reviewed and corrected. The actual AIOS pilot, assisted recovery and negative results are retained separately from textual cases. All 48 selected installed files matched the release; the native lead explicitly reread all six changed instructions and verified their hashes.
This completes one documented improvement cycle, not a measured claim of universal reliability. The remaining owner-decision/recovery proof stays in #3, and Claude instruction/toolchain qualification in #7. No new runtime, mandatory skill catalog or automatic improvement issue was introduced.