Skip to content

Harden release packaging and strengthen editing behavior - #2

Merged
altmn merged 2 commits into
masterfrom
codex/engineering-review-fixes
Oct 12, 2026
Merged

altmn merged 2 commits into
masterfrom
codex/engineering-review-fixes

Conversation

@altmn

@altmn altmn commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

The package validator could accept malformed metadata or images, and failed builds could replace a good archive. This PR makes ZIP publication atomic, validates the supported YAML string subset, rejects duplicate JSON keys and non-ASCII semantic versions, counts physical Unicode characters including CRLF, and validates WebP structure with a separate CI pixel-decode check. Synthetic regression tests cover these failures and staging recovery. Root plugin.json remains canonical and the runtime allowlist remains exactly six files.

The behavioral investigation confirmed the 42 recorded Luna failures: shortening exceeded the requested levels, Changes explanations omitted bold quotation formatting, and explicit fenced-payload editing lost or added source content. The skill now orders source separation, calculated shortening budgets/fact inventory, composition, a full edit/count audit, and safe-fence construction. It retains the exact pronoun rule, minimal edits/no-op response, literal formatting, source-instruction isolation, truthful complete Changes, and fresh-draft output. The skill is 6,990 Unicode characters, below the 8,000 limit. Package 1.0.1 remains an unreleased proposal.

All original 18 fixtures/83 assertions and the original Luna runner are byte-for-byte unchanged. Frozen snapshots bind source, manifest, fixture text and runner; captured prompts, responses and manual grades remain inspectable. No failed candidate or prior response was rewritten or discarded. The known draft-facts gap is an additional separately recorded regression check, not a replacement assertion or holdout.

Current source / simulation profile Passing assertions Failing assertions
gpt-6.1-sol, high, three runs 249 0
gpt-6-luna, high, three runs 218 31
gpt-6-luna, max, three runs 248 1

Independent Astra grading and a separate Sol review agree on the three final Sol fixture passes. All nine current records validate with current source binding and complete grading; all three final Sol records additionally pass --require-pass. Current supplementary draft-facts checks pass 9/9. CI now requires those three Sol passes and verifies all current records, while retaining the Luna failures.

The original Luna high failures are not fully resolved. A stronger-model old-source control already passed 83/83, so stronger-profile success cannot be attributed solely to the prompt changes. One maximum-effort Luna response still shortens 36% for the about-30% request. Single-request native Luna probes also failed light trimming, showing batching alone does not explain the observed issue. Sol run 3 has an additional unscored gap: redundant terminal periods after some quoted Changes spans. Passing the selected assertions does not certify every skill rule or every model. These results and all controls/failed candidates are documented in tests/RESULTS.md.

Validation: all 52 standard-library unit tests pass; release validate/build and git diff --check pass; all 19 frozen full-suite records pass complete provenance validation; supplementary and diagnostic hash bindings verified; six-file archive allowlist, CRC, source-byte equality and compatibility version verified. ZIP SHA-256: 6779f8b50f79728947ee3ed5f652d21cec5ffac709e51af9d5435663d48f2603. The unchanged icon was previously successfully decoded with libwebp 1.6.0.

Intended-host acceptance remains pending for this exact ZIP. The plugin is instruction-only and cannot enforce host model selection. No live plugin installation/update, release, listing publication, or merge is performed by this PR. Historical ChatGPT observations remain historical. Local verification is complete; hosted CI status must be checked separately.

@altmn altmn changed the title Fix release validation, archive recovery, and evaluation provenance Harden release packaging and strengthen editing behavior Oct 12, 2026
@altmn
altmn merged commit 9d699b7 into master Oct 12, 2026
1 check passed
@altmn
altmn deleted the codex/engineering-review-fixes branch October 12, 2026 23:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant