Skip to content

test(ci): cover the QA sealing gate in finalize_qa.py - #25

Merged
denson9874 merged 3 commits into
mainfrom
test/finalize-qa-seal-gate
Sep 30, 2026
Merged

denson9874 merged 3 commits into
mainfrom
test/finalize-qa-seal-gate

Conversation

@denson9874

Copy link
Copy Markdown
Owner

Summary

ci/finalize_qa.py is the last gate before a Jenkins QA build is sealed for publication, but it was the only release-path CI script without tests. This adds ci/tests/test_finalize_qa.py (test-only; no production code changes).

Covered:

  • ✅ Seals a passing build: seal.json digests match the sealed files, unit-test totals are summed across JUnit files, and no .preparing-* staging dir is left behind
  • 🚫 Refuses QA results that failed, are non-boolean ("true"), or whose SHA-256 doesn't match — including tampered APK bytes
  • 🚫 Refuses QA from a different source revision
  • 🚫 Refuses zero/failing/erroring/skipped/missing unit-test results
  • 🚫 Never overwrites an existing sealed release directory

Testing

  • python3 -m unittest discover -s ci/tests → 127 tests OK (122 existing + 5 new). Jenkins' Jenkinsfile.build already runs this discovery, so the new file is picked up automatically.
  • Mutation check: removing the skipped guard from finalize_qa.py makes the new tests fail.

🤖 Generated with Claude Code

denson9874 and others added 3 commits September 27, 2026 23:12
finalize_qa.py decides which APK bytes are sealed for publication but had
no tests. Cover the success path (seal digests, unit-test totals, no
leftover staging dir) and each refusal: failed or mismatched QA/APK
digests, a different source revision, missing/failing/erroring/skipped
unit tests, and an already-sealed release directory.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mismatched-bytes case tampered the APK, which also breaks the QA digest,
so the separate `sha != metadata['sha256']` check was never exercised alone.
Add a case where the APK and qa-result.json agree but metadata.json names
other bytes; removing that clause from finalize_qa.py now fails the test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@denson9874
denson9874 merged commit 8d9e073 into main Sep 30, 2026
8 checks passed
denson9874 added a commit that referenced this pull request Sep 30, 2026
…ize_qa

Adapted to the product finalize_qa.py (metadata fixture carries channel=qa);
passes against unmodified product code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
denson9874 pushed a commit that referenced this pull request Nov 6, 2026
Source revision a1e2796, the Full APK sha256 from Jenkins build #25, and
what was checked on the emulator and on a Pixel 7 Pro.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Csz3tZ1byCNq2yKBWWuS8X
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant