Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,26 @@ This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.htm
`[tool.coverage.report] fail_under`, so local runs enforce it too.
- `codecov/codecov-action` pinned to the v7.1.1 commit SHA, and it
uses `CODECOV_TOKEN` when one is configured.
- The shimkit-specific release facts from `docs/shipping-checklist.md`
now live in `docs/release.md`: a release-readiness table (ruleset on
`main`, the `pypi` environment, the pending pypi.org trusted
publisher, TestPyPI, first upload), the `invalid-publisher` history
and the `gh workflow run release.yml --ref <tag>` republish recipe,
the CI gates, the `verify-attestation` job, the abandoned Homebrew
tap and the optional future items. The branch-protection item,
recorded as pending, is corrected to done: an active ruleset on
`main` requires a pull request and every CI check (checked
2026-10-05). Links in the README index and `docs/` point at
`docs/release.md` instead.
- `codecov.yml` added: carryforward flags, a blocking project status
with 1% slack, and an advisory patch status.

### Removed

- `docs/shipping-checklist.md`. Its generic steps are covered by the
org-wide Simtabi shipping checklist, and everything specific to
shimkit moved to `docs/release.md` (see Changed).

### Fixed

- A rejected Codecov upload was silent: with `fail_ci_if_error: false`
Expand Down
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,8 +202,7 @@ the full per-version release-notes index — see
| Config layer, schema, examples | [`docs/configuration.md`](docs/configuration.md) |
| Architecture, the load-bearing rules, how to add a new tool | [`docs/architecture.md`](docs/architecture.md) |
| Onboarding: setup, the 5 rules, recipe for adding a tool, debugging | [`docs/onboarding.md`](docs/onboarding.md) |
| Cutting a release, what each CI job does | [`docs/release.md`](docs/release.md) |
| Shipping checklist (what's done vs what's pending) | [`docs/shipping-checklist.md`](docs/shipping-checklist.md) |
| Cutting a release, what each CI job does, release readiness (what's done vs pending) | [`docs/release.md`](docs/release.md) |
| Validation scope: what's gated, what's deliberately out of scope | [`docs/validation-scope.md`](docs/validation-scope.md) |
| Known issues + pending items (un-automatable checks, deferrals) | [`docs/plans/known-issues.md`](docs/plans/known-issues.md) |
| Future additions (no-demand, naturally-extensible surface) | [`docs/plans/future-additions.md`](docs/plans/future-additions.md) |
Expand Down
6 changes: 3 additions & 3 deletions docs/onboarding.md
Original file line number Diff line number Diff line change
Expand Up @@ -479,8 +479,8 @@ adguard-integration ← real AGH on ubuntu-latest. JSON-asserted output.
adguard-mutating-integration ← real `shimkit adguard fix` inside a privileged systemd container.
```

All must pass before merge to `main` (once branch protection is
configured — see `docs/shipping-checklist.md` row 1.8).
All must pass before merge to `main`: the ruleset on `main` requires
them (see [release.md](release.md#release-readiness), item 2).

The release workflow only triggers on `v*` tags. Pushing to `main`
is a no-op for releases.
Expand All @@ -495,4 +495,4 @@ is a no-op for releases.
- For validation-scope questions ("is this thing tested?"), see
[validation-scope.md](validation-scope.md).
- For "what's left to ship?" questions, see
[shipping-checklist.md](shipping-checklist.md).
[release.md](release.md#release-readiness).
7 changes: 3 additions & 4 deletions docs/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,9 @@ Top-level utilities (not tools):
productive: setup, the 5 rules with grep recipes, the canonical
recipe for adding a new tool, common dev tasks, debugging guide.
- **[Release process](release.md)** — cutting a new version, the CI
pipeline, what each release job does.
- **[Shipping checklist](shipping-checklist.md)** — every step from
"code ready" to "users can install", in dependency order. Tracks
what's done vs what still needs your action.
pipeline, what each release job does, and the release-readiness
table: every step from "code ready" to "users can install", in
dependency order, with what's done vs what still needs your action.
- **[Validation scope](validation-scope.md)** — what's in scope for
automated + manual gates, what's deliberately out of scope, and
how to expand the envelope.
Expand Down
2 changes: 1 addition & 1 deletion docs/plans/known-issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ for a future release. Tracked here so contributors and operators can
see what's known and decide if it matters for their use case.

For the release-blocking checklist see
[`shipping-checklist.md`](../shipping-checklist.md). For the broader
[`release.md`](../release.md#release-readiness). For the broader
validation envelope see [`validation-scope.md`](../validation-scope.md).

---
Expand Down
10 changes: 6 additions & 4 deletions docs/plans/shipping-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,9 @@ Cross-references:
with concrete patterns but no current demand.
- [`known-issues.md`](known-issues.md) — runtime checks that
can't be containerised.
- [`../shipping-checklist.md`](../shipping-checklist.md) — the
release-readiness operational checklist (different from this
- [`../release.md`](../release.md#release-readiness) — the
release-readiness operational checklist (formerly
`docs/shipping-checklist.md`) (different from this
audit; that file tracks "ship readiness", this file tracks
"feature completeness against plans").
- [`../../.design/plans/feature-gap-analysis.md`](../../.design/plans/feature-gap-analysis.md)
Expand Down Expand Up @@ -95,15 +96,16 @@ Source: [`docs/plans/cleanup-2026-05-14.md`](cleanup-2026-05-14.md).

## E. Shipping-checklist phases

Source: [`../shipping-checklist.md`](../shipping-checklist.md).
Source: the former `docs/shipping-checklist.md`, now the release-readiness
table in [`../release.md`](../release.md#release-readiness).

| Phase | Item | Status |
|-------|------|--------|
| 1.1-1.7 | Code/repo setup | ✅ done |
| **1.8** | **Branch protection on `main`** | **⏳ pending — user action; see section G** |
| 2.1-2.3 | GitHub remote setup | ✅ done |
| **2.4** | **`pypi` GitHub Environment** | **⏳ pending — user action; see section G** |
| 3 | Homebrew tap | 🗑 abandoned — see [`../shipping-checklist.md`](../shipping-checklist.md) Phase 3 |
| 3 | Homebrew tap | 🗑 abandoned — see [`../release.md`](../release.md#homebrew-tap-abandoned) |
| **4.1** | **PyPI account + 2FA** | **⏳ pending — user action; see section G** |
| **4.2** | **Configure PyPI trusted publisher** | **⏳ pending — user action; see section G** |
| 4.3 | TestPyPI (dry-run) | ⏸ optional |
Expand Down
172 changes: 144 additions & 28 deletions docs/release.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Release process

Releases are **tag-driven**. Push a `vX.Y.Z` tag, GitHub Actions does
the rest.
Releases are tag-driven: push a `vX.Y.Z` tag and GitHub Actions builds, attests and publishes it.

> **Current channels (as of v0.11.0):** GitHub Release wheel + sdist
> + SBOM, AND PyPI (when the trusted-publisher config below is in
Expand All @@ -10,31 +9,102 @@ the rest.
> next tag after the trusted-publisher prerequisites are in place.
>
> The Homebrew tap path and the GHCR container were removed
> earlier — Homebrew tap is in `shipping-checklist.md` Phase 5
> if anyone wants to re-add it; the GHCR container was always
> meant as a testing artifact, not a distribution channel.
> earlier. The tap is recorded under
> [Homebrew tap (abandoned)](#homebrew-tap-abandoned) if anyone
> wants to re-add it; the GHCR container was removed in v0.2.2
> because it was always a testing artifact, not a documented
> install method.

For the one-time PyPI / npm / Docker setup (trusted publishers,
environments, tokens), this repo follows the standard
[PyPI Trusted Publishers](https://docs.pypi.org/trusted-publishers/)
flow. The recipe is below; ask the team if you need the org-wide
reference for cross-project conventions.
flow. The recipe is below. The generic, org-wide release gate (writing
review, security, CI hardening, supply chain, first-release steps) is
the Simtabi shipping checklist, which is kept outside this repository;
this page carries only what is specific to shimkit.

## Ship checklist
## Release readiness

Do these in dependency order:
What stands between "code is ready" and "users can `pip install
shimkit`", in dependency order. Each state below says when it was last
checked.

| # | Step | Where |
| # | Item | State |
|---|------|-------|
| 1 | Claim `shimkit` on PyPI + configure trusted publisher | <https://pypi.org/manage/account/publishing/> |
| 2 | Create `pypi` GitHub Environment | `simtabi/shimkit` Settings → Environments |
| 3 | Create `simtabi/homebrew-tap` (public repo, empty `Formula/` dir) | `gh repo create simtabi/homebrew-tap --public` |
| 4 | Create `TAP_GITHUB_TOKEN` secret (fine-grained PAT, Contents R/W on tap repo) | Settings → Secrets → Actions |
| 5 | Cut `v0.1.0` (or whatever's next) | `git tag vX.Y.Z && git push origin vX.Y.Z` |
| 1 | GitHub Release channel (wheel + sdist + SBOM) | Live. Every release from v0.5.0 onward is on the GitHub Release page. |
| 2 | Ruleset on `main` requiring a pull request and passing checks | Active (checked 2026-10-05 with `gh api repos/simtabi/shimkit/rulesets`): ruleset "default branch: pull request and passing checks", requiring every `test` matrix cell, `security`, `build`, both `smoke` cells and both `adguard` integration jobs. |
| 3 | `pypi` GitHub Environment | Exists (checked 2026-10-05 with `gh api repos/simtabi/shimkit/environments`). The OIDC token every `publish-pypi` run presents already carries `sub: repo:simtabi/shimkit:environment:pypi`. Optionally add a Required Reviewer rule for a human gate before upload. |
| 4 | PyPI account with 2FA | Pending, owner action: <https://pypi.org/account/register/>. |
| 5 | PyPI trusted publisher for `shimkit` | **Pending, owner action, and the one blocker left.** See [PyPI trusted-publisher setup](#pypi-trusted-publisher-setup). |
| 6 | TestPyPI rehearsal | Optional. Configure a `testpypi` trusted publisher at <https://test.pypi.org/manage/account/publishing/> and a `testpypi` GitHub environment to rehearse a release on TestPyPI before real PyPI. |
| 7 | First PyPI upload | Automatic: the next tagged release after item 5 publishes to PyPI. Then verify with `pip install shimkit` (no `--index-url`), which should resolve to the latest PyPI version. |

Step 3's `bump-homebrew-tap` workflow step `continue-on-error`s, so
step 5 will succeed even if step 3 isn't done yet — the formula just
won't be auto-bumped on that release.
Critical path for the next release to reach PyPI:

```
3 Create the pypi GitHub Environment (done)
└─► 4 PyPI account + 2FA
└─► 5 Configure trusted publisher
└─► next tag → auto-publishes
```

If only the GitHub Release wheel is acceptable, **none of these are
required**: that channel works today and has shipped every release
since v0.5.0.

### Why PyPI has never received an upload

Every `publish-pypi` run from v0.11.0 through v0.19.0 (nine releases)
failed the token exchange with
`invalid-publisher: valid token, but no corresponding publisher`, and `https://pypi.org/pypi/shimkit/json`
answers 404 (checked 2026-10-05). The GitHub side is correct; pypi.org
has no publisher registered.

### Publishing past tags after the publisher exists

Failed uploads on past tags can **not** be fixed with "Re-run failed
jobs": `publish-pypi` downloads the `dist` artifact from its run's
`build` job, and those artifacts expire after 90 days (v0.11.0 to
v0.19.0 ran 2026-05-15/16). Dispatch the whole workflow on the tag
instead, which rebuilds and uploads:

```bash
gh workflow run release.yml --repo simtabi/shimkit --ref v0.19.0
```

To publish older releases retroactively, dispatch `release.yml` the
same way on each tag (`--ref vX.Y.Z`). `publish-pypi` sets
`skip-existing: true`, so a version already on PyPI is skipped rather
than failing the run.

## Gates a release depends on

CI (`ci.yml`) runs on macOS + Ubuntu × Python 3.10/3.11/3.12/3.13 with
the jobs `test`, `security`, `build`, `smoke`, `adguard-integration`
and `adguard-mutating-integration`. The `test` job carries ruff
(strict), mypy (strict, pydantic plugin), shellcheck, bandit (`-ll`,
fail on medium and above) and pip-audit (`--skip-editable`); at v0.12.0
the suite was 1027 pytest cases at 85% coverage. Coverage uploads to
codecov.io (wired in v0.12.0, with the coverage badge in the README).
Dependabot opens weekly PRs for pip and GitHub Actions updates.

Repository setup that the release path assumes is in place: the
`src/` layout with the hatchling backend, `py.typed` and the bundled
`defaults.json` (the wheel builds cleanly with `python -m build`); the
org-style docs (`README.md`,
`docs/{installation,configuration,architecture,release,tools/*}.md`
with a `docs/tools/<name>.md` page for every shipped tool,
`CHANGELOG.md`, `CONTRIBUTING.md`, `SECURITY.md`,
`CODE_OF_CONDUCT.md`); issue and PR templates, Dependabot and
pre-commit; and the public `simtabi/shimkit` repository with `main` as
default branch, description, homepage, topics, Issues and Discussions.
Commits use the noreply identity `imanimanyara@users.noreply.github.com`;
the ID-prefixed form `19682005+imanimanyara@users.noreply.github.com`
recorded in the old checklist is no longer what the checkout is
configured with.

What each gate validates, and what is deliberately left out, is in
[`validation-scope.md`](validation-scope.md).

## Cutting a release

Expand All @@ -45,10 +115,17 @@ won't be auto-bumped on that release.

# 2. Update CHANGELOG.md with the new entry (move [Unreleased] to [X.Y.Z]).

# 3. Commit and tag:
# 3. Commit on a branch and open a pull request; main only changes through a PR
# (its ruleset requires one, with passing checks):
git switch -c release/vX.Y.Z
git commit -am "release: vX.Y.Z"
git tag vX.Y.Z
git push origin main vX.Y.Z
git push -u origin release/vX.Y.Z
gh pr create --title "Release vX.Y.Z" --body "..."

# 4. After the PR merges, tag the merge commit and push only the tag:
git switch main && git pull --ff-only
git tag -a vX.Y.Z -m "Release vX.Y.Z"
git push origin vX.Y.Z
```

## What `release.yml` does
Expand Down Expand Up @@ -76,12 +153,19 @@ push tag v*
│ (skip-existing: true for reruns)
│
└──► github-release ────────── create the GH Release with
wheel + sdist + SBOM attached.
│ wheel + sdist + SBOM attached.
│
▼
verify-attestation ────── download the released wheel + sdist
and run `gh attestation verify`
on each (wired in v0.12.0).
```

`publish-pypi` and `github-release` run in parallel after `build`
— a PyPI outage doesn't block the GitHub Release upload, and vice
versa.
versa. `verify-attestation` reads the provenance back from Sigstore's
transparency log after the release exists, so a misconfigured publish
flow fails visibly instead of shipping unattested artifacts.

### PyPI trusted-publisher setup

Expand All @@ -96,7 +180,7 @@ Required once per project, by a maintainer with PyPI write access:
| PyPI Project Name | `shimkit` |
| Owner | `simtabi` |
| Repository name | `shimkit` |
| Workflow filename | `release.yml` |
| Workflow name | `release.yml` |
| Environment name | `pypi` |

2. **GitHub side** — Settings → Environments → New environment
Expand All @@ -110,8 +194,11 @@ Required once per project, by a maintainer with PyPI write access:

If either side is missing, the workflow fails at the `publish-pypi`
step with `invalid-publisher`. The `github-release` job still
succeeds, so you can fix the PyPI side at your leisure and re-run
the workflow against the same tag via `workflow_dispatch`.
succeeds, so you can fix the PyPI side at your leisure and then
dispatch the workflow against the same tag (`workflow_dispatch`, see
[Publishing past tags](#publishing-past-tags-after-the-publisher-exists));
"Re-run failed jobs" only works while the run's `dist` artifact has
not expired.

## Verifying a release

Expand Down Expand Up @@ -140,10 +227,39 @@ jq .name /tmp/sbom.json # should print "shimkit"
Once the first release has gone through, every subsequent one is just:

```bash
# Bump versions, commit, tag, push:
git commit -am "release: vX.Y.Z" && git tag vX.Y.Z && git push origin main vX.Y.Z
# Bump versions and the CHANGELOG on a release branch, merge it through a PR,
# then tag the merge commit and push only the tag:
git switch main && git pull --ff-only && git tag -a vX.Y.Z -m "Release vX.Y.Z" && git push origin vX.Y.Z
```

CI does everything else. If the `pypi` environment has Required
Reviewers configured, you'll get a GitHub notification asking you to
approve before publish.

## Homebrew tap (abandoned)

Distribution through a Homebrew tap was scoped in the original release
plan but never restored after PyPI was deferred, and `release.yml`
has no tap job today. The live channels are PyPI and the GitHub
Release page.

If a tap is wanted in future:

1. Create `simtabi/homebrew-tap` (public repo, empty `Formula/` dir):
`gh repo create simtabi/homebrew-tap --public`.
2. Create a `TAP_GITHUB_TOKEN` secret: a fine-grained PAT scoped to
that repo with Contents read/write (Settings → Secrets → Actions).
3. Add a `bump-homebrew-tap` job to `release.yml` after
`github-release`, marked `continue-on-error` so a tap failure
never fails the release; the formula is simply not bumped on that
release.

## Optional and future

- Move the docs to mkdocs-material on GitHub Pages if SEO and
discoverability come to matter more than plain GitHub rendering.
- Add a `CITATION.cff` if the tool gets cited academically.

---

[← Docs index](../README.md#documentation)
Loading
Loading