Skip to content

test: real-GitHub validation across trunk models #27

Closed

Description

@joshua-temple

Goal

Validate orchestrate + promote on real GitHub (not just the local act-based harness) across every trunk model the generator supports. The local e2e suite proves generator output and runs workflows under act; this issue proves the framework end-to-end against the real GitHub Actions runtime, environment gates, and API surface.

Approach

Stand up example test repos under the stablekernel org, one per trunk model:

  • no-env - environments: [], single trunk, release-only lifecycle
  • 2-env - minimal promotion chain
  • 3-env - intermediate cascade
  • 4-env cascade - full dev -> test -> uat -> prod promotion cascade

Coverage

Validate the real-GitHub behavior below, in the trunk models where each path is meaningful. Not every path applies to every model (a no-env repo has no promotion or rollback), so coverage is allocated rather than run blanket in every repo:

  • Orchestrate on merge to trunk (all models): change detection, build/deploy callback dispatch, state write-back
  • Dry-run codepaths (all models): emulated outputs without skipping
  • Failure handling and idempotency (all models): re-running a dispatch is safe
  • Release lifecycle (no-env, plus the terminal stage of the multi-env models): tag + changelog + GitHub release
  • Promotion between environments (2-env, 3-env, 4-env): SHA-pinned promotion, breaking-change gate, state propagation
  • Hotfix / re-promote flows (one multi-env model, e.g. the 4-env cascade)
  • Rollback flows (validate in the 2-env model for the core paths, and the 4-env cascade for the full divergence/rejoin behavior across a chain; not in every repo):
    • Auto-rollback on a failed deploy (the env reverts to its prior SHA), for both promote and hotfix
    • Manual arbitrary-depth rollback via the generated cascade-rollback workflow, re-deploying a prior version resolved from the per-environment history ring (git-history fallback / explicit --to)
    • The downgrade gate (promoting an older version onto an env is blocked unless allow_downgrade is set; the prod/terminal env always requires it)
    • Divergence + rejoin (a rolled-back env is marked diverged so an onward promote of the downgrade is guarded, and a roll-forward rejoins and clears it)

Scope notes

  • These repos double as living examples for adopters; keep their manifests clean and minimal.
  • Each repo wires the published composite actions and pins a cli_version.
  • The local act+gitea harness already covers rollback (five scenarios: re-deploy prior, deployable-scoped, marks-diverged-blocks-promote, failed-deploy-leaves-state, roll-forward rejoins). The real-GitHub validation re-proves the allocated subset above against the live runtime.
  • Capture any divergence from the local act harness as follow-up issues.

Impact

This is the credibility gate: it proves the framework works on real GitHub for every supported topology, not just under local emulation.

Activity

  1. added this to the v1.0.0 milestone on Jun 10, 2026
  2. joshua-temple commented on Jun 18, 2026

    @joshua-temple
    CollaboratorAuthor

    Real-GitHub validation: closing status

    The v0.2.1-rc.2 Fleet E2E run (27771438666, 8/8 example suites green) drove repin -> 8 green suites -> Auto-promote (27772440057) publishing v0.2.1. A per-suite log review (not just the aggregate result) confirmed these acceptance flows ran green on real GitHub across the trunk models:

    • Orchestrate on merge to trunk (all suites)
    • Dry-run codepaths (4env)
    • Failure handling and idempotency (3env abort path; 2env re-dispatch)
    • Release lifecycle (release-only; single-env)
    • Promotion between environments (2env, 3env, 4env chains; breaking-change gate in 4env)
    • Hotfix and re-promote (3env)
    • Manual arbitrary-depth rollback (4env)
    • Downgrade gate (4env)
    • Mark-diverged on rollback (4env)

    Three rollback assertions were proven on the local act + gitea harness but were not yet exercised by the fleet suites: auto-rollback leaving env state unchanged on a failed re-deploy, divergence rejoin clearing the ref and base SHA, and core rollback under the 2-env model. Coverage for all three is now merged into the example suites, each mirroring its proven local scenario and independently reviewed:

    The example suites run their current main on every fleet run, so these steps execute on the next release-candidate fleet run against the published rc.

    Closing: cascade is validated across all trunk models on real GitHub, and the rollback-coverage completeness is merged and runs on the next release-candidate fleet run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions