ci: demo smoke test that runs make demo end to end by cbrgm · Pull Request #736 · opendefensecloud/solution-arsenal

cbrgm · 2026-08-06T09:29:11Z

Background

Part of #644, the context is one command to spin up SolAr locally with the demo app deployed -> make demo. Small PRs:

✅ make make dev-cluster reliably populate the catalog (fix(dev): make dev-cluster reliably populate the catalog #731)
✅ single source of truth for the OCM transfer step (refactor(dev): single source of truth for the OCM transfer step #733)
✅ make demo-app / make demo / make demo-clean (feat(dev): make demo / demo-app / demo-clean for a one-command demo #734)
(this PR) a CI smoke test so the demo flow cant silently rot

This is the last one, it closes out the story

What

A Demo Smoke Test workflow that runs make demo (dev-cluster + demo-app) end to end and asserts the demo application actually comes up.

Why

The demo shares its inputs with the e2e suite (same test/fixtures/e2e manifests), but the ordering and the make targets live outside the e2e Go test. Without a check they can drift and nobody notices until someone tries the demo. This job is the backstop: it walks the real transfer -> discover -> release -> render -> bootstrap pipeline and fails if the workload doesnt come up.

demo-app already waits for the bootstrap HelmRelease to become Ready and exits non-zero otherwise, so a green make demo is the assertion. The assert step then polls for the demo pod and waits for it to be Ready.

It builds SolAr from source rather than reusing the ghcr images, because dev-cluster.sh is wired for local images and adding a ghcr values + pull-secret path just for a smoke test wasnt worth it.

Gating

The demo is heavy (builds the images + full cluster), so it follows the same opt-in pattern as docker/helm/e2e:

on PRs -> only runs when the ok-to-demo label is set
on main -> runs automatically, but only when demo-relevant files change

Testing

actionlint clean on the workflow.
The full flow (make demo -> running workload) is verified locally end to end in feat(dev): make demo / demo-app / demo-clean for a one-command demo #734.
Self-validating: this PR carries ok-to-demo, so the smoke job runs on it (an earlier revision without the label gating already passed green in CI).

Checklist

Tests added/updated (this is the test, a CI smoke check)
No breaking changes (new workflow only)
Readable commit history (one focused commit)
AI code review considered and comments resolved

Part of #644

Summary by CodeRabbit

Tests
- Added automated demo validation for relevant changes.
- Demo checks verify that required demo pods reach a ready state.

Runs 'make demo' (dev-cluster + demo-app) and asserts the demo application comes up, as the backstop that keeps the demo flow from silently drifting from what the e2e suite covers. It builds SolAr from source and reuses the same test/fixtures/e2e manifests the demo applies, so it exercises the real render -> bootstrap pipeline. Gated to demo-relevant paths to keep CI time down.

coderabbitai · 2026-08-06T09:29:30Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 45c12550-e7f3-4128-930a-39f26f58ef79

📥 Commits

Reviewing files that changed from the base of the PR and between 9be4737 and 75de573.

📒 Files selected for processing (1)

.github/workflows/test-demo.yaml

🚧 Files skipped from review as they are similar to previous changes (1)

.github/workflows/test-demo.yaml

📝 Walkthrough

Walkthrough

The pull request adds a GitHub Actions workflow for demo smoke tests. The workflow uses Nix and Cachix, runs make demo, and checks pod readiness in the demo namespace.

Changes

Demo smoke-test validation

Layer / File(s)	Summary
Workflow execution and readiness checks `.github/workflows/test-demo.yaml`	The workflow runs on relevant `main` pushes and pull requests labeled `ok-to-demo`. It installs Nix, enables Cachix, runs `make demo`, and verifies Kubernetes pod readiness.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

opendefensecloud/solution-arsenal#734: Introduces the make demo target validated by this workflow.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description covers the change, motivation, testing, gating, and checklist, with sufficient detail for review.
Title check	✅ Passed	The title clearly and concisely describes the new CI smoke test that runs the demo flow end to end.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch ci/demo-smoke

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/test-demo.yaml:
- Around line 68-70: Update the workflow step before the existing kubectl wait
to wait until at least one demo pod has been created, using the existing
kind-solar-dev context and demo namespace. Then retain the kubectl wait for
Ready and subsequent pod listing, ensuring the readiness check cannot fail with
no matching resources.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 09bdbefe-4cfe-4c11-bb29-0081868d9b2b

📥 Commits

Reviewing files that changed from the base of the PR and between 875449e and 3bea4d5.

📒 Files selected for processing (1)

.github/workflows/test-demo.yaml

coderabbitai · 2026-08-07T07:15:29Z

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

cbrgm · 2026-08-07T07:55:27Z

In the daily, @olzemal mentioned that it would make sense to switch from file paths to an "ok-to-" label. I think this is a way better suited solution compared to providing filepaths. So now one can just add the label "ok-to-demo" and then see check the demo runs through without issues

The demo is heavy (it builds the images and stands up a full cluster), so on PRs it is opt-in via the 'ok-to-demo' label, matching how the docker/helm/e2e jobs are gated. On main it runs automatically, but only when demo-relevant files change, to keep CI time down

cbrgm · 2026-08-07T07:56:15Z

@coderabbitai review

coderabbitai · 2026-08-07T07:56:22Z

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

olzemal

lgtm

coderabbitai Bot reviewed Aug 6, 2026

View reviewed changes

Comment thread .github/workflows/test-demo.yaml

cbrgm force-pushed the ci/demo-smoke branch from 3bea4d5 to 9be4737 Compare August 7, 2026 07:15

cbrgm force-pushed the ci/demo-smoke branch from 9be4737 to 9557b8a Compare August 7, 2026 07:53

cbrgm added the ok-to-demo PR is allowed to run the demo smoke test label Aug 7, 2026

Merge branch 'main' into ci/demo-smoke

62d32df

cbrgm force-pushed the ci/demo-smoke branch from 9557b8a to 62d32df Compare August 7, 2026 07:58

olzemal approved these changes Aug 7, 2026

View reviewed changes

cbrgm enabled auto-merge August 7, 2026 10:29

Merge branch 'main' into ci/demo-smoke

75de573

cbrgm merged commit 9c5af75 into main Aug 10, 2026
21 checks passed

cbrgm deleted the ci/demo-smoke branch August 10, 2026 09:53

Navigation Menu

Repository navigation

ci: demo smoke test that runs make demo end to end - #736

ci: demo smoke test that runs make demo end to end#736
cbrgm merged 3 commits into
mainfrom
ci/demo-smoke

cbrgm commented Aug 6, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Aug 6, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

coderabbitai Bot commented Aug 7, 2026

Uh oh!

cbrgm commented Aug 7, 2026 •

edited

Loading

Uh oh!

cbrgm commented Aug 7, 2026

Uh oh!

coderabbitai Bot commented Aug 7, 2026 •

edited

Loading

Uh oh!

olzemal left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

cbrgm commented Aug 6, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

What

Why

Gating

Testing

Checklist

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coderabbitai Bot commented Aug 7, 2026

Uh oh!

cbrgm commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cbrgm commented Aug 7, 2026

Uh oh!

coderabbitai Bot commented Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

olzemal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cbrgm commented Aug 6, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Aug 6, 2026 •

edited

Loading

cbrgm commented Aug 7, 2026 •

edited

Loading

coderabbitai Bot commented Aug 7, 2026 •

edited

Loading