Skip to content

feat(smoke): add gpd:smoke pre-commit sniff-test command - #253

Open
ayushchopra96 wants to merge 1 commit into
mainfrom
feat/smoke-command
Open

feat(smoke): add gpd:smoke pre-commit sniff-test command#253
ayushchopra96 wants to merge 1 commit into
mainfrom
feat/smoke-command

Conversation

@ayushchopra96

@ayushchopra96 ayushchopra96 commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds gpd:smoke, a pre-commit gate that reproduces the smallest viable version of a published quantitative claim, or
verifies an in-project assumption from PLAN.md, before committing to gpd:new-project, gpd:plan-phase, or
gpd:execute-phase.

Three invocation forms, no existing command needs modification:

  • gpd:smoke "<claim>" — anchor mode
  • gpd:smoke --assumption "<text>" — assumption mode
  • gpd:smoke --from-plan — read an assumption from PLAN.md (parser tolerant of both flat ## Assumptions and the
    hierarchical ## My Assumptions for Phase X + ### <category> shape gpd:plan-phase emits today)

Verdict is PASS / FAIL / INCONCLUSIVE; smoke refuses to fabricate, auto-route, or rewrite PLAN.md.

Worked example

See docs/smoke-tutorial.md — verifies the Lyapunov exponent of the logistic map at r=4 (butterfly effect, λ = ln(2)),
then scaffolds a real project to measure Feigenbaum's universal constant δ ≈ 4.6692.

Test plan

  • pytest tests/core/test_smoke_command.py — 3 tests, pass
  • pytest tests/core/test_command_*.py — all 90 existing command-surface tests still pass
  • Manual: invoke /gpd:smoke "..." in Claude Code after install, confirm verdict block emits
  • Manual: /gpd:smoke --from-plan against a real PLAN.md produced by gpd:plan-phase

Summary by CodeRabbit

Release Notes

  • New Features

    • Added gpd:smoke command as a pre-commit sniff-test for validating quantitative claims and assumptions with three operation modes (claim anchor, --assumption, --from-plan) producing PASS/FAIL/INCONCLUSIVE verdicts.
  • Documentation

    • Added comprehensive tutorial and changelog documentation for smoke workflow usage and behavior.
  • Tests

    • Added test coverage for the smoke command.

`gpd:smoke` is a pre-commit gate that reproduces the smallest viable version
of a published quantitative claim, or verifies an in-project assumption from
PLAN.md, before committing to gpd:new-project, gpd:plan-phase, or
gpd:execute-phase.

Three invocation forms:
- gpd:smoke "<claim>"               anchor mode (reproduce a published number)
- gpd:smoke --assumption "<text>"   assumption mode
- gpd:smoke --from-plan             read an assumption from GPD/phases/*/PLAN.md
                                    with a tolerant parser that accepts both
                                    flat `## Assumptions` and hierarchical
                                    `## My Assumptions for Phase X` + `### <category>`
                                    shapes (the latter is what gpd:plan-phase
                                    emits today).

No existing command requires modification for either mode to work.

Adds:
- src/gpd/commands/smoke.md          command surface (Validation and analysis, order 290)
- src/gpd/specs/workflows/smoke.md   workflow with 8 steps + a tolerant from-plan parser
- tests/core/test_smoke_command.py   3 tests: command frontmatter, workflow steps,
                                     from-plan parser fixture
- docs/smoke-tutorial.md             worked example: butterfly effect -> chaos project
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Ayush Chopra seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@coderabbitai

coderabbitai Bot commented Jun 4, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR introduces gpd:smoke, a new pre-commit workflow command that validates quantitative claims and assumptions at minimal scale through local scratch execution. It includes command metadata, a complete workflow specification with multi-step execution logic, surface tests confirming command structure, and a comprehensive tutorial for users.

Changes

Smoke Command Implementation

Layer / File(s) Summary
Command interface and input handling
src/gpd/commands/smoke.md, src/gpd/specs/workflows/smoke.md (lines 1–61)
Defines the gpd:smoke command metadata and purpose, then specifies argument routing logic for three input modes—<claim> anchor, --assumption "<text>", and --from-plan—including ordered plan-file search rules and tolerant assumption extraction that parses flat/hierarchical H2–H3 bullet structures while keeping PLAN.md read-only.
Workflow execution and verdict flow
src/gpd/specs/workflows/smoke.md (lines 63–170)
Specifies the complete workflow steps: gather_the_claim to capture structured quantitative anchors and reject vague claims, identify_minimal_setup with user confirmation gate, scaffold_and_run to create a local scratch Python script without GPD state or git changes, compare_and_verdict to map outcomes to PASS/FAIL/INCONCLUSIVE, gate_next_step to prevent automatic routing into project/plan/execute workflows, and guardrails constraining scope with explicit success-criteria checklist.
Command structure and parsing tests
tests/core/test_smoke_command.py
Tests verify that smoke.md exists and parses with correct frontmatter fields (name, context_mode, help metadata), workflow steps are present, and the --from-plan assumption-bullet extraction logic correctly identifies - bullets under ### subheadings, tagging each with its parent H3 category.
User tutorial and changelog
docs/smoke-tutorial.md, CHANGELOG.md
Complete tutorial with worked logistic-map "butterfly effect" example covering claim anchoring, minimal setup, smoke execution for anchor and assumption modes, and verdict interpretation; changelog documents the command modes, verdict behavior, and constraints.

🎯 2 (Simple) | ⏱️ ~12 minutes

🐰 A smoke test hops into view,

To catch the flaws before they're true,

With tiny scripts and claims so bright,

It verdicts quick: PASS, FAIL, or "not quite!" ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'feat(smoke): add gpd:smoke pre-commit sniff-test command' is clear, concise, and directly describes the main change—a new command feature.
Description check ✅ Passed The description covers the main change (adds gpd:smoke), motivation (pre-commit gate), implementation details (three invocation forms), and testing done. Checklist items are partially addressed but not in template format.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/smoke-command

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/core/test_smoke_command.py (1)

78-142: ⚡ Quick win

Consider adding test coverage for flat-form assumptions.

The test validates hierarchical assumptions (H2 + H3 + bullets), which matches the canonical gpd:plan-phase output. However, the workflow specification at lines 39-40 in src/gpd/specs/workflows/smoke.md explicitly requires the parser to tolerate flat form (H2 + bullets directly, without H3 subheadings). The current test helper _extract_assumption_bullets_with_categories only extracts bullets when current_category is not None (line 140), which means it would skip flat-form bullets.

🧪 Suggested additional test case
def test_smoke_from_plan_assumption_parser_extracts_flat_bullets() -> None:
    """The --from-plan parser must also tolerate flat form: H2 + bullets."""
    fixture = """# PLAN
    
## Assumptions
- The system is ergodic.
- Burn-in of 1000 steps suffices.

## Next section
Some content.
"""
    extracted = _extract_assumption_bullets_with_categories(fixture)
    # For flat form, use empty string as category or "(uncategorized)"
    assert len(extracted) == 2
    assert extracted[0][1] == "The system is ergodic."
    assert extracted[1][1] == "Burn-in of 1000 steps suffices."

And update the helper to handle flat form:

     for line in section.splitlines():
         h3 = h3_re.match(line)
         if h3:
             current_category = h3.group(1).strip()
             continue
         b = bullet_re.match(line)
-        if b and current_category is not None:
+        if b:
+            # Use empty string for flat-form bullets without a category
+            category = current_category if current_category is not None else ""
-            out.append((current_category, b.group(1).strip()))
+            out.append((category, b.group(1).strip()))
     return out
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/core/test_smoke_command.py` around lines 78 - 142, The helper
_extract_assumption_bullets_with_categories currently ignores flat-form bullets
because it only appends a bullet when current_category is not None; update it to
treat bullets seen before any H3 as unclassified (e.g., use "" or
"(uncategorized)") and append them regardless of current_category, and add a new
unit test test_smoke_from_plan_assumption_parser_extracts_flat_bullets that
supplies an H2 "Assumptions" section with bullets (no H3) and asserts both
bullets are returned with the chosen unclassified category; ensure the change
still preserves existing hierarchical behavior for H3-tagged bullets in the same
function.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/core/test_smoke_command.py`:
- Around line 78-142: The helper _extract_assumption_bullets_with_categories
currently ignores flat-form bullets because it only appends a bullet when
current_category is not None; update it to treat bullets seen before any H3 as
unclassified (e.g., use "" or "(uncategorized)") and append them regardless of
current_category, and add a new unit test
test_smoke_from_plan_assumption_parser_extracts_flat_bullets that supplies an H2
"Assumptions" section with bullets (no H3) and asserts both bullets are returned
with the chosen unclassified category; ensure the change still preserves
existing hierarchical behavior for H3-tagged bullets in the same function.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 323054b3-1e53-47a4-8a45-54c94046a935

📥 Commits

Reviewing files that changed from the base of the PR and between 0f41769 and b7f3ef1.

📒 Files selected for processing (5)
  • CHANGELOG.md
  • docs/smoke-tutorial.md
  • src/gpd/commands/smoke.md
  • src/gpd/specs/workflows/smoke.md
  • tests/core/test_smoke_command.py

@marcpickett1

Copy link
Copy Markdown
Collaborator

🤖 RoastBot: A sniff test. For a physics research tool. That is exactly where we are as a field.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants