Skip to content

Add plugin-quality rule manifest (SSOT) + validator (M0) - #100

Open
canvas-investigator[bot] wants to merge 1 commit into
mainfrom
plugin-quality-m0-rule-manifest
Open

Add plugin-quality rule manifest (SSOT) + validator (M0)#100
canvas-investigator[bot] wants to merge 1 commit into
mainfrom
plugin-quality-m0-rule-manifest

Conversation

@canvas-investigator

@canvas-investigator canvas-investigator Bot commented Aug 21, 2026

Copy link
Copy Markdown

KOALA-6917

🧵 Aaryan Shah cc @aaryanxshah

M0 of the Studio/CPA plugin-quality program: the single-source-of-truth (SSOT) rule manifest, its validator, and a CI check. This milestone records the taxonomy and validates it — it does not change lint behavior or author skills (detector code-generation is M4; skills are M2).

What's here

  • canvas-plugin-assistant/quality/rules.yaml — SSOT, 12 rules. Schema: rule_id, title, axis, severity, stages[], delivery, always_on, detector{type,pattern}, teach_ref, fix, runtime_signal.
  • canvas-plugin-assistant/quality/validate_rules.py — manifest validator (importable + CLI).
  • tests/canvas-plugin-assistant/quality/test_validate_rules.py — 24 tripwire tests; 100% branch coverage of the validator.
  • .github/workflows/quality-rules.yml — runs the validator + tests on every PR/push.
  • README ### Skills fixed 8 → 11 (adds custom-data-patterns, fhir-api-client-development, instance-analyze); the validator now guards this list against future drift.
  • pyyaml added to the dev dependency group.

Rules (12)

block severity on the three patient-safety holes (SEC-XSS-001, SEC-AUTH-TRUE-001, SEC-PHI-LLM-001); the rest nudge. always_on: true on the three core one-liners (PERF-UNBOUNDED-001, SEC-AUTH-TRUE-001, SEC-PHI-LLM-001). Axes: performance ×5, security ×4, memory ×1, correctness ×1. teach_refs to the not-yet-authored llm-integration / memory skills are forward-refs (recorded now; skills land in M2).

Validator checks

Unique rule_ids; all required fields; enum values; every regex detector compiles; teach_ref resolves to an existing-or-forward-ref skill; skill→manifest cross-check (a rule_id cited in a SKILL.md must exist in the manifest); README ### Skills == skills/ on disk.

Decisions recorded

  • HOME: manifest sourced here next to the skills its teach_refs reference; a generated lint artifact is vendored into canvas-plugins in M4 (keeps canvas validate's hot path free of a YAML parse). Both repos are public, so rules.yaml is kept public-safe (generic OWASP-standard patterns, no customer detail).
  • DELIVERY: lint detectors are code-generated from the manifest (M4), not read at runtime.

Cross-check semantics

skill→manifest must resolve; manifest→skill existing-or-forward-ref; section anchors recorded but not validated in M0. To be tightened in M2 with a machine-readable rules: front-matter list + anchor validation.

Verification

uv run python canvas-plugin-assistant/quality/validate_rules.pyOK: 12 rules validated. 24 tests pass at 100% branch coverage. Red→green proven: injecting a duplicate rule_id (or a README skill mismatch) exits 1 with the specific error; the real manifest exits 0.

Follow-ups (not in this PR)

  • M2: author the llm-integration + memory skills; tighten the cross-check (machine-readable rules: front-matter + anchor validation).
  • M4: the ast block rules (SEC-AUTH-TRUE-001, SEC-PHI-LLM-001) can't run on a half-written file, so they belong at checkpoint/publish time (the fail-closed backstop), not as per-edit push nudges — design the generator accordingly.

Attribution: Aaryan + Claude.

Generated by the Investigator

@canvas-sisyphus

Copy link
Copy Markdown

Rebased plugin-quality-m0-rule-manifest onto main (resolved conflicts locally) and force-pushed, preserving the branch's individual commits. Force-pushed rebased plugin-quality-m0-rule-manifest to 2cd6dcd (1 commit, 8 file changes)

@canvas-sisyphus
canvas-sisyphus Bot force-pushed the plugin-quality-m0-rule-manifest branch from 97f8cc8 to 2cd6dcd Compare September 1, 2026 01:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant