Skip to content

Commit bbf489a

Browse files
committed
feat(quality-scan): add quality-scan skill and zizmor security scanner
- Added quality-scan skill with tailored architecture description - Added zizmor as pinned devDependency (1.16.3) - Removed .claude from .gitignore to track skills - Security scan runs zizmor on GitHub Actions workflows - Detects template injection, cache poisoning, and other workflow vulnerabilities
1 parent b408ecc commit bbf489a

3 files changed

Lines changed: 1179 additions & 1 deletion

File tree

Lines changed: 375 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,375 @@
1+
---
2+
name: quality-scan
3+
description: Performs comprehensive quality scans across codebase to identify critical bugs, logic errors, caching issues, and workflow problems. Spawns specialized agents for targeted analysis and generates prioritized improvement tasks. Use when improving code quality, before releases, or investigating issues.
4+
---
5+
6+
# quality-scan
7+
8+
<task>
9+
Your task is to perform comprehensive quality scans across the socket-lib codebase using specialized agents to identify critical bugs, logic errors, caching issues, and workflow problems. Generate a prioritized report with actionable improvement tasks.
10+
</task>
11+
12+
<context>
13+
**What is Quality Scanning?**
14+
Quality scanning uses specialized AI agents to systematically analyze code for different categories of issues. Each agent type focuses on specific problem domains and reports findings with severity levels and actionable fixes.
15+
16+
**socket-lib Architecture:**
17+
Core utilities and infrastructure library for Socket.dev security tools that:
18+
- Provides shared utilities used across Socket.dev projects
19+
- Contains logger, fs helpers, process utilities
20+
- Manages build infrastructure and tooling
21+
- Provides testing utilities and helpers
22+
- Supports cross-platform compatibility
23+
24+
**Scan Types Available:**
25+
1. **critical** - Crashes, security vulnerabilities, resource leaks, data corruption
26+
2. **logic** - Algorithm errors, edge cases, type guards, off-by-one errors
27+
3. **cache** - Cache staleness, race conditions, invalidation bugs
28+
4. **workflow** - Build scripts, CI issues, cross-platform compatibility
29+
5. **security** - GitHub Actions workflow security (zizmor scanner)
30+
31+
**Why Quality Scanning Matters:**
32+
- Catches bugs before they reach production
33+
- Identifies security vulnerabilities early
34+
- Improves code quality systematically
35+
- Provides actionable fixes with file:line references
36+
- Prioritizes issues by severity for efficient remediation
37+
38+
**Agent Prompts:**
39+
All agent prompts are embedded in `reference.md` with structured <context>, <instructions>, <pattern>, and <output_format> tags following Claude best practices.
40+
</context>
41+
42+
<constraints>
43+
**CRITICAL Requirements:**
44+
- Read-only analysis (no code changes during scan)
45+
- Must complete all enabled scans before reporting
46+
- Findings must be prioritized by severity (Critical → High → Medium → Low)
47+
- Must generate actionable tasks with file:line references
48+
- All findings must include suggested fixes
49+
50+
**Do NOT:**
51+
- Fix issues during scan (analysis only - report findings)
52+
- Skip critical scan types without user permission
53+
- Report findings without file/line references
54+
- Proceed if codebase has uncommitted changes (warn but continue)
55+
56+
**Do ONLY:**
57+
- Run enabled scan types in priority order (critical → logic → cache → workflow → security)
58+
- Generate structured findings with severity levels
59+
- Provide actionable improvement tasks with specific code changes
60+
- Report statistics and coverage metrics
61+
- Deduplicate findings across scans
62+
</constraints>
63+
64+
<instructions>
65+
66+
## Process
67+
68+
Execute the following phases sequentially to perform comprehensive quality analysis.
69+
70+
### Phase 1: Validate Environment
71+
72+
<prerequisites>
73+
Verify the environment before starting scans:
74+
</prerequisites>
75+
76+
```bash
77+
git status
78+
```
79+
80+
<validation>
81+
**Expected State:**
82+
- Working directory should be clean (warn if dirty but continue)
83+
- On a valid branch
84+
- Node modules installed
85+
86+
**If working directory dirty:**
87+
- Warn user: "Working directory has uncommitted changes - continuing with scan"
88+
- Continue with scans (quality scanning is read-only)
89+
90+
</validation>
91+
92+
---
93+
94+
### Phase 2: Determine Scan Scope
95+
96+
<action>
97+
Ask user which scans to run:
98+
</action>
99+
100+
**Default Scan Types** (run all unless user specifies):
101+
1. **critical** - Critical bugs (crashes, security, resource leaks)
102+
2. **logic** - Logic errors (algorithms, edge cases, type guards)
103+
3. **cache** - Caching issues (staleness, races, invalidation)
104+
4. **workflow** - Workflow problems (scripts, CI, git hooks)
105+
5. **security** - GitHub Actions security (template injection, cache poisoning, etc.)
106+
107+
**User Interaction:**
108+
Use AskUserQuestion tool:
109+
- Question: "Which quality scans would you like to run?"
110+
- Header: "Scan Types"
111+
- multiSelect: true
112+
- Options:
113+
- "All scans (recommended)" → Run all 5 scan types
114+
- "Critical only" → Run critical scan only
115+
- "Critical + Logic" → Run critical and logic scans
116+
- "Custom selection" → Ask user to specify which scans
117+
118+
**Default:** If user doesn't specify, run all scans.
119+
120+
<validation>
121+
Validate selected scan types exist in reference.md:
122+
- critical-scan → reference.md line ~5
123+
- logic-scan → reference.md line ~100
124+
- cache-scan → reference.md line ~200
125+
- workflow-scan → reference.md line ~300
126+
- security-scan → reference.md line ~400
127+
128+
If user requests non-existent scan type, report error and suggest valid types.
129+
</validation>
130+
131+
---
132+
133+
### Phase 3: Execute Scans
134+
135+
<action>
136+
For each enabled scan type, spawn a specialized agent using Task tool:
137+
</action>
138+
139+
```typescript
140+
// Example: Critical scan
141+
Task({
142+
subagent_type: "general-purpose",
143+
description: "Critical bugs scan",
144+
prompt: `${CRITICAL_SCAN_PROMPT_FROM_REFERENCE_MD}
145+
146+
Focus on src/, packages/.
147+
148+
Report findings in this format:
149+
- File: path/to/file.mts:lineNumber
150+
- Issue: Brief description
151+
- Severity: Critical/High/Medium/Low
152+
- Pattern: Code snippet
153+
- Trigger: What input triggers this
154+
- Fix: Suggested fix
155+
- Impact: What happens if triggered
156+
157+
Scan systematically and report all findings. If no issues found, state that explicitly.`
158+
})
159+
```
160+
161+
**For each scan:**
162+
1. Load agent prompt template from `reference.md`
163+
2. Customize for socket-lib context (focus on src/, packages/)
164+
3. Spawn agent with Task tool using "general-purpose" subagent_type
165+
4. Capture findings from agent response
166+
5. Parse and categorize results
167+
168+
**Execution Order:** Run scans sequentially in priority order:
169+
- critical (highest priority)
170+
- logic
171+
- cache
172+
- workflow
173+
- security (lowest priority)
174+
175+
**Agent Prompt Sources:**
176+
- Critical scan: reference.md starting at line ~12
177+
- Logic scan: reference.md starting at line ~100
178+
- Cache scan: reference.md starting at line ~200
179+
- Workflow scan: reference.md starting at line ~300
180+
- Security scan: reference.md starting at line ~400
181+
182+
<validation>
183+
For each scan completion:
184+
- Verify agent completed without errors
185+
- Extract findings from agent output
186+
- Parse into structured format (file, issue, severity, fix)
187+
- Track scan coverage (files analyzed)
188+
</validation>
189+
190+
---
191+
192+
### Phase 4: Aggregate Findings
193+
194+
<action>
195+
Collect all findings from agents and aggregate:
196+
</action>
197+
198+
```typescript
199+
interface Finding {
200+
file: string // "src/api/client.mts:89"
201+
issue: string // "Potential null pointer access"
202+
severity: "Critical" | "High" | "Medium" | "Low"
203+
scanType: string // "critical"
204+
pattern: string // Code snippet showing the issue
205+
trigger: string // What causes this issue
206+
fix: string // Suggested code change
207+
impact: string // What happens if triggered
208+
}
209+
```
210+
211+
**Deduplication:**
212+
- Remove duplicate findings across scans (same file:line, same issue)
213+
- Keep the finding from the highest priority scan
214+
- Track which scans found the same issue
215+
216+
**Prioritization:**
217+
- Sort by severity: Critical → High → Medium → Low
218+
- Within same severity, sort by scanType priority
219+
- Within same severity+scanType, sort alphabetically by file path
220+
221+
<validation>
222+
**Checkpoint:** Verify aggregation:
223+
- Total findings count
224+
- Breakdown by severity (N critical, N high, N medium, N low)
225+
- Breakdown by scan type
226+
- Duplicate removal count (if any)
227+
</validation>
228+
229+
---
230+
231+
### Phase 5: Generate Report
232+
233+
<action>
234+
Create structured quality report with all findings:
235+
</action>
236+
237+
```markdown
238+
# Quality Scan Report
239+
240+
**Date:** YYYY-MM-DD
241+
**Repository:** socket-lib
242+
**Scans:** [list of scan types run]
243+
**Files Scanned:** N
244+
**Findings:** N critical, N high, N medium, N low
245+
246+
## Critical Issues (Priority 1) - N found
247+
248+
### src/api/client.mts:89
249+
- **Issue**: Potential null pointer access when calling API
250+
- **Pattern**: `const result = response.data.items[0].value`
251+
- **Trigger**: When API returns empty items array
252+
- **Fix**: `const items = response.data?.items ?? []; const value = items[0]?.value ?? null`
253+
- **Impact**: Crashes API client, breaks user workflows
254+
- **Scan**: critical
255+
256+
## High Issues (Priority 2) - N found
257+
258+
[Similar format for high severity issues]
259+
260+
## Medium Issues (Priority 3) - N found
261+
262+
[Similar format for medium severity issues]
263+
264+
## Low Issues (Priority 4) - N found
265+
266+
[Similar format for low severity issues]
267+
268+
## Scan Coverage
269+
270+
- **Critical scan**: N files analyzed in src/, packages/
271+
- **Logic scan**: N files analyzed
272+
- **Cache scan**: N files analyzed (if applicable)
273+
- **Workflow scan**: N files analyzed (package.json, scripts/, .github/)
274+
- **Security scan**: N workflow files analyzed (.github/workflows/)
275+
276+
## Recommendations
277+
278+
1. Address N critical issues immediately before next release
279+
2. Review N high-severity logic errors
280+
3. Schedule N medium issues for next sprint
281+
4. Low-priority items can be addressed during refactoring
282+
283+
## No Findings
284+
285+
[If a scan found no issues, list it here:]
286+
- Critical scan: ✓ Clean
287+
- Logic scan: ✓ Clean
288+
- Security scan: ✓ Clean
289+
```
290+
291+
**Output Report:**
292+
1. Display report to console (user sees it)
293+
2. Offer to save to file (optional): `reports/quality-scan-YYYY-MM-DD.md`
294+
295+
<validation>
296+
**Report Quality Checks:**
297+
- All findings include file:line references
298+
- All findings include suggested fixes
299+
- Findings are grouped by severity
300+
- Scan coverage statistics included
301+
- Recommendations are actionable
302+
</validation>
303+
304+
---
305+
306+
### Phase 6: Complete
307+
308+
<completion_signal>
309+
```xml
310+
<promise>QUALITY_SCAN_COMPLETE</promise>
311+
```
312+
</completion_signal>
313+
314+
<summary>
315+
Report these final metrics to the user:
316+
317+
**Quality Scan Complete**
318+
========================
319+
✓ Scans completed: [list of scan types]
320+
✓ Total findings: N (N critical, N high, N medium, N low)
321+
✓ Files scanned: N
322+
✓ Report generated: Yes
323+
✓ Scan duration: [calculated from start to end]
324+
325+
**Critical Issues Requiring Immediate Attention:**
326+
- N critical issues found
327+
- Review report above for details and fixes
328+
329+
**Next Steps:**
330+
1. Address critical issues immediately
331+
2. Review high-severity findings
332+
3. Schedule medium/low issues appropriately
333+
4. Re-run scans after fixes to verify
334+
335+
All findings include file:line references and suggested fixes.
336+
</summary>
337+
338+
</instructions>
339+
340+
## Success Criteria
341+
342+
-`<promise>QUALITY_SCAN_COMPLETE</promise>` output
343+
- ✅ All enabled scans completed without errors
344+
- ✅ Findings prioritized by severity (Critical → Low)
345+
- ✅ All findings include file:line references
346+
- ✅ Actionable suggestions provided for all findings
347+
- ✅ Report generated with statistics and coverage metrics
348+
- ✅ Duplicate findings removed
349+
350+
## Scan Types
351+
352+
See `reference.md` for detailed agent prompts with structured tags:
353+
354+
- **critical-scan** - Null access, promise rejections, race conditions, resource leaks
355+
- **logic-scan** - Off-by-one errors, type guards, edge cases, algorithm correctness
356+
- **cache-scan** - Invalidation, key generation, memory management, concurrency
357+
- **workflow-scan** - Scripts, package.json, git hooks, CI configuration
358+
- **security-scan** - GitHub Actions workflow security (runs zizmor scanner)
359+
360+
All agent prompts follow Claude best practices with <context>, <instructions>, <pattern>, <output_format>, and <quality_guidelines> tags.
361+
362+
## Commands
363+
364+
This skill is self-contained. No external commands needed.
365+
366+
## Context
367+
368+
This skill provides systematic code quality analysis for socket-lib by:
369+
- Spawning specialized agents for targeted analysis
370+
- Using Task tool to run agents autonomously
371+
- Embedding agent prompts in reference.md following best practices
372+
- Generating prioritized, actionable reports
373+
- Supporting partial scans (user can select specific scan types)
374+
375+
For detailed agent prompts with best practices structure, see `reference.md`.

0 commit comments

Comments
 (0)