|
| 1 | +--- |
| 2 | +name: quality-scan |
| 3 | +description: Performs comprehensive quality scans across codebase to identify critical bugs, logic errors, caching issues, and workflow problems. Spawns specialized agents for targeted analysis and generates prioritized improvement tasks. Use when improving code quality, before releases, or investigating issues. |
| 4 | +--- |
| 5 | + |
| 6 | +# quality-scan |
| 7 | + |
| 8 | +<task> |
| 9 | +Your task is to perform comprehensive quality scans across the socket-lib codebase using specialized agents to identify critical bugs, logic errors, caching issues, and workflow problems. Generate a prioritized report with actionable improvement tasks. |
| 10 | +</task> |
| 11 | + |
| 12 | +<context> |
| 13 | +**What is Quality Scanning?** |
| 14 | +Quality scanning uses specialized AI agents to systematically analyze code for different categories of issues. Each agent type focuses on specific problem domains and reports findings with severity levels and actionable fixes. |
| 15 | + |
| 16 | +**socket-lib Architecture:** |
| 17 | +Core utilities and infrastructure library for Socket.dev security tools that: |
| 18 | +- Provides shared utilities used across Socket.dev projects |
| 19 | +- Contains logger, fs helpers, process utilities |
| 20 | +- Manages build infrastructure and tooling |
| 21 | +- Provides testing utilities and helpers |
| 22 | +- Supports cross-platform compatibility |
| 23 | + |
| 24 | +**Scan Types Available:** |
| 25 | +1. **critical** - Crashes, security vulnerabilities, resource leaks, data corruption |
| 26 | +2. **logic** - Algorithm errors, edge cases, type guards, off-by-one errors |
| 27 | +3. **cache** - Cache staleness, race conditions, invalidation bugs |
| 28 | +4. **workflow** - Build scripts, CI issues, cross-platform compatibility |
| 29 | +5. **security** - GitHub Actions workflow security (zizmor scanner) |
| 30 | + |
| 31 | +**Why Quality Scanning Matters:** |
| 32 | +- Catches bugs before they reach production |
| 33 | +- Identifies security vulnerabilities early |
| 34 | +- Improves code quality systematically |
| 35 | +- Provides actionable fixes with file:line references |
| 36 | +- Prioritizes issues by severity for efficient remediation |
| 37 | + |
| 38 | +**Agent Prompts:** |
| 39 | +All agent prompts are embedded in `reference.md` with structured <context>, <instructions>, <pattern>, and <output_format> tags following Claude best practices. |
| 40 | +</context> |
| 41 | + |
| 42 | +<constraints> |
| 43 | +**CRITICAL Requirements:** |
| 44 | +- Read-only analysis (no code changes during scan) |
| 45 | +- Must complete all enabled scans before reporting |
| 46 | +- Findings must be prioritized by severity (Critical → High → Medium → Low) |
| 47 | +- Must generate actionable tasks with file:line references |
| 48 | +- All findings must include suggested fixes |
| 49 | + |
| 50 | +**Do NOT:** |
| 51 | +- Fix issues during scan (analysis only - report findings) |
| 52 | +- Skip critical scan types without user permission |
| 53 | +- Report findings without file/line references |
| 54 | +- Proceed if codebase has uncommitted changes (warn but continue) |
| 55 | + |
| 56 | +**Do ONLY:** |
| 57 | +- Run enabled scan types in priority order (critical → logic → cache → workflow → security) |
| 58 | +- Generate structured findings with severity levels |
| 59 | +- Provide actionable improvement tasks with specific code changes |
| 60 | +- Report statistics and coverage metrics |
| 61 | +- Deduplicate findings across scans |
| 62 | +</constraints> |
| 63 | + |
| 64 | +<instructions> |
| 65 | + |
| 66 | +## Process |
| 67 | + |
| 68 | +Execute the following phases sequentially to perform comprehensive quality analysis. |
| 69 | + |
| 70 | +### Phase 1: Validate Environment |
| 71 | + |
| 72 | +<prerequisites> |
| 73 | +Verify the environment before starting scans: |
| 74 | +</prerequisites> |
| 75 | + |
| 76 | +```bash |
| 77 | +git status |
| 78 | +``` |
| 79 | + |
| 80 | +<validation> |
| 81 | +**Expected State:** |
| 82 | +- Working directory should be clean (warn if dirty but continue) |
| 83 | +- On a valid branch |
| 84 | +- Node modules installed |
| 85 | + |
| 86 | +**If working directory dirty:** |
| 87 | +- Warn user: "Working directory has uncommitted changes - continuing with scan" |
| 88 | +- Continue with scans (quality scanning is read-only) |
| 89 | + |
| 90 | +</validation> |
| 91 | + |
| 92 | +--- |
| 93 | + |
| 94 | +### Phase 2: Determine Scan Scope |
| 95 | + |
| 96 | +<action> |
| 97 | +Ask user which scans to run: |
| 98 | +</action> |
| 99 | + |
| 100 | +**Default Scan Types** (run all unless user specifies): |
| 101 | +1. **critical** - Critical bugs (crashes, security, resource leaks) |
| 102 | +2. **logic** - Logic errors (algorithms, edge cases, type guards) |
| 103 | +3. **cache** - Caching issues (staleness, races, invalidation) |
| 104 | +4. **workflow** - Workflow problems (scripts, CI, git hooks) |
| 105 | +5. **security** - GitHub Actions security (template injection, cache poisoning, etc.) |
| 106 | + |
| 107 | +**User Interaction:** |
| 108 | +Use AskUserQuestion tool: |
| 109 | +- Question: "Which quality scans would you like to run?" |
| 110 | +- Header: "Scan Types" |
| 111 | +- multiSelect: true |
| 112 | +- Options: |
| 113 | + - "All scans (recommended)" → Run all 5 scan types |
| 114 | + - "Critical only" → Run critical scan only |
| 115 | + - "Critical + Logic" → Run critical and logic scans |
| 116 | + - "Custom selection" → Ask user to specify which scans |
| 117 | + |
| 118 | +**Default:** If user doesn't specify, run all scans. |
| 119 | + |
| 120 | +<validation> |
| 121 | +Validate selected scan types exist in reference.md: |
| 122 | +- critical-scan → reference.md line ~5 |
| 123 | +- logic-scan → reference.md line ~100 |
| 124 | +- cache-scan → reference.md line ~200 |
| 125 | +- workflow-scan → reference.md line ~300 |
| 126 | +- security-scan → reference.md line ~400 |
| 127 | + |
| 128 | +If user requests non-existent scan type, report error and suggest valid types. |
| 129 | +</validation> |
| 130 | + |
| 131 | +--- |
| 132 | + |
| 133 | +### Phase 3: Execute Scans |
| 134 | + |
| 135 | +<action> |
| 136 | +For each enabled scan type, spawn a specialized agent using Task tool: |
| 137 | +</action> |
| 138 | + |
| 139 | +```typescript |
| 140 | +// Example: Critical scan |
| 141 | +Task({ |
| 142 | + subagent_type: "general-purpose", |
| 143 | + description: "Critical bugs scan", |
| 144 | + prompt: `${CRITICAL_SCAN_PROMPT_FROM_REFERENCE_MD} |
| 145 | +
|
| 146 | +Focus on src/, packages/. |
| 147 | +
|
| 148 | +Report findings in this format: |
| 149 | +- File: path/to/file.mts:lineNumber |
| 150 | +- Issue: Brief description |
| 151 | +- Severity: Critical/High/Medium/Low |
| 152 | +- Pattern: Code snippet |
| 153 | +- Trigger: What input triggers this |
| 154 | +- Fix: Suggested fix |
| 155 | +- Impact: What happens if triggered |
| 156 | +
|
| 157 | +Scan systematically and report all findings. If no issues found, state that explicitly.` |
| 158 | +}) |
| 159 | +``` |
| 160 | + |
| 161 | +**For each scan:** |
| 162 | +1. Load agent prompt template from `reference.md` |
| 163 | +2. Customize for socket-lib context (focus on src/, packages/) |
| 164 | +3. Spawn agent with Task tool using "general-purpose" subagent_type |
| 165 | +4. Capture findings from agent response |
| 166 | +5. Parse and categorize results |
| 167 | + |
| 168 | +**Execution Order:** Run scans sequentially in priority order: |
| 169 | +- critical (highest priority) |
| 170 | +- logic |
| 171 | +- cache |
| 172 | +- workflow |
| 173 | +- security (lowest priority) |
| 174 | + |
| 175 | +**Agent Prompt Sources:** |
| 176 | +- Critical scan: reference.md starting at line ~12 |
| 177 | +- Logic scan: reference.md starting at line ~100 |
| 178 | +- Cache scan: reference.md starting at line ~200 |
| 179 | +- Workflow scan: reference.md starting at line ~300 |
| 180 | +- Security scan: reference.md starting at line ~400 |
| 181 | + |
| 182 | +<validation> |
| 183 | +For each scan completion: |
| 184 | +- Verify agent completed without errors |
| 185 | +- Extract findings from agent output |
| 186 | +- Parse into structured format (file, issue, severity, fix) |
| 187 | +- Track scan coverage (files analyzed) |
| 188 | +</validation> |
| 189 | + |
| 190 | +--- |
| 191 | + |
| 192 | +### Phase 4: Aggregate Findings |
| 193 | + |
| 194 | +<action> |
| 195 | +Collect all findings from agents and aggregate: |
| 196 | +</action> |
| 197 | + |
| 198 | +```typescript |
| 199 | +interface Finding { |
| 200 | + file: string // "src/api/client.mts:89" |
| 201 | + issue: string // "Potential null pointer access" |
| 202 | + severity: "Critical" | "High" | "Medium" | "Low" |
| 203 | + scanType: string // "critical" |
| 204 | + pattern: string // Code snippet showing the issue |
| 205 | + trigger: string // What causes this issue |
| 206 | + fix: string // Suggested code change |
| 207 | + impact: string // What happens if triggered |
| 208 | +} |
| 209 | +``` |
| 210 | + |
| 211 | +**Deduplication:** |
| 212 | +- Remove duplicate findings across scans (same file:line, same issue) |
| 213 | +- Keep the finding from the highest priority scan |
| 214 | +- Track which scans found the same issue |
| 215 | + |
| 216 | +**Prioritization:** |
| 217 | +- Sort by severity: Critical → High → Medium → Low |
| 218 | +- Within same severity, sort by scanType priority |
| 219 | +- Within same severity+scanType, sort alphabetically by file path |
| 220 | + |
| 221 | +<validation> |
| 222 | +**Checkpoint:** Verify aggregation: |
| 223 | +- Total findings count |
| 224 | +- Breakdown by severity (N critical, N high, N medium, N low) |
| 225 | +- Breakdown by scan type |
| 226 | +- Duplicate removal count (if any) |
| 227 | +</validation> |
| 228 | + |
| 229 | +--- |
| 230 | + |
| 231 | +### Phase 5: Generate Report |
| 232 | + |
| 233 | +<action> |
| 234 | +Create structured quality report with all findings: |
| 235 | +</action> |
| 236 | + |
| 237 | +```markdown |
| 238 | +# Quality Scan Report |
| 239 | + |
| 240 | +**Date:** YYYY-MM-DD |
| 241 | +**Repository:** socket-lib |
| 242 | +**Scans:** [list of scan types run] |
| 243 | +**Files Scanned:** N |
| 244 | +**Findings:** N critical, N high, N medium, N low |
| 245 | + |
| 246 | +## Critical Issues (Priority 1) - N found |
| 247 | + |
| 248 | +### src/api/client.mts:89 |
| 249 | +- **Issue**: Potential null pointer access when calling API |
| 250 | +- **Pattern**: `const result = response.data.items[0].value` |
| 251 | +- **Trigger**: When API returns empty items array |
| 252 | +- **Fix**: `const items = response.data?.items ?? []; const value = items[0]?.value ?? null` |
| 253 | +- **Impact**: Crashes API client, breaks user workflows |
| 254 | +- **Scan**: critical |
| 255 | + |
| 256 | +## High Issues (Priority 2) - N found |
| 257 | + |
| 258 | +[Similar format for high severity issues] |
| 259 | + |
| 260 | +## Medium Issues (Priority 3) - N found |
| 261 | + |
| 262 | +[Similar format for medium severity issues] |
| 263 | + |
| 264 | +## Low Issues (Priority 4) - N found |
| 265 | + |
| 266 | +[Similar format for low severity issues] |
| 267 | + |
| 268 | +## Scan Coverage |
| 269 | + |
| 270 | +- **Critical scan**: N files analyzed in src/, packages/ |
| 271 | +- **Logic scan**: N files analyzed |
| 272 | +- **Cache scan**: N files analyzed (if applicable) |
| 273 | +- **Workflow scan**: N files analyzed (package.json, scripts/, .github/) |
| 274 | +- **Security scan**: N workflow files analyzed (.github/workflows/) |
| 275 | + |
| 276 | +## Recommendations |
| 277 | + |
| 278 | +1. Address N critical issues immediately before next release |
| 279 | +2. Review N high-severity logic errors |
| 280 | +3. Schedule N medium issues for next sprint |
| 281 | +4. Low-priority items can be addressed during refactoring |
| 282 | + |
| 283 | +## No Findings |
| 284 | + |
| 285 | +[If a scan found no issues, list it here:] |
| 286 | +- Critical scan: ✓ Clean |
| 287 | +- Logic scan: ✓ Clean |
| 288 | +- Security scan: ✓ Clean |
| 289 | +``` |
| 290 | + |
| 291 | +**Output Report:** |
| 292 | +1. Display report to console (user sees it) |
| 293 | +2. Offer to save to file (optional): `reports/quality-scan-YYYY-MM-DD.md` |
| 294 | + |
| 295 | +<validation> |
| 296 | +**Report Quality Checks:** |
| 297 | +- All findings include file:line references |
| 298 | +- All findings include suggested fixes |
| 299 | +- Findings are grouped by severity |
| 300 | +- Scan coverage statistics included |
| 301 | +- Recommendations are actionable |
| 302 | +</validation> |
| 303 | + |
| 304 | +--- |
| 305 | + |
| 306 | +### Phase 6: Complete |
| 307 | + |
| 308 | +<completion_signal> |
| 309 | +```xml |
| 310 | +<promise>QUALITY_SCAN_COMPLETE</promise> |
| 311 | +``` |
| 312 | +</completion_signal> |
| 313 | + |
| 314 | +<summary> |
| 315 | +Report these final metrics to the user: |
| 316 | + |
| 317 | +**Quality Scan Complete** |
| 318 | +======================== |
| 319 | +✓ Scans completed: [list of scan types] |
| 320 | +✓ Total findings: N (N critical, N high, N medium, N low) |
| 321 | +✓ Files scanned: N |
| 322 | +✓ Report generated: Yes |
| 323 | +✓ Scan duration: [calculated from start to end] |
| 324 | + |
| 325 | +**Critical Issues Requiring Immediate Attention:** |
| 326 | +- N critical issues found |
| 327 | +- Review report above for details and fixes |
| 328 | + |
| 329 | +**Next Steps:** |
| 330 | +1. Address critical issues immediately |
| 331 | +2. Review high-severity findings |
| 332 | +3. Schedule medium/low issues appropriately |
| 333 | +4. Re-run scans after fixes to verify |
| 334 | + |
| 335 | +All findings include file:line references and suggested fixes. |
| 336 | +</summary> |
| 337 | + |
| 338 | +</instructions> |
| 339 | + |
| 340 | +## Success Criteria |
| 341 | + |
| 342 | +- ✅ `<promise>QUALITY_SCAN_COMPLETE</promise>` output |
| 343 | +- ✅ All enabled scans completed without errors |
| 344 | +- ✅ Findings prioritized by severity (Critical → Low) |
| 345 | +- ✅ All findings include file:line references |
| 346 | +- ✅ Actionable suggestions provided for all findings |
| 347 | +- ✅ Report generated with statistics and coverage metrics |
| 348 | +- ✅ Duplicate findings removed |
| 349 | + |
| 350 | +## Scan Types |
| 351 | + |
| 352 | +See `reference.md` for detailed agent prompts with structured tags: |
| 353 | + |
| 354 | +- **critical-scan** - Null access, promise rejections, race conditions, resource leaks |
| 355 | +- **logic-scan** - Off-by-one errors, type guards, edge cases, algorithm correctness |
| 356 | +- **cache-scan** - Invalidation, key generation, memory management, concurrency |
| 357 | +- **workflow-scan** - Scripts, package.json, git hooks, CI configuration |
| 358 | +- **security-scan** - GitHub Actions workflow security (runs zizmor scanner) |
| 359 | + |
| 360 | +All agent prompts follow Claude best practices with <context>, <instructions>, <pattern>, <output_format>, and <quality_guidelines> tags. |
| 361 | + |
| 362 | +## Commands |
| 363 | + |
| 364 | +This skill is self-contained. No external commands needed. |
| 365 | + |
| 366 | +## Context |
| 367 | + |
| 368 | +This skill provides systematic code quality analysis for socket-lib by: |
| 369 | +- Spawning specialized agents for targeted analysis |
| 370 | +- Using Task tool to run agents autonomously |
| 371 | +- Embedding agent prompts in reference.md following best practices |
| 372 | +- Generating prioritized, actionable reports |
| 373 | +- Supporting partial scans (user can select specific scan types) |
| 374 | + |
| 375 | +For detailed agent prompts with best practices structure, see `reference.md`. |
0 commit comments