Skip to content

Commit e0fe61f

Browse files
DavertMikDavertMikclaude
authored
feat(mcp): snapshot tool + run_code return/console capture + storage state + HTML cleanup (#5537)
* feat(mcp): snapshot tool, run_code return/console capture, storage state, HTML cleanup Adds a `snapshot` MCP tool that captures the current browser state without performing any action. `run_code` now also returns the value the code produced and captures `console.*` output (level + message + relative time) using the existing `safeStringify` from lib/utils.js with a `truncateString` helper, so models can use debug statements and inspect return values. Captures cookies + localStorage via Playwright's `grabStorageState()` (with a `grabCookie + executeScript` fallback for Puppeteer/WebDriver), normalized to a single shape and saved as `_storage.json` alongside the other artifacts. Centralizes the artifact-grab + trace-record glue: - New `lib/utils/captureSnapshot.js` is the single funnel for HTML / ARIA / screenshot / console / storage capture. - New `lib/utils/trace.js` holds shared helpers (`pickActingHelper`, `traceDirFor`, `snapshotDirFor`, `artifactLinks`, `artifactsToFileUrls`) used by `aiTrace`, `pageInfo`, and the MCP server. - aiTrace and pageInfo now go through the same funnel — pageInfo's three serial recorder steps collapse to one delegated capture. HTML processing pipeline (`formatHtml` in `lib/html.js`): minify -> cleanHtml -> beautify. `cleanHtml` (new) drops `<script>` (no src) / `<style>` / `<noscript>`, strips trash class names (Tailwind utilities, hashed/scoped/framework-generated), and removes inline `style=""`. Built on `js-beautify` (already a dep) instead of a hand-rolled pretty-printer. Browser-log normalization fixes a latent bug where `JSON.stringify` was called on Playwright `ConsoleMessage` objects, producing files full of empty objects. Logs are now coerced to plain `{type, text}` (or kept as strings for WebDriver) before serialization. `safeStringify` in lib/utils.js is extended to also coerce Function/Error/BigInt/Symbol values — universal improvements that close a JSON.stringify(BigInt) crash and don't regress the existing 12 mocha serialization tests. Tests: 72 unit tests passing (12 circular_reference + 23 html (incl. new `#cleanHtml`, `#formatHtml`, `#isTrashClass` describes) + 13 aiTrace + 32 mcp + 1 pending). Docs updated in `docs/mcp.md` and `docs/aitrace.md`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(utils): unify captureSnapshot + trace into lib/utils/trace.js Both files belonged to the same flow (capture artifacts -> render trace.md). Splitting them across two files made the import sites awkward, so they're now consolidated: - `lib/utils/captureSnapshot.js` deleted; its `captureSnapshot`, `normalizeBrowserLogs`, and `captureStorageState` moved into `lib/utils/trace.js`. - `writeTraceMarkdown` (was inlined in `bin/mcp-server.js`) moved into `lib/utils/trace.js` too — it's the same trace-record concern. - All three call sites (`bin/mcp-server.js`, `lib/plugin/aiTrace.js`, `lib/plugin/pageInfo.js`) now import from a single module. - Stale `writeFileSync` import dropped from `bin/mcp-server.js`. 72 unit tests still passing. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(utils): add unit coverage for trace.js + safeStringify extensions + truncateString 49 new test cases across two files: - `test/unit/utils/trace_test.js` (new, 37 cases) — direct coverage of every export from `lib/utils/trace.js`: - `pickActingHelper` (all 3 paths) - `traceDirFor` (determinism, uniqueness, sanitization, missing args) - `snapshotDirFor` (uniqueness, embedded timestamp) - `artifactLinks` (each artifact, console-count override, storage counts, custom indent, empty input, ordering invariant) - `fileToUrl` / `artifactsToFileUrls` - `writeTraceMarkdown` (golden output, error block, optional commands, default `file`) - `captureSnapshot` (default options, full opt-out matrix, fullPage, formatHtml integration, ConsoleMessage normalization, Playwright grabStorageState path, Puppeteer/WebDriver fallback path, empty storage handled, missing helper methods, error swallowing, default prefix) - `test/unit/circular_reference_test.js` (extended, 12 new cases): - `safeStringify` Function/BigInt/Symbol/Error coercions including nested mixed types (BigInt was a real bug; JSON.stringify(BigInt) used to throw and trigger the legacy fallback path) - `truncateString` under / equal / over maxBytes, non-string coercion, empty string Total unit suite: 121 passing (was 72). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(trace): use pathToFileURL for Windows-safe URLs; scan raw HTML in pageInfo - Replace `file://${path}` template literals in lib/utils/trace.js fileToUrl() and bin/mcp-server.js with pathToFileURL().href so paths like `C:\foo\bar` are encoded correctly on Windows. - captureSnapshot now also returns out.htmlRaw (pre-cleanHtml source) so consumers can scan classes that the trash-class filter would strip. - pageInfo's error-class scan now reads htmlRaw, restoring detection of errorClasses containing digits/prefixes like `text-error` or `alert-1`. --------- Co-authored-by: DavertMik <davert@testomat.io> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 87a25ad commit e0fe61f

11 files changed

Lines changed: 1308 additions & 236 deletions

File tree

bin/mcp-server.js

Lines changed: 160 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,22 @@ import { CallToolRequestSchema, ListToolsRequestSchema } from '@modelcontextprot
44
import Codecept from '../lib/codecept.js'
55
import container from '../lib/container.js'
66
import { getParamsToString } from '../lib/parser.js'
7-
import { methodsOfObject } from '../lib/utils.js'
7+
import { methodsOfObject, safeStringify, truncateString } from '../lib/utils.js'
8+
import {
9+
captureSnapshot,
10+
pickActingHelper,
11+
traceDirFor,
12+
snapshotDirFor,
13+
artifactsToFileUrls,
14+
writeTraceMarkdown,
15+
} from '../lib/utils/trace.js'
816
import event from '../lib/event.js'
9-
import { fileURLToPath } from 'url'
17+
import { fileURLToPath, pathToFileURL } from 'url'
1018
import { dirname, resolve as resolvePath } from 'path'
1119
import path from 'path'
12-
import crypto from 'crypto'
1320
import { spawn } from 'child_process'
1421
import { createRequire } from 'module'
15-
import { existsSync, readdirSync, writeFileSync } from 'fs'
22+
import { existsSync, readdirSync } from 'fs'
1623
import { mkdirp } from 'mkdirp'
1724

1825
const require = createRequire(import.meta.url)
@@ -224,15 +231,8 @@ async function resolveTestToFile({ cli, root, configPath, test }) {
224231
return fsFound ? normalizePath(fsFound) : null
225232
}
226233

227-
function clearString(str) {
228-
return str.replace(/[^a-zA-Z0-9]/g, '_')
229-
}
230-
231-
function getTraceDir(testTitle, testFile) {
232-
const hash = crypto.createHash('sha256').update(testFile + testTitle).digest('hex').slice(0, 8)
233-
const cleanTitle = clearString(testTitle).slice(0, 200)
234-
const outputDir = global.output_dir || resolvePath(process.cwd(), 'output')
235-
return resolvePath(outputDir, `trace_${cleanTitle}_${hash}`)
234+
function outputBaseDir() {
235+
return global.output_dir || resolvePath(process.cwd(), 'output')
236236
}
237237

238238
async function initCodecept(configPath) {
@@ -337,6 +337,17 @@ server.setRequestHandler(ListToolsRequestSchema, async () => ({
337337
description: 'Stop the browser session.',
338338
inputSchema: { type: 'object', properties: {} },
339339
},
340+
{
341+
name: 'snapshot',
342+
description: 'Capture current browser state (HTML, ARIA, screenshot, console, URL) without performing any action.',
343+
inputSchema: {
344+
type: 'object',
345+
properties: {
346+
config: { type: 'string' },
347+
fullPage: { type: 'boolean' },
348+
},
349+
},
350+
},
340351
],
341352
}))
342353

@@ -416,74 +427,132 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
416427
return { content: [{ type: 'text', text: JSON.stringify({ status: 'Browser stopped successfully' }, null, 2) }] }
417428
}
418429

430+
case 'snapshot': {
431+
const { config: configPath, fullPage = false } = args || {}
432+
await initCodecept(configPath)
433+
434+
const helper = pickActingHelper(container.helpers())
435+
if (!helper) throw new Error('No supported acting helper available (Playwright, Puppeteer, WebDriver).')
436+
437+
const dir = snapshotDirFor(outputBaseDir())
438+
mkdirp.sync(dir)
439+
440+
const captured = await captureSnapshot(helper, { dir, prefix: 'snapshot', fullPage })
441+
const traceFile = writeTraceMarkdown({
442+
dir,
443+
title: 'snapshot',
444+
file: 'mcp',
445+
durationMs: 0,
446+
commands: [],
447+
captured,
448+
})
449+
450+
return {
451+
content: [{
452+
type: 'text',
453+
text: JSON.stringify({
454+
status: 'success',
455+
dir,
456+
traceFile: pathToFileURL(traceFile).href,
457+
artifacts: artifactsToFileUrls(captured, dir),
458+
}, null, 2),
459+
}],
460+
}
461+
}
462+
419463
case 'run_code': {
420464
const { code, timeout = 60000, config: configPath, saveArtifacts = true } = args
421465
await initCodecept(configPath)
422466

423467
const I = container.support('I')
424468
if (!I) throw new Error('I object not available. Make sure helpers are configured.')
425469

426-
const result = { status: 'unknown', output: '', error: null, artifacts: {} }
470+
const result = { status: 'unknown', output: '', error: null, commands: [], artifacts: {} }
427471

472+
const commands = []
473+
const onStepAfter = step => {
474+
try { commands.push(step.toString()) } catch {}
475+
}
476+
event.dispatcher.on(event.step.after, onStepAfter)
477+
478+
const traceDir = traceDirFor(`mcp_${Date.now()}`, 'run_code', outputBaseDir())
479+
mkdirp.sync(traceDir)
480+
const startedAt = Date.now()
481+
482+
const MAX_LOG_ENTRIES = 100
483+
const MAX_LOG_MSG_BYTES = 2000
484+
const MAX_RETURN_BYTES = 20000
485+
const consoleLogs = []
486+
const consoleMethods = ['log', 'info', 'warn', 'error', 'debug']
487+
const origConsoleMethods = {}
488+
const captureLog = level => (...args) => {
489+
if (consoleLogs.length >= MAX_LOG_ENTRIES) return
490+
const message = args.map(a => {
491+
if (typeof a === 'string') return a
492+
return truncateString(safeStringify(a, [], 2), MAX_LOG_MSG_BYTES).value
493+
}).join(' ')
494+
consoleLogs.push({ level, message, t: Date.now() - startedAt })
495+
}
496+
for (const m of consoleMethods) {
497+
origConsoleMethods[m] = console[m]
498+
console[m] = captureLog(m)
499+
}
500+
501+
let returnValue
428502
try {
429503
const asyncFn = new Function('I', `return (async () => { ${code} })()`)
430-
await Promise.race([
504+
returnValue = await Promise.race([
431505
asyncFn(I),
432506
new Promise((_, reject) => setTimeout(() => reject(new Error(`Timeout after ${timeout}ms`)), timeout)),
433507
])
434508

435509
result.status = 'success'
436510
result.output = 'Code executed successfully'
437-
438-
if (saveArtifacts) {
439-
const helpers = container.helpers()
440-
const helper = Object.values(helpers)[0]
441-
if (helper) {
442-
try {
443-
const traceDir = getTraceDir('mcp', 'run_code')
444-
mkdirp.sync(traceDir)
445-
446-
if (helper.grabAriaSnapshot) {
447-
const aria = await helper.grabAriaSnapshot()
448-
const ariaFile = path.join(traceDir, 'aria.txt')
449-
writeFileSync(ariaFile, aria)
450-
result.artifacts.aria = `file://${ariaFile}`
451-
}
452-
453-
if (helper.grabCurrentUrl) {
454-
result.artifacts.url = await helper.grabCurrentUrl()
455-
}
456-
457-
if (helper.grabBrowserLogs) {
458-
const logs = (await helper.grabBrowserLogs()) || []
459-
const logsFile = path.join(traceDir, 'console.json')
460-
writeFileSync(logsFile, JSON.stringify(logs, null, 2))
461-
result.artifacts.consoleLogs = `file://${logsFile}`
462-
}
463-
464-
if (helper.grabSource) {
465-
const html = await helper.grabSource()
466-
const htmlFile = path.join(traceDir, 'page.html')
467-
writeFileSync(htmlFile, html)
468-
result.artifacts.html = `file://${htmlFile}`
469-
}
470-
471-
if (helper.saveScreenshot) {
472-
const screenshotFile = path.join(traceDir, 'screenshot.png')
473-
await helper.saveScreenshot(screenshotFile)
474-
result.artifacts.screenshot = `file://${screenshotFile}`
475-
}
476-
} catch (e) {
477-
result.output += ` (Warning: ${e.message})`
478-
}
479-
}
480-
}
481511
} catch (error) {
482512
result.status = 'failed'
483513
result.error = error.message
484514
result.output = error.stack || error.message
515+
} finally {
516+
for (const m of consoleMethods) console[m] = origConsoleMethods[m]
517+
try { event.dispatcher.removeListener(event.step.after, onStepAfter) } catch {}
518+
}
519+
520+
result.commands = commands
521+
result.logs = consoleLogs
522+
if (consoleLogs.length === MAX_LOG_ENTRIES) result.logsTruncated = true
523+
524+
if (returnValue !== undefined) {
525+
const json = typeof returnValue === 'string' ? returnValue : safeStringify(returnValue, [], 2)
526+
const stringified = truncateString(json, MAX_RETURN_BYTES)
527+
result.returnValue = stringified.value
528+
if (stringified.truncated) result.returnValueTruncated = true
485529
}
486530

531+
let captured = {}
532+
if (saveArtifacts) {
533+
const helper = pickActingHelper(container.helpers())
534+
if (helper) {
535+
try {
536+
captured = await captureSnapshot(helper, { dir: traceDir, prefix: 'mcp' })
537+
result.artifacts = artifactsToFileUrls(captured, traceDir)
538+
} catch (e) {
539+
result.output += ` (Warning: ${e.message})`
540+
}
541+
}
542+
}
543+
544+
const traceFile = writeTraceMarkdown({
545+
dir: traceDir,
546+
title: 'run_code',
547+
file: 'mcp',
548+
durationMs: Date.now() - startedAt,
549+
commands,
550+
captured,
551+
error: result.error,
552+
})
553+
result.dir = traceDir
554+
result.traceFile = pathToFileURL(traceFile).href
555+
487556
return { content: [{ type: 'text', text: JSON.stringify(result, null, 2) }] }
488557
}
489558

@@ -549,27 +618,56 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
549618

550619
const results = []
551620
const currentSteps = {}
621+
const traceDirs = {}
552622
let currentTestTitle = null
553623
const testFile = testFiles[0]
554624

555625
const onBefore = (t) => {
556-
const traceDir = getTraceDir(t.title, t.file)
626+
const traceDir = traceDirFor(t.file, t.title, outputBaseDir())
557627
currentTestTitle = t.title
558628
currentSteps[t.title] = []
629+
traceDirs[t.title] = traceDir
559630
results.push({
560631
test: t.title,
561632
file: t.file,
562-
traceFile: `file://${resolvePath(traceDir, 'trace.md')}`,
563633
status: 'running',
564634
steps: [],
565635
})
566636
}
567637

568-
const onAfter = (t) => {
638+
const onAfter = async (t) => {
569639
const r = results.find(x => x.test === t.title)
570640
if (r) {
571641
r.status = t.err ? 'failed' : 'completed'
572642
if (t.err) r.error = t.err.message
643+
644+
if (t.artifacts?.aiTrace) {
645+
r.traceFile = pathToFileURL(t.artifacts.aiTrace).href
646+
}
647+
if (t.artifacts?.har) r.har = pathToFileURL(t.artifacts.har).href
648+
if (t.artifacts?.trace) r.trace = pathToFileURL(t.artifacts.trace).href
649+
650+
if (!t.artifacts?.aiTrace) {
651+
try {
652+
const helper = pickActingHelper(container.helpers())
653+
const dir = traceDirs[t.title]
654+
if (helper && dir) {
655+
mkdirp.sync(dir)
656+
const captured = await captureSnapshot(helper, { dir, prefix: 'final' })
657+
r.artifacts = artifactsToFileUrls(captured, dir)
658+
const tracePath = writeTraceMarkdown({
659+
dir,
660+
title: t.title,
661+
file: t.file,
662+
durationMs: 0,
663+
commands: (currentSteps[t.title] || []).map(s => s.step),
664+
captured,
665+
error: r.error,
666+
})
667+
r.traceFile = pathToFileURL(tracePath).href
668+
}
669+
} catch {}
670+
}
573671
}
574672
currentTestTitle = null
575673
}

docs/aitrace.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,16 +53,18 @@ For each test, a `trace_<sha256>` directory is created with the following files:
5353

5454
**0000_step_name_screenshot.png** - Screenshot for each step (file names include step names)
5555

56-
**0000_step_name_page.html** - Full HTML of the page at each step
56+
**0000_step_name_page.html** - Full HTML of the page at each step. Processed through a `minify -> clean -> beautify` pipeline so the file is multi-line indented, free of `<script>` / `<style>` / `<noscript>` content, free of inline `style=""` attributes, and free of trash class names (Tailwind utilities, framework-generated `v-*` / `ember-*`, hashed classes, scoped `xl:hidden`-style classes). Semantic attributes (`id`, `aria-*`, `data-*`, `role`, etc.) are preserved.
5757

58-
**0000_step_name_aria.txt** - ARIA accessibility snapshot (AI-readable structure without HTML noise)
58+
**0000_step_name_aria.txt** - ARIA accessibility snapshot (AI-readable structure without HTML noise; Playwright only)
5959

60-
**0000_step_name_console.json** - Browser console logs
60+
**0000_step_name_console.json** - Browser console logs, normalized to plain `{ type, text }` objects (Playwright `ConsoleMessage` instances are coerced via their `.type()` / `.text()` methods so the JSON file is genuinely useful — not full of empty objects).
6161

6262
When HAR or trace recording is enabled in your helper config, links to those files are also included.
6363

6464
**Note:** Artifact files are named using step names for easier identification (e.g., `0000_I_see_Product_screenshot.png` instead of just `0000_screenshot.png`).
6565

66+
**Storage state:** Cookies and `localStorage` are intentionally **not** captured per-step by `aiTrace` (they rarely change between actions, so per-step `_storage.json` files would be noise). Use the `pageInfo` plugin or the MCP `snapshot()` tool when you need a storage snapshot.
67+
6668
## Trace File Format
6769

6870
The `trace.md` file contains a structured execution log with links to all artifacts:

0 commit comments

Comments
 (0)