From cb4de2c29684f5b2bccae625a70e35384196416d Mon Sep 17 00:00:00 2001 From: suchintan <3853670+suchintan@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:30:27 +0000 Subject: [PATCH 1/6] =?UTF-8?q?=F0=9F=94=84=20synced=20local=20'bindings/'?= =?UTF-8?q?=20with=20remote=20'bindings/'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit https://github.com/Skyvern-AI/rustwright-cloud/pull/232 --- bindings/CONTRACT.md | 55 +++++++++++++++++++++++++++++++++++--------- 1 file changed, 44 insertions(+), 11 deletions(-) diff --git a/bindings/CONTRACT.md b/bindings/CONTRACT.md index 0f40be2..4998558 100644 --- a/bindings/CONTRACT.md +++ b/bindings/CONTRACT.md @@ -23,7 +23,10 @@ as one JSON value, and returns the core JSON wire value decoded to a native value. `screenshot` returns encoded bytes. Timeout values are milliseconds. The language-facing option names and behavior match the Node alpha surface. -The C launch JSON is the normalized core wire shape: `headless`, +The core launch parser accepts canonical snake_case fields and the current +camelCase aliases `executablePath`, `ignoreAllDefaultArgs`, `ignoreDefaultArgs`, +`userDataDir`, and `chromiumSandbox`. C callers should emit canonical snake_case +keys. The C launch JSON uses the normalized core wire shape: `headless`, `executable_path`, `channel`, `args`, `ignore_all_default_args`, `ignore_default_args`, `timeout`, `user_data_dir`, `env`, `chromium_sandbox`, and `proxy`. Screenshot JSON uses the Node names `path`, `fullPage`, `clip`, @@ -46,6 +49,9 @@ const char *rw_last_error(void); void rw_string_free(char *s); void rw_bytes_free(uint8_t *buf, size_t len); +/* evaluate wire */ +int32_t rw_decode_wire(const char *wire_json, char **out_json); + /* Chromium */ int32_t rw_chromium_executable_path(char **out_path); int32_t rw_chromium_launch(const char *options_json, @@ -57,6 +63,16 @@ int32_t rw_browser_close(RwBrowser *b); char *rw_browser_ws_endpoint(RwBrowser *b); void rw_browser_free(RwBrowser *b); +/* page timeout defaults */ +int32_t rw_page_set_default_timeout(RwPage *p, double timeout_ms_or_nan); +int32_t rw_page_set_default_navigation_timeout(RwPage *p, + double timeout_ms_or_nan); +int32_t rw_page_set_context_default_timeout(RwPage *p, + double timeout_ms_or_nan); +int32_t rw_page_set_context_default_navigation_timeout( + RwPage *p, + double timeout_ms_or_nan); + /* page */ char *rw_page_target_id(RwPage *p); int32_t rw_page_goto(RwPage *p, @@ -149,20 +165,37 @@ arrays as `{ "__rustwright_cdp_array__": id, "items": [...] }` and objects as `{ "__rustwright_cdp_object__": id, "entries": {...} }`; recursively unwrap `items` and `entries`. It uses `__rustwright_cdp_ref__` for repeated/cyclic references and tagged objects for undefined, non-finite numbers, dates, -regular expressions, URLs, errors, symbols, and functions. The core -serializer is the single source of truth for this vocabulary; a binding maps -the core's tags to its closest native representation and must not invent or -assume tags the core does not emit. Manifest v1 expected/captured values are -JSON-compatible and never require cycles; a runner must at least recursively -decode array/object wrappers before capture or `assertEval` comparison. +regular expressions, URLs, errors, symbols, and functions. + +The legacy core `decode_wire_value` and C ABI `rw_decode_wire` contract is +flattened plain JSON. Array and object wrappers are removed, repeated +non-cyclic references are duplicated, and references that form cycles become +`{"__rustwright_cdp_cycle__": true}`. Leaf scalar tags remain in the output for +binding-specific native mapping. This behavior is compatibility-stable. + +The PyO3 and napi graph adapters use the core graph representation instead. +They allocate host arrays and objects with stable identity, preserve repeated +references and cycles, and keep leaf conversion policy in the adapter. These +graph adapters must not be replaced with the legacy flattened C ABI decoder. + +The core serializer is the single source of truth for this vocabulary; a +binding maps the core's tags to its closest native representation and must not +invent or assume tags the core does not emit. Manifest v1 expected/captured +values are JSON-compatible and never require cycles; a runner must at least +recursively decode array/object wrappers before capture or `assertEval` +comparison. ### Thin-shim rule (single source of logic) Any behavior expressible as a pure function of JSON-in/JSON-out — launch and -screenshot option normalization and defaulting, evaluate-wire decoding, -timeout-precedence resolution, data-URL construction, and structural result -comparison — is implemented once in `rustwright-core` and exposed through the -C ABI (and napi/PyO3). A binding limits itself to: +screenshot option normalization and defaulting, evaluate-wire parsing and +decoding, timeout-precedence resolution, data-URL construction, and structural +result comparison — is implemented once in `rustwright-core` and exposed +through the C ABI and native PyO3/napi adapters. + +The legacy C ABI decoder intentionally remains a flattened JSON compatibility +path. Native graph adapters use core graph parsing to preserve host identity +and cycles. A binding limits itself to: - marshalling native values to and from the documented JSON wire shapes, - handle, memory, and thread ownership per this contract, and From adf3858166e9a43e94aee7f5fa23a6734038f43e Mon Sep 17 00:00:00 2001 From: suchintan <3853670+suchintan@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:30:27 +0000 Subject: [PATCH 2/6] =?UTF-8?q?=F0=9F=94=84=20synced=20local=20'node/'=20w?= =?UTF-8?q?ith=20remote=20'node/'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit https://github.com/Skyvern-AI/rustwright-cloud/pull/232 --- node/index.cjs | 55 +----------- node/native.d.ts | 2 + node/src/lib.rs | 170 +++++++++++++++++++++++++++++++++++- node/test/decode.test.mjs | 59 ++++++++----- node/test/timeouts.test.mjs | 2 +- 5 files changed, 213 insertions(+), 75 deletions(-) diff --git a/node/index.cjs b/node/index.cjs index 2d831aa..11e426d 100644 --- a/node/index.cjs +++ b/node/index.cjs @@ -65,61 +65,12 @@ function encodeEvaluateArg(arg) { return JSON.stringify(arg); } -function decodeWireValue(value, seen = new Map()) { - if (Array.isArray(value)) return value.map((item) => decodeWireValue(item, seen)); - if (!value || typeof value !== 'object') return value; - - if (hasOwn(value, '__rustwright_cdp_ref__')) { - return seen.get(value.__rustwright_cdp_ref__); - } - if (hasOwn(value, '__rustwright_cdp_array__')) { - const ref = value.__rustwright_cdp_array__; - const result = []; - seen.set(ref, result); - for (const item of value.items || []) result.push(decodeWireValue(item, seen)); - return result; - } - if (hasOwn(value, '__rustwright_cdp_object__')) { - const ref = value.__rustwright_cdp_object__; - const result = {}; - seen.set(ref, result); - for (const [key, item] of Object.entries(value.entries || {})) { - result[key] = decodeWireValue(item, seen); - } - return result; - } - if (hasOwn(value, '__rustwright_cdp_undefined__')) return undefined; - if (hasOwn(value, '__rustwright_cdp_symbol__')) return undefined; - if (hasOwn(value, '__rustwright_cdp_function__')) return undefined; - if (hasOwn(value, '__rustwright_cdp_date__')) return new Date(value.__rustwright_cdp_date__); - if (hasOwn(value, '__rustwright_cdp_regexp__')) { - const spec = value.__rustwright_cdp_regexp__ || {}; - return new RegExp(String(spec.p || ''), String(spec.f || '')); - } - if (hasOwn(value, '__rustwright_cdp_url__')) return new URL(value.__rustwright_cdp_url__); - if (hasOwn(value, '__rustwright_cdp_error__')) { - const spec = value.__rustwright_cdp_error__ || {}; - const error = new Error(String(spec.message || '')); - error.name = String(spec.name || 'Error'); - error.stack = String(spec.stack || ''); - return error; - } - if (hasOwn(value, '__rustwright_cdp_unserializable_value__')) { - const marker = value.__rustwright_cdp_unserializable_value__; - if (marker === 'NaN') return NaN; - if (marker === 'Infinity') return Infinity; - if (marker === '-Infinity') return -Infinity; - if (marker === '-0') return -0; - if (typeof marker === 'string' && marker.endsWith('n')) return BigInt(marker.slice(0, -1)); - } - - return Object.fromEntries( - Object.entries(value).map(([key, item]) => [key, decodeWireValue(item, seen)]) - ); +function decodeWireValue(wireJson) { + return native.decodeWire(wireJson); } function parseRustJson(json) { - return decodeWireValue(JSON.parse(json)); + return decodeWireValue(json); } class Browser { diff --git a/node/native.d.ts b/node/native.d.ts index 97b6141..3793a8e 100644 --- a/node/native.d.ts +++ b/node/native.d.ts @@ -24,4 +24,6 @@ export declare class Page { export declare function chromiumExecutablePath(): Promise +export declare function decodeWire(wireJson: string): unknown + export declare function launchChromium(optionsJson: string): Promise diff --git a/node/src/lib.rs b/node/src/lib.rs index 18a6600..5393994 100644 --- a/node/src/lib.rs +++ b/node/src/lib.rs @@ -1,4 +1,6 @@ -use napi::bindgen_prelude::Buffer; +use napi::bindgen_prelude::{ + Array, Buffer, Env, FnArgs, Function, JsObjectValue, Null, Object, ToNapiValue, Unknown, +}; use napi::{Error, Result, Status}; use napi_derive::napi; use rustwright_core as rw; @@ -24,6 +26,172 @@ where })? .map_err(napi_error) } +fn to_unknown<'env, T: ToNapiValue>(env: &'env Env, value: T) -> Result> { + value.into_unknown(env) +} + +struct WireMaterializer<'env, 'graph> { + env: &'env Env, + graph: &'graph rw::WireGraph, + values: Vec>>, +} + +impl<'env, 'graph> WireMaterializer<'env, 'graph> { + fn new(env: &'env Env, graph: &'graph rw::WireGraph) -> Self { + Self { + env, + graph, + values: vec![None; graph.nodes().len()], + } + } + + fn first_pass(&mut self) -> Result<()> { + for (index, kind) in self.graph.nodes().iter().enumerate() { + let value = match kind { + rw::WireNodeKind::Null => to_unknown(self.env, Null)?, + rw::WireNodeKind::Bool(value) => to_unknown(self.env, *value)?, + rw::WireNodeKind::Number(rw::WireNumber::Signed(value)) => { + to_unknown(self.env, *value as f64)? + } + rw::WireNodeKind::Number(rw::WireNumber::Unsigned(value)) => { + to_unknown(self.env, *value as f64)? + } + rw::WireNodeKind::Number(rw::WireNumber::Float(value)) => { + to_unknown(self.env, *value)? + } + rw::WireNodeKind::String(value) => { + to_unknown(self.env, self.env.create_string(value.clone())?)? + } + rw::WireNodeKind::Array(items) => self + .env + .create_array(items.len() as u32)? + .into_unknown(self.env)?, + rw::WireNodeKind::Object(_) => Object::new(self.env)?.into_unknown(self.env)?, + rw::WireNodeKind::Leaf(leaf) => leaf_to_unknown(self.env, leaf)?, + }; + self.values[index] = Some(value); + } + Ok(()) + } + + fn value(&self, id: rw::WireNodeId) -> Result> { + self.values + .get(id.index()) + .and_then(|value| value.as_ref()) + .copied() + .ok_or_else(|| napi_error(format!("wire graph node {} has no value", id.index()))) + } + + fn second_pass(&mut self) -> Result<()> { + for (index, kind) in self.graph.nodes().iter().enumerate() { + let value = self + .values + .get(index) + .and_then(|value| value.as_ref()) + .copied() + .ok_or_else(|| napi_error(format!("wire graph node {index} has no value")))?; + match kind { + rw::WireNodeKind::Array(items) => { + // SAFETY: first_pass creates an Array at this dense graph index. + let mut array = unsafe { value.cast::()? }; + for (item_index, child) in items.iter().enumerate() { + array.set(item_index as u32, self.value(*child)?)?; + } + } + rw::WireNodeKind::Object(entries) => { + // SAFETY: first_pass creates an Object at this dense graph index. + let mut object = unsafe { value.cast::()? }; + let mut properties = Vec::with_capacity(entries.len()); + for (key, child) in entries { + let child = self.value(*child)?; + properties.push( + napi::Property::new() + .with_name(self.env, key)? + .with_value(&child), + ); + } + object.define_properties(&properties)?; + } + _ => {} + } + } + Ok(()) + } + + fn materialize(&mut self) -> Result> { + self.first_pass()?; + self.second_pass()?; + self.value(self.graph.root()) + } +} +fn unserializable_wrapper<'env>(env: &'env Env, value: &str) -> Result> { + let mut object = Object::new(env)?; + let value = to_unknown(env, env.create_string(value)?)?; + let property = napi::Property::new() + .with_name(env, "__rustwright_cdp_unserializable_value__")? + .with_value(&value); + object.define_properties(&[property])?; + object.into_unknown(env) +} + +fn bigint_to_unknown<'env>(env: &'env Env, value: &str) -> Result> { + let payload = value.strip_suffix('n').unwrap_or(value).to_string(); + let constructor: Function = + env.get_global()?.get_named_property_unchecked("BigInt")?; + constructor.call(payload) +} + +fn leaf_to_unknown<'env>(env: &'env Env, leaf: &rw::WireLeaf) -> Result> { + match leaf { + rw::WireLeaf::Unserializable(value) => match value.as_str() { + "NaN" => to_unknown(env, f64::NAN), + "Infinity" => to_unknown(env, f64::INFINITY), + "-Infinity" => to_unknown(env, f64::NEG_INFINITY), + "-0" => to_unknown(env, -0.0_f64), + value if value.ends_with('n') => bigint_to_unknown(env, value), + value => unserializable_wrapper(env, value), + }, + rw::WireLeaf::BigInt(value) => bigint_to_unknown(env, value), + rw::WireLeaf::Date(value) => { + let constructor: Function = + env.get_global()?.get_named_property_unchecked("Date")?; + constructor.new_instance(value.clone()) + } + rw::WireLeaf::RegExp { pattern, flags } => { + let constructor: Function, Unknown> = + env.get_global()?.get_named_property_unchecked("RegExp")?; + constructor.new_instance(FnArgs::from((pattern.clone(), flags.clone()))) + } + rw::WireLeaf::Url(value) => { + let constructor: Function = + env.get_global()?.get_named_property_unchecked("URL")?; + constructor.new_instance(value.clone()) + } + rw::WireLeaf::Error { + name, + message, + stack, + } => { + let constructor: Function = + env.get_global()?.get_named_property_unchecked("Error")?; + let value = constructor.new_instance(message.clone())?; + // SAFETY: the Error constructor returns an object for every message. + let mut object = unsafe { value.cast::()? }; + object.set("name", if name.is_empty() { "Error" } else { name })?; + object.set("stack", stack.clone())?; + Ok(value) + } + rw::WireLeaf::Undefined | rw::WireLeaf::Symbol | rw::WireLeaf::Function => { + to_unknown(env, ()) + } + } +} + +#[napi(js_name = "decodeWire")] +pub fn decode_wire<'env>(env: &'env Env, wire_json: String) -> Result> { + let graph = rw::parse_wire_graph(&wire_json).map_err(napi_error)?; + WireMaterializer::new(env, &graph).materialize() +} #[napi(js_name = "chromiumExecutablePath")] pub async fn chromium_executable_path() -> Result> { diff --git a/node/test/decode.test.mjs b/node/test/decode.test.mjs index a1176bc..26eebb2 100644 --- a/node/test/decode.test.mjs +++ b/node/test/decode.test.mjs @@ -3,27 +3,10 @@ import { createRequire } from 'node:module'; import test from 'node:test'; const require = createRequire(import.meta.url); -const nativePath = require.resolve('../native.cjs'); -const cachedNative = require.cache[nativePath]; - -// The decoder is pure, so stub the native binding while loading its CJS export. -require.cache[nativePath] = { - id: nativePath, - filename: nativePath, - loaded: true, - exports: {} -}; - -let decodeWireValue; -try { - ({ _decodeWireValue: decodeWireValue } = require('../index.cjs')); -} finally { - if (cachedNative) require.cache[nativePath] = cachedNative; - else delete require.cache[nativePath]; -} +const { _decodeWireValue: decodeWireValue } = require('../index.cjs'); function decodeFixture(json) { - return decodeWireValue(JSON.parse(json)); + return decodeWireValue(json); } test('decodes core unserializable number and bigint markers', () => { @@ -42,11 +25,18 @@ test('decodes core unserializable number and bigint markers', () => { assert.equal(decodeFixture( '{"__rustwright_cdp_unserializable_value__":"9007199254740993n"}' ), 9007199254740993n); + assert.equal(decodeFixture( + '{"__rustwright_cdp_bigint__":"9007199254740993n"}' + ), 9007199254740993n); + assert.deepEqual( + decodeFixture('{"__rustwright_cdp_unserializable_value__":"future-marker"}'), + { __rustwright_cdp_unserializable_value__: 'future-marker' } + ); }); test('decodes RegExp, Date, URL, and Error wrappers', () => { const regexp = decodeFixture( - '{"__rustwright_cdp_regexp__":{"p":"a+b\\\\s","f":"gi"}}' + '{"__rustwright_cdp_regexp__":{"f":"gi","p":"a+b\\\\s"}}' ); assert.ok(regexp instanceof RegExp); assert.equal(regexp.source, 'a+b\\s'); @@ -65,7 +55,7 @@ test('decodes RegExp, Date, URL, and Error wrappers', () => { assert.equal(url.href, 'https://example.com/path?q=wire#value'); const error = decodeFixture( - '{"__rustwright_cdp_error__":{"name":"TypeError","message":"boom","stack":"TypeError: boom\\n at fixture.js:1:1"}}' + '{"__rustwright_cdp_error__":{"stack":"TypeError: boom\\n at fixture.js:1:1","message":"boom","name":"TypeError"}}' ); assert.ok(error instanceof Error); assert.equal(error.name, 'TypeError'); @@ -88,6 +78,8 @@ test('decodes nested array items and object entries wrappers', () => { const decoded = decodeFixture(`{ "__rustwright_cdp_object__": 1, "entries": { + "z": "last", + "a": "first", "label": "root", "items": { "__rustwright_cdp_array__": 2, @@ -106,6 +98,7 @@ test('decodes nested array items and object entries wrappers', () => { } }`); + assert.deepEqual(Object.keys(decoded).slice(0, 2), ['z', 'a']); assert.equal(decoded.label, 'root'); assert.equal(decoded.items.length, 3); assert.equal(decoded.items[0], 1); @@ -114,6 +107,30 @@ test('decodes nested array items and object entries wrappers', () => { assert.equal(decoded.items[2].date.toISOString(), '2026-01-02T03:04:05.000Z'); }); +test('defines decoded object entries as own data properties', () => { + const decoded = decodeFixture(`{ + "__rustwright_cdp_object__": 1, + "entries": { + "__proto__": { + "__rustwright_cdp_object__": 2, + "entries": { + "isAdmin": true + } + }, + "nul\\u0000key": "nul" + } + }`); + const nulKey = `nul${String.fromCharCode(0)}key`; + + assert.equal(Object.hasOwn(decoded, '__proto__'), true); + assert.equal(Object.hasOwn(decoded.__proto__, 'isAdmin'), true); + assert.equal(decoded.__proto__.isAdmin, true); + assert.equal(Object.getPrototypeOf(decoded), Object.prototype); + assert.equal(decoded.isAdmin, undefined); + assert.equal(Object.hasOwn(decoded, nulKey), true); + assert.equal(decoded[nulKey], 'nul'); +}); + test('resolves ref wrappers for shared values and cycles', () => { const decoded = decodeFixture(`{ "__rustwright_cdp_object__": 1, diff --git a/node/test/timeouts.test.mjs b/node/test/timeouts.test.mjs index 70b1c2c..f26f22f 100644 --- a/node/test/timeouts.test.mjs +++ b/node/test/timeouts.test.mjs @@ -11,7 +11,7 @@ require.cache[nativePath] = { id: nativePath, filename: nativePath, loaded: true, - exports: {} + exports: { decodeWire: JSON.parse } }; let Browser; From 6b670f06add7dfe02f95d93bc5ee9683cf865b69 Mon Sep 17 00:00:00 2001 From: suchintan <3853670+suchintan@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:30:27 +0000 Subject: [PATCH 3/6] =?UTF-8?q?=F0=9F=94=84=20synced=20local=20'python/'?= =?UTF-8?q?=20with=20remote=20'python/'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit https://github.com/Skyvern-AI/rustwright-cloud/pull/232 --- python/rustwright/async_api.py | 4 +- python/rustwright/sync_api.py | 212 ++++++++++++++------------------- 2 files changed, 92 insertions(+), 124 deletions(-) diff --git a/python/rustwright/async_api.py b/python/rustwright/async_api.py index efd4c5b..0e8bb31 100644 --- a/python/rustwright/async_api.py +++ b/python/rustwright/async_api.py @@ -70,7 +70,7 @@ _MISSING, _UNSET, backend_marker, - _decode_json_result, + _decode_json_result_json, _copy_wire_error_metadata, _default_timeout_for_method, _emit_event, @@ -2637,7 +2637,7 @@ async def evaluate(self, expression: str, arg: Any = None) -> Any: self._sync._default_timeout, ) ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) async def add_script_tag( self, diff --git a/python/rustwright/sync_api.py b/python/rustwright/sync_api.py index 3917b9d..b4f24c3 100644 --- a/python/rustwright/sync_api.py +++ b/python/rustwright/sync_api.py @@ -1651,7 +1651,7 @@ def _decode_wire_error(message: str) -> Optional[Error]: ): return None try: - info = _decode_json_result(json.loads(payload["last_info_json"])) + info = _decode_json_result_json(payload["last_info_json"]) except (TypeError, ValueError, json.JSONDecodeError): return None info_key = payload["last_info_key"] @@ -2840,78 +2840,52 @@ def _normalize_nth_index(index: Any) -> Any: return "NaN" -def _decode_json_result(value: Any) -> Any: - references: dict[Any, Any] = {} - - def decode(item: Any) -> Any: - if isinstance(item, dict) and set(item) == {"__rustwright_cdp_ref__"}: - return references.get(item["__rustwright_cdp_ref__"]) - if isinstance(item, dict) and {"__rustwright_cdp_array__", "items"}.issubset(item.keys()): - ref = item.get("__rustwright_cdp_array__") - result: list[Any] = [] - references[ref] = result - values = item.get("items") - if isinstance(values, list): - result.extend(decode(value) for value in values) - return result - if isinstance(item, dict) and {"__rustwright_cdp_object__", "entries"}.issubset(item.keys()): - ref = item.get("__rustwright_cdp_object__") - result: dict[str, Any] = {} - references[ref] = result - entries = item.get("entries") - if isinstance(entries, dict): - for key, value in entries.items(): - result[key] = decode(value) - return result - if isinstance(item, dict) and set(item) == {"__rustwright_cdp_unserializable_value__"}: - marker = item["__rustwright_cdp_unserializable_value__"] - if marker == "NaN": - return float("nan") - if marker == "Infinity": - return float("inf") - if marker == "-Infinity": - return float("-inf") - if marker == "-0": - return -0.0 - if isinstance(marker, str) and marker.endswith("n"): - try: - return int(marker[:-1]) - except ValueError: - pass - if isinstance(item, dict) and set(item) == {"__rustwright_cdp_date__"}: - date_value = item["__rustwright_cdp_date__"] - if isinstance(date_value, str): - return datetime.fromisoformat(date_value.replace("Z", "+00:00")) - if isinstance(item, dict) and set(item) == {"__rustwright_cdp_regexp__"}: - regexp_value = item["__rustwright_cdp_regexp__"] - if isinstance(regexp_value, dict): - return {"r": regexp_value} - if isinstance(item, dict) and set(item) == {"__rustwright_cdp_url__"}: - url_value = item["__rustwright_cdp_url__"] - if isinstance(url_value, str): - return url_parse.urlparse(url_value) - if isinstance(item, dict) and set(item) == {"__rustwright_cdp_error__"}: - error_value = item["__rustwright_cdp_error__"] - if isinstance(error_value, dict): - message = str(error_value.get("message") or "") - error = Error(message) - error._message = message - error._name = str(error_value.get("name") or "Error") - error._stack = str(error_value.get("stack") or "") - return error - if isinstance(item, dict) and set(item) == {"__rustwright_cdp_symbol__"}: - return None - if isinstance(item, dict) and set(item) == {"__rustwright_cdp_function__"}: - return None - if isinstance(item, dict) and set(item) == {"__rustwright_cdp_undefined__"}: - return None - if isinstance(item, list): - return [decode(value) for value in item] - if isinstance(item, dict): - return {key: decode(value) for key, value in item.items()} - return item +def _decode_wire_leaf(tag: str, payload: Any) -> Any: + if tag == "unserializable": + if payload == "NaN": + return float("nan") + if payload == "Infinity": + return float("inf") + if payload == "-Infinity": + return float("-inf") + if payload == "-0": + return -0.0 + return {"__rustwright_cdp_unserializable_value__": payload} + if tag == "bigint": + try: + return int(payload) + except (TypeError, ValueError): + return {"__rustwright_cdp_bigint__": payload} + if tag == "date": + return datetime.fromisoformat(str(payload).replace("Z", "+00:00")) + if tag == "regexp": + if isinstance(payload, dict): + pattern = payload.get("pattern", payload.get("p")) + flags = payload.get("flags", payload.get("f")) + return {"r": {"p": pattern, "f": flags}} + return {"__rustwright_cdp_regexp__": payload} + if tag == "url": + return url_parse.urlparse(str(payload)) + if tag == "error": + if isinstance(payload, dict): + message = str(payload.get("message") or "") + error = Error(message) + error._message = message + error._name = str(payload.get("name") or "Error") + error._stack = str(payload.get("stack") or "") + return error + return {"__rustwright_cdp_error__": payload} + if tag in {"undefined", "symbol", "function"}: + return None + raise ValueError(f"unknown wire leaf tag: {tag}") - return decode(value) + +def _decode_json_result_json(wire_json: str) -> Any: + return _rustwright._decode_wire_value(wire_json, _decode_wire_leaf) + + +def _decode_json_result(value: Any) -> Any: + return _decode_json_result_json(json.dumps(value, separators=(",", ":"))) def _selector_strict(page: "Page", options: Optional[dict[str, Any]] = None) -> bool: @@ -10125,14 +10099,12 @@ def _ensure_not_disposed(self, method: str) -> None: def json_value(self) -> Any: self._ensure_not_disposed("json_value") if self._object_id: - return _decode_json_result( - json.loads( - _call( - self._page._core.js_handle_json_value, - self._object_id, - self._page._default_timeout, - *self._serialized_owner_args(), - ) + return _decode_json_result_json( + _call( + self._page._core.js_handle_json_value, + self._object_id, + self._page._default_timeout, + *self._serialized_owner_args(), ) ) if self._payload.get("type") == "undefined": @@ -10156,7 +10128,7 @@ def _truthy(self, timeout_ms: Optional[float] = None) -> bool: self._page._default_timeout if timeout_ms is None else timeout_ms, *self._serialized_owner_args(), ) - return bool(_decode_json_result(json.loads(result))) + return bool(_decode_json_result_json(result)) if self._payload.get("type") == "undefined" or self._payload.get("subtype") == "null": return False if self._payload.get("unserializableValue") in {"NaN", "-0"}: @@ -10271,7 +10243,7 @@ def _evaluate_with_method( timeout_ms, *self._serialized_owner_args(), ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) finally: prepared.dispose_temporaries() arg = prepared.value if prepared is not None else arg @@ -10286,7 +10258,7 @@ def _evaluate_with_method( timeout_ms, *self._serialized_owner_args(), ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) def evaluate(self, expression: str, arg: Any = None) -> Any: self._ensure_not_disposed("evaluate") @@ -10484,9 +10456,9 @@ def _normalize_launch_options( options = _clean_options( { "headless": ( - True - if headless is None - else _normalize_boolean_option(headless, method=method, name="headless") + _normalize_boolean_option(headless, method=method, name="headless") + if headless is not None + else None ), "executable_path": normalized_executable_path, "args": normalized_args, @@ -12317,7 +12289,7 @@ def _end_action( def _capture_dom_snapshot(self, page: "Page", call_id: str, snapshot_name: str) -> bool: try: result = _call(page._core.evaluate, _TRACE_DOM_SNAPSHOT_JS, None, page._default_timeout) - payload = _decode_json_result(json.loads(result)) + payload = _decode_json_result_json(result) except Exception: return False if not isinstance(payload, dict): @@ -14104,7 +14076,7 @@ def evaluate(self, expression: str, arg: Any = None) -> Any: arg_json, None, ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) if self._frame_id and arg is None: result = _call_with_method_prefix( "Frame.evaluate", @@ -14115,7 +14087,7 @@ def evaluate(self, expression: str, arg: Any = None) -> Any: ) if result is not None: self._uses_direct_evaluation = True - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) if arg is not None and _argument_contains_handle(arg): prepared = _prepare_evaluate_argument(self._page, arg) try: @@ -14130,7 +14102,7 @@ def evaluate(self, expression: str, arg: Any = None) -> Any: None, *anchor._serialized_owner_args(), ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) finally: prepared.dispose_temporaries() if self._frame_spec is not None: @@ -16051,7 +16023,7 @@ def _wait_for_document_state( while time.monotonic() < deadline: remaining = max(1.0, (deadline - time.monotonic()) * 1000) try: - snapshot = _decode_json_result(json.loads(_call( + snapshot = _decode_json_result_json(_call( self._core.evaluate, """() => ({ readyState: document.readyState, @@ -16061,7 +16033,7 @@ def _wait_for_document_state( })""", None, min(250.0, remaining), - ))) + )) except Error: time.sleep(0.02) continue @@ -16578,7 +16550,7 @@ def forward(message: ConsoleMessage) -> None: def _evaluate_history_buffer(self, expression: str, arg: Any) -> Any: try: payload = _call(self._core.evaluate, expression, json.dumps(arg), 250.0) - return _decode_json_result(json.loads(payload)) + return _decode_json_result_json(payload) except (Error, TypeError, ValueError): return None @@ -17313,7 +17285,7 @@ def set_content( None, timeout_ms, ) - content_type = _decode_json_result(json.loads(content_type_result)) + content_type = _decode_json_result_json(content_type_result) except Exception: content_type = None if isinstance(content_type, str): @@ -17888,7 +17860,7 @@ def _evaluate_with_method(self, expression: str, arg: Any = None, *, method: str True, None, ) - value = _decode_json_result(json.loads(result)) + value = _decode_json_result_json(result) finally: prepared.dispose_temporaries() else: @@ -17907,7 +17879,7 @@ def _evaluate_with_method(self, expression: str, arg: Any = None, *, method: str raise value = None else: - value = _decode_json_result(json.loads(result)) + value = _decode_json_result_json(result) except Exception as exc: self._trace_end_action(call_id, error=exc) raise @@ -20506,7 +20478,7 @@ def _eval(self, body: str, timeout: Optional[float] = None, *, method: Optional[ self._page._default_timeout if timeout is None else timeout, ) result = _call_with_method_prefix(method, *args) if method is not None else _call(*args) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) def _try_immediate_aria_snapshot(self, body: str, timeout: Optional[float]) -> Any: if self._simple_css_indexed_read_payload() is None: @@ -20717,7 +20689,7 @@ def _evaluate_simple_css_fast_path( json_module_dumps(payload), self._page._default_timeout if timeout is None else timeout, ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) def _native_locator_fast_path( self, @@ -20742,7 +20714,7 @@ def _native_locator_fast_path( _json(payload), self._page._default_timeout if timeout is None else timeout, ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) def _simple_css_indexed_read_payload(self) -> Optional[dict[str, Any]]: if getattr(self._page, "_locator_handlers", None): @@ -21842,7 +21814,7 @@ def _target_state( _json(options), self._page._default_timeout if timeout is None else timeout, ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) def _fill_apply(self, value: str, *, strict: bool, forced: bool, timeout: float) -> dict[str, Any]: result = _call( @@ -21854,7 +21826,7 @@ def _fill_apply(self, value: str, *, strict: bool, forced: bool, timeout: float) forced, timeout, ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) def _select_apply( self, @@ -21875,7 +21847,7 @@ def _select_apply( _json(indexes), timeout, ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) @staticmethod def _state_matches(info: dict[str, Any], state: str) -> bool: @@ -23930,15 +23902,13 @@ def _checked_state(self, method: str, *, timeout: Optional[float]) -> dict[str, return state def _checked_state_now(self, method: str, *, timeout: Optional[float]) -> dict[str, Any]: - state = _decode_json_result( - json.loads( - _call_with_method_prefix( - method, - self._page._core.locator_check_apply, - _json(self._spec), - self._index, - self._page._default_timeout if timeout is None else timeout, - ) + state = _decode_json_result_json( + _call_with_method_prefix( + method, + self._page._core.locator_check_apply, + _json(self._spec), + self._index, + self._page._default_timeout if timeout is None else timeout, ) ) if not isinstance(state, dict) or state.get("valid") is False: @@ -25999,14 +25969,12 @@ def _up_impl(self, key: str) -> None: self._uncertain_pressed_key = None def _send(self, method: str, params: dict[str, Any]) -> str: - outcome = _decode_json_result( - json.loads( - _call( - self._page._core.keyboard_dispatch_native, - method, - json.dumps(params), - self._page._default_timeout, - ) + outcome = _decode_json_result_json( + _call( + self._page._core.keyboard_dispatch_native, + method, + json.dumps(params), + self._page._default_timeout, ) ) error_message = outcome.get("error") @@ -27081,13 +27049,13 @@ def evaluate(self, expression: str, arg: Any = None) -> Any: True, None, ) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) finally: prepared.dispose_temporaries() arg = prepared.value if prepared is not None else arg arg_json = None if arg is None else json.dumps(arg) result = _call(self._core.evaluate, expression, arg_json, None) - return _decode_json_result(json.loads(result)) + return _decode_json_result_json(result) def evaluate_handle(self, expression: str, arg: Any = None) -> JSHandle: if self._core is None: @@ -27357,7 +27325,7 @@ def _native_locator_poll( raise result: dict[str, Any] = {"passed": False, "actual": None, "log": ""} else: - decoded = _decode_json_result(json.loads(result_json)) + decoded = _decode_json_result_json(result_json) if not isinstance(decoded, dict): raise Error(f"{api_method}: native assertion returned an invalid result") result = decoded From 9aaf5979c0bcd830058aa9e40ac8a5b53d635051 Mon Sep 17 00:00:00 2001 From: suchintan <3853670+suchintan@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:30:27 +0000 Subject: [PATCH 4/6] =?UTF-8?q?=F0=9F=94=84=20synced=20local=20'src/'=20wi?= =?UTF-8?q?th=20remote=20'src/'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit https://github.com/Skyvern-AI/rustwright-cloud/pull/232 --- src/lib.rs | 858 ++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 851 insertions(+), 7 deletions(-) diff --git a/src/lib.rs b/src/lib.rs index cd12936..9aca07b 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -28,8 +28,11 @@ use pyo3::exceptions::{PyRuntimeError, PyValueError}; #[cfg(feature = "python")] use pyo3::prelude::*; #[cfg(feature = "python")] -use pyo3::types::{PyAny, PyBytes, PyModule}; -use serde::{Deserialize, Serialize}; +use pyo3::types::{PyAny, PyBytes, PyDict, PyList, PyModule}; +#[cfg(feature = "python")] +use pyo3::IntoPyObjectExt; +use serde::de::{MapAccess, SeqAccess, Visitor}; +use serde::{Deserialize, Deserializer, Serialize}; use serde_json::{json, Value}; use tempfile::{NamedTempFile, TempDir}; use thiserror::Error; @@ -22724,6 +22727,15 @@ return this.dataset.mainWorldOverride === "observed"; assert!(error.to_string().contains("duplicate field")); } + + #[cfg(feature = "python")] + #[test] + fn python_launch_option_parse_errors_remain_value_errors() { + Python::attach(|py| { + let error = parse_python_launch_options("{").unwrap_err(); + assert!(error.is_instance_of::(py)); + }); + } #[test] fn remote_discovery_url_derivation_preserves_query_and_removes_known_suffix() { for endpoint in [ @@ -23051,6 +23063,14 @@ struct LaunchOptions { proxy: Option, } +fn parse_launch_options(json: &str) -> RwResult { + serde_json::from_str(json).map_err(RwError::from) +} +#[cfg(feature = "python")] +fn parse_python_launch_options(json: &str) -> PyResult { + parse_launch_options(json).map_err(|error| PyValueError::new_err(error.to_string())) +} + #[derive(Debug, Deserialize, PartialEq)] struct ProxyOptions { server: String, @@ -40881,8 +40901,7 @@ async fn wait_for_launch_cancellation(cancelled: Arc) { #[cfg(feature = "python")] #[pyfunction] fn launch_chromium(py: Python<'_>, options_json: &str) -> PyResult { - let options: LaunchOptions = serde_json::from_str(options_json) - .map_err(|error| PyValueError::new_err(error.to_string()))?; + let options = parse_python_launch_options(options_json)?; let inner = py .detach(move || { launch_chromium_with_options_cancellation( @@ -40899,8 +40918,7 @@ fn launch_chromium(py: Python<'_>, options_json: &str) -> PyResult { #[cfg(feature = "python")] #[pyfunction] fn launch_chromium_async(py: Python<'_>, options_json: &str) -> PyResult> { - let options: LaunchOptions = serde_json::from_str(options_json) - .map_err(|error| PyValueError::new_err(error.to_string()))?; + let options = parse_python_launch_options(options_json)?; python_future_on_thread( py, move |cancelled| { @@ -41786,7 +41804,7 @@ pub fn rustwright_launch_chromium_with_cancel( options_json: &str, cancel: Option<&CancelToken>, ) -> RwResult { - let options: LaunchOptions = serde_json::from_str(options_json)?; + let options = parse_launch_options(options_json)?; let result = match cancel { Some(cancel) => launch_chromium_with_options_token(options, cancel.clone()), None => launch_chromium_with_options(options), @@ -53163,6 +53181,628 @@ const WIRE_LEAF_TAGS: [&str; 9] = [ "__rustwright_cdp_function__", ]; +#[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)] +pub struct WireNodeId(usize); + +impl WireNodeId { + pub fn index(self) -> usize { + self.0 + } +} + +#[derive(Clone, Copy, Debug, PartialEq)] +pub enum WireNumber { + Signed(i64), + Unsigned(u64), + Float(f64), +} + +#[derive(Clone, Debug, PartialEq)] +pub enum WireLeaf { + Unserializable(String), + BigInt(String), + Date(String), + RegExp { + pattern: String, + flags: String, + }, + Url(String), + Error { + name: String, + message: String, + stack: String, + }, + Undefined, + Symbol, + Function, +} + +#[derive(Clone, Debug, PartialEq)] +pub enum WireNodeKind { + Null, + Bool(bool), + Number(WireNumber), + String(String), + Array(Vec), + Object(Vec<(String, WireNodeId)>), + Leaf(WireLeaf), +} + +#[derive(Clone, Debug, PartialEq)] +pub struct WireGraph { + root: WireNodeId, + nodes: Vec, +} + +impl WireGraph { + pub fn root(&self) -> WireNodeId { + self.root + } + + pub fn nodes(&self) -> &[WireNodeKind] { + &self.nodes + } + + pub fn node(&self, id: WireNodeId) -> Option<&WireNodeKind> { + self.nodes.get(id.index()) + } +} + +enum WireJsonValue { + Null, + Bool(bool), + Number(WireNumber), + String(String), + Array(Vec), + Object(Vec<(String, WireJsonValue)>), +} + +impl<'de> Deserialize<'de> for WireJsonValue { + fn deserialize(deserializer: D) -> Result + where + D: Deserializer<'de>, + { + struct WireJsonValueVisitor; + + impl<'de> Visitor<'de> for WireJsonValueVisitor { + type Value = WireJsonValue; + + fn expecting(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter.write_str("a JSON value") + } + + fn visit_unit(self) -> Result + where + E: serde::de::Error, + { + Ok(WireJsonValue::Null) + } + + fn visit_bool(self, value: bool) -> Result + where + E: serde::de::Error, + { + Ok(WireJsonValue::Bool(value)) + } + + fn visit_i64(self, value: i64) -> Result + where + E: serde::de::Error, + { + Ok(WireJsonValue::Number(WireNumber::Signed(value))) + } + + fn visit_u64(self, value: u64) -> Result + where + E: serde::de::Error, + { + Ok(WireJsonValue::Number(WireNumber::Unsigned(value))) + } + + fn visit_f64(self, value: f64) -> Result + where + E: serde::de::Error, + { + Ok(WireJsonValue::Number(WireNumber::Float(value))) + } + + fn visit_str(self, value: &str) -> Result + where + E: serde::de::Error, + { + Ok(WireJsonValue::String(value.to_string())) + } + + fn visit_string(self, value: String) -> Result + where + E: serde::de::Error, + { + Ok(WireJsonValue::String(value)) + } + + fn visit_seq(self, mut sequence: A) -> Result + where + A: SeqAccess<'de>, + { + let mut values = Vec::with_capacity(sequence.size_hint().unwrap_or(0)); + while let Some(value) = sequence.next_element()? { + values.push(value); + } + Ok(WireJsonValue::Array(values)) + } + + fn visit_map(self, mut map: A) -> Result + where + A: MapAccess<'de>, + { + let mut entries = Vec::with_capacity(map.size_hint().unwrap_or(0)); + while let Some(entry) = map.next_entry()? { + entries.push(entry); + } + Ok(WireJsonValue::Object(entries)) + } + } + + deserializer.deserialize_any(WireJsonValueVisitor) + } +} + +type WireJsonObject = Vec<(String, WireJsonValue)>; + +#[derive(Clone, Copy)] +enum WireStructuralKind { + Ref, + Array, + Object, +} + +fn wire_json_object_get<'a>(object: &'a WireJsonObject, key: &str) -> Option<&'a WireJsonValue> { + object + .iter() + .rev() + .find_map(|(name, value)| (name == key).then_some(value)) +} + +fn wire_json_object_contains(object: &WireJsonObject, key: &str) -> bool { + object.iter().any(|(name, _)| name == key) +} + +fn wire_structural_kind(object: &WireJsonObject) -> RwResult> { + let mut result = None; + for (tag, kind) in [ + (WIRE_REF_TAG, WireStructuralKind::Ref), + (WIRE_ARRAY_TAG, WireStructuralKind::Array), + (WIRE_OBJECT_TAG, WireStructuralKind::Object), + ] { + if wire_json_object_contains(object, tag) { + if result.is_some() { + return Err(RwError::InvalidInput( + "wire wrapper contains multiple structural tags".to_string(), + )); + } + result = Some(kind); + } + } + Ok(result) +} + +fn wire_leaf_value(object: &WireJsonObject) -> RwResult> { + let tags = WIRE_LEAF_TAGS + .iter() + .copied() + .filter(|tag| wire_json_object_contains(object, tag)) + .collect::>(); + if tags.is_empty() { + return Ok(None); + } + if tags.len() != 1 || object.len() != 1 { + return Err(RwError::InvalidInput( + "wire leaf wrapper must contain exactly one leaf tag".to_string(), + )); + } + let tag = tags[0]; + let payload = wire_json_object_get(object, tag).expect("leaf tag was found"); + let string_payload = |message: &str| { + if let WireJsonValue::String(value) = payload { + Ok(value.clone()) + } else { + Err(RwError::InvalidInput(message.to_string())) + } + }; + let leaf = match tag { + "__rustwright_cdp_unserializable_value__" => { + let payload = string_payload("wire unserializable leaf payload must be a string")?; + if let Some(value) = payload.strip_suffix('n') { + WireLeaf::BigInt(value.to_string()) + } else { + WireLeaf::Unserializable(payload) + } + } + "__rustwright_cdp_bigint__" => { + let payload = string_payload("wire bigint leaf payload must be a string")?; + WireLeaf::BigInt(payload.strip_suffix('n').unwrap_or(&payload).to_string()) + } + "__rustwright_cdp_date__" => { + WireLeaf::Date(string_payload("wire date leaf payload must be a string")?) + } + "__rustwright_cdp_regexp__" => { + let regexp = match payload { + WireJsonValue::Object(regexp) => regexp, + _ => { + return Err(RwError::InvalidInput( + "wire regexp leaf payload must be an object".to_string(), + )); + } + }; + if regexp.len() != 2 { + return Err(RwError::InvalidInput( + "wire regexp leaf payload must contain pattern and flags".to_string(), + )); + } + let string_field = |name: &str| { + wire_json_object_get(regexp, name).and_then(|value| match value { + WireJsonValue::String(value) => Some(value.clone()), + _ => None, + }) + }; + let pattern = string_field("p") + .or_else(|| string_field("pattern")) + .ok_or_else(|| { + RwError::InvalidInput("wire regexp pattern must be a string".to_string()) + })?; + let flags = string_field("f") + .or_else(|| string_field("flags")) + .ok_or_else(|| { + RwError::InvalidInput("wire regexp flags must be a string".to_string()) + })?; + WireLeaf::RegExp { pattern, flags } + } + "__rustwright_cdp_url__" => { + WireLeaf::Url(string_payload("wire url leaf payload must be a string")?) + } + "__rustwright_cdp_error__" => { + let error = match payload { + WireJsonValue::Object(error) => error, + _ => { + return Err(RwError::InvalidInput( + "wire error leaf payload must be an object".to_string(), + )); + } + }; + if error.len() != 3 { + return Err(RwError::InvalidInput( + "wire error leaf payload must contain name, message, and stack".to_string(), + )); + } + let string_field = |name: &str| { + wire_json_object_get(error, name) + .and_then(|value| match value { + WireJsonValue::String(value) => Some(value.clone()), + _ => None, + }) + .ok_or_else(|| { + RwError::InvalidInput(format!("wire error field must be a string: {name}")) + }) + }; + WireLeaf::Error { + name: string_field("name")?, + message: string_field("message")?, + stack: string_field("stack")?, + } + } + "__rustwright_cdp_undefined__" => WireLeaf::Undefined, + "__rustwright_cdp_symbol__" => WireLeaf::Symbol, + "__rustwright_cdp_function__" => WireLeaf::Function, + _ => unreachable!("all wire leaf tags are listed in WIRE_LEAF_TAGS"), + }; + Ok(Some(leaf)) +} + +fn wire_graph_reference_key(value: &WireJsonValue) -> RwResult { + match value { + WireJsonValue::Number(WireNumber::Signed(value)) => Ok(value.to_string()), + WireJsonValue::Number(WireNumber::Unsigned(value)) => Ok(value.to_string()), + WireJsonValue::Number(WireNumber::Float(value)) => { + serde_json::to_string(value).map_err(RwError::from) + } + WireJsonValue::String(value) => serde_json::to_string(value).map_err(RwError::from), + _ => Err(RwError::InvalidInput( + "wire reference id must be a number or string".to_string(), + )), + } +} + +fn wire_wrapper_id(object: &WireJsonObject, tag: &str) -> RwResult { + wire_graph_reference_key(wire_json_object_get(object, tag).ok_or_else(|| { + RwError::InvalidInput(format!("wire wrapper is missing its id tag: {tag}")) + })?) +} + +fn wire_array_items(object: &WireJsonObject) -> RwResult<&[WireJsonValue]> { + if object.len() != 2 { + return Err(RwError::InvalidInput( + "wire array wrapper must contain only id and items".to_string(), + )); + } + match wire_json_object_get(object, "items") { + Some(WireJsonValue::Array(items)) => Ok(items), + _ => Err(RwError::InvalidInput( + "wire array wrapper must contain an items array".to_string(), + )), + } +} + +fn wire_object_entries(object: &WireJsonObject) -> RwResult<&WireJsonObject> { + if object.len() != 2 { + return Err(RwError::InvalidInput( + "wire object wrapper must contain only id and entries".to_string(), + )); + } + match wire_json_object_get(object, "entries") { + Some(WireJsonValue::Object(entries)) => Ok(entries), + _ => Err(RwError::InvalidInput( + "wire object wrapper must contain an entries object".to_string(), + )), + } +} + +struct WireGraphParser { + definitions: HashMap, + built: HashSet, + nodes: Vec, +} + +impl WireGraphParser { + fn new() -> Self { + Self { + definitions: HashMap::new(), + built: HashSet::new(), + nodes: Vec::new(), + } + } + + fn reserve_definition(&mut self, key: String) -> RwResult { + if self.definitions.contains_key(&key) { + return Err(RwError::InvalidInput(format!( + "wire reference id is defined more than once: {key}" + ))); + } + let id = WireNodeId(self.nodes.len()); + self.nodes.push(WireNodeKind::Null); + self.definitions.insert(key, id); + Ok(id) + } + + fn collect_definitions(&mut self, value: &WireJsonValue) -> RwResult<()> { + match value { + WireJsonValue::Array(values) => { + for value in values { + self.collect_definitions(value)?; + } + } + WireJsonValue::Object(object) => { + if wire_leaf_value(object)?.is_some() { + return Ok(()); + } + match wire_structural_kind(object)? { + None => { + for (_, value) in object { + self.collect_definitions(value)?; + } + } + Some(WireStructuralKind::Ref) => { + if object.len() != 1 { + return Err(RwError::InvalidInput( + "wire ref wrapper must contain only its id".to_string(), + )); + } + wire_wrapper_id(object, WIRE_REF_TAG)?; + } + Some(WireStructuralKind::Array) => { + let key = wire_wrapper_id(object, WIRE_ARRAY_TAG)?; + let items = wire_array_items(object)?; + self.reserve_definition(key)?; + for item in items { + self.collect_definitions(item)?; + } + } + Some(WireStructuralKind::Object) => { + let key = wire_wrapper_id(object, WIRE_OBJECT_TAG)?; + let entries = wire_object_entries(object)?; + self.reserve_definition(key)?; + for (_, value) in entries { + self.collect_definitions(value)?; + } + } + } + } + _ => {} + } + Ok(()) + } + + fn allocate(&mut self, kind: WireNodeKind) -> WireNodeId { + let id = WireNodeId(self.nodes.len()); + self.nodes.push(kind); + id + } + + fn definition(&self, key: &str) -> RwResult { + self.definitions.get(key).copied().ok_or_else(|| { + RwError::InvalidInput(format!("wire reference points to unknown id: {key}")) + }) + } + + fn build_value(&mut self, value: &WireJsonValue) -> RwResult { + match value { + WireJsonValue::Null => Ok(self.allocate(WireNodeKind::Null)), + WireJsonValue::Bool(value) => Ok(self.allocate(WireNodeKind::Bool(*value))), + WireJsonValue::Number(value) => Ok(self.allocate(WireNodeKind::Number(*value))), + WireJsonValue::String(value) => Ok(self.allocate(WireNodeKind::String(value.clone()))), + WireJsonValue::Array(values) => { + let id = self.allocate(WireNodeKind::Array(Vec::new())); + let children = values + .iter() + .map(|value| self.build_value(value)) + .collect::>>()?; + self.nodes[id.index()] = WireNodeKind::Array(children); + Ok(id) + } + WireJsonValue::Object(object) => { + if let Some(leaf) = wire_leaf_value(object)? { + return Ok(self.allocate(WireNodeKind::Leaf(leaf))); + } + match wire_structural_kind(object)? { + None => { + let id = self.allocate(WireNodeKind::Object(Vec::new())); + let mut entries = Vec::with_capacity(object.len()); + for (key, value) in object { + entries.push((key.clone(), self.build_value(value)?)); + } + self.nodes[id.index()] = WireNodeKind::Object(entries); + Ok(id) + } + Some(WireStructuralKind::Ref) => { + if object.len() != 1 { + return Err(RwError::InvalidInput( + "wire ref wrapper must contain only its id".to_string(), + )); + } + let key = wire_wrapper_id(object, WIRE_REF_TAG)?; + self.definition(&key) + } + Some(WireStructuralKind::Array) => { + let key = wire_wrapper_id(object, WIRE_ARRAY_TAG)?; + let id = self.definition(&key)?; + if self.built.insert(id) { + let items = wire_array_items(object)?; + let children = items + .iter() + .map(|value| self.build_value(value)) + .collect::>>()?; + self.nodes[id.index()] = WireNodeKind::Array(children); + } + Ok(id) + } + Some(WireStructuralKind::Object) => { + let key = wire_wrapper_id(object, WIRE_OBJECT_TAG)?; + let id = self.definition(&key)?; + if self.built.insert(id) { + let entries = wire_object_entries(object)?; + let mut children = Vec::with_capacity(entries.len()); + for (key, value) in entries { + children.push((key.clone(), self.build_value(value)?)); + } + self.nodes[id.index()] = WireNodeKind::Object(children); + } + Ok(id) + } + } + } + } + } +} + +pub fn parse_wire_graph(json: &str) -> RwResult { + let value = serde_json::from_str::(json)?; + let mut parser = WireGraphParser::new(); + parser.collect_definitions(&value)?; + let root = parser.build_value(&value)?; + Ok(WireGraph { + root, + nodes: parser.nodes, + }) +} +#[cfg(feature = "python")] +fn py_wire_leaf(py: Python<'_>, callback: &Py, leaf: &WireLeaf) -> PyResult> { + let (tag, payload) = match leaf { + WireLeaf::Unserializable(value) => ( + "unserializable", + value.clone().into_pyobject(py)?.unbind().into_any(), + ), + WireLeaf::BigInt(value) => ( + "bigint", + value.clone().into_pyobject(py)?.unbind().into_any(), + ), + WireLeaf::Date(value) => ("date", value.clone().into_pyobject(py)?.unbind().into_any()), + WireLeaf::RegExp { pattern, flags } => { + let payload = PyDict::new(py); + payload.set_item("pattern", pattern)?; + payload.set_item("flags", flags)?; + ("regexp", payload.unbind().into_any()) + } + WireLeaf::Url(value) => ("url", value.clone().into_pyobject(py)?.unbind().into_any()), + WireLeaf::Error { + name, + message, + stack, + } => { + let payload = PyDict::new(py); + payload.set_item("name", name)?; + payload.set_item("message", message)?; + payload.set_item("stack", stack)?; + ("error", payload.unbind().into_any()) + } + WireLeaf::Undefined => ("undefined", py.None()), + WireLeaf::Symbol => ("symbol", py.None()), + WireLeaf::Function => ("function", py.None()), + }; + Ok(callback + .bind(py) + .call1((tag, payload.bind(py)))? + .unbind() + .into_any()) +} + +#[cfg(feature = "python")] +fn py_wire_node(py: Python<'_>, callback: &Py, kind: &WireNodeKind) -> PyResult> { + match kind { + WireNodeKind::Null => Ok(py.None()), + WireNodeKind::Bool(value) => (*value).into_py_any(py), + WireNodeKind::Number(WireNumber::Signed(value)) => (*value).into_py_any(py), + WireNodeKind::Number(WireNumber::Unsigned(value)) => (*value).into_py_any(py), + WireNodeKind::Number(WireNumber::Float(value)) => (*value).into_py_any(py), + WireNodeKind::String(value) => value.clone().into_py_any(py), + WireNodeKind::Array(_) => Ok(PyList::empty(py).unbind().into_any()), + WireNodeKind::Object(_) => Ok(PyDict::new(py).unbind().into_any()), + WireNodeKind::Leaf(leaf) => py_wire_leaf(py, callback, leaf), + } +} + +#[cfg(feature = "python")] +#[pyfunction] +fn _decode_wire_value( + py: Python<'_>, + wire_json: &str, + leaf_callback: Py, +) -> PyResult> { + let graph = parse_wire_graph(wire_json).map_err(py_err)?; + let callback = leaf_callback; + let values = graph + .nodes() + .iter() + .map(|kind| py_wire_node(py, &callback, kind)) + .collect::>>()?; + for (index, kind) in graph.nodes().iter().enumerate() { + match kind { + WireNodeKind::Array(children) => { + let list = values[index].bind(py).cast::()?; + for child in children { + list.append(values[child.index()].bind(py))?; + } + } + WireNodeKind::Object(entries) => { + let object = values[index].bind(py).cast::()?; + for (key, child) in entries { + object.set_item(key, values[child.index()].bind(py))?; + } + } + _ => {} + } + } + Ok(values[graph.root().index()].clone_ref(py)) +} + #[derive(Clone)] enum WireDefinition { Array(Vec), @@ -53423,6 +54063,209 @@ mod wire_decode_tests { assert!(matches!(error, RwError::Json(_))); } + #[test] + fn parses_nested_values_and_all_wire_leaves() { + let graph = parse_wire_graph( + r#"{ + "__rustwright_cdp_object__": 1, + "entries": { + "nested": { + "__rustwright_cdp_array__": 2, + "items": [ + null, + true, + -7, + 1.25, + "text", + {"__rustwright_cdp_unserializable_value__": "NaN"}, + {"__rustwright_cdp_bigint__": "123n"}, + {"__rustwright_cdp_date__": "2026-07-21T12:34:56.789Z"}, + {"__rustwright_cdp_regexp__": {"p": "a+b", "f": "gi"}}, + {"__rustwright_cdp_url__": "https://example.com/path"}, + {"__rustwright_cdp_error__": { + "name": "TypeError", + "message": "broken", + "stack": "TypeError: broken" + }}, + {"__rustwright_cdp_undefined__": true}, + {"__rustwright_cdp_symbol__": true}, + {"__rustwright_cdp_function__": true} + ] + } + } + }"#, + ) + .unwrap(); + let WireNodeKind::Object(entries) = graph.node(graph.root()).unwrap() else { + panic!("root must be an object"); + }; + let nested = entries + .iter() + .find(|(key, _)| key == "nested") + .map(|(_, id)| *id) + .unwrap(); + let WireNodeKind::Array(items) = graph.node(nested).unwrap() else { + panic!("nested value must be an array"); + }; + assert!(matches!(graph.node(items[0]), Some(WireNodeKind::Null))); + assert!(matches!( + graph.node(items[1]), + Some(WireNodeKind::Bool(true)) + )); + assert!(matches!( + graph.node(items[2]), + Some(WireNodeKind::Number(WireNumber::Signed(-7))) + )); + assert!(matches!( + graph.node(items[3]), + Some(WireNodeKind::Number(WireNumber::Float(value))) if (*value - 1.25).abs() < f64::EPSILON + )); + assert!(matches!( + graph.node(items[4]), + Some(WireNodeKind::String(value)) if value == "text" + )); + assert!(matches!( + graph.node(items[5]), + Some(WireNodeKind::Leaf(WireLeaf::Unserializable(value))) if value == "NaN" + )); + assert!(matches!( + graph.node(items[6]), + Some(WireNodeKind::Leaf(WireLeaf::BigInt(value))) if value == "123" + )); + assert!(matches!( + graph.node(items[7]), + Some(WireNodeKind::Leaf(WireLeaf::Date(value))) if value == "2026-07-21T12:34:56.789Z" + )); + assert!(matches!( + graph.node(items[8]), + Some(WireNodeKind::Leaf(WireLeaf::RegExp { pattern, flags })) + if pattern == "a+b" && flags == "gi" + )); + assert!(matches!( + graph.node(items[9]), + Some(WireNodeKind::Leaf(WireLeaf::Url(value))) if value == "https://example.com/path" + )); + assert!(matches!( + graph.node(items[10]), + Some(WireNodeKind::Leaf(WireLeaf::Error { name, message, stack })) + if name == "TypeError" && message == "broken" && stack == "TypeError: broken" + )); + assert!(matches!( + graph.node(items[11]), + Some(WireNodeKind::Leaf(WireLeaf::Undefined)) + )); + assert!(matches!( + graph.node(items[12]), + Some(WireNodeKind::Leaf(WireLeaf::Symbol)) + )); + assert!(matches!( + graph.node(items[13]), + Some(WireNodeKind::Leaf(WireLeaf::Function)) + )); + } + + #[test] + fn preserves_plain_and_wrapped_object_insertion_order() { + let graph = parse_wire_graph( + r#"{ + "z": 1, + "a": 2, + "wrapped": { + "__rustwright_cdp_object__": "obj", + "entries": { + "z": { + "__rustwright_cdp_regexp__": {"f": "gi", "p": "a+b"} + }, + "a": { + "__rustwright_cdp_error__": { + "stack": "TypeError: broken", + "message": "broken", + "name": "TypeError" + } + } + } + } + }"#, + ) + .unwrap(); + + let WireNodeKind::Object(entries) = graph.node(graph.root()).unwrap() else { + panic!("root must be an object"); + }; + assert_eq!( + entries + .iter() + .map(|(key, _)| key.as_str()) + .collect::>(), + ["z", "a", "wrapped"] + ); + let wrapped = entries + .iter() + .find(|(key, _)| key == "wrapped") + .map(|(_, id)| *id) + .unwrap(); + let WireNodeKind::Object(wrapped_entries) = graph.node(wrapped).unwrap() else { + panic!("wrapped value must be an object"); + }; + assert_eq!( + wrapped_entries + .iter() + .map(|(key, _)| key.as_str()) + .collect::>(), + ["z", "a"] + ); + assert!(matches!( + graph.node(wrapped_entries[0].1), + Some(WireNodeKind::Leaf(WireLeaf::RegExp { pattern, flags })) + if pattern == "a+b" && flags == "gi" + )); + assert!(matches!( + graph.node(wrapped_entries[1].1), + Some(WireNodeKind::Leaf(WireLeaf::Error { name, message, stack })) + if name == "TypeError" && message == "broken" && stack == "TypeError: broken" + )); + } + + #[test] + fn preserves_repeated_identity_cycles_and_forward_references() { + let graph = parse_wire_graph( + r#"{ + "__rustwright_cdp_array__": 1, + "items": [ + {"__rustwright_cdp_ref__": 2}, + { + "__rustwright_cdp_object__": 2, + "entries": {"self": {"__rustwright_cdp_ref__": 2}} + } + ] + }"#, + ) + .unwrap(); + let WireNodeKind::Array(items) = graph.node(graph.root()).unwrap() else { + panic!("root must be an array"); + }; + assert_eq!(items[0], items[1]); + let WireNodeKind::Object(entries) = graph.node(items[0]).unwrap() else { + panic!("repeated value must be an object"); + }; + assert_eq!(entries[0].1, items[0]); + } + + #[test] + fn rejects_duplicate_unknown_and_malformed_wire_wrappers() { + for input in [ + r#"[{"__rustwright_cdp_array__": 1, "items": []}, {"__rustwright_cdp_array__": 1, "items": []}]"#, + r#"{"__rustwright_cdp_ref__": 999}"#, + r#"{"__rustwright_cdp_array__": 1}"#, + r#"{"__rustwright_cdp_object__": 1, "entries": [], "extra": true}"#, + r#"{"__rustwright_cdp_ref__": 1, "extra": true}"#, + ] { + assert!(matches!( + parse_wire_graph(input), + Err(RwError::InvalidInput(_)) + )); + } + } } const RUNTIME_VALUE_SERIALIZER: &str = r#"(function (value) { @@ -56317,6 +57160,7 @@ fn _rustwright(py: Python<'_>, module: &Bound<'_, PyModule>) -> PyResult<()> { module.add_class::()?; module.add_class::()?; module.add_class::()?; + module.add_function(wrap_pyfunction!(_decode_wire_value, module)?)?; module.add_function(wrap_pyfunction!(launch_chromium, module)?)?; module.add_function(wrap_pyfunction!(launch_chromium_async, module)?)?; module.add_function(wrap_pyfunction!(connect_over_cdp, module)?)?; From d959764c12b2d5bdf2805a22b883776804fd14ca Mon Sep 17 00:00:00 2001 From: suchintan <3853670+suchintan@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:30:27 +0000 Subject: [PATCH 5/6] =?UTF-8?q?=F0=9F=94=84=20synced=20local=20'tests/'=20?= =?UTF-8?q?with=20remote=20'tests/'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit https://github.com/Skyvern-AI/rustwright-cloud/pull/232 --- tests/test_rustwright_sync_api.py | 63 +++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/tests/test_rustwright_sync_api.py b/tests/test_rustwright_sync_api.py index c0a5014..e207fdb 100644 --- a/tests/test_rustwright_sync_api.py +++ b/tests/test_rustwright_sync_api.py @@ -47682,3 +47682,66 @@ async def run() -> None: await browser.close() asyncio.run(run()) + + +def test_native_wire_decoder_preserves_identity_cycles_and_leaf_types(): + from rustwright import _rustwright + from rustwright.sync_api import _decode_json_result_json + + wire = { + "__rustwright_cdp_array__": 1, + "items": [ + { + "__rustwright_cdp_object__": 2, + "entries": { + "z": 1, + "a": 2, + "self": {"__rustwright_cdp_ref__": 2}, + "leaves": { + "__rustwright_cdp_array__": 3, + "items": [ + {"__rustwright_cdp_unserializable_value__": "NaN"}, + {"__rustwright_cdp_bigint__": "123n"}, + {"__rustwright_cdp_date__": "2026-07-21T12:34:56.789Z"}, + {"__rustwright_cdp_regexp__": {"f": "gi", "p": "a+b"}}, + {"__rustwright_cdp_url__": "https://example.com/path?q=1"}, + { + "__rustwright_cdp_error__": { + "stack": "TypeError: broken", + "message": "broken", + "name": "TypeError", + } + }, + {"__rustwright_cdp_undefined__": True}, + {"__rustwright_cdp_symbol__": True}, + {"__rustwright_cdp_function__": True}, + ], + }, + }, + }, + {"__rustwright_cdp_ref__": 2}, + ], + } + + decoded = _decode_json_result_json(json.dumps(wire)) + assert decoded[0] is decoded[1] + assert list(decoded[0])[:2] == ["z", "a"] + assert decoded[0]["self"] is decoded[0] + leaves = decoded[0]["leaves"] + assert math.isnan(leaves[0]) + assert leaves[1] == 123 + assert leaves[2] == datetime(2026, 7, 21, 12, 34, 56, 789000, tzinfo=timezone.utc) + assert leaves[3] == {"r": {"p": "a+b", "f": "gi"}} + assert leaves[4] == urlparse("https://example.com/path?q=1") + assert leaves[5]._name == "TypeError" + assert leaves[5]._message == "broken" + assert leaves[5]._stack == "TypeError: broken" + assert leaves[6:] == [None, None, None] + assert hasattr(_rustwright, "_decode_wire_value") + + +def test_native_launch_parser_preserves_value_error(): + from rustwright import _rustwright + + with pytest.raises(ValueError): + _rustwright.launch_chromium("{") From c0cf185401197a3b9d32303cfd178eca77980a61 Mon Sep 17 00:00:00 2001 From: suchintan <3853670+suchintan@users.noreply.github.com> Date: Sun, 30 Aug 2026 18:30:27 +0000 Subject: [PATCH 6/6] =?UTF-8?q?=F0=9F=94=84=20synced=20local=20'CODE=5FARC?= =?UTF-8?q?HITECTURE.md'=20with=20remote=20'CODE=5FARCHITECTURE.md'?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit https://github.com/Skyvern-AI/rustwright-cloud/pull/232 --- CODE_ARCHITECTURE.md | 159 ++++++++++++++++++++----------------------- 1 file changed, 72 insertions(+), 87 deletions(-) diff --git a/CODE_ARCHITECTURE.md b/CODE_ARCHITECTURE.md index 4fa8bd7..170ee8e 100644 --- a/CODE_ARCHITECTURE.md +++ b/CODE_ARCHITECTURE.md @@ -22,27 +22,34 @@ Last updated: 2026-07-21 ## Shim Lightness Principle The single most expensive architecture failure mode in this repo is engine -logic accreting inside one language shim. It gets rewritten N times as other -bindings mature, and the copies drift. Both failure modes are no longer -hypothetical: +logic accreting inside one language shim. It gets rewritten as other bindings +mature, and the copies drift. Two concrete failures shaped the current split: - The remote-CDP premature-timeout bug (#96) existed because the actionability deadline lived in Python while the per-probe cap lived in the core — two - timeout engines disagreeing across the FFI boundary. -- The Node evaluate decoder silently drifted from the core serializer (it read - `__rustwright_cdp_number__` and `pattern`/`flags` where the core emits - `__rustwright_cdp_unserializable_value__` and `p`/`f`), so NaN/BigInt - results leaked as raw wrapper objects and every RegExp decoded as `//`. - Seven hand-written decoder copies existed across the bindings when this was - caught. + timeout engines disagreed across the FFI boundary. +- The Node evaluate decoder used an outdated tag vocabulary. That Node tag drift + is fixed. + +The compatibility split is intentional: + +- Core `decode_wire_value` and C ABI `rw_decode_wire` remain the flattened plain + JSON path. They duplicate repeated references and use the cycle sentinel. +- Go and rust-native already use core structural decoding. +- Python and Node use native graph adapters that preserve host object identity + and cycles. +- Java, C#/.NET, Ruby, and PHP retain full structural copies until the + remaining C-ABI binding migration. The rule, applied to every change: 1. Anything expressible as a pure function of JSON-in/JSON-out — option normalization and defaulting, evaluate-wire encoding/decoding, timeout-precedence resolution, data-URL construction, structural result - comparison — is implemented once in `rustwright-core` and exposed through - the PyO3/napi/C-ABI surfaces. + comparison, and graph parsing — is implemented once in + `rustwright-core` and exposed through the relevant binding surfaces. The C + ABI keeps its flattened decoder for compatibility; native graph adapters + preserve identity and cycles. 2. Anything that owns a deadline, poll cadence, retry policy, or CDP round-trip sequencing lives in the core. A shim never contains a wait loop whose correctness depends on transport latency. @@ -76,88 +83,68 @@ C-ABI bindings. | `capi/` | Shared C ABI (`librustwright_capi`) over `rustwright-core`; the boundary for the Go/Java/C#/Ruby/PHP bindings. | | `go/`, `java/`, `csharp/`, `ruby/`, `php/` | C-ABI language bindings (alpha surface) + per-language conformance runners. | | `bindings/` | Cross-binding contract (`CONTRACT.md`) and shared conformance case data. | -| `benchmarks/automation_cases.py` | 408 shared Playwright-style automation/parity cases and the 15-case benchmark subset, including WebVoyager/Mind2Web-style workflow cases. | -| `benchmarks/run_benchmarks.py` | Rustwright and Playwright benchmark runner for the 15-case comparable workload. | +| `benchmarks/automation_cases.py` | Shared Playwright-style automation/parity cases and benchmark workflows, including WebVoyager/Mind2Web-style workflow cases. | +| `benchmarks/run_benchmarks.py` | Rustwright and Playwright benchmark runner for a comparable workload. | | `tests/test_rustwright_sync_api.py` | Main behavior/regression suite. | | `tests/test_playwright_parity_cases.py` | Shared parity harness test entry point. | | `tools/api_surface_audit.py` | Public API surface comparison against reference Playwright. | | `tools/run_parity_cases.py` | Runs shared parity cases against Rustwright or reference Playwright. | -| `tools/run_antibot_benchmarks.py` | Anti-bot benchmark runner covering Tier 0 local smoke signals, Tier 1 public fingerprint adapters for SannySoft, CreepJS, BrowserScan, and DeviceAndBrowserInfo, and local Tier 4 fresh/warm profile matrix checks across Rustwright and Playwright. | +| `tools/run_antibot_benchmarks.py` | Anti-bot benchmark runner covering local smoke signals, public fingerprint adapters, and fresh/warm profile matrix checks across Rustwright and Playwright. | ## Known Architecture Debt The current implementation intentionally optimized for fast parity iteration. The largest monoliths are now large enough to slow development: -| File | Current size | Debt | -| --- | ---: | --- | -| `src/lib.rs` | 19,794 lines | CDP transport, browser state, event routing, DOM helpers, stealth/dedicated-worker identity wiring, network shaping, facade promotions, and PyO3 exports are all colocated. | -| `python/rustwright/sync_api.py` | 29,310 lines | Public API classes, option validators, event waiters, routing, locators, assertions, artifacts, and request helpers are colocated — and a large engine-in-shim share (see audit below). | -| `python/rustwright/async_api.py` | 6,447 lines | Hand-written async mirror of the sync API; ~242 of 404 methods are pure mechanical delegations that drift when the sync surface changes. | -| `benchmarks/automation_cases.py` | 16,158 lines | Shared parity cases and benchmark workflows are useful but increasingly hard to scan by subsystem. | -| `tests/test_rustwright_sync_api.py` | 29,011+ lines | Broad regression coverage is useful but hard to navigate by subsystem. | +| File | Debt | +| --- | --- | +| `src/lib.rs` | CDP transport, browser state, event routing, DOM helpers, stealth/dedicated-worker identity wiring, network shaping, facade promotions, and PyO3 exports are all colocated. | +| `python/rustwright/sync_api.py` | Public API classes, option validators, event waiters, routing, locators, assertions, artifacts, and request helpers are colocated. | +| `python/rustwright/async_api.py` | Hand-written async control paths sit beside generated mechanical delegation mixins; ownership rules must stay synchronized with the generator. | +| `benchmarks/automation_cases.py` | Shared parity cases and benchmark workflows are useful but increasingly hard to scan by subsystem. | +| `tests/test_rustwright_sync_api.py` | Broad regression coverage is useful but hard to navigate by subsystem. | This is acceptable for alpha while behavior is moving quickly, but the beta bar should include splitting by stable ownership boundaries. -## Shim Weight Audit (2026-07-21) - -Measured shim weight per binding (hand-maintained lines, excluding tests): - -| Binding | Lines | Mechanism | Engine-in-shim findings | -| --- | ---: | --- | --- | -| Python | ~39,100 | PyO3 (in-process) | ~95% of all shim code; details below. | -| Go | ~850 | C ABI (purego) | Re-defaults `headless=true`; own wire decoder; own launch normalizer. | -| rust-native | ~820 | rlib (in-process) | Re-defaults `headless` + injects a 30s launch timeout; partial wire decoder. | -| C ABI (`capi/`) | ~540 | is the boundary | Passes launch JSON through raw, forcing every C-ABI binding to normalize. | -| Java / C# / Ruby / PHP | ~300–1,300 each | C ABI | Each: own launch/screenshot normalizer, own wire decoder, own harness helpers. | -| Node | ~420 | napi-rs (in-process) | Own launch/screenshot normalizer; wire decoder had drifted into a live bug. | - -Python engine-in-shim inventory (`sync_api.py` unless noted): - -- ~4,955 lines of the file are JavaScript inside Python strings; the - actionability probe (`_target_state`) is rebuilt via string `.replace()` on - every poll iteration. -- 34 `_try_fast_*` DOM fast-paths totalling ~2,027 lines — pure engine - performance shortcuts, cleanly excisable as a unit. -- 31 `while True` poll loops; ~1,700 lines of deadline/poll/retry code - (`_wait_for_single`, `_wait_for_fill_ready`, fill/select apply loops, 16 - near-identical page event-waiter loops, 21 event context-manager classes). -- The `expect()` assertion engine: ~1,000 lines of poll loop + probe JS. -- `APIRequestContext`: a hand-rolled urllib HTTP/proxy/redirect stack - (~1,500 lines) parallel to the core's reqwest stack. -- Error classification by string-sniffing: the core maps every failure to - `PyRuntimeError`, and Python re-derives timeout/crash/closed semantics by - matching message substrings. -- The whole shim drives the core through ~104 native methods, most of which - reduce to "evaluate this JS against a locator" — the core exposes few - semantic DOM operations, which is the root cause of the accretion. - -Cross-shim duplication (non-Python): - -| Concern | Copies | ~LOC | Resolution | -| --- | ---: | ---: | --- | -| Evaluate wire decoder | 7 | 820 | Decode once core-side; shims map leaf scalars only. | -| Launch normalize + defaults | 7 | 366 | Core accepts camelCase aliases; shim re-defaults deleted. | -| Screenshot normalize | 5 | 120 | Already parsed core-side (`capi`); delete shim copies. | -| Data-URL / JSON-equality / manifest validation (harness) | 5–6 each | ~1,700 | Move behind C-ABI helpers when harness work next opens. | -| Error mapping | 7 | thin | Already correct (core-owned strings) — the model to follow. | +## Binding Ownership Audit + +The core graph parser establishes the evaluate-wire and launch-parser ownership split: + +| Binding | Current ownership | +| --- | --- | +| Python | PyO3 native graph adapter for evaluate results; Playwright-shaped validation and ergonomics remain in Python. | +| Go | C ABI binding that already uses the core's flattened structural decoder and maps leaf values. | +| rust-native | In-process facade that already uses core structural decoding and maps leaf values. | +| C ABI | Legacy flattened JSON decoder and core launch parser; it does not preserve host graph identity. | +| Java / C#/.NET / Ruby / PHP | C-ABI bindings with full structural copies pending migration to the core decoder. | +| Node | napi native graph adapter; the typed camelCase coercion veneer remains in Node. | + +The remaining cross-shim concerns have these owners: + +| Concern | Current owner | Remaining work | +| --- | --- | --- | +| Evaluate wire | Core parser; C ABI uses its flattened compatibility decoder, while Python and Node use native graph adapters. | Migrate Java, C#/.NET, Ruby, and PHP to the core decoder. | +| Launch options | Core parser accepts canonical snake_case and current camelCase aliases. | Keep C callers on canonical snake_case; retain language-specific validation and coercion only where required for compatibility. | +| Screenshot options | Core parser. | Keep language veneers limited to marshalling and native coercion. | +| Data URL / JSON equality / manifest validation | Binding harnesses. | Move behind C-ABI helpers when harness work next opens. | +| Error mapping | Core-owned error strings and structured payloads where available. | Add a structured taxonomy in the later error track. | ## Shim Lightening Roadmap Ordered tracks; each is independently landable and keeps parity tests green. -1. **Fix + centralize the evaluate wire decoder.** Repair the Node decoder - drift against the core serializer (regression-tested), then add a - canonical core-side decode so per-language decoders reduce to leaf-scalar - mapping. -2. **Centralize launch-option normalization/defaulting.** serde camelCase - aliases on `LaunchOptions`; delete shim-side re-defaults (Go, rust-native) - and redundant key-mapping (Node). -3. **Generate the async Python facade.** Machine-generate the mechanical - delegation methods in `async_api.py` from `sync_api.py` signatures with a - checked-in-output freshness test; hand-written code shrinks to the ~160 - methods with real async semantics. +1. **Core evaluate-wire graph and launch parsing (this change).** Keep the + legacy C ABI decoder flattened for compatibility. Use native graph adapters + in Python and Node, and retain core structural decoding in Go and + rust-native. +2. **Migrate the remaining C-ABI bindings.** Move Java, C#/.NET, Ruby, and PHP + from full structural copies to the core graph contract. This remains + pending; do not treat those migrations as complete. +3. **Maintain the generated async Python facade.** Keep mechanical delegation + methods generated from `sync_api.py` signatures, keep hand-written code + limited to methods with real async semantics, and keep the checked-in + freshness contract synchronized with public context-manager methods. 4. **Native actionability.** Land the in-flight native-actionability branch (Tokio-side waits, shared probe templates, trusted CDP mouse dispatch, structured `ActionTimeoutError`), reconciled with the #96 probe-budget @@ -168,22 +155,21 @@ Ordered tracks; each is independently landable and keeps parity tests green. Python formats parity messages without string-sniffing. Trusted keyboard dispatch (`Input.dispatchKeyEvent`/`insertText`) is the missing input primitive. -5. **Move the 34 `_try_fast_*` DOM fast-paths into the core** as semantic - native operations. +5. **Move the DOM fast paths into the core** as semantic native operations. 6. **Bundle the injected probe/action JavaScript core-side** (single injected script, no per-poll string assembly). -7. **Consolidate event waiters** behind one generic native waiter + a small +7. **Consolidate event waiters** behind one generic native waiter plus a small Python descriptor table. 8. **Native `expect()` polling** returning `(passed, actual)`; Python keeps - assertion API + message formatting. + assertion API plus message formatting. 9. **Structured error taxonomy across the boundary** (typed timeout/crash/ closed payloads; retire substring classification). 10. **Core-side default-timeout register** (page/context), so contexts/default timeouts land once in the core instead of per shim. -Sequencing rule: tracks 1–3 are independent and safe now; track 4 gates 5–8 -(they reuse its probe/loop machinery); 9 rides along with 4; 10 pairs with -the first binding that needs contexts. +Sequencing rule: track 1 is the contract baseline for track 2. Track 3 is +independent. Track 4 gates tracks 5–8; track 9 rides along with track 4; track +10 pairs with the first binding that needs contexts. ## Target Rust Module Split @@ -293,11 +279,10 @@ Chrome-for-Testing installer is linux x86_64-only. ## Current Refactor Priority -1. Shim Lightening Roadmap tracks 1–3 (decoder fix/centralization, launch - normalization, async generation) — independent, safe, in flight. -2. Land the native-actionability branch and extend it to the sync path - (roadmap track 4); it unlocks tracks 5–8. -3. Extract Python option validators into `python/rustwright/options.py`. +1. Preserve and extend the core evaluate-wire and launch-parser contract + without changing the legacy C ABI behavior. +2. Migrate the remaining C-ABI bindings; this is not complete. +3. Maintain the generated async Python facade and its ownership checks. 4. Extract Python event waiters/context managers into `python/rustwright/events.py` (pairs with roadmap track 7). 5. Extract Rust launch/process code into `src/browser/launch.rs`, then CDP