feat(trackers,browser): ProcessTracker (heap, main-thread lag, idle, user idle, uptime) (TR5)

.changeset/browser-process-tracker.md

@@ -0,0 +1,7 @@
+---
+'@bugsee/browser': minor
+'@bugsee/webview': minor
+'@bugsee/trackers': minor
+---
+
+Browser and WebView system traces gain `main_thread_idle_ms` (the browser's granted idle period, 0-50 ms, from one `requestIdleCallback` probe per sampling tick on visible pages; omitted where `requestIdleCallback` is absent, and for a tick in which no idle period was granted) and `user_idle_state` (`active`, `idle` or `locked`, from the Idle Detection API). `user_idle_state` is OPT-IN: it is produced only with the new `captureUserIdle: true` launch option (browser and WebView; default false, only boolean `true` enables it, anything else stays off and is reported) AND when the page's `idle-detection` permission is already granted. The SDK never prompts for it, and a page's own grant is not consent to record and upload idle/locked state. The heap (`ram_js_heap_*`) and `main_thread_lag_ms` traces are unchanged but now come from one per-window `ProcessTracker` (new in `@bugsee/trackers/browser`), whose stages (`heap`, `mainThreadLag`, `idle`, `userIdle`, `uptime`) run only while they have listeners.

docs/PROGRESS.md

@@ -411,6 +411,7 @@ solid), in slices:
 - **TR1 (DONE, `feat/application-tracker`; design §15):** `PageLifecycleTracker` is now the domain **`ApplicationTracker`** (`packages/trackers/src/browser/application-tracker.ts`). Stages: `pagehide`, `visibility`, `freeze`, `resume`, plus the NEW aggregated `state` (`{state: 'foreground'|'background'}`, Android vocabulary, emitted when a visibility change CHANGES it) and `current()` (read live from the document, else the retained value; undefined when never known — an unreadable document emits no `state`, never a fabricated background). **Singleton per key** via `createTrackerRegistry`: `applicationTrackerFor(env, onError?)` keys by the document, else the window, else one shared key; a window-only env and a window+document env are DISTINCT trackers (pair a document with its own window); `resetApplicationTracker(env)` is exported from `@bugsee/trackers/browser` for internal/test use and is NOT re-exported by `@bugsee/browser`. Browser + webview launch and the events-source fallback obtain it through the registry. **Retention rule (tested):** DOM listeners attach on the first subscriber and detach on the last, the last known `state` is retained and recomputed silently from the document on re-activation, so the first subscriber after a gap is correct at once and the de-dup baseline is never stale. `AppStateProducer` now takes the tracker only (`on('state')` + `current()` for the initial value); trace/event names and wire shapes are unchanged. **Keying rule:** document, else window, else one module key (design §15.1 #5 will be amended from "one per window" to this). **Latest launch wins** through the registry `rebind`: a defined `onError` replaces the shared reporter (undefined keeps it); while IDLE the window/document are merged (a target the request omits is kept), while ACTIVE the binding is kept and a window/document mismatch (only a target the request supplies is compared; the error names which) is reported once. The shared sink is never released on stop: a relaunch with no `onError` keeps the previous launch's (accepted limitation). `resetApplicationTracker` is `@internal` test/teardown use: exported from `@bugsee/trackers/browser` but not re-exported by `@bugsee/browser` (`reset()` removes all subscribers). An unreadable document emits neither `visibility` nor `state` (no fabricated `process_background`/fps pause). **Stays for later:** `online`/`offline` (move to Connectivity, TR3) and `orientationchange` was a stage here until TR2: the UiTracker owns it now and it is removed from this tracker.
 - **TR3 (DONE, `feat/connectivity-tracker`):** the per-signal `ConnectionTracker` (and its test) is deleted; the domain **`ConnectivityTracker`** (`packages/trackers/src/browser/connectivity-tracker.ts`, `connectivityTrackerFor(env, onError)`, `resetConnectivityTracker`) is a per-window singleton via the registry (keyed on `env.window`, else `env.navigator`, else the env object) with `rebind` (latest `onError` wins; a defined `window`/`navigator` replaces ours, a partial env never drops them; a RUNNING tracker moves its listeners to a changed window/navigator and re-reads). It owns ALL connectivity listeners (window `online`/`offline` + `navigator.connection` `change`) and does NOT depend on the page-lifecycle tracker. Stages: `online` `{online}`, `networkInfo` `{type?, effectiveType?, downlink?, downlinkMax?, rtt?, saveData?}` and the aggregate `state` `{online, networkInfo?}`; activation emits a baseline of every stage. Fields are omitted when absent/invalid; zero/non-finite `downlink`/`downlinkMax`/`rtt` count as "no estimate" (Chromium reports 0 offline and rounds `rtt` to 25 ms, so a 0 rtt is an unmeasured one; the spec reports `downlinkMax` 0 for type `none`) and are omitted, `saveData` only when boolean. Every host read goes through `hostRead` (a hostile getter costs only its field; reported). The emitter's reporter and every DOM listener are guarded, so a throwing subscriber plus a throwing `onError` never escapes a dispatch or `on()`. A refused `addEventListener` is reported once and the listeners that did attach stay attached (no silent death; released on deactivation). A THROWING `navigator.onLine` getter is NOT online: `online` is omitted from the state (no `online` emission, no fabricated true; `ConnectionProducer` writes no row), unless the window `online`/`offline` event that triggered the read supplies the value (the event is evidence; a READABLE `onLine` always wins), and an absent `onLine` property still counts as online. `connectivityTrackerFor` reads `window`/`navigator` once per call (a flaky getter is reported once and the key matches the host it built); `resetConnectivityTracker` is best-effort for such envs (first of window/navigator/env that has a tracker). The `addEventListener` READ itself is guarded too (cross-origin proxies). `state.online` is therefore optional. **Retention:** there is no cached snapshot to lose: state lives in `navigator`, which the tracker keeps and READS LIVE (`current()` and a fresh baseline on each activation); only the listeners are released on the last unsubscribe. `ConnectionProducer` is rewired onto it with UNCHANGED trace name/wire (`connection`: `type`, `effective_type`, `link_downstream_kbps`, `rtt`, `save_data`); `downlinkMax` is added to the open trace data as `link_downstream_max_kbps` (the connection trace is a free-form object with no schema, so this is additive). Consequence: `online`/`offline` are now bound twice on a window while capturing (the lifecycle tracker still serves the events source; the connectivity tracker owns its own), until the Application tracker absorbs the events source.
 - **TR6 (DONE, `feat/storage-tracker`):** domain **`StorageTracker`** (`packages/trackers/src/browser/storage-tracker.ts`), singleton per window via `createTrackerRegistry` (`storageTrackerFor(env, onError?)`, `resetStorageTracker`; keyed window -> navigator -> env object, `rebind` as PowerTracker). Wraps `navigator.storage.estimate()` + `persisted()`; ONE stage `storageEstimate` `{usageBytes?, quotaBytes?, persisted?}`. This is an ORIGIN quota ESTIMATE (browsers fuzz/pad it), NOT device free space. No change event, so it polls sparsely: on activation, on `visibilitychange` -> visible, then a slow timer (default 5 min via an injectable scheduler; ticks while the page is hidden are skipped). A field that is non-finite/negative/non-number is omitted (never 0); `persisted` only when boolean; an estimate with no usable field emits nothing. A rejected/throwing `estimate()`/`persisted()` is reported via `onError` and costs only its own field (host getters through `hostRead`); launch is never failed. **Retention rule (tested):** the LAST estimate survives the last unsubscribe (`current()` answers at once; a failed poll never erases it); timer and visibility listener do not (released, re-armed on the next subscriber, which also polls afresh). A single epoch counter is bumped by every poll/stop/reset, so a stale in-flight promise (after stop/reset, or superseded by a newer poll) is inert. `rebind` keeps every defined seam (undefined never clobbers) and applies it from the NEXT activation; a tracker that is attached keeps the navigator/document/scheduler it armed (only `onError` changes at once). Visibility-triggered polls are throttled to one per 60 s (`VISIBILITY_MIN_GAP_MS`, injectable `now`); the slow timer is not. Poll failures are reported ONCE per failure streak (reset by a poll without failures, and by `reset()`), not on every tick. A PARTIAL poll (one of `estimate()`/`persisted()` failed) merges: the failed call's retained fields are kept, answered fields replace, and the estimate's age advances only when usage/quota were refreshed. `currentFresh()` returns the retained estimate only while younger than 2x the poll interval (`current()` still returns it regardless). `StorageProducer` (browser `trace-producers.ts`, built after the `sensors:false` gate like the other sensor producers, so a `systemMetricsSampler` replaces it) emits the NEW JS-specific trace `storage_estimate` `{usage, quota}` in integer MB (`bytesToMegabytes`, floored; a missing key is omitted, `persisted` is not a trace) - NOT the Android `disk_mem_*` group. It is held/de-duplicated (`emitIfChanged`) because the poll period (5 min) far exceeds the default 60 s recording window; a non-held value would almost never appear in a report. `StorageProducer.onStart` seeds only from `currentFresh()` (never a snapshot older than 2x the interval). **OWNER DECISION PENDING:** a held polled value cannot be retracted, so after failed polls the last known estimate keeps being re-recorded every `refreshMs`. Alternatives: plain `emit` (the value would rarely land in a 60 s window) or a provider retract/release mechanism (same open question as the TR4 time traces). Usage under 1 MiB floors to 0 MB (an honest floor, not a measurement of zero). Shared by the WebView launch via the browser producers. Launch-level test proves launch goes through `storageTrackerFor` (the window's singleton holds the estimate).
+- **TR5 (implemented on `feat/process-tracker`, PR #83; multi-agent review rounds still in progress, one owner decision open — see "Known limit" below):** domain **`ProcessTracker`** (`packages/trackers/src/browser/process-tracker.ts`), singleton per window via `createTrackerRegistry` (`processTrackerFor(env, onError?)`, `resetProcessTracker`; keyed window -> performance -> env object, `rebind` merge-not-drop: a seam change on a RUNNING tracker first detaches every armed stage with the seams it was armed with, then swaps and restarts). It ABSORBS and replaces the per-signal `MemoryTracker` and `MainThreadLagTracker` (files + tests deleted; `MemoryProducer`/`MainThreadLagProducer` now subscribe to it; trace names and wire shapes unchanged). Five stages, each armed ONLY while it has listeners (via the `listenerCount(name)` / `onListenersChanged()` hooks `MultiKeyEmitterBase` already has on `main` from TR2; this PR does not touch the emitter; `start()`/`stop()` do nothing here): `heap` `{used, limit}` (`performance.memory`, polled; a figure that is not a finite non-negative number omits the reading), `mainThreadLag` `{lagMs}` (TIMER DRIFT, algorithm and visible-only gating unchanged: block >= 1 s always shows, shorter that crosses no due tick reads 0, reported once, fresh baseline each visible start), `idle` `{idleMs}` (NEW, ONE `requestIdleCallback` probe per sampling tick, never a loop and at most one in flight, visible pages only, `timeRemaining()` clamped to <= 50 ms; a tick whose callback never runs yields NO sample, never 0; omitted without rIC, e.g. Safari), `userIdle` `{userState: 'active'|'idle', screenState?: 'locked'|'unlocked'}` (NEW, Idle Detection API), `uptime` `{uptimeMs}` (`performance.now()`, omitted without a clock; no trace consumes it yet). **Idle Detection permission rule (binding): the SDK NEVER prompts.** `IdleDetector.requestPermission()` is never called; the detector starts ONLY when `navigator.permissions.query({name:'idle-detection'})` ALREADY reports `granted` (an engine that does not know the name rejects = stage inert, silent). The permission is re-queried on every activation and watched through its `change` event (granted later -> starts; revoked -> aborts and drops the tracker's retained state). Threshold floored at the spec's 60 s; stop aborts via `AbortController`; every async continuation is guarded by its run. **Retention rule (tested):** the last `userIdle` state survives the last unsubscribe (`currentUserIdle()`; `currentUserIdleFresh()`, which `UserIdleProducer` seeds from, answers only once the ARMED stage's detector has PUBLISHED in this run (permission re-confirmed and its `start()` settled; no age bound, a live detector reports changes only) and is undefined while the async permission re-query or `start()` is pending, or after a refusal/revoke; with NO armed stage it serves the state only while younger than 2 detector thresholds, via a new injectable `now` seam defaulting to `Date.now`, mirroring TR6's `currentFresh()`. A revoke while no stage is armed is therefore only noticed by the next activation's re-query (the plain `currentUserIdle()` read keeps the snapshot until then), but it can never seed a held trace); heap is read live; idle/lag/uptime keep nothing (a stale baseline would be a lie); timers, lifecycle subscription, pending probe and detector are released. New JS-specific traces (`BrowserTraceEnv` gains `requestIdleCallback`/`cancelIdleCallback`/`IdleDetector`/`AbortController`, bound/taken from the injected window by `browserTraceEnvFrom`): `main_thread_idle_ms` (ms, polled/not held; built under the same lifecycle gate as lag/fps) and `user_idle_state` (a STRING like `app_state`: `active` | `idle` | `locked`, where a locked screen wins over `userState`; held, de-duplicated; no trace at all unless the permission was already granted). **`user_idle_state` is OPT-IN (review P2; the maintainer's decision, PENDING OWNER CONFIRMATION): the host page's `idle-detection` grant shows only that the page was given the permission, not consent to record and upload idle/locked state. So `createBrowserSystemTraceProducers` builds `UserIdleProducer` only for `BrowserTraceEnv.captureUserIdle === true`, fed by the new browser + WebView launch option `captureUserIdle` (default false, strict boolean: any other value stays off and is reported; local key `com.bugsee.option.browser.capture-user-idle`, exported as `CAPTURE_USER_IDLE_OPTION_KEY`). Even when on, the tracker starts a detector only if the permission ALREADY reports `granted` (the SDK still never prompts). Without the opt-in the permission is not even queried. This applies ONLY to `user_idle_state`; private-mode detection is opt-out by a separate owner decision.** The viewer has no definition for either name yet (generic value block, design §13.3). **Known limit (OWNER DECISION PENDING — no retract mechanism invented):** `user_idle_state` is a HELD trace and the tracker emits nothing when the permission is revoked, so a producer that already published a value keeps having it re-recorded every `refreshMs` until the producer stops (the revoke only drops the tracker's retained copy and so stops NEW seeds; the seed is gated as above). Resolving it needs a decision on a retract/clear signal for held traces (same family as TR6's held `storage_estimate`). Other hardening from review: a hostile `performance.now` getter is reported exactly once in total (the factory reads silently, the tracker reports; only a SUCCESSFUL clock read is memoised per env, a failed one is retried on the next activation); `ProcessTracker.reset()` now drops every listener (like `ApplicationTracker`), so a later listener change cannot re-arm a stage for a gone subscriber; `browserTraceEnvFrom` now hands back stable bound closures per (window, function), so a no-op relaunch no longer restarts armed stages; a non-finite lag baseline omits that tick's `main_thread_lag_ms` instead of sending NaN; the gated teardown always closes the run. Launch-level test proves browser launch goes through `processTrackerFor` (the window's singleton can read the heap through launch's seams); the WebView launch shares the producers. `sensors:false` (a `systemMetricsSampler`) still builds only the battery producer.
 - **ST0′ (DONE, `feat/trace-producers`):** `traces.system` is now producers over shared trackers.
   `@bugsee/capture` has `TraceProducer`/`TraceProducerBase`; the traces provider is a producer container
   (per-producer start/stop isolation, last-value refresh so a rolling window never loses a rarely-changing

packages/browser/src/index.ts

@@ -66,6 +66,7 @@ export {
 export {
   type Bugsee,
   type BugseeLaunchOptions,
+  CAPTURE_USER_IDLE_OPTION_KEY,
   getStatus,
   type LaunchInternals,
   type LaunchResult,

packages/browser/src/launch-options.invalid.test.ts

@@ -4,14 +4,19 @@ import { createMemoryCaptureStore, type HttpResponse, type HttpTransport } from
 import { BugseeOption } from '@bugsee/protocol';
 import { afterEach, describe, expect, it, vi } from 'vitest';
 import type { BrowserProbe } from './environment';
-import { type Bugsee, type BugseeLaunchOptions, launch } from './launch';
+import {
+  type Bugsee,
+  type BugseeLaunchOptions,
+  CAPTURE_USER_IDLE_OPTION_KEY,
+  launch,
+} from './launch';
 
 // Every browser-owned boolean launch option, every kind of invalid value. Read by plain
 // truthiness these used to be wrong in BOTH directions: `persistUserIdentifier: 'false'` persisted PII,
 // `captureInteractions: 0` kept capturing input. The rules, per option:
 //   captureInteractions, captureViewHierarchy (default ON, capture)  -> invalid = OFF (fail-closed) + onError
 //   startInBlackout (default OFF, but ON is the safe side)             -> invalid = ON (fail-on) + onError
-//   persistUserIdentifier, report* UI flags (default OFF)              -> only boolean true turns on
+//   persistUserIdentifier, captureUserIdle, report* UI flags (default OFF) -> only boolean true turns on
 
 const probe: BrowserProbe = {
   userAgent: () => 'Mozilla/5.0 Chrome/119.0.0.0 Safari/537.36',
@@ -68,6 +73,7 @@ const failClosed: Array<[string, string]> = [
 ];
 const defaultOff: Array<[string, string]> = [
   ['persistUserIdentifier', BugseeOption.PersistUserIdentifier],
+  ['captureUserIdle', CAPTURE_USER_IDLE_OPTION_KEY],
   ['reportSummaryRequired', BugseeOption.ReportingUiSummaryRequired],
   ['reportDescriptionRequired', BugseeOption.ReportingUiDescriptionRequired],
   ['reportEmailRequired', BugseeOption.ReportingUiEmailRequired],