Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
f82d5e7
fix(ts, sourcemaps): a command that built removes its build worktree …
genesiscz Oct 13, 2026
90d9868
fix(ax-tool): the recording border frames the recorded windows, also …
genesiscz Oct 14, 2026
4b8196f
fix(tools): ./tools never runs bun install by itself in a linked work…
genesiscz Oct 14, 2026
bcfb1ac
fix(github): the merge restack borrows the local object store, so its…
genesiscz Oct 14, 2026
1918089
fix(doctor): the security check warns when the terminal app that runs…
genesiscz Oct 14, 2026
26e06ec
perf(doctor): the Developer Tools check walks its ancestors through t…
genesiscz Oct 14, 2026
80e2c21
refactor(tools): every .genesis-tools path is built by storage/root (…
genesiscz Oct 16, 2026
3cdac22
docs(debugging-master): put back the llm-log usage lines 985e6b5f31 d…
genesiscz Oct 14, 2026
5497a76
fix(utils): export @genesiscz/utils/storage/root, because the youtube…
genesiscz Oct 14, 2026
991ec6a
feat(ax-tool, control): act --action drag --allow-outside drags a win…
genesiscz Oct 14, 2026
8aaaccb
fix(ax-tool): the act parser accepts --allow-outside for a drag, beca…
genesiscz Oct 14, 2026
d06d5c9
fix(control, ts): #537 followup - allow outside drags and stop build …
genesiscz Oct 16, 2026
426f2bb
fix(control, ts): #537 followup - refresh host-dependent security fin…
genesiscz Oct 16, 2026
62c6846
fix(ts): #537 followup - the build group signal carries its pid-verif…
genesiscz Oct 16, 2026
4f5c1d6
fix(control, ts): #537 followup - cancel every build and preserve unk…
genesiscz Oct 16, 2026
d335cc7
fix(tools): #537 followup - reject cancelled builds and enforce stora…
genesiscz Oct 17, 2026
977c39f
fix(tools): #537 followup - refresh drag release coordinates after wi…
genesiscz Oct 17, 2026
d852571
fix(tools): #537 followup - fail safely and report partial operations
genesiscz Oct 17, 2026
5d5c6b9
fix(tools): #537 followup - preserve refresh caches and drag scope bo…
genesiscz Oct 17, 2026
b0be163
fix(tools): #537 followup - normalize session roots and surface cache…
genesiscz Oct 17, 2026
d898f93
fix(tools): #537 followup - use shared environment for auto-install o…
genesiscz Oct 17, 2026
b992e7a
fix(tools): #537 followup - report cleanup failures and preserve drag…
genesiscz Oct 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/rules/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ When creating a new tool and writing helper functions, check if the utility is *
- `src/utils/string.ts` - String utilities (glob matching, ANSI stripping)
- `src/utils/cli/executor.ts` - CLI helpers: `suggestCommand()`, `isInteractive()`, `buildCommand()`, `Executor`, `enhanceHelp()`
- `src/utils/storage/storage.ts` - Config & cache management
- `src/utils/storage/root.ts` - `toolDataDir(tool, ...)` / `genesisToolsDir(...)`: every `~/.genesis-tools/...` path (honours `GENESIS_TOOLS_HOME`, no logger import). Never hand-join `homedir()` or `env.paths.getHome()` with `".genesis-tools"`
- `src/utils/storage/root.ts` - `toolDataDir(tool, ...)` / `genesisToolsDir(...)`: every `~/.genesis-tools/...` path (honours `GENESIS_TOOLS_HOME`, no logger import); `genesisToolsDirIn(home, ...)` for an explicit sandbox or fixture home. Never hand-join any home with `".genesis-tools"`: lint-rules `no-hand-joined-genesis-tools` fails it outside tests and `plugins/**`
Comment thread
genesiscz marked this conversation as resolved.
- `src/utils/async.ts` - Async helpers (concurrency, retry, etc.)
- `src/utils/json-schema.ts` - JSON schema inference: `inferSchema()`, `formatSchema(value, "skeleton"|"typescript"|"schema")`
- `src/utils/ai/local/device.ts` - ONNX Runtime device detection: `detectDevice()`, `resolveDevice()` (CoreML/CUDA/DML/CPU)
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Run `bash scripts/ci/placeholder-check.sh`. The pre-push hook runs it first, and
Every teammate/subagent given its own worktree MUST, before ANY other work:

1. **Verify the base commit.** Isolated worktrees are often cut from `origin/master`, NOT the campaign/feature branch you were briefed on. Run `git log --oneline -1`; if the briefed base commit is not an ancestor, `git fetch origin <branch> && git reset --hard <base-sha>`. Building on the wrong base silently invalidates every anchor in your brief.
2. **Run `bun install` in the worktree.** A worktree without its own `node_modules` resolves imports against the MAIN repo's dependency tree and fabricates failures that look like "the branch broke the world" (verified repeatedly; see also Tests above).
2. **Run `bun install` in the worktree.** A worktree without its own `node_modules` resolves imports against the MAIN repo's dependency tree and fabricates failures that look like "the branch broke the world" (verified repeatedly; see also Tests above). Install once, yourself: `./tools` never installs by itself in a linked worktree (`TOOLS_AUTOINSTALL=1` opts in), because every fresh install writes new native binaries that macOS checks at their first start, and with several worktrees that check queued for minutes (2026-10-14).
3. **Never bare `bun test`** — always `bun run test` (the wrapper repairs the dependency tree first).

Skipping any of these has cost real sessions hours; there are no exceptions.
Expand Down
37 changes: 29 additions & 8 deletions native/ax-tool/SnapshotSupport/WindowEvents.swift
Original file line number Diff line number Diff line change
Expand Up @@ -71,28 +71,46 @@ public final class WindowEventFactory {

public func drag(start: CGPoint, points: [CGPoint], stepDelay: Double,
verify: (CGPoint) throws -> Void, post: (CGEvent) -> Void) throws {
try drag(start: start, points: points, stepDelay: stepDelay, liveBounds: nil, verify: verify, post: post)
}

/// `liveBounds` is the `--allow-outside` drag: the path may leave the snapshot window, and the window may move with
/// it (a panel dragged by its body). Each step reads where the window is now and computes the window-local location
/// against that, so an app that converts `locationInWindow` to screen gets the global point; against the snapshot
/// bounds a moving window saw every step off by the distance it had already moved. Only the start must lie inside.
public func drag(start: CGPoint, points: [CGPoint], stepDelay: Double, liveBounds: (() -> CGRect?)?,
verify: (CGPoint) throws -> Void, post: (CGEvent) -> Void) throws {
let down = try mouse(type: .leftMouseDown, point: start, clickCount: 1)
var release = try mouse(type: .leftMouseUp, point: start, clickCount: 1)
func postRelease() {
if let liveBounds {
// Posting the last move can move the window after its release event was allocated.
// Refresh the local field without risking a new allocation after mouse-down.
setLocation(release, snapshotWindowPoint(release.location, in: liveBounds() ?? bounds))
}
post(release)
}
try verify(start)
post(down)
do {
for point in points {
Thread.sleep(forTimeInterval: stepDelay)
try verify(point)
let move = try mouse(type: .leftMouseDragged, point: point, clickCount: 1)
let nextRelease = try mouse(type: .leftMouseUp, point: point, clickCount: 1)
let frame = liveBounds.map { $0() ?? bounds }
let move = try mouse(type: .leftMouseDragged, point: point, clickCount: 1, frame: frame)
let nextRelease = try mouse(type: .leftMouseUp, point: point, clickCount: 1, frame: frame)
Comment thread
genesiscz marked this conversation as resolved.
post(move)
release = nextRelease
}
} catch {
post(release)
postRelease()
// A takeover keeps its own type, so the caller reports it as the terminal refusal it is.
if error is UserTakeoverError {
throw error
}
throw WindowEventError.unavailable("drag interrupted after mouse-down: \(error.localizedDescription); inspect the partial outcome")
}
post(release)
postRelease()
}

public func scroll(point: CGPoint, deltaX: Int32, deltaY: Int32) throws -> CGEvent {
Expand Down Expand Up @@ -129,29 +147,32 @@ public final class WindowEventFactory {
return event
}

public func mouse(type: NSEvent.EventType, point: CGPoint, clickCount: Int) throws -> CGEvent {
/// `frame` nil: the point must lie in the snapshot window and is located against it. A frame: the window's bounds
/// right now, for a step of an `--allow-outside` drag; the point may lie anywhere.
public func mouse(type: NSEvent.EventType, point: CGPoint, clickCount: Int, frame: CGRect? = nil) throws -> CGEvent {
// Split from the bounds check on purpose: Int(Double.nan) traps, so a non-finite point must
// never reach the formatter that describes the rectangle.
guard point.x.isFinite, point.y.isFinite else {
throw WindowEventError.unavailable("mouse point \(point.x),\(point.y) is not a finite coordinate")
}

guard bounds.contains(point) else {
let window = frame ?? bounds
guard frame != nil || bounds.contains(point) else {
let received = "\(Int(point.x.rounded())),\(Int(point.y.rounded()))"
let rectangle = "\(Int(bounds.origin.x.rounded())),\(Int(bounds.origin.y.rounded())) "
+ "\(Int(bounds.width.rounded()))x\(Int(bounds.height.rounded()))"
throw WindowEventError.unavailable(
"mouse point \(received) is outside snapshot window \(windowID), whose bounds are \(rectangle)")
}
let local = NSPoint(x: point.x - bounds.minX, y: bounds.maxY - point.y)
let local = NSPoint(x: point.x - window.minX, y: window.maxY - point.y)
guard let event = NSEvent.mouseEvent(with: type, location: local, modifierFlags: [],
timestamp: ProcessInfo.processInfo.systemUptime, windowNumber: windowID, context: nil,
eventNumber: 0, clickCount: clickCount, pressure: 0)?.cgEvent else {
throw WindowEventError.unavailable("could not allocate window-addressed mouse event")
}
// Setting the global position clears the private window-local field.
event.location = point
setLocation(event, snapshotWindowPoint(point, in: bounds))
setLocation(event, snapshotWindowPoint(point, in: window))
event.setIntegerValueField(.mouseEventClickState, value: Int64(clickCount))
event.setIntegerValueField(.mouseEventWindowUnderMousePointer, value: Int64(windowID))
event.setIntegerValueField(.mouseEventWindowUnderMousePointerThatCanHandleThisEvent, value: Int64(windowID))
Expand Down
10 changes: 9 additions & 1 deletion native/ax-tool/SnapshotSupport/WorkflowArguments.swift
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,13 @@ public struct WorkflowArguments {
public let values: [String: String]
public let flags: Set<String>

/// Outside geometry must not bypass the chrome token's omitted-web-content boundary.
public static func validateDragScope(scope: String, allowOutside: Bool) throws {
guard !allowOutside || scope != "chrome" else {
throw WorkflowArgumentError.invalid("--allow-outside requires window scope; chrome scope omits web content")
}
}

public init(_ args: [String], command: String) throws {
let valueOptions: Set<String>
let flagOptions: Set<String>
Expand All @@ -30,7 +37,7 @@ public struct WorkflowArguments {
"--revalidate-scope", "--frame", "--by-identifier", "--window-index", "--depth", "--modifiers",
"--budget-ms", "--target-ordinal", "--expect-value-sha256",
]
flagOptions = ["--background", "--double", "--refresh", "--no-cursor", "--no-image", "--prepare", "--replace", "--hold", "--no-activate"]
flagOptions = ["--background", "--allow-outside", "--double", "--refresh", "--no-cursor", "--no-image", "--prepare", "--replace", "--hold", "--no-activate"]
default:
throw WorkflowArgumentError.invalid("unknown workflow command \(command)")
}
Expand Down Expand Up @@ -184,6 +191,7 @@ public struct WorkflowArguments {
// hover deliberately excluded from --background: the whole point is to move the REAL
// pointer, and a window-addressed event does not.
try reject(["--background"], unless: ["click", "move", "drag", "scroll"])
try reject(["--allow-outside"], unless: ["drag"])
try reject(["--coords", "--region"], unless: ["click", "move", "drag", "scroll", "hover"])
try reject(["--frame"], unless: ["click", "move", "drag", "scroll", "hover"])
if let frame = values["--frame"] {
Expand Down
6 changes: 4 additions & 2 deletions native/ax-tool/Sources/IsolatedCapture.swift
Original file line number Diff line number Diff line change
Expand Up @@ -512,7 +512,9 @@ func captureIsolatedWindows(_ options: RecordOptions) {
compositor.updateSizes()
}
let indicator = RecordingIndicator(enabled: options.selection.indicator)
indicator.update(displayCanvas.map { [$0] } ?? selected.map(\.bounds))
// The border marks what the movie holds: the selected windows, also on a display canvas, where framing the whole
// display read as "the whole screen is recorded" (Martin, 2026-10-14).
indicator.update(selected.map(\.bounds))
defer { indicator.close() }
var startedStreams: [SCStream] = []
do {
Expand Down Expand Up @@ -541,7 +543,7 @@ func captureIsolatedWindows(_ options: RecordOptions) {
($0[kCGWindowOwnerPID] as? pid_t) == target.pid }) else { return nil }
return captureWindowBounds(row)
}
indicator.update(displayCanvas.map { [$0] } ?? bounds)
indicator.update(bounds)
compositor.refresh(rows)
return compositor.queue.sync { compositor.error == nil }
}
Expand Down
50 changes: 42 additions & 8 deletions native/ax-tool/Sources/SnapshotWorkflow.swift
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,20 @@ private func describeOutsideWindow(point: CGPoint, window: ObservedWindow) -> St
+ "both are global logical points, and a window that moved since see needs a fresh see"
}

/// Where a window is right now, while it is on screen and still owned by `pid`; nil once it closed, hid or changed owner.
/// An `--allow-outside` drag reads it at every step: the window may move with the drag, but it must stay this window.
private func onscreenWindowBounds(_ id: CGWindowID, pid: pid_t) -> CGRect? {
let listed = CGWindowListCopyWindowInfo(.optionIncludingWindow, id) as? [[CFString: Any]] ?? []
for info in listed {
guard (info[kCGWindowNumber] as? CGWindowID) == id, (info[kCGWindowOwnerPID] as? pid_t) == pid,
(info[kCGWindowIsOnscreen] as? Bool) == true,
let raw = info[kCGWindowBounds] as? NSDictionary,
let bounds = CGRect(dictionaryRepresentation: raw) else { continue }
return bounds
}
return nil
}

private var workflowInput: WorkflowArguments?

/// The caller's deadline (`--budget-ms`), less a margin for printing the result and exiting.
Expand Down Expand Up @@ -1431,7 +1445,7 @@ func cmdAct(appName _: String) {
// error — just a click in the wrong place. A caller who wants a point that survives the
// window moving opts in with --frame window, and that is the one this documents.
let coordinateFrame = workflowArgument("--frame") ?? "screen"
func parsePoint(_ raw: String) throws -> CGPoint {
func parsePoint(_ raw: String, requireInside: Bool = true) throws -> CGPoint {
let parts = raw.split(separator: ",", omittingEmptySubsequences: false)
let numbers = parts.compactMap { Double($0.trimmingCharacters(in: .whitespaces)) }
guard parts.count == 2, numbers.count == 2, numbers.allSatisfy({ $0.isFinite }) else {
Expand All @@ -1444,7 +1458,7 @@ func cmdAct(appName _: String) {
let point = coordinateFrame == "window"
? CGPoint(x: window.bounds.origin.x + given.x, y: window.bounds.origin.y + given.y)
: given
guard window.bounds.contains(point) else {
guard !requireInside || window.bounds.contains(point) else {
throw WindowEventError.unavailable(describeOutsideWindow(point: point, window: window))
}

Expand Down Expand Up @@ -1616,7 +1630,7 @@ func cmdAct(appName _: String) {
guard rawCoords != nil || row["visible"] as? Bool == true || visiblePart != nil else {
throw WindowEventError.unavailable("element is outside its window/scroll clip")
}
let point = try rawCoords.map(parsePoint) ?? visiblePart ?? CGPoint(x: frame.midX, y: frame.midY)
let point = try rawCoords.map { try parsePoint($0) } ?? visiblePart ?? CGPoint(x: frame.midX, y: frame.midY)
// A caller that passed a window-relative point cannot otherwise tell WHERE it landed,
// and that is the number to compare against a screenshot or a later observation.
if let raw = rawCoords {
Expand Down Expand Up @@ -1725,10 +1739,17 @@ func cmdAct(appName _: String) {
Thread.sleep(forTimeInterval: 0.1)
} else if action == "drag" {
guard let destination = workflowArgument("--to") else {
throw WindowEventError.unavailable("drag requires --to x,y in the snapshot window")
throw WindowEventError.unavailable("drag requires --to x,y in the snapshot window, or anywhere with --allow-outside")
}
// --allow-outside: the destination and the path may leave the window, and the window may move with
// the drag (a panel dragged by its body). The start keeps every check below; each later step checks
// that the user has not taken the pointer and that the window is still this window, on screen.
let allowOutside = workflowFlag("--allow-outside")
try WorkflowArguments.validateDragScope(scope: token.effectiveScope, allowOutside: allowOutside)
let end = try parsePoint(destination, requireInside: !allowOutside)
if !allowOutside {
_ = try verifyPoint(end, pin: .window)
}
let end = try parsePoint(destination)
_ = try verifyPoint(end, pin: .window)
let duration = Double(workflowArgument("--duration") ?? "0.3") ?? 0
guard duration.isFinite, (0.1...5).contains(duration) else {
throw WindowEventError.unavailable("--duration must be 0.1–5 seconds")
Expand All @@ -1742,16 +1763,29 @@ func cmdAct(appName _: String) {
try validateAfterFeedback()
// The takeover check runs in verify, which precedes every post but the closing
// release; the factory posts that release itself when verify throws.
try factory.drag(start: point, points: points, stepDelay: duration / Double(steps),
let liveBounds: (() -> CGRect?)? = allowOutside ? { onscreenWindowBounds(window.id, pid: pid) } : nil
try factory.drag(start: point, points: points, stepDelay: duration / Double(steps), liveBounds: liveBounds,
verify: {
try inputGate.check()
_ = try verifyPoint($0, pin: WindowEventFactory.dragVerifyTarget(point: $0, start: point))
if allowOutside && $0 != point {
Comment thread
genesiscz marked this conversation as resolved.
guard onscreenWindowBounds(window.id, pid: pid) != nil else {
throw WindowEventError.unavailable("window \(window.id) closed or left the screen during the drag")
}
} else {
_ = try verifyPoint($0, pin: WindowEventFactory.dragVerifyTarget(point: $0, start: point))
}
try admitVisualAction()
}, post: { event in
ActionCursor.emit("drag", point: event.location, background: background, target: rawCoords == nil ? "ax" : "pixel", waitForPresentation: false)
inputGate.deliver { event.postToPid(pid) }
})
Thread.sleep(forTimeInterval: 0.05)
if allowOutside {
// Nothing checked what lay under the path, so say where the window ended: a drag that moved it
// shows here, and a caller compares it with the bounds it saw in see.
let after = onscreenWindowBounds(window.id, pid: pid)
actionExtras["windowBoundsAfter"] = after.map { ["x": $0.minX, "y": $0.minY, "width": $0.width, "height": $0.height] } ?? NSNull()
}
} else if secondaryClick {
// 🛑 A right-click is delivered in the background three ways, in this order, and each later way
// runs only while the app shows no menu at all: the NSEvent-built window event, a plain CGEvent
Expand Down
Loading
Loading