Skip to content

Add subtree search, property filtering, and targeted DataContext inspection #69

Closed

Description

@mkajander

Report

A repeated WPF control set required one find_elements call followed by roughly 120 get_computed_properties calls. Selection state was visible through a binding and the computed Fill value, but find_elements could not use either value as a predicate.

The same investigation exposed two related limits:

  • get_data_context serialized nested objects as type names instead of allowing selected nested fields.
  • find_elements could not restrict traversal to a known subtree.

Together these limits produced a noisy N+1 inspection workflow that was slow to cancel and hard to audit.

Requested behavior

Bounded property inspection

Allow an exact computed-property predicate in find_elements, or add a bounded batch operation.

{
  "query": {
    "type": "SelectableShape",
    "computedProperty": {
      "name": "Fill",
      "equals": "#FFA8D7E5"
    }
  },
  "maxResults": 10
}

A batch alternative could accept a bounded set of element IDs and property names:

{
  "elementIds": ["id-1", "id-2", "id-3"],
  "propertyNames": ["Fill", "Points", "IsSelected"],
  "maxElements": 120
}

Targeted DataContext paths

Allow callers to request named paths without recursively serializing the full object graph:

{
  "elementId": "...",
  "propertyPaths": [
    "CurrentSelection.Kind",
    "CurrentSelection.Key"
  ]
}

Subtree search

Let find_elements start from the same kind of root locator used by get_visual_tree:

{
  "root": {
    "automationId": "ITEMS_CANVAS"
  },
  "query": {
    "type": "SelectableShape"
  },
  "maxResults": 10
}

Safety and response requirements

  • Keep element traversal, returned results, property predicates, property paths, and batch sizes bounded.
  • Reject invalid or oversized requests before traversal.
  • Report scanned and returned element counts.
  • Report truncation and the limits applied to the request.
  • Preserve cancellation throughout traversal and property evaluation.
  • Do not turn DataContext inspection into an unrestricted recursive serializer.

Evaluation

Implementation approach and final scope still need verification against the current WPF agent and public tool contracts.

Activity

  1. mkajander commented on Aug 26, 2026

    @mkajander
    ContributorAuthor

    Implementation assessment:

    • Subtree search is accepted. The diagnostics profile already supports root; the default compact profile drops it. The fix is to expose the existing bounded path in the compact schema.
    • Targeted DataContext paths are accepted. The agent already has bounded dotted-path rules for state observation. get_data_context can reuse the same limits for a one-shot projection without recursively serializing unrelated objects.
    • Batch computed-property inspection is accepted instead of a property predicate in this change. A predicate would be WPF-only inside a tool that can fall back to UIA, and string equality for brushes, unset values, converters, and arbitrary objects needs a separate comparison contract. A single bounded agent request over element IDs and property names removes the N+1 MCP calls without inventing those semantics.

    The implementation will cap path count and depth, batch element count, properties per element, and total work. New agent fields and methods will be capability-gated so an older injected agent cannot silently ignore them.

  2. mkajander commented on Aug 26, 2026

    @mkajander
    ContributorAuthor

    Resolved in #70 and published as v0.1.0-preview.28.

    Delivered:

    • find_elements.root in the core profile for bounded subtree search.
    • get_computed_properties_batch with element and property limits, counts, truncation metadata, cancellation, capability checks, and stable public ID mapping.
    • get_data_context.propertyPaths for bounded dotted-path projections. Shared prefixes are read once per request.

    Computed-property predicates in find_elements remain deferred. They need a separate comparison contract for WPF-only routing, unset values, converters, brushes, and application-defined values. The batch operation resolves the reported N+1 call problem without guessing at those equality rules.

    CI passed all 784 tests. The release workflow built, tested, packed, and published the package, and NuGet now lists preview.28.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriority: P1Important user feedbackuser-feedbackCaptured from real user experience

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions