feat(agents): port the Delegate agent as a built-in - #191
Merged
Merged
Conversation
akshaylive
requested review from
CarlesUIPath,
bai-uipath,
tmatup and
uipreliga
as code owners
September 21, 2026 22:47
akshaylive
changed the title
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
akshaylive
force-pushed
the
akshaya/delegate_agent
branch
7 times, most recently
from
September 22, 2026 00:12
8ac6ce5 to
22b592d
Compare
…lt-in Adds `delegate` as a built-in agent (AgentKind.DELEGATE / DelegateAgentConfig), registered unconditionally like codex/antigravity, wrapping the now-public @uipath/delegate-sdk via a first-party Node stdio host (src/coder_eval/agents/delegate/delegate_host.mjs) since the internal-only @uipath/delegate-stdio host isn't public. Full turn lifecycle, transcript/ token accounting, pricing table, docs (docs/agents/DELEGATE.md), example tasks (tasks/delegate/*.yaml), and unit/registration/live test suites — SDK event field shapes are marked `# UNVERIFIED` pending a real backend run (excluded from golden-master coverage, covered by tests/test_delegate_agent_live.py under -m live). Also adds a Delegate live-integration CI job (pr-checks.yml) that mints a fresh access token via the OAuth2 Resource Owner Password Credentials (ROPC) grant (mirroring coder_eval_uipath's refresh-auth.sh) and runs tasks/delegate/fizzbuzz_delegate.yaml end to end against a real backend on every PR, with a presence-check gate so a missing secret skips cleanly instead of failing a required check. The mint destination is resolved from a fixed choice-input allowlist rather than a free-form URL, closing a credential-exfiltration path a free-form workflow_dispatch string would otherwise open. Bumps anyio 4.13.0 -> 4.15.1 for two newly-disclosed CVEs (CVE-2026-63374, CVE-2026-64847), both fixed in 4.14.2+. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
akshaylive
force-pushed
the
akshaya/delegate_agent
branch
from
September 22, 2026 00:16
22b592d to
4ee1bcc
Compare
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
uipreliga
self-requested a review
September 22, 2026 02:03
uipreliga
approved these changes
Sep 22, 2026
uipreliga
left a comment
uipreliga
left a comment
Collaborator
There was a problem hiding this comment.
Fix what oyu agree with and 🚢
Fixes the review's blockers and most non-blocking findings: - Crash/timeout turns now fan out ToolEnd(UNRESOLVED)/TurnEnd/AgentEnd to the caller's stream_callback (not the EventCollector only), and every mid-turn crash path force-kills the host so a retry can never reuse a live host's stale queue. - Spawn the Node host in its own process group and SIGKILL the group on force-kill, so the SDK's bundled interop process is reaped instead of orphaned. - _parse_usage warns instead of silently returning None on an unrecognized usage shape; added tests for the bucket spellings, model/cost wiring, and crash/timeout streaming. - A transient init-handshake timeout on a mid-run respawn now raises the retryable AgentCrashError instead of the non-retryable AgentConfigError. - Fixed the non-hermetic SDK-resolution test (patched cwd but not Path.home()), which passed only for developers who had NOT installed the SDK as documented. - CI: the live-tests job now fails loudly (not silently green) on a missing secret outside a fork PR, runs tests/test_delegate_agent_live.py under -m live with an "actually ran" assertion, keeps ROPC credentials out of curl's argv, derives DELEGATE_ENV from the same dispatch input the token mint uses, and the diagnostic script never gates the real run it exists to explain. - Removed dead _TurnState attributes, narrowed an over-broad exception suppression, fixed the Node host's process.exit() truncating its own `fatal` line under backpressure, added DELEGATE_-namespaced env var fallbacks, and closed the delegate/USER_GUIDE/EXTENDING/default.yaml roster-doc gaps (with CE047 extended to catch a future one). Deliberately not touched: the pricing table's hyphenated/dotted twin divergence (already documented in .claude/notes/agents.md as a known, pre-existing state, not something to silently "fix" by picking a number), docker-driver support for the new agent (a real feature addition, not a review fix), and the ROPC OAuth scope list / lockfile+npm-ci switch (both touch the CI job that was just brought to green; left as-is to avoid re-breaking it without a live run to verify against). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
test_delegate_live_token_usage_populated failed live (crashed=False, real text/tool output, token_usage=None): _parse_usage was never even reached with a dict, because it was looking in the wrong place entirely. Confirmed by reading the installed @uipath/delegate-sdk's bundled dist/index.mjs directly: sendMessage() always resolves to a plain string, never an object, and no event forwarded through agent.onEvent() ever carries a `usage` field. The SDK's per-turn token accounting lives only in its internal store, reachable through DelegateAgent.getLastTurnUsage() (and the session id through getSessionId()) -- called nowhere in this agent before now. - delegate_host.mjs: call both getters right after sendMessage() resolves and attach them to the send_ok message (usage, sessionId). - delegate_agent.py: _handle_send_ok reads them off send_ok's own top level instead of a nonexistent nested result dict. _parse_usage now reads the getter's real (confirmed, not guessed) shape -- promptTokens/completionTokens/promptTokensCached/cacheCreationTokens -- computing uncached_input_tokens = promptTokens - promptTokensCached (promptTokens is the OpenAI-style total, promptTokensCached the cache-read subset). - Updated tests to the real send_ok shape and confirmed bucket names. - .claude/notes/agents.md § Delegate agent records the finding so it isn't re-litigated as a guess next time. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
delegateas a built-in agent (AgentKind.DELEGATE/DelegateAgentConfig), registered unconditionally likecodex/antigravity, wrapping the now-public@uipath/delegate-sdkvia a first-party Node stdio host (src/coder_eval/agents/delegate/delegate_host.mjs) since the internal-only@uipath/delegate-stdiohost isn't public.docs/agents/DELEGATE.md), example tasks (tasks/delegate/*.yaml), and unit/registration/live test suites — SDK event field shapes are marked# UNVERIFIEDpending a real backend run (excluded from golden-master coverage, covered bytests/test_delegate_agent_live.pyunder-m live).delegate-live-testsCI job (pr-checks.yml), manual-dispatch only, that ROPC-mints a fresh access token (mirroringcoder_eval_uipath'srefresh-auth.shflow) and runstasks/delegate/fizzbuzz_delegate.yamlend to end against a real backend. Skips cleanly until its 6DELEGATE_ROPC_*/DELEGATE_ORG_ID/DELEGATE_TENANT_IDsecrets are configured — they now are.Test plan
make verifypasses (5990 passed, 4 pre-existing/unrelated skips, 92.34% coverage)delegate-live-testsonce to confirm the ROPC mint + live SDK run actually pass against alpha, then lift the# UNVERIFIEDmarkers / add golden-master coverage if field shapes need correcting🤖 Generated with Claude Code