Prerequisites
Background / Description
GoalPipeline.__init__ documents max_retries as the maximum number of retries for the executor and verifier to generate valid structured outputs, and stores it as self.max_retries. However, neither structured-output retry loop reads that value.
When either agent repeatedly completes without structured_output, GoalPipeline.reply_stream() invokes it indefinitely. The verifier has another unbounded path when its stream ends without a completed final message: if final_msg is None: continue.
This is separate from max_iters: max_iters limits executor-verifier goal attempts after an explicit fail verdict, while malformed or missing structured output is retried inside a single goal attempt.
Expected behavior
max_retries should bound malformed/missing structured-output retries independently for the executor and verifier. Following the existing AgentScope retry convention, max_retries=1 should allow the initial call plus one retry. Once exhausted, the pipeline should raise a clear error instead of silently appearing to finish.
HITL and external-execution events should continue to park the pipeline without consuming the malformed-output retry budget.
Actual behavior
With max_retries=1, both executor and verifier can be called more than two times and continue indefinitely if they never produce structured output.
Error Messages
No exception is raised by GoalPipeline. A guarded reproducer stopped each loop after the fifth call:
executor: guard stopped executor; calls=5; configured_retries=1
verifier: guard stopped verifier; calls=5; configured_retries=1
Steps to Reproduce
The existing StubAgent and _no_output helpers in tests/pipeline_goal_test.py can reproduce the behavior:
executor = StubAgent("executor", [[_no_output("executor")]])
verifier = StubAgent("verifier", [[_verdict("pass")]])
pipe = GoalPipeline(executor, verifier, max_retries=1)
await self._run(pipe, self.query) # never returns
The same occurs on the verifier path:
executor = StubAgent("executor", [[_report()]])
verifier = StubAgent("verifier", [[_no_output("verifier")]])
pipe = GoalPipeline(executor, verifier, max_retries=1)
await self._run(pipe, self.query) # never returns
Proposed fix
- Enforce
max_retries + 1 total attempts separately in the executor and verifier structured-output loops.
- Treat a verifier stream with no completed final message as an invalid attempt too.
- Preserve existing HITL and
max_iters behavior.
- Add regression tests for executor exhaustion, verifier exhaustion, and success on the last allowed attempt.
I would be happy to prepare a focused PR for this.
Environment
- AgentScope Version:
main at 10eaaac269101e25ad9f808353c45e305c6d7231
- Python Version: 3.11
- OS: Windows 11
Prerequisites
Background / Description
GoalPipeline.__init__documentsmax_retriesas the maximum number of retries for the executor and verifier to generate valid structured outputs, and stores it asself.max_retries. However, neither structured-output retry loop reads that value.When either agent repeatedly completes without
structured_output,GoalPipeline.reply_stream()invokes it indefinitely. The verifier has another unbounded path when its stream ends without a completed final message:if final_msg is None: continue.This is separate from
max_iters:max_iterslimits executor-verifier goal attempts after an explicitfailverdict, while malformed or missing structured output is retried inside a single goal attempt.Expected behavior
max_retriesshould bound malformed/missing structured-output retries independently for the executor and verifier. Following the existing AgentScope retry convention,max_retries=1should allow the initial call plus one retry. Once exhausted, the pipeline should raise a clear error instead of silently appearing to finish.HITL and external-execution events should continue to park the pipeline without consuming the malformed-output retry budget.
Actual behavior
With
max_retries=1, both executor and verifier can be called more than two times and continue indefinitely if they never produce structured output.Error Messages
Steps to Reproduce
The existing
StubAgentand_no_outputhelpers intests/pipeline_goal_test.pycan reproduce the behavior:The same occurs on the verifier path:
Proposed fix
max_retries + 1total attempts separately in the executor and verifier structured-output loops.max_itersbehavior.I would be happy to prepare a focused PR for this.
Environment
mainat10eaaac269101e25ad9f808353c45e305c6d7231