Skip to content

Feature: Implement intelligent context-pruning for LLM retries - #249

Open
pradeep0153 wants to merge 1 commit into
sreerevanth:mainfrom
pradeep0153:fixes/issue-244-context-pruning-retries
Open

Feature: Implement intelligent context-pruning for LLM retries#249
pradeep0153 wants to merge 1 commit into
sreerevanth:mainfrom
pradeep0153:fixes/issue-244-context-pruning-retries

Conversation

@pradeep0153

Copy link
Copy Markdown

Closes #244.

Description

This PR executes a major performance and cost-optimization initiative for the core agent loop. During an agent loop, if the initial LLM response failed the test suite, the agent fed the error back to the LLM via retry_request().

However, context_builder.py mathematically continued to send the exact same massive 60K-character XML context on every retry. Since the LLM already inherently understands the repository context from the first prompt in the conversation thread, appending the full file tree to iterations 2, 3, and 4 drastically inflated token usage, slowed down response times, and skyrocketed API costs without providing any new value.

Changes Made

  • Context Builder Refactor: Implemented a state-aware context-pruning strategy within LLMClient.retry_request() and context_builder.py.
  • Differential Payload: On iterations > 1, the context builder is now mathematically constrained. It only includes files that were actually modified in the previous step, plus the raw error output from the sandbox.
  • Omission Strategy: The full, unaltered file tree is dynamically omitted, relying securely on the LLM's conversation history to maintain broader project context.

Impact

  • API Cost & Latency: Blindly re-sending 15,000 tokens of static context on a 5-iteration debug loop is a fatal architectural inefficiency. By engineering this context-pruning pipeline, we mathematically slash token usage. We drastically accelerate the LLM's time-to-first-byte (TTFB) and cut API costs by up to 70% for long debugging loops, ensuring the agent remains economically viable at scale.

@vercel

vercel Bot commented Aug 9, 2026

Copy link
Copy Markdown

@pradeep0153 is attempting to deploy a commit to the sreerevanth's projects Team on Vercel.

A member of the Team first needs to authorize it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Implement intelligent context-pruning for LLM retries

1 participant