Skip to content

Missing hook points for LLM input/output interception inside the agent loop #253

Description

@badbye

Problem

Bub's hook system is rich at the Framework layer (inbound processing, state management, outbound rendering) but almost entirely absent from the Agent loop layer — the multi-step cycle where LLM calls are dispatched, tool results are processed, and follow-up prompts are constructed.

Framework layer (hook-rich)
  ├── resolve_session        ← hook
  ├── load_state             ← hook
  ├── build_prompt           ← hook (only handles the first user input)
  └── run_model              ← hook (delegates to Agent.run())
        │
        ▼
Agent layer (hook-desert)
  ├── _agent_loop
  │     ├── _run_once
  │     │     ├── _system_prompt()  → system_prompt hook (only hook here)
  │     │     └── tape.run_tools_async()  → no hooks
  │     ├── CONTINUE_PROMPT constructed  → no hooks
  │     ├── tool auto-result resolved   → no hooks
  │     └── auto-handoff decision       → no hooks
  │
  ▼
Framework layer (hook-rich)
  ├── save_state             ← hook
  ├── render_outbound        ← hook
  └── dispatch_outbound      ← hook

Specifically:

1. build_prompt only handles the first user input

build_prompt fires once in framework.process_inbound(). Inside the agent loop, subsequent prompts are constructed with zero hook involvement:

2. No hook before each LLM call

_run_once() assembles prompt, system_prompt, tools, and model, then calls Republic directly. No hook exists between assembly and dispatch. Plugins cannot inspect, audit, or modify the request.

3. No hook after each LLM response

After _run_once() returns, the output is classified by _resolve_tool_auto_result() and drives loop control directly. Plugins cannot observe intermediate outputs, transform responses, or export per-step metrics.

Proposal: Add before_llm_call and after_llm_call hooks

New hook specifications in hookspecs.py:

  • before_llm_call(prompt, system_prompt, model, tools, state) — called before each LLM API request. Return a dict with optional keys (prompt, system_prompt, model, tools) to override, or None to pass through.
  • after_llm_call(prompt, system_prompt, model, tools, state, output, step, status, elapsed_ms) — observer hook after each LLM call, regardless of outcome. Return value ignored.

Framework delegation in framework.py — add public methods so the Agent doesn't touch _hook_runtime directly:

def before_llm_call(self, **kwargs) -> dict | None:
    return self._hook_runtime.call_first_sync("before_llm_call", **kwargs)

def after_llm_call(self, **kwargs) -> None:
    self._hook_runtime.call_many_sync("after_llm_call", **kwargs)

Integration in agent.py:

  • _run_once(): call self.framework.before_llm_call(...) after assembling parameters but before the Republic call. Apply overrides if returned.
  • _run_tools_with_auto_handoff() / _stream_events_with_auto_handoff(): call self.framework.after_llm_call(...) after each step outcome is resolved.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions