Skip to content

Add local-model variant of the opportunities agent - #8

Open
makegov-mark[bot] wants to merge 2 commits into
mainfrom
add-local-opportunities-agent
Open

Add local-model variant of the opportunities agent#8
makegov-mark[bot] wants to merge 2 commits into
mainfrom
add-local-opportunities-agent

Conversation

@makegov-mark

@makegov-mark makegov-mark Bot commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • New examples/opportunities-agent/local_agent.py — the mirror of agent.py swapped onto any OpenAI-compatible endpoint (Ollama, LM Studio, vLLM, llama.cpp). Same tool catalog, same loop shape; the two files diff cleanly so the diff itself is the lesson on where the Anthropic and OpenAI tool-use protocols differ.
  • Hardened LOCAL_SYSTEM_PROMPT + tighter MAX_TURNS=6 — soft guidance like "two empty searches is a signal to stop" that Sonnet self-regulates on lets a ~30B local model burn every available turn keyword-grinding. Numbered hard rules and per-phase call caps turn a failed local run into a working demo.
  • sam_url now comes from Tango (in both agent.py and local_agent.py) — the brief includes a clickable SAM.gov link. Tango computes it using the latest notice id with hyphens stripped, which is what SAM.gov actually accepts; the original opportunity_id (hyphenated) silently 404s on SAM. Done by requesting sam_url via the shape= parameter rather than constructing the URL client-side.
  • As a side effect of correcting the field names in the new shape= string (naics_code/psc_code, not naics/psc), six fields the original _trim_opportunity was aspirationally requesting actually show up in the tool result for the first time.
  • just local-agent, openai>=1.50 in the examples dependency group, and LOCAL_MODEL/LOCAL_BASE_URL/LOCAL_API_KEY documented in .env.example.

Test plan

  • just agent (default question) — still works against Anthropic, sam_url now appears in the brief
  • just local-agent against LM Studio (google/gemma-4-26b-a4b-qat) — clean 3-call run, brief includes correct SAM.gov link (hyphens stripped, latest notice id)
  • just local-agent "<harder SDVOSB question>" against remote vLLM endpoint — completes under MAX_TURNS, honest "no incumbent found"
  • Confirmed generated SAM URLs return 200 (curl -I https://sam.gov/opp/<id>/view)

Caveats

  • Local-model tool-calling on smaller models is stochastic. LM Studio's chat template for some Gemma builds leaks gpt-oss-style `<|channel|>` markers into the assistant content, which produces empty `tool_calls` and stalls the loop. Worked around at the prompt level, but ultimately a template-side issue you may want to fix in LM Studio before demoing.
  • This example is not run in CI (paid LLM API + non-deterministic output).

vdavez added 2 commits June 5, 2026 20:23
`local_agent.py` is the mirror of `agent.py` swapped onto any OpenAI-compatible
endpoint (Ollama, LM Studio, vLLM, llama.cpp). Same tool catalog and same loop
shape so the two files diff cleanly — the diff itself is the lesson on where the
Anthropic and OpenAI tool-use protocols differ.

Carries a separate, hardened `LOCAL_SYSTEM_PROMPT` because soft guidance like
"two empty searches is a signal to stop" that Sonnet self-regulates on lets a
~30B local model burn every available turn keyword-grinding. Numbered hard
rules, per-phase call caps, and a tighter MAX_TURNS=6 turn a failed local run
into a working demo.

Also tightens the SAM.gov link surfaced in the brief — both the Anthropic and
local variants now ask Tango for the `sam_url` field via the `shape=` parameter
rather than constructing it client-side. Tango computes it using the latest
notice id with hyphens stripped, which is what SAM.gov actually accepts; the
original opportunity_id (hyphenated) silently 404s on SAM. As a side effect,
the corrected `shape` field names (`naics_code`/`psc_code`, not `naics`/`psc`)
make six fields the original `_trim_opportunity` was aspirationally requesting
actually show up in the tool result for the first time.
`{{question}}` substituted the literal string into the recipe, so apostrophes
in questions like "that's likely the incumbent I'd be competing against"
landed in /bin/sh as an unmatched single quote and the recipe died with a
syntax error before python ever ran. `{{ quote(question) }}` wraps the value
with single quotes and escapes embedded singles as `'\''`, so the user can
pass any question they want.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant