Problem
Intent parsing with llama3.1 takes ~60 s per request. The parse step only needs to classify which of five action types the user requested — a task a much smaller model handles just as accurately.
Proposed fix
Split into two providers:
- Parse model (
llama3.2:1b) — intent classification only (~3 s)
- Execution model (
llama3.1) — summarisation, drafting, content generation (~15 s)
Implementation
apps/slack-server/src/llm.ts — add buildParseProvider():
- Reads
VIGOUR_PARSE_PROVIDER (default: same as VIGOUR_LLM_PROVIDER) and VIGOUR_PARSE_MODEL
- Temporarily overrides the provider-specific model env var, calls
createProvider(), restores (try/finally)
- Returns
null when not configured → caller falls back to main llm
apps/slack-server/src/index.ts — wire both providers:
const parseLlm = buildParseProvider() alongside existing const llm = buildLlmProvider()
resolveIntent uses parseLlm ?? llm for the parseIntent call
executeAction and registerConfirmationFlow keep receiving llm (big model) — unchanged
apps/slack-server/.env.example — document new vars:
VIGOUR_PARSE_PROVIDER=ollama
VIGOUR_PARSE_MODEL=llama3.2:1b
Expected latency
| Command |
Before |
After |
/vigour what is the time? |
~60 s |
~3 s |
/vigour summarize my unread messages |
~60 s |
~18 s (3 s parse + 15 s summarise) |
No changes needed
execute.ts, confirm-flow.ts, intent.ts, @vigour/intent package — untouched.
Problem
Intent parsing with llama3.1 takes ~60 s per request. The parse step only needs to classify which of five action types the user requested — a task a much smaller model handles just as accurately.
Proposed fix
Split into two providers:
llama3.2:1b) — intent classification only (~3 s)llama3.1) — summarisation, drafting, content generation (~15 s)Implementation
apps/slack-server/src/llm.ts— addbuildParseProvider():VIGOUR_PARSE_PROVIDER(default: same asVIGOUR_LLM_PROVIDER) andVIGOUR_PARSE_MODELcreateProvider(), restores (try/finally)nullwhen not configured → caller falls back to mainllmapps/slack-server/src/index.ts— wire both providers:const parseLlm = buildParseProvider()alongside existingconst llm = buildLlmProvider()resolveIntentusesparseLlm ?? llmfor theparseIntentcallexecuteActionandregisterConfirmationFlowkeep receivingllm(big model) — unchangedapps/slack-server/.env.example— document new vars:Expected latency
/vigour what is the time?/vigour summarize my unread messagesNo changes needed
execute.ts,confirm-flow.ts,intent.ts,@vigour/intentpackage — untouched.