Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
88855f2
feat(chat): one chip row for documents, folders and knowledge bases
ViaJables Sep 1, 2026
47eea5f
feat(certification): meet the time-strapped RA where she starts (DGA …
ViaJables Sep 1, 2026
437ed8b
fix(workflows): say where a conflicting workflow name actually is
ViaJables Sep 1, 2026
2ff1005
build(makefile): pin the review-graph target to the main working tree
arhyneRWU Sep 2, 2026
ce60bca
docs(review-graph): say that semantic search degrades silently
arhyneRWU Sep 2, 2026
d0894d6
docs: changelog for the review-graph worktree and search-mode fixes
arhyneRWU Sep 2, 2026
4eaeaee
feat(budget): a margin measured by the provider, not guessed
arhyneRWU Sep 2, 2026
a83c4cf
fix(routing): size a request on the ruler it was measured with
arhyneRWU Sep 2, 2026
5e028a2
feat(budget): ask Anthropic and Google what a request actually costs
arhyneRWU Sep 2, 2026
9a1a2c0
feat(chat): measure the budget against the provider, once per turn
arhyneRWU Sep 2, 2026
ca2e0a4
docs: changelog for provider-native token counting
arhyneRWU Sep 2, 2026
72bc542
fix(branding): stop showing a deployment's logo above our gold button
arhyneRWU Sep 2, 2026
e472d21
feat(branding): the dark chrome follows the brand colour
arhyneRWU Sep 2, 2026
5905bd8
fix(models): make vLLM structured output work, and make the Test butt…
ViaJables Sep 4, 2026
35f3521
fix(branding): keep a visible focus ring on the branded auth pages
ViaJables Sep 4, 2026
0cba64d
fix(budget): stop retrying a dead count endpoint on every chat turn
ViaJables Sep 4, 2026
2943488
fix(review-graph): a path with a space, and embed pointing at the wor…
ViaJables Sep 4, 2026
cf28c8e
Merge origin/main into stack for #795
ViaJables Sep 8, 2026
e40ee12
Merge stack/795 into stack for #799
ViaJables Sep 8, 2026
88b8745
fix(certification): the pop-out window opens the course fullscreen, n…
ViaJables Sep 8, 2026
10e57a8
Merge stack/799 into stack for #800
ViaJables Sep 8, 2026
1a61fe2
fix(workflows): the 'still in your library' check looks at the user's…
ViaJables Sep 8, 2026
ed956e0
Merge stack/800 into stack for #815
ViaJables Sep 8, 2026
f920a90
chore(review-graph): ignore the graph directory the Makefile now crea…
ViaJables Sep 8, 2026
69d7b08
Merge stack/815 into stack for #857
ViaJables Sep 8, 2026
f3cf484
fix(diagnostics): the structured-output failure summary names what ac…
ViaJables Sep 8, 2026
0712093
Merge stack/857 into stack for #818
ViaJables Sep 8, 2026
6578cf3
fix(routing): log the rejected boolean margin itself, not None
ViaJables Sep 8, 2026
bb47870
Merge stack/818 into stack for #821
ViaJables Sep 8, 2026
33645b8
Merge stack/821 into stack for #824
ViaJables Sep 8, 2026
cdc1123
fix(changelog): the provider-native token counting entry belongs unde…
ViaJables Sep 8, 2026
90bddbf
Merge branch 'refs/heads/stack/818' into stack/821
ViaJables Sep 8, 2026
f19d8b8
Merge branch 'refs/heads/stack/821' into stack/824
ViaJables Sep 8, 2026
9de25fd
fix(branding): a malformed brand colour falls back to the neutral chrome
ViaJables Sep 8, 2026
451c6dc
fix(test): restore the describe closing braces lost in the merge of #…
ViaJables Sep 8, 2026
babbe5e
Merge branch 'refs/heads/stack/821' into stack/824
ViaJables Sep 8, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,8 @@ scripts/ui_endpoint_map.*
# Per-developer AI/editor tool config, written into the repo root by tools like
# code-review-graph's installer. Personal, not project config.
.mcp.json
# The graph itself (and the lock file `make review-graph` takes before building it).
.code-review-graph/
.cursor/
.kiro/
.qoder/
10 changes: 10 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

33 changes: 26 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -51,18 +51,37 @@ backend-audit:
# wrongly. Hence the explicit build branch. ORIG_HEAD (set by git across a
# pull) is the right base for a refresh; HEAD~1, the tool's default, would
# index only the last commit of a multi-commit pull.
#
# --repo is passed explicitly because the tool otherwise discovers the repo
# from the working directory, which under `git worktree` is the *worktree* --
# so a run from one builds a second index there, holding only the files that
# worktree has touched, and answers every query from it with status "ok". A
# partial index returns 0 for anything it never parsed, and a true 0 looks
# identical: `callers_of` on a live symbol reports no callers, which reads as
# "safe to change". The first `git worktree list` entry is always the main
# working tree, so one shared graph serves every worktree.
review-graph:
@command -v code-review-graph >/dev/null 2>&1 || { \
printf 'review-graph: code-review-graph is not installed.\n' >&2; \
printf ' install it with: uv tool install code-review-graph\n' >&2; \
printf " install it with: uv tool install 'code-review-graph[embeddings]'\n" >&2; \
printf ' it is optional -- no other make target needs it.\n' >&2; \
exit 1; }
@if [ ! -f .code-review-graph/graph.db ]; then \
code-review-graph build; \
else \
base=$${CRG_BASE:-$$(git rev-parse --verify --quiet ORIG_HEAD || echo HEAD~1)}; \
code-review-graph update --base "$$base"; \
fi
@root=$$(git worktree list --porcelain | sed -n '1s/^worktree //p'); \
if [ -z "$$root" ]; then \
printf 'review-graph: cannot resolve the main working tree.\n' >&2; exit 1; \
fi; \
mkdir -p "$$root/.code-review-graph"; \
if [ -f "$$root/.code-review-graph/graph.db" ]; then \
base=$${CRG_BASE:-$$(git -C "$$root" rev-parse --verify --quiet ORIG_HEAD || echo HEAD~1)}; \
set -- update --repo "$$root" --base "$$base"; \
else \
set -- build --repo "$$root"; \
fi; \
if command -v flock >/dev/null 2>&1; then \
flock "$$root/.code-review-graph/.lock" code-review-graph "$$@"; \
else \
code-review-graph "$$@"; \
fi

# Writes scripts/ui_endpoint_map.md and .json for reading; regenerate on demand,
# never commit (see .gitignore).
Expand Down
199 changes: 181 additions & 18 deletions backend/app/services/chat_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,17 @@
from app.services.config_service import get_llm_model_by_name, get_user_model_name
from app.services.context_budget import (
DocumentSegment,
NativeCount,
estimate_input_tokens,
plan_and_compact_context,
token_safety_margin,
)
from app.services.model_routing import (
RoutingDecision,
choose_document_model,
suggest_document_model,
)
from app.services.native_token_count import count_natively, native_count_for
from app.services.page_locator import (
annotate_chunk_pages,
cited_pages,
Expand Down Expand Up @@ -420,12 +423,137 @@ def select_chat_system_prompt(
return NO_DOCUMENT_SYSTEM_PROMPT


def _build_chat_prompt(
message: str,
documents: list[DocumentSegment],
attachments: list[DocumentSegment],
*,
have_context: bool,
include_onboarding_context: bool,
) -> str:
"""The single user prompt this turn sends, documents and attachments included.

Extracted so the native pre-flight can count the request chat is actually
about to make. The provider is asked for one figure over one payload, and
in document chat the documents *are* the payload — they are also the
digit-dense content the 1.5 default margin was sized for. Measuring the
divergence on the question and the history alone would sample flowing prose
and then apply the result to a budget table.

The onboarding-only branch keeps its own wording deliberately: it is the
prompt the model sees, not merely one we count, and collapsing it into the
reference-documents shape would change an answer.
"""
if not (have_context or include_onboarding_context):
return message

context_pieces: list[str] = [s.text for s in documents]
context_pieces.extend(s.text for s in attachments)
context_block = "\n\n".join(context_pieces)

if include_onboarding_context and not have_context:
return f"{context_block}\n\nUser question: {message}"
return (
f"{message}\n\n"
"--- BEGIN REFERENCE DOCUMENTS (provided for context only) ---\n"
f"{context_block}\n"
"--- END REFERENCE DOCUMENTS ---"
)


async def _measure_native_count(
*,
model_name: str,
model_config: Optional[dict],
sys_config_doc: Optional[dict],
system_prompt: str,
prompt: str,
history: list,
) -> Optional[NativeCount]:
"""Ask the provider what this turn costs, or return None.

None is the ordinary answer: most deployments are OpenAI-compatible, and
`count_natively` gates those out before building anything, so this costs
nothing at all there. No gating happens here — one gate, in one place.

``prompt`` must be the assembled user prompt, not the bare message, and the
baseline is taken over the same components by `native_count_for`. The two
have to describe the same characters or the ratio they form is not a
measurement.

Wrapped even though `count_natively` promises never to raise. This sits in
front of the first token of a chat response, and a budget refinement is
never worth a failed answer; the promise is a property of today's providers,
not of every one that will be added.
"""
try:
result = await count_natively(
model_name=model_name,
model_config=model_config,
system_config_doc=sys_config_doc,
system_prompt=system_prompt,
user_message=prompt,
history=history,
)
return native_count_for(
result,
model_name=model_name,
model_config=model_config,
system_prompt=system_prompt,
user_message=prompt,
history=history,
)
except Exception:
logger.debug(
"native token count unavailable for %s", model_name, exc_info=True
)
return None


def _native_for_model(
native: Optional[NativeCount], model_name: str
) -> Optional[NativeCount]:
"""The count, but only when it was measured for *this* model.

Routing can reassign the model between the count and the plan. Vocabularies
differ per model, so carrying a measurement across is the wrong-ruler
mistake the safety margin exists to correct. `context_budget._native_margin`
does reject on the name mismatch, so this fails closed either way — but a
stale count travelling silently through the planner on the strength of a
check two modules away is not something to leave implicit.
"""
if native is None or native.model_name != model_name:
return None
return native


def _margin_for(
native: Optional[NativeCount], model_name: str, model_config: Optional[dict]
) -> Optional[float]:
"""The safety margin an estimate measured with ``native`` actually carries.

`token_safety_margin` and not ``tokens / baseline_tokens`` computed here:
that function weighs exactness, a deployment's configured margin and
tiktoken-is-the-real-tokenizer *ahead* of the measurement, and it is what
`estimate_input_tokens` just applied. A ratio re-derived at this call site
would hand routing a number the request was never sized with.

None when there is no count, which is what keeps the routing call identical
to what it was: ``current_margin=None`` makes `model_routing._sized_for`
derive the margin from the name and config exactly as before.
"""
if native is None:
return None
return token_safety_margin(model_name, model_config, native=native)


def _suggest_model_for_overflow(
compacted,
model_name: str,
model_config: Optional[dict],
sys_config_doc: dict,
input_tokens: int,
current_margin: Optional[float] = None,
) -> Optional[dict]:
"""A larger model to offer, or None when nothing needs offering.

Expand All @@ -439,6 +567,12 @@ def _suggest_model_for_overflow(
is always within the current model's budget. Passing it made
:func:`suggest_document_model` return None every time and the dialog's
fourth option could never appear.

``current_margin`` is the margin ``input_tokens`` was measured with, when
the caller knows it — a provider-native count carries a much tighter one
than :func:`token_safety_margin` can derive from a name and a config, and
dividing by a factor never applied would understate the request and offer a
model it would then fail on. Omitted, the margin is derived as before.
"""
if not compacted.actions:
return None
Expand All @@ -447,6 +581,7 @@ def _suggest_model_for_overflow(
current_config=model_config,
models=(sys_config_doc or {}).get("available_models") or [],
input_tokens=input_tokens,
current_margin=current_margin,
)
if not suggestion:
return None
Expand Down Expand Up @@ -728,6 +863,28 @@ async def chat_stream(
# is an upper bound rather than an optimistic one. Routing decides off this
# number: an estimate that reads low makes the router see headroom that is
# not there and decline to move a request that does not fit.
#
# Before estimating, ask the provider what this turn really costs. Anthropic
# and Google will say; everything else is gated out inside `count_natively`
# at no cost. The payload counted is the prompt this turn is about to send —
# built from the *uncompacted* segments, so it covers the same text
# `estimate_input_tokens` is about to size, and so the ratio is measured
# over the documents that dominate it rather than over the question alone.
native = await _measure_native_count(
model_name=model_name,
model_config=model_config,
sys_config_doc=sys_config_doc,
system_prompt=system_prompt or "",
prompt=_build_chat_prompt(
message,
doc_segments,
attachment_segments,
have_context=have_context,
include_onboarding_context=include_onboarding_context,
),
history=previous_messages,
)

requested_input_tokens = estimate_input_tokens(
model_name=model_name,
system_prompt=system_prompt or "",
Expand All @@ -736,7 +893,14 @@ async def chat_stream(
documents=doc_segments,
attachments=attachment_segments,
model_config=model_config,
native=native,
)
# The margin `requested_input_tokens` carries. A property of that number,
# not of whichever model is current — so it is computed once here and is
# deliberately *not* recomputed after routing switches models below: the
# measurement it describes does not change when the model does, and
# `_sized_for` needs it to restate that number in a candidate's units.
requested_margin = _margin_for(native, model_name, model_config)

routing = RoutingDecision(model_name, False, "")
candidate_name = (sys_config_doc or {}).get("long_document_model") or ""
Expand All @@ -748,6 +912,7 @@ async def chat_stream(
candidate_name=candidate_name,
candidate_config=candidate_config,
input_tokens=requested_input_tokens,
current_margin=requested_margin,
)
if routing.switched:
logger.info(
Expand All @@ -763,6 +928,11 @@ async def chat_stream(
history=previous_messages,
documents=doc_segments,
attachments=attachment_segments,
# Dropped when routing switched models above: the count was measured
# against the previous model's tokenizer and says nothing about this
# one. `_native_margin` would refuse it on the name mismatch, but a
# stale measurement should not reach the planner and depend on that.
native=_native_for_model(native, model_name),
)

# Tell the client what we planned (and whether we had to compact).
Expand All @@ -777,7 +947,7 @@ async def chat_stream(
# would walk around that gate.
"suggested_model": _suggest_model_for_overflow(
compacted, model_name, model_config, sys_config_doc,
requested_input_tokens,
requested_input_tokens, requested_margin,
),
}) + "\n"
# Switching the model without saying so is the same failure as trimming a
Expand Down Expand Up @@ -861,23 +1031,16 @@ async def chat_stream(

previous_messages = compacted.history

# Rebuild the final prompt from compacted segments.
if have_context or include_onboarding_context:
context_pieces: list[str] = [s.text for s in compacted.documents]
context_pieces.extend(s.text for s in compacted.attachments)
context_block = "\n\n".join(context_pieces)
if include_onboarding_context and not have_context:
# Preserve the original onboarding wording when that's the only context.
prompt = f"{context_block}\n\nUser question: {message}"
else:
prompt = (
f"{message}\n\n"
"--- BEGIN REFERENCE DOCUMENTS (provided for context only) ---\n"
f"{context_block}\n"
"--- END REFERENCE DOCUMENTS ---"
)
else:
prompt = message
# Rebuild the final prompt from compacted segments. Same assembly the
# native pre-flight counted above, over the trimmed segments this time —
# sharing it is what keeps the counted payload the shape of the real one.
prompt = _build_chat_prompt(
message,
compacted.documents,
compacted.attachments,
have_context=have_context,
include_onboarding_context=include_onboarding_context,
)

agent = create_chat_agent(model_name, system_prompt=system_prompt, system_config_doc=sys_config_doc)

Expand Down
Loading
Loading