Skip to content

feat(agent): use footnote-style conversation citations to reduce chat clutter - #981

Merged
ussaama merged 2 commits into
mainfrom
sam/clean-up-citation-clutter
Aug 10, 2026
Merged

feat(agent): use footnote-style conversation citations to reduce chat clutter#981
ussaama merged 2 commits into
mainfrom
sam/clean-up-citation-clutter

Conversation

@dembrane-sam-bot

Copy link
Copy Markdown
Contributor

What this changes

  • Agentic chat uses standard GFM footnotes (superscripts) inline, moving all raw [conversation_id:...] citation tags to the bottom under a "Sources" list — echo/agent/agent.py
  • Inline chat text is clean, readable, and free of repeated citation link blocks — echo/agent/agent.py
  • Footnotes mapped inside footnote definitions render as rich clickable links pointing to specific transcripts or transcript excerpts on the frontend seamlessly — echo/agent/agent.py

Confidence

Confidence: High. The system prompt changes explicitly direct the model to use GFM footnotes ([^1]) and group multiple claims referencing the same conversation/chunk under unique footnotes at the end. The frontend regex (AgenticChatPanel.tsx) replaces [conversation_id:...] on the entire message text, meaning it will perfectly convert the footnote definition tags into rich links. Standard GFM footnotes are natively parsed by our existing Markdown.tsx component which uses remarkGfm. Added a unit test assertion to verify the revised system prompt.

Discussion: https://dembraneworkspace.slack.com/archives/C0884QPQF6W/p1786381186184369

@ussaama
ussaama enabled auto-merge August 10, 2026 17:20
@ussaama
ussaama added this pull request to the merge queue Aug 10, 2026
Merged via the queue into main with commit 8a75a39 Aug 10, 2026
7 checks passed
spashii added a commit that referenced this pull request Aug 10, 2026
The regex that removed a model-written "Sources"/"Footnotes" heading
from already-persisted messages was a shim: it compensated in render
code, forever, for a few rows written in the hours between #981 and
this fix, and it only knew the English words. The handful of affected
rows get a one-off data cleanup instead; the renderer keeps owning the
header for everything written from now on.
spashii added a commit that referenced this pull request Aug 10, 2026
)

### What this fixes

Follow-up to #981. On the reported chat (`57493241`, dashboard) the
agentic reply showed **two headings** ("Sources" and "Footnotes"), the
**superscript citation links gave no feedback** when clicked, and every
source line stacked **several arrow icons** (the external-link arrow
plus one ↩ back-reference per citation).

### Why it happened

- The prompt told the model to write a literal "Sources" header above
its footnote definitions. remark-gfm then moves the definitions into its
own trailing section headed "Footnotes", so the model's header stays
behind as an empty duplicate.
- The renderer's "Footnotes" heading ships with an `sr-only` class,
which our Tailwind build never generates (no source file uses it), so
the heading that is meant to be invisible is visible.
- The superscripts use fragment navigation. Every message mints the same
ids (`#user-content-fn-1`), a repeated click on the same fragment does
nothing, and when the footnote list is already on screen (the exact case
in the reported chat, verified against the stored row in prod) nothing
visibly happens at all.

### What changed

- `Markdown.tsx` takes an optional footnote label and id prefix and
passes them through remark-rehype: the section heading is ours now
(visible, localised) and footnote ids are unique per message. Surfaces
that pass no label keep the default "Footnotes" heading but hidden
behind `dembrane-sr-only`, a visually-hidden class we define ourselves
in `index.css` (our own name so it cannot collide with Tailwind's
`sr-only` utility).
- `ChatHistoryMessage.tsx` passes `Sources` (per locale, `Bronnen` in
Dutch, other catalogs fall back to English until translated) plus a
per-message id prefix. A superscript click scrolls to the source line
and flashes it with the same cyan highlight the transcript page uses for
a deep-linked chunk — feedback even when the target was already on
screen, no URL rewrite, no history entries. The ↩ back-references are
dropped in agentic chat: one icon per source line.
- `agent.py` no longer asks the model for a header above the
definitions; the renderer owns the heading. Prompt test updated to pin
this.

### Already-persisted rows

Exactly one row in all of production stores the old model-written header
(verified by query): the beta test chat this was reported on. Agentic
chat is beta; the row stays as is. No migration, no render-time
compatibility code.

### Tests

- `AgenticChatPanel.test.tsx`: 15 passed, including a new case pinning
that footnote definition tags become rich transcript links.
- `agent/tests/test_agent_tools.py`: 65 passed.
- `tsc --noEmit` and `biome lint` clean; end-to-end renders verified:
one "Sources" heading in chat, hidden "Footnotes" heading on label-less
surfaces, unique prefixed ids.

Discussion:
https://dembraneworkspace.slack.com/archives/C0884QPQF6W/p1786381186184369
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants