Skip to content

feat(embed): resolve encoder model and width from config - #12

Merged
josaum merged 1 commit into
mainfrom
feat/configurable-embedding-dim
Nov 4, 2026
Merged

josaum merged 1 commit into
mainfrom
feat/configurable-embedding-dim

Conversation

@josaum

@josaum josaum commented Nov 4, 2026

Copy link
Copy Markdown
Owner

Why

EMBED_DIM was pinned to 1024 and EMBED_MODEL to BAAI/bge-m3. All three ingest paths compared the response width against the constant (src/embed.rs OpenAI / TEI / Gemini parsers) and the Gemini path forced outputDimensionality: 1024, so no other encoder could be used at all.

Concretely: google/embeddinggemma-2 is 768-d (text_config.embedding_dim = 768; Matryoshka cuts are 512/256/128, never 1024), so it was rejected outright.

What changed

  • [embed] dim and [embed] model are now load-bearing. config::embed_dim() mirrors the existing embed_model(): LEIO_CODE_EMBED_DIM → [embed] dim → DEFAULT_EMBED_DIM (1024). Zero and unparseable values fall back rather than producing empty vectors.
  • Width is threaded through ingest instead of read from a constant — embed_texts_remote and the three parsers take dim, and Gemini sends it as outputDimensionality. Mismatches are still hard errors, now naming both widths.
  • Provenance is no longer false. Rows stamped embed_model from the EMBED_MODEL constant, so a row embedded by any other encoder still claimed BAAI/bge-m3. Rows now carry the pair that produced them.
  • The Gemini substitution moved out of the request builder into resolved_model(), called once before the stamp. Previously a Gemini URL with no [embed] model would send models/gemini-embedding-2 and stamp BAAI/bge-m3 — the same bug class in a different place.

Compatibility

Defaults are unchanged, so a repo with no [embed] table behaves exactly as before. Changing model or dim makes existing stored vectors incomparable; rebuild with export arrow-nodes.

Notes for review

  • embed_query_url() is untouched. It stays env-only so an inspected tree cannot choose the host receiving query text or LEIO_CODE_EMBED_API_KEY.
  • embed_texts() (query path) resolves dim from the inspected repo, mirroring the pre-existing embed_model(repo_root) call on the same line. The isolation property is about the URL and key, not the width — flagging it so it is a decision rather than an oversight.
  • No sidecar header field was added. Cross-width mixing is already prevented per-vector at src/local_nodes.rs:405, which only scores stored vectors whose length equals the query's; others fall through to on-demand embedding or lexical ranking. A header check would be redundant with that guard.
  • Known residual gap, documented not fixed: two different models at the same width are not distinguished by the width guard. docs/CLI-GUIDE.md says plainly not to point one store at two encoders.

Verification

  • cargo test — 900 passed, 0 failed
  • cargo clippy --all-targets — clean
  • cargo fmt applied

New tests cover a 768-d encoder accepted end-to-end through all three parsers, a 1024-d response rejected against a 768-d store, config resolution and its zero/default fallbacks, and that the stamped model equals the model actually sent.

🤖 Generated with Claude Code

`EMBED_DIM` was pinned to 1024 and `EMBED_MODEL` to `BAAI/bge-m3`, so any
other encoder was rejected outright: all three ingest paths compared the
response width against the constant, and the Gemini path forced
`outputDimensionality: 1024`. A locally served 768-d encoder such as
`google/embeddinggemma-2` could not be used at all.

Both values now come from `[embed] model` / `[embed] dim` in
`.leio-code/config.toml`, with `LEIO_CODE_EMBED_MODEL` / `LEIO_CODE_EMBED_DIM`
overriding and the old pair as defaults, so existing stores are unaffected.

Also fixes false provenance: embedded rows stamped `embed_model` from the
constant rather than the configured id, so a row embedded by any other encoder
still claimed to be bge-m3. Rows now carry the pair that actually produced
them. The Gemini substitution is resolved once in `resolved_model` before the
stamp is taken, rather than inside the request builder, so the id sent and the
id recorded cannot drift apart.

`embed_query_url()` stays env-only — an inspected tree must not choose the
host that receives query text or `LEIO_CODE_EMBED_API_KEY`.

Width mismatches remain errors, now against the configured width; the search
path already refuses to score vectors whose width differs from the query's.

900 tests pass, clippy clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@josaum
josaum merged commit 23b67c2 into main Nov 4, 2026
4 checks passed
@josaum
josaum deleted the feat/configurable-embedding-dim branch November 4, 2026 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant