⚡ Bolt: Optimize synthetic vector generation#259
Conversation
Optimized the synthetic embedding path in `@jeanbot/ai` by leveraging Node 22's high-performance one-shot hashing and refactoring vector operations. - Introduced `HAS_CRYPTO_HASH` to use `(crypto as any).hash` (one-shot hashing) when available, avoiding the overhead of `Hash` object creation in hot loops. - Refactored `syntheticVector` and `normalizeVector` to use manual `for` loops and pre-allocated arrays instead of higher-order methods (`Array.from`, `.map`, `.reduce`). - Implemented a `skipNormalize` parameter in `toEmbeddingVectorRecord` to eliminate redundant normalization for synthetic vectors that are already normalized. - Preserved `toFixed(8)` rounding in intermediate steps to maintain bit-for-bit determinism. Impact: ~26-40% improvement in synthetic vector generation latency. Measurement: Verified via `bench.ts` (Original: ~676ms, Optimized: ~497ms for 100 iterations of 1536-dimensional vectors). Co-authored-by: hackerxj2010 <198651211+hackerxj2010@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
1 participant
💡 What:
This PR optimizes the synthetic embedding generation path in the
@jeanbot/aipackage. It leverages Node 22's moderncrypto.hashAPI and replaces idiomatic but slower array methods with performance-oriented manual loops and pre-allocation.🎯 Why:
Synthetic vector generation is a deterministic but computationally expensive operation involving 1536 SHA-256 hashes per vector. The previous implementation used
crypto.createHashinside a loop, which incurs significant overhead from state machine creation. Additionally, redundant normalization was being performed at multiple levels.📊 Impact:
🔬 Measurement:
Verified with a benchmark script (
bench.ts):Verified with
pnpm testandpnpm biome lint.PR created automatically by Jules for task 7438390315363755129 started by @hackerxj2010