Skip to content

chore(bench): publish proxy and memory HNSW and ranking-quality and write-pipeline benchmark data - #683

Merged
Rick1330 merged 9 commits into
mainfrom
chore/bench-data-publish
Sep 3, 2026
Merged

chore(bench): publish proxy and memory HNSW and ranking-quality and write-pipeline benchmark data#683
Rick1330 merged 9 commits into
mainfrom
chore/bench-data-publish

Conversation

@ibex-harness-benchmark

@ibex-harness-benchmark ibex-harness-benchmark Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

IBEX Benchmark Bot IBEX Benchmark Bot

What and Why

Automated publish of public benchmark history for ibex-harness. Suites in this PR: Proxy, Memory HNSW, Ranking quality, Write pipeline.

Keeps /benchmarks and /benchmarks/memory history current without contributor-authored commits.

Tracking issue

N/A (GitHub App data publish)

How

Proxy suite

Status:PASS

Field Value
Run number 886
Head SHA 078d837
Proxy p99 (k6) 1.31 ms
Proxy overhead (Go) 137.69 µs/op
Paths web/public/benchmarks/benchmark-data.json, web/public/benchmarks/badge.svg

k6 p99 is the authoritative SLA metric. Stage breakdown and ns/op values are synthetic Go microbenchmarks.

Memory HNSW suite

Field Value
Run number 166
Head SHA 9dcefa8
Mean recall@10 1.0000
Path web/public/benchmarks/hnsw-benchmark-data.json

Corpus cells

Size Recall@10 p95 ef min_sim iterative build
10000 1.0000 5.7ms 40 0.70 off bulk
100000 1.0000 8.8ms 40 0.70 off bulk

Production-like knobs expected (ef_search=40, min_similarity≈0.70, iterative_scan=off, bulk index build).

Ranking-quality suite

Field Value
Run number 166
Head SHA 9dcefa8
Precision@5 1.0000
Recall@10 1.0000
Path web/public/benchmarks/ranking-quality-benchmark-data.json

Write-pipeline suite

Field Value
Run number 166
Head SHA 9dcefa8
p95 latency 33.57 ms
Path web/public/benchmarks/write-pipeline-benchmark-data.json

SLA: p95 <= 200 ms.

Testing

  • Bot workflow validation passed (suite(s) above)
  • Harness CI green on this PR
  • /benchmarks and/or /benchmarks/memory preview updated

Performance

No runtime change — history JSON / badge only. Review suite tables above for SLA deltas.

Security

No secrets; App-signed commits; artifacts re-verified via Actions API before publish.

Migrations / Ops

None.

Docs

Public benchmark history only (web/public/benchmarks/*). Suite JSON files stay separate.

Checklist (Definition of Done)

  • Correct files only (proxy JSON+badge and/or suite JSON files)
  • Labels match files (benchmark-data, hnsw-benchmark-data, ranking-quality-benchmark-data, write-pipeline-benchmark-data)
  • Single shared data PR branch chore/bench-data-publish

ibex-harness-benchmark[bot] added 5 commits September 2, 2026 18:54
Signed-off-by: ibex-harness-benchmark[bot] <4215784+ibex-harness-benchmark[bot]@users.noreply.github.com>
Signed-off-by: ibex-harness-benchmark[bot] <4215784+ibex-harness-benchmark[bot]@users.noreply.github.com>
Signed-off-by: ibex-harness-benchmark[bot] <4215784+ibex-harness-benchmark[bot]@users.noreply.github.com>
Signed-off-by: ibex-harness-benchmark[bot] <4215784+ibex-harness-benchmark[bot]@users.noreply.github.com>
Signed-off-by: ibex-harness-benchmark[bot] <4215784+ibex-harness-benchmark[bot]@users.noreply.github.com>
@github-actions github-actions Bot added area/web Next.js web app, public site, dashboard area/benchmarks Benchmark collection, dashboard, published data labels Sep 3, 2026
codescene-delta-analysis[bot]

This comment was marked as outdated.

@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@ibex-harness-benchmark

ibex-harness-benchmark Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

View IBEX dashboard →

✅ All benchmarks passed

TL;DR: All collected suites passed. Proxy P99 vs baseline: 📉 -93.9%.

Suite matrix

Suite Primary SLA vs Baseline Status
Proxy 1.22 ms (P99) 📉 -93.9% ✅ PASS
Deep Dive: Proxy

Load test (k6)

  • P99: 1.22 ms (SLA < 20 ms)
  • Throughput: 1.4k req/s
  • Error rate: 0.00%

Synthetic overhead (Go)

  • Total: 257.02 µs/op
  • Stages: Auth LRU 0.38 µs · gRPC 0.16 µs · Rate limit 0.17 ms · Directive 0.08 ms · Prompt 47 ns

k6 P99 is the SLA. Stage / ns/op values are synthetic Go microbenchmarks, not live traces.

Environment & meta
Generated by IBEX Benchmark Bot

ibex-harness-benchmark[bot] added 3 commits September 3, 2026 11:52
Signed-off-by: ibex-harness-benchmark[bot] <4215784+ibex-harness-benchmark[bot]@users.noreply.github.com>
Signed-off-by: ibex-harness-benchmark[bot] <4215784+ibex-harness-benchmark[bot]@users.noreply.github.com>
Signed-off-by: ibex-harness-benchmark[bot] <4215784+ibex-harness-benchmark[bot]@users.noreply.github.com>
codescene-delta-analysis[bot]

This comment was marked as outdated.

codescene-delta-analysis[bot]

This comment was marked as outdated.

codescene-delta-analysis[bot]

This comment was marked as outdated.

@sonarqubecloud

sonarqubecloud Bot commented Sep 3, 2026

Copy link
Copy Markdown

@sonarqubecloud

sonarqubecloud Bot commented Sep 3, 2026

Copy link
Copy Markdown

@codescene-delta-analysis codescene-delta-analysis Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No application code in the PR — skipped Code Health checks.

See analysis details in CodeScene

Quality Gate Profile: Pay Down Tech Debt
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.

@Rick1330
Rick1330 merged commit 8fcf88d into main Sep 3, 2026
72 checks passed
@Rick1330
Rick1330 deleted the chore/bench-data-publish branch September 3, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/benchmarks Benchmark collection, dashboard, published data area/web Next.js web app, public site, dashboard automated Bot-generated commits benchmark-data Automated benchmark data commits hnsw-benchmark-data ranking-quality-benchmark-data write-pipeline-benchmark-data

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant