Skip to content

docs: add PR E benchmarks and missing telecom entries to adapter refences - #112

Merged
julpayne merged 1 commit into
mainfrom
docs/pr-e-benchmarks
Sep 8, 2026
Merged

julpayne merged 1 commit into
mainfrom
docs/pr-e-benchmarks

Conversation

@j-mok-dev

Copy link
Copy Markdown
Contributor

What and why

Update adapter benchmark reference pages to reflect PR E curated collections benchmarks and fill in previously missing entries.

Changes

inspect/benchmarks.md

  • Add 4 new PR E benchmarks: HLE, BFCL, CyberSecEval-2-PI, DocVQA
  • Add 4 previously missing telecom benchmarks: 3GPP-TSG, TeleLogs, TeleMath, TeleQnA (found by diffing against config/providers/inspect.yaml)
  • Add new sections: Tool Use and Function Calling, Telecom, Multimodal
  • Update total benchmark count: 75 → 83

lighteval/benchmarks.md

  • Replace "Coming Soon" placeholder with full 28-benchmark catalog
  • Include GPQA-Diamond and LiveCodeBench (PR E benchmarks)

.gitignore

  • Add .cursor/ and .cursorignore/

Prerequisites

This PR should be merged after the following companion PRs land:

Refs: eval-hub/eval-hub#941

@coderabbitai

coderabbitai Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 52 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 39aa59f3-679b-4824-b0e9-bcbc5c9c25e0

📥 Commits

Reviewing files that changed from the base of the PR and between 624bd37 and c488045.

📒 Files selected for processing (3)
  • .gitignore
  • src/content/docs/adapters/inspect/benchmarks.md
  • src/content/docs/adapters/lighteval/benchmarks.md

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 3, 2026 •

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-08 12:24 UTC

@julpayne

julpayne commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@julpayne julpayne left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@julpayne
julpayne merged commit 263e084 into main Sep 8, 2026
4 checks passed
@julpayne
julpayne deleted the docs/pr-e-benchmarks branch September 8, 2026 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants