Skip to content

docs: add IFBench adapter references and Phase 2 collection catalog - #113

Open
j-mok-dev wants to merge 1 commit into
mainfrom
docs/ifbench-adapter
Open

j-mok-dev wants to merge 1 commit into
mainfrom
docs/ifbench-adapter

Conversation

@j-mok-dev

@j-mok-dev j-mok-dev commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Document the new IFBench community adapter and sync the Collections guide built-in table with Phase 2 curated collections from eval-hub.

home.mdx

  • Add IFBench to Multi-framework card and Available Adapters table

bring-your-own-framework.mdx

  • Add IFBench row (quay.io/evalhub/community-ifbench:latest)

collections.mdx

  • Add 7 Phase 2 system collections to built-in table, including instruction-output-v1 (JSONSchemaBench + IFBench)

Companion PRs:

Refs: eval-hub/eval-hub#942

Type

  • feat
  • fix
  • docs
  • refactor / chore
  • test / ci

Testing

  • Tests added or updated

  • Tested manually

  • npm run build (or site preview) — verify pages render

Breaking changes

None.

Summary by CodeRabbit

  • Documentation
    • Added IFBench to the list of supported frameworks and available adapters.
    • Added the AllenAI IFBench adapter to the framework adapter reference.
    • Documented seven new built-in collections covering instruction output, knowledge reasoning, document understanding, tool use, software, multimodal, and trustworthiness evaluations.

- List IFBench on home and bring-your-own-framework adapter tables
- Add Phase 2 curated collections to collections guide built-in table
  (including instruction-output-v1 with IFBench)
Refs: eval-hub/eval-hub#942

Signed-off-by: Jooyeon Mok <jmok@redhat.com>
@coderabbitai

coderabbitai Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 70031305-dd53-4dff-81f5-e8243b457959

📥 Commits

Reviewing files that changed from the base of the PR and between d265364 and 257c20f.

📒 Files selected for processing (3)
  • src/content/docs/guides/bring-your-own-framework.mdx
  • src/content/docs/guides/collections.mdx
  • src/content/docs/home.mdx

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The documentation adds IFBench to framework and adapter references. It also adds seven built-in system collections to the Collections guide.

Changes

Documentation updates

Layer / File(s) Summary
Document IFBench support
src/content/docs/guides/bring-your-own-framework.mdx, src/content/docs/home.mdx
The documentation adds the AllenAI IFBench adapter, its community image, supported-framework listing, and evaluation details.
Document built-in collections
src/content/docs/guides/collections.mdx
The Collections guide adds seven system collections with categories, pass thresholds, benchmark counts, and descriptions.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to 257c2

This updates documentation for IFBench and seven built-in collections without changing runtime behavior. No current merge-readiness risk remains.

Suggested reviewers: julpayne

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the two main documentation changes: IFBench adapter references and the Phase 2 collection catalog.
Description check ✅ Passed The description covers the purpose, changed files, documentation type, manual testing, and lack of breaking changes. The issue-closing field is not completed, and the site build check is left unchecke…
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch docs/ifbench-adapter

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

🚀 View preview at
https://eval-hub.github.io/pr-preview/pr-113/

Built to branch gh-pages at 2026-09-08 22:55 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant