Skip to content

docs(lm-eval): add provider integration guide - #116

Merged
ppadashe-psp merged 1 commit into
mainfrom
docs/lm-evaluation-harness-guide
Sep 16, 2026
Merged

ppadashe-psp merged 1 commit into
mainfrom
docs/lm-evaluation-harness-guide

Conversation

@j-mok-dev

Copy link
Copy Markdown
Contributor

What and why

Add LM Evaluation Harness provider documentation covering:

  • Provider/task model and RHOAI execution flow
  • incubation-based task integration and image publication workflow
  • Common benchmark parameters and task-specific image requirements
  • Local validation checklist and task catalog maintenance
  • HumanEval/MBPP code-execution safety guidance

This documents the integration work and operational requirements behind
RHOAIENG-85393 and RHOAIENG-85410.

Type

  • feat
  • fix
  • docs
  • refactor / chore
  • test / ci

Testing

  • Tests added or updated
  • Tested manually

Validated with:

  • npm run check — 0 errors, warnings, or hints
  • npm run build — production site build completed successfully

Breaking changes

None.

Signed-off-by: Jooyeon Mok <jmok@redhat.com>
@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 27fe7261-7af6-49c0-b7d6-50bad6fab43c


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 15, 2026 •

Copy link
Copy Markdown
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-16 03:19 UTC

@ppadashe-psp ppadashe-psp left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ppadashe-psp
ppadashe-psp merged commit dc1bebc into main Sep 16, 2026
4 checks passed
@ppadashe-psp
ppadashe-psp deleted the docs/lm-evaluation-harness-guide branch September 16, 2026 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants