Skip to content

Utilizing the HELM Benchmark and Running the whole package #6

Description

@Mahnaz19
  1. I haven't been able to obtain any output, even for the initial step. Could you please advise on the recommended computer hardware requirements for running the package effectively? I do appreciate any advice and guidance from you. the image below is running first step of Readme file after 8 hours!
    image 1

  2. The paper states that users can access the benchmark through the code provided in your open-source GitHub repository or by downloading the benchmark and implementing it themselves. I am Master student in AI program, to ensure proper ethical clearance for my dissertation, I would like to formally request your approval to use the HELM benchmark. Is there a specific process or any conditions for obtaining your approval? or by responding this issue could you please provide the official approval for using HELM benchmark? I sent an email also but i didn't get any response.

  3. My research plan involves applying all chosen techniques on a single dataset. Is the HELM benchmark suitable for evaluating post-processing techniques in addition to the non-post-processing techniques I am considering?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions