Evaluate the benchmark against other open-source LLM's (eg. gpt-oss, etc)

[0xdevalias]

Looking at the current leaderboard / paper, it seems like a number of open source / open weight models were evaluated (eg. deepseek-chat-20250301, Codestral-22B-v0.1, DeepSeek-Coder-V2, Llama-3.1, Mixtral-v3). It would be interesting to see how some other / newer models (eg. gpt-oss) rank in this benchmark too.

See Also

• Evaluate the benchmark against other open-source LLM's (eg. gpt-oss, etc) jsdeobf/jsdeobf.github.io#11
• Evaluate the benchmark against other hosted LLM API's, not just OpenAI GPT-4o (eg. Claude, Gemini, etc) #8
• Evaluate the benchmark against other hosted LLM API's, not just OpenAI GPT-4o (eg. Claude, Gemini, etc) jsdeobf/jsdeobf.github.io#10
• Include LLM evaluation costs in the leaderboard jsdeobf/jsdeobf.github.io#9

[0xdevalias]

OpenAI's gpt-oss open weight models:

• https://openai.com/open-models/

  • Open models by OpenAI

  • Advanced open-weight reasoning models to customize for any use case and run anywhere.

• https://openai.com/index/introducing-gpt-oss/

  • Introducing gpt-oss

  • gpt-oss-120b and gpt-oss-20b push the frontier of open-weight reasoning models

  • https://platform.openai.com/docs/models/gpt-oss-20b

    • gpt-oss-20b

    • Medium-sized open-weight model for low latency

  • https://platform.openai.com/docs/models/gpt-oss-120b

    • gpt-oss-120b

    • Most powerful open-weight model, fits into an H100 GPU

  • https://github.com/openai/gpt-oss

    • gpt-oss

    • We're releasing two flavors of these open models:

      • gpt-oss-120b — for production, general purpose, high reasoning use cases that fit into a single 80GB GPU (like NVIDIA H100 or AMD MI300X) (117B parameters with 5.1B active parameters)
      • gpt-oss-20b — for lower latency, and local or specialized use cases (21B parameters with 3.6B active parameters)

      Both models were trained using our harmony response format and should only be used with this format; otherwise, they will not work correctly.