[疑问] MMLU_PRO如何看总分 #199

Activity

1. SJTUyh

   added

   questionFurther information is requestedFurther information is requested

   on Mar 18, 2026
2. github-actions commented on Mar 18, 2026

   

   github-actionsBot

   on Mar 18, 2026 – with GitHub Actions

   More actions

   🌟欢迎给AISBench benchmark测评工具提issue，您可以先尝试以下途径寻求issue的解决方法：

   1. 【强烈推荐❤️‍🔥】确保issue描述完整后，可以试着将issue交给回答，deepwiki包含了和工具相关的所有知识库
   2. 在 工具历史issue 中通过问题的关键日志片段或关键词检索相似issue
   3. 在 官方文档 的搜索栏中通过问题的关键日志片段或关键词检索
   4. 寻求仓库维护者的帮助（issue评论区中 @SJTUyh）
3. github-actions commented on Mar 18, 2026

   

   github-actionsBot

   on Mar 18, 2026 – with GitHub Actions

   More actions

   🤖 基于AI机器人的issue内容完整性检查结果:

   问题内容检查结果

   检查状态

   PASS

   详细检查

   • 疑问描述: COMPLETE

   改进建议

   所有内容已完备！

   补充说明

   无

   👉 如果想重新检查，在评论区@issue_checker即可。

   【强烈推荐❤️‍🔥】确保issue描述完整后，可以试着将issue交给回答，deepwiki包含了和工具相关的所有知识库
4. github-actions

   added

   content_check_passedissue content check passedissue content check passed

   on Mar 18, 2026
5. SJTUyh commented on Mar 18, 2026

   

   SJTUyh

   on Mar 18, 2026

   Last edited by SJTUyh

   CollaboratorAuthor

   More actions

   group/mmlu_pro.py

   categories = ['math', 'physics', 'chemistry', 'law', 'engineering', 'other', 'economics', 'health', 'psychology', 'business', 'biology', 'philosophy', 'computer science', 'history']
   
   _mmlu_pro_all = ['mmlu_pro_' + c.replace(' ', '_') for c in categories]
   
   _mmlu_pro_weights = {
       'math': 1351,
       'physics': 1299,
       'chemistry': 1132,
       'law': 1101,
       'engineering': 969,
       'other': 924,
       'economics': 844,
       'health': 818,
       'psychology': 798,
       'business': 789,
       'biology': 717,
       'philosophy': 499,
       'computer science': 410,
       'history': 381,
   }
   _mmlu_pro_weights = {'mmlu_pro_' + k.replace(' ', '_'): v for k, v in _mmlu_pro_weights.items()}
   
   mmlu_pro_summary_groups = [
       {'name': 'mmlu_pro', 'subsets': _mmlu_pro_all},
       {'name': 'mmlu_pro-weighted', 'subsets': _mmlu_pro_all, 'weights': _mmlu_pro_weights},
   ]
6. Keithwwa

   added this to the AISBench 2026 Q4 Roadmap milestone on Sep 20, 2026
7. Keithwwa

   closed this as completedon Sep 23, 2026