QBench

An executable, bilingual benchmark for evaluating LLMs and AI agents on quantitative-investment work.

量化投资大模型与 AI Agent 综合评测。

• 中文说明
• English documentation
• 提交测试题目教程 / Task contribution tutorial
• Benchmark configuration
• Official Qlib data contract

QBench contains a 100-point Theory Track and a 100-point Engineering Track. Engineering uses consecutive Q1–Q5 numbering and keeps R1–R3 as separately stored reserves. Main tasks test code repair, implementation, testing, cold starts, and artifact recomputation; configuration is input rather than the answer. Every engineering task runs exactly one public case, and the public Q1 evaluator directly exercises repaired Python APIs and artifacts.

Theory Track Results / 理论评测结果

Model / 模型	Theory Score / 理论得分
gpt-5.6-sol high	100
gpt-5.6-luna high	100
claude-opus-5	100
claude-sonnet-5	100
gpt-5.6-terra high	96.5
deepseek-v4-pro	94
deepseek-v4-flash-free	91.5

Composite Results / 综合评测结果（2026-08-08）

The engineering scores below are the reconciled public-case results after deterministic evaluator recomputation and a code/artifact audit. Equivalent artifact schemas are normalized before scoring; independently recomputed causality, constraints, risk, returns, and hashes still take precedence over candidate-reported values. No challenge cases or 40% public-case cap are used.

以下工程分为公开case的复核结果：先由确定性evaluator复算，再结合代码与artifact审计；等价schema不会造成误扣，但因果、约束、风险、收益与hash仍以独立复算为准。不使用挑战case，也没有“只做公开case最高40%”规则。

Rank	Model / 模型	Theory / 理论	Engineering / 工程	Overall / 综合
1	gpt-5.6-sol high	100	87.5	91.25
2	gpt-5.6-terra high	96.5	84.5	88.10
3	gpt-5.6-luna high	100	75.5	82.85
4	deepseek-v4-flash-free	91.5	76.5	81.00
5	deepseek-v4-pro	94	63.5	72.65

Overall = 0.30 × Theory + 0.70 × Engineering.

Model / 模型	Q1 / 18	Q2 / 22	Q3 / 18	Q4 / 18	Q5 / 24	Engineering / 100
gpt-5.6-sol high	18	21	9	16.5	23	87.5
gpt-5.6-terra high	18	16.5	15	16	19	84.5
deepseek-v4-flash-free	18	18	9	14.5	17	76.5
gpt-5.6-luna high	18	13	8	16.5	20	75.5
deepseek-v4-pro	14.5	16	8	14	11	63.5