Add models

[Yuki-Imajuku]

No description provided.

[Copilot]

Pull request overview

Adds new model configurations and extends the local pricing fallback table so evaluations can run (and costs can be computed) for additional LLMs/providers.

Changes:

• Added fallback pricing entries for several new models in calc_cost.py.
• Added new llm_configs/*.json model presets for OpenAI, OpenRouter, and Google providers.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/ale_bench_eval/calc_cost.py	Extends FALLBACK_DICT with prices for newly added models.
llm_configs/mistral-small-4.json	Adds OpenRouter config for mistral-small-2603.
llm_configs/minimax-m2.7.json	Adds OpenRouter config for minimax-m2.7.
llm_configs/mimo-v2-pro.json	Adds OpenRouter config for mimo-v2-pro.
llm_configs/grok-4.20-beta.json	Adds OpenRouter config for grok-4.20-beta.
llm_configs/gpt-5.4-xhigh.json	Adds OpenAI config preset for gpt-5.4 with xhigh reasoning effort.
llm_configs/gpt-5.4-none.json	Adds OpenAI config preset for gpt-5.4 with none reasoning effort.
llm_configs/gpt-5.4-nano-high.json	Adds OpenAI config preset for gpt-5.4-nano with high reasoning effort.
llm_configs/gpt-5.4-mini-high.json	Adds OpenAI config preset for gpt-5.4-mini with high reasoning effort.
llm_configs/gpt-5.4-high.json	Adds OpenAI config preset for gpt-5.4 with high reasoning effort.
llm_configs/glm-5-turbo.json	Adds OpenRouter config for glm-5-turbo.
llm_configs/gemini-3.1-flash-lite-preview-high.json	Adds Google config for gemini-3.1-flash-lite-preview with high thinking.

Comments suppressed due to low confidence (1)

src/ale_bench_eval/calc_cost.py:27

• FALLBACK_DICT defines the key "gpt-5.4-2026-03-05" twice. In a Python dict literal the latter entry silently overwrites the former, which can hide mistakes (e.g., accidentally duplicating a key instead of adding a new model). Remove the duplicate or replace one of them with the intended distinct model name.

    "gpt-5.3-codex": ModelPrice(
        input_mtok=Decimal(175) / Decimal(100),
        cache_read_mtok=Decimal(175) / Decimal(1000),
        output_mtok=Decimal(14),
    ),

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

This OpenAI model config omits temperature, while other OpenAI configs in llm_configs/ consistently set it (commonly to 1.0). If temperature defaults differ across providers/SDK versions, this makes runs harder to reproduce/compare. Consider explicitly setting temperature here for consistency (or document why it's intentionally omitted).

Suggested change

	"openai_reasoning_summary": "detailed"
	"openai_reasoning_summary": "detailed",
	"temperature": 1.0

[Copilot]

This OpenAI model config omits temperature, while other OpenAI configs in llm_configs/ consistently set it (commonly to 1.0). If temperature defaults differ across providers/SDK versions, this makes runs harder to reproduce/compare. Consider explicitly setting temperature here for consistency (or document why it's intentionally omitted).

Suggested change

	"openai_reasoning_summary": "detailed"
	"openai_reasoning_summary": "detailed",
	"temperature": 1.0

[Copilot]

This OpenRouter config doesn't set temperature, while other Grok OpenRouter configs in this repo (e.g. grok-4.json, grok-4.1-fast.json) do. Consider explicitly setting temperature here as well to keep experiments reproducible/comparable across Grok variants.