Issues

• at_least() reports 0 when unscored panel members leave fewer than k scored values

  UKGovernmentBEIS/inspect_ai#5696 · Filter by author feiiiiii5 opened on Oct 5, 2026

• frequency() reports 0.0 for every category when no samples were scored

  UKGovernmentBEIS/inspect_ai#5695 · Filter by author feiiiiii5 opened on Oct 5, 2026
  1

• Improve tutorial consistency and add custom solver coverage

  UKGovernmentBEIS/inspect_ai#5691 · Filter by author Alg2338 opened on Oct 5, 2026

• OpenAI Responses: readable (unsummarised) reasoning is marked redacted=True when no summary part is returned

  UKGovernmentBEIS/inspect_ai#5689 · Filter by author thjread-aisi opened on Oct 5, 2026
  1

• Sandbox agent bridge denies every host tool call from Codex code-mode models (exec-nested calls have no proposal since #5428)

  UKGovernmentBEIS/inspect_ai#5687 · Filter by author sjawhar opened on Oct 5, 2026

• Dataclass tool parameters lose their types and fail with TypeError when the dataclass module uses from __future__ import annotations

  UKGovernmentBEIS/inspect_ai#5681 · Filter by author JoeyTan21 opened on Oct 4, 2026
  1

• ACP CLI tests fail on Windows before exercising their intended contracts

  UKGovernmentBEIS/inspect_ai#5678 · Filter by author XCODESSS opened on Oct 4, 2026

• Let tasks and samples carry a description recorded in the log

  UKGovernmentBEIS/inspect_ai#5666 · Filter by author dragonstyle opened on Oct 3, 2026

• eval_retry() appears to reshuffle samples when sample_shuffle=False

  UKGovernmentBEIS/inspect_ai#5659 · Filter by author kausthubhk opened on Oct 2, 2026
  1

• eval_set can reuse a successful result after the task scorer changes

  UKGovernmentBEIS/inspect_ai#5655 · Filter by author camerontjs-dot opened on Oct 1, 2026
  1

• inspect log convert --stream drops the error from failed eval logs

  accepted
  UKGovernmentBEIS/inspect_ai#5639 · Filter by author Zhuoxi2000 opened on Oct 1, 2026
  1

• Give human_approver context on previous escalation

  UKGovernmentBEIS/inspect_ai#5618 · Filter by author nix-aisi opened on Sep 30, 2026

• Remove the legacy inspect_tool_support package (once web_browser is removed)

  UKGovernmentBEIS/inspect_ai#5617 · Filter by author craig-meridian opened on Sep 30, 2026
  

• Sample and Task silently drop sandbox values they cannot resolve, including Sample's own serialised form

  UKGovernmentBEIS/inspect_ai#5616 · Filter by author MattFisher opened on Sep 30, 2026

• test_deployment_name_normalization has no assertion and cannot fail

  UKGovernmentBEIS/inspect_ai#5615 · Filter by author johnny-glitch12 opened on Sep 30, 2026
  3

• Epoch reducer arguments other than k are lost when metrics are recomputed from a log

  UKGovernmentBEIS/inspect_ai#5612 · Filter by author David-Wu1119 opened on Sep 29, 2026

• Implicit mean epoch reducer converts scores before the metric's own to_float sees them

  UKGovernmentBEIS/inspect_ai#5611 · Filter by author David-Wu1119 opened on Sep 29, 2026

• Agent bridge has no route for Codex's standalone web search, so codex_cli gets no search above GPT 5.5

  UKGovernmentBEIS/inspect_ai#5604 · Filter by author technophile-04 opened on Sep 29, 2026
  2

• Report judge reliability over repeated trials: flip rate, and routing an unstable verdict to Score.unscored()

  UKGovernmentBEIS/inspect_ai#5599 · Filter by author Abelo9996 opened on Sep 28, 2026
  1

• HF task loading reads eval.yaml with the locale encoding, so non-ASCII task configs crash or silently corrupt on Windows

  acceptedbuggood first issue
  UKGovernmentBEIS/inspect_ai#5597 · Filter by author kittimzhe opened on Sep 28, 2026
  7

• answer("line") scores a correct answer INCORRECT on trailing whitespace

  acceptedbug
  UKGovernmentBEIS/inspect_ai#5595 · Filter by author feiiiiii5 opened on Sep 28, 2026
  7

• Document an openai-api service example for uncensored-model endpoints (red-teaming use case)

  UKGovernmentBEIS/inspect_ai#5592 · Filter by author haterfactory opened on Sep 27, 2026
  1

• math() records both timeout outcomes with reason=None, so a slow host looks like a wrong answer

  UKGovernmentBEIS/inspect_ai#5590 · Filter by author feiiiiii5 opened on Sep 27, 2026

• The scalar mean/median reducers convert each score twice, against the invariant the rest of the layer follows

  UKGovernmentBEIS/inspect_ai#5589 · Filter by author feiiiiii5 opened on Sep 27, 2026

• exact() and f1() ignore the sign of a negative number, so a sign-flipped answer scores CORRECT

  UKGovernmentBEIS/inspect_ai#5587 · Filter by author IvanSFlowGit opened on Sep 27, 2026