Skip to content

fix(benchmark): isolate native profile runtime temporary paths - #5226

Merged
huangruiteng merged 2 commits into
mainfrom
codex/release-v1.2.2-profile-isolation
Sep 27, 2026
Merged

huangruiteng merged 2 commits into
mainfrom
codex/release-v1.2.2-profile-isolation

Conversation

@huangruiteng

@huangruiteng huangruiteng commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

The formal native Codex profile isolated HOME and CODEX_HOME but left the installer, CLI and app-server on the shared process temporary directory. Equal-source profiles consequently reused one Effect runtime, so stopping one fixture could affect another and its directory could still be written during parallel full-public cleanup.

Bind TMPDIR, TMP and TEMP to the existing profile-owned HOME in both established environment builders. Add a real two-profile install/CLI test that stops one runtime and reads the other as still running; retain the existing explicit runtime teardown and provider/privacy assertions. No scoring, task semantics, model turn or benchmark job changes.

Placement: built-in benchmark-toolkit provider, existing native Codex transport/profile owner. Python continues to adapt the installed host environment; the typed runtime authority and global defaults remain unchanged.

Validation passed on the exact head before authorized merge: immutable base/head temp-scope readback, two real isolated profiles, the formal installed-profile/app-server smoke, focused release qualification tests, static/boundary checks and native premerge.

15 focused tests and actual two-profile runtime/shutdown pass. Final clean-head formal install and Codex app-server skills/Goal readback pass with no model turn. Ruff, configured mypy19, TypeScript typecheck, docs governance, boundary and exact quality pass; all19 native premerge checks pass. The conservative benchmark-path manual review is covered by the published exact-head review and maintainer authorization; original classification remains recorded. No frontend consumer or setting changes: this is derived environment in the existing explicit host/provider adapter. Final merged-source full release qualification remains required.

Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>
Signed-off-by: huangruiteng <14976749+huangruiteng@users.noreply.github.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (author-owned PR; GitHub blocks formal self-approval)

Reviewed exact head 5226@e8ee52003a7554bd2f5ab2c26d4f67c87335f747.

动机

1.2.2 的并行全量公开验证仍遇到正式安装 profile 清理失败。此前 #5224 增加停止 callback,但 HOME/CODEX_HOME 隔离并不能证明 runtime 独占:现有内容寻址 runtime locator 使用系统临时目录,相同源码 profile 因而能共享 writer;其中一个退出后另一个仍可重新写入已清理目录。原始失败记录保留,维护者已授权本次发行的必要修复和合并。

改动思路

由既有 benchmark-toolkit 原生 Codex profile 安装/环境适配器拥有这项边界。两个现有环境 builder 都将 TMPDIR/TMP/TEMP 指向已存在的私有 HOME,安装和后续 CLI/app-server 采用相同 scope。复用原 TypeScript runtime locator、doctor 停止结果和标准临时目录解析;没有新增状态键、全局环境设置、进程 supervisor 或第二个决策 owner。Python 留在已有专用 host/安装 transport,TypeScript 权威逻辑不变。

具体改动

三个文件、57 行增加和 1 行删除:生产适配器 8 行;一项真实双安装 profile 测试 41 行;现有 provider 文档更新 8 行、替换 1 行。实际相同源码的两个正式安装 profile 分别启动 runtime,停止 first 返回 stopped 且 locator 位于 first HOME,second 仍 running;最后只停止各自 owner。15 项相关测试通过,生产代码在已测试功能提交与最终文档提交之间完全相同;最终干净 head 的正式安装/app-server 技能及 Goal 读回再次通过,无 model turn。对同一个实际已安装 fixture,immutable base 的导出环境使 Python tempdir 位于共享系统目录,最终 head 则位于私有 HOME。Ruff、mypy(19 源文件)、TypeScript typecheck、文档治理、公开边界、精确质量以及全部 19 项 native premerge 通过。原始 benchmark-sensitive 人工门禁不被改写;本评审核对没有评分、task、runner 提交、benchmark 作业或生产 Goal 操作,结合维护者授权满足此人工要求。

对主干的风险

变更仅对显式创建的资格验证 profile 生效;通用 runtime、正常 CLI/App 环境、profile schema、技能内容和权限 owner 均保持原字节。主要风险是安装与后续调用 scope 不一致,双 builder 同步和真实安装测试覆盖了这个路径;停止 pending/失败仍会拒绝,不能伪装清理成功。HOME 由已有安装路径预先创建;临时目录不携带上游凭证。实际模型工作仍需原 OS/provider isolation,环境隔离本身不授予凭证、网络沙箱或执行权限。没有 frontend 配置项或应用 consumer:这是已有显式 provider 适配器派生的环境边界,实际安装 CLI 和 app-server 已验证。此局部通过不代替合入后的全量 CI、真实模型、公开下载及签名验收。

我的整体评价

当前精确 head 可以批准。相关未来改进检查选择直接复用已有 HOME 和两个 builder,没有新增字段或抽象;回归测试保护真实安装后的并行 shutdown ownership。#5224 的串行清理结论需补充这项并行边界修正,最终发行仍在当前任务中继续完成。

English verdict: APPROVE

@huangruiteng
huangruiteng merged commit d0bc16a into main Sep 27, 2026
26 of 30 checks passed
@huangruiteng
huangruiteng deleted the codex/release-v1.2.2-profile-isolation branch September 27, 2026 21:29
@huangruiteng

Copy link
Copy Markdown
Collaborator Author

Qualification readback update for the existing native-profile lifecycle owner:

The private HOME and runtime-locator scope verified by this PR still holds, but a later Linux full-public attempt on ee9dad81b14c6d15d32b95851b5d38fcc485e732 failed during the disposable installed-profile smoke's TemporaryDirectory cleanup (Directory not empty: profile). Original attempt.

The complete five-shard rerun of the same immutable source passes all 514 public smokes with the original budgets, and eight additional concurrent local runs of the real installed-profile smoke pass. Same-source complete rerun. No cleanup error was ignored, assertion removed, timeout raised, or production Goal changed.

This successful rerun qualifies the current source; it does not prove elimination of the intermittent Linux fixture-quiescence race. The remaining maintenance boundary stays with this existing profile lifecycle owner: identify the exact late writer/process under Linux, then prove it is quiescent before removing the owned fixture. The final bilingual release validation preserves the first failure and this limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant