Skip to content

server: a refused API key file logs the path and the mode it read - #2365

Merged
ashhart merged 1 commit into
ashhart:mainfrom
Whq02:mercury/key-file-refusal-mode
Nov 7, 2026
Merged

ashhart merged 1 commit into
ashhart:mainfrom
Whq02:mercury/key-file-refusal-mode

Conversation

@Whq02

@Whq02 Whq02 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

--api-key-file refuses a file with any group or other read or write bit (auth.zig, mode & 0o066). The refusal
says what the file must be and the remedy, and nothing about the file it read. On main at 53311e7, with a key file at
0640:

tensorfold: API key file must be a regular file unreadable and unwritable by other users; use chmod 600

Mercury CLI integrates TensorFold and hit this on a rented GPU pod whose working directory is a mounted network
volume: the same key file was refused after chmod 600, while on the local disk it passed. With neither the path nor
the mode in the refusal there was nothing to read; the message said the remedy had not been applied when it had.

After this change the refusal is preceded by one log line with the path and the octal mode the file system reported,
and which of the two tests it failed:

[tensorfold] API key file keys.txt: mode 0640, readable or writable by other users
tensorfold: API key file must be a regular file unreadable and unwritable by other users; use chmod 600

(or not a regular file). The message itself, the check and the exit code are unchanged; a file at 0600 serves as
before. --slide's key check, which reads the file through the same store before the model loads, writes the same
line before its own refusal. The line is also written once when a reload refuses the file, next to the existing "API
key file reload refused" line, which until now gave no reason either; the rereads that follow, at most once a second
while requests come in and the routes stay closed, write nothing more, as that line does not.

What the network volume reports for a 0600 file, and whether the check should read something other than the mode bits
there, is a question for you; this change only makes the refusal show it. The pod's volume was not at hand to record
the mode it reports.

  • Issue: n/a: no issue filed; the refusal and the change are above.
  • Stacked on: main (5daafaf), one commit (rebased from 53311e7 for the review asks; patch-id unchanged).
  • atc overlap --json: run 2026-11-06T15:58:08Z on this head against origin/main at 5daafaf (ATC 0.9.0). changedFiles 2; the conflicts named are CHANGELOG.md only, with the other open PRs that add a line under Unreleased (cli: tensorfold models --json, the listed checkpoints as one document #2364 among them); auth.zig overlaps no open PR.

Checks on this head

  • Head commit: 3fb39f9
  • Lean checker, python3 tools/zig/lean_check.py, on the base and on this head: 252 on 5daafaf, 252 on this head, 0 new (tools/zig/lean_new.py: "lean: 252 problems, 252 on the base, 0 new"; the 252 are inherited, and the checker exits 1 on both for them).
  • Tests run, with the command and pass, fail and skip counts:
    • The server unit-test module (zig/src/server/root.zig, the zig test line zig build test-key-scopes-host --verbose prints, without its --test-filter and --listen=-): 1108 pass, 5 skip, 0 fail on this head; 1107 pass, 5 skip, 0 fail on the base. The new test, the key file refusal logs the path and the mode the file system reported, once at startup and once a reload, fails on the base (expects the line, finds no log output; 1107 pass, 5 skip, 1 fail, exit 1) and passes here; it also fails on this change without the once guard (the first retry writes the line again, a third line where the test expects two; 1107 pass, 5 skip, 1 fail, exit 1). It skips on Windows with the other key-file tests, whose guard it keeps. The 5 skips are the same on both: three voice-tool tests that run on Linux only, and the Clef tokenizer and Qwen3 rerank tests without their checkpoints. zig build test-key-scopes-host itself: 29/29 on this head.
    • zig build test-golden --summary all: 5/5 steps, 322/335 and 55/55 equal to the server goldens, 0 unexpected differences (the 13 are its recorded known differences), on the base and on this head. Its fake server runs the real serve.run: this head's binary printed the two lines above with a 0640 file and exit 1 (the base's, the one line); a 0600 file served /health 200 and a keyed /v1/models 200; a file read at 0600 and then set to 0640 while serving wrote the line once and "reload refused" once across six keyed requests over seven seconds, each answered 401 (the base's binary wrote only "reload refused").
    • zig build test-server-security: 24 passed, 0 failed, 0 skipped over the wire and 20/20 security: unit tests, on the base and on this head.
    • zig fmt --check build.zig zig and git diff --check: clean.
  • CI: runs when the pull request opens; not run on this head yet. Of its steps, the lean check, zig fmt, test-golden and test-server-security pass locally as above.
  • Not run, and why: zig build test on macOS, which compiles the Metal shaders; the build host has no Metal toolchain (xcrun --find metal exits 72). The server module unfiltered, the goldens and the security step above are where auth.zig runs on the CPU. The --slide sentence above rests on a test written for this check and not committed (slideKeyless with a 0640 file: the line, then its refusal naming the file and the unchanged message); the native binary, which needs the Metal toolchain on macOS, was not built.

Receipt

n/a: a log line at startup and on a refused reload; nothing reaches inference.

Checklist

  • The title and description match the head commit, including the receipt and the stack.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency.
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Thanks @Whq02. We read this with the secrets question in mind:

  • The line is written before any of the file is read, so no key bytes can reach it.
  • Its path is the --api-key-file value exactly as given, with nothing resolved, the same value the --slide refusal already prints.
  • It goes to stdout only.

The once guard on valid reads right, and the reload test covers it.

One ask: a CHANGELOG line under Unreleased, e.g. "Server: a refused --api-key-file logs the path it was given, the mode the file system reported and which test failed, once at startup and once when a reload is refused (#2365). Thanks to @Whq02."

On the network volume: this PR doesn't change the check, and that's right for this PR. If you get the pod again, the new line's output for a file you've just set to 0600 would be the useful thing to paste into an issue. It would show whether the volume reports other-user bits regardless of chmod.

@Whq02
Whq02 force-pushed the mercury/key-file-refusal-mode branch from b3a2033 to 3fb39f9 Compare November 6, 2026 16:09
@Whq02

Whq02 commented Nov 6, 2026

Copy link
Copy Markdown
Contributor Author

Done as one commit on 5daafaf, 3fb39f9: your CHANGELOG line under Unreleased; auth.zig is the version you read.

On that tip: zig fmt --check and git diff --check clean, lean 252 on base and head (0 new), test-golden 322/335 and 55/55 equal with 0 unexpected, the server module unfiltered 1124 passed / 5 skipped / 0 failed (test-key-scopes-host 29/29), and test-server-security 24 passed over the wire and 20/20. atc overlap after the rebase: the only conflicts are in CHANGELOG.md, with the other open PRs that add a line under Unreleased (#2364 among them).

@ashhart

ashhart commented Nov 6, 2026

Copy link
Copy Markdown
Owner

Thanks @Whq02, the CHANGELOG line is in and the code is the version we read. Ready to land. If #2364 lands first, the line needs a one-line re-place under Unreleased, which we'll do at landing.

The key file's refusal said only that the file must be a regular file unreadable and unwritable by other users, with
chmod 600 as the remedy. On a file system that reports a mode the check rejects after chmod 600, as a mounted network
volume can, that left nothing to go on: the message named neither the path nor the mode. The refusal now logs
`API key file PATH: mode 0640, readable or writable by other users` (or `not a regular file`) before the unchanged
message, at startup and on a reload that refuses the file.

refresh rereads a refused key file at most once a second while requests come in and the routes stay closed, so the line
is written while the store is still valid, as "API key file reload refused" is: at startup, and on the reload that
refuses the file, not on the retries after it. The test reads the line for a 0640 file at startup, nothing for a 0600
file, and the line once, with the reload's own, across two reloads after that file is set to 0640.

From the review: the CHANGELOG line under Unreleased.
@ashhart
ashhart force-pushed the mercury/key-file-refusal-mode branch from 3fb39f9 to 9f2f9f1 Compare November 7, 2026 02:59
@ashhart
ashhart merged commit cbe46a6 into ashhart:main Nov 7, 2026
0 of 7 checks passed
@ashhart

ashhart commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Thank you @Whq02. This is merged into main with your commits under your own name, so you're listed as a TensorFold contributor. It ships in the next release, credited to you in the changelog and the release notes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants