Environment
- lh-harness 0.1.7, Python 3.12, Windows 11
- Runner stdout is piped (not an interactive console) — e.g. under a launcher, a supervisor, or the MCP bridge
What happens
adapters/deepseek_runner.py → _emit_result does:
sys.stdout.write(json.dumps(record, ensure_ascii=False))
On Windows, a piped sys.stdout defaults to the ANSI code page (cp1252), not UTF-8. The moment the model emits any non-ASCII character (math symbols like ≠ U+2260, arrows, accented text — routine for these models), the write raises:
UnicodeEncodeError: 'charmap' codec can't encode character '\u2260' in position …
and the runner dies mid-episode. Interactive-console runs can mask this (a real console may negotiate UTF-8), which is why it surfaces specifically under piped/launched invocations.
Why it matters
Any Windows deployment that pipes the runner's stdout (launchers, supervisors, CI, MCP bridges — i.e. most automated setups) is one non-ASCII token away from a crash. Our models emit UTF-8 symbols routinely, so this is not an edge case.
Local mitigation (workaround, not a fix)
- P-4 patch in the pinned venv: a
_force_utf8_stdio() called at the top of run() reconfigures sys.stdout/sys.stderr to UTF-8 (errors="replace"), kept as a no-op when the stream is not reconfigurable.
- Launcher sets
PYTHONIOENCODING=utf-8 (defense in depth for every Python child in the tree).
Validated by A/B: the exact ≠ (U+2260) round-trip that crashed pre-patch runs passes post-patch.
Ask
Make the runner encoding-safe on Windows by default — pick one (or both):
- Reconfigure
sys.stdout/sys.stderr to UTF-8 inside the runner (same as our P-4 patch), guarded so it is a no-op where not applicable; or
- Use
ensure_ascii=True in _emit_result so emitted JSON is pure-ASCII and immune to the consumer's console/pipe code page.
And document that Windows consumers must run the runner with UTF-8 stdio (PYTHONIOENCODING=utf-8) if they rely on raw UTF-8 output.
Reproduction: any task whose model output contains a non-ANSI character, run with a piped stdout on Windows.
Environment
What happens
adapters/deepseek_runner.py→_emit_resultdoes:On Windows, a piped
sys.stdoutdefaults to the ANSI code page (cp1252), not UTF-8. The moment the model emits any non-ASCII character (math symbols like≠U+2260, arrows, accented text — routine for these models), the write raises:and the runner dies mid-episode. Interactive-console runs can mask this (a real console may negotiate UTF-8), which is why it surfaces specifically under piped/launched invocations.
Why it matters
Any Windows deployment that pipes the runner's stdout (launchers, supervisors, CI, MCP bridges — i.e. most automated setups) is one non-ASCII token away from a crash. Our models emit UTF-8 symbols routinely, so this is not an edge case.
Local mitigation (workaround, not a fix)
_force_utf8_stdio()called at the top ofrun()reconfiguressys.stdout/sys.stderrto UTF-8 (errors="replace"), kept as a no-op when the stream is not reconfigurable.PYTHONIOENCODING=utf-8(defense in depth for every Python child in the tree).Validated by A/B: the exact
≠(U+2260) round-trip that crashed pre-patch runs passes post-patch.Ask
Make the runner encoding-safe on Windows by default — pick one (or both):
sys.stdout/sys.stderrto UTF-8 inside the runner (same as our P-4 patch), guarded so it is a no-op where not applicable; orensure_ascii=Truein_emit_resultso emitted JSON is pure-ASCII and immune to the consumer's console/pipe code page.And document that Windows consumers must run the runner with UTF-8 stdio (
PYTHONIOENCODING=utf-8) if they rely on raw UTF-8 output.Reproduction: any task whose model output contains a non-ANSI character, run with a piped stdout on Windows.