Skip to content

fix(cli): treat json v2 truncated-input error as unexpected EOF (Go 1.27) (#301) - #302

Open
chiliec wants to merge 1 commit into
itchyny:mainfrom
chiliec:fix/go1.27-stream-unexpected-eof
Open

fix(cli): treat json v2 truncated-input error as unexpected EOF (Go 1.27) (#301)#302
chiliec wants to merge 1 commit into
itchyny:mainfrom
chiliec:fix/go1.27-stream-unexpected-eof

Conversation

@chiliec

@chiliec chiliec commented Aug 23, 2026

Copy link
Copy Markdown

Fixes #301.

Problem

Go 1.27 backs encoding/json with the v2 implementation. Marshal/unmarshal behavior is preserved, but the exact error values differ. Specifically, when (*json.Decoder).Token() hits a stream that is truncated in the middle of a value, Go ≤1.26 returned io.EOF, while Go 1.27 returns a *json.SyntaxError ("unexpected end of JSON input").

cli/stream.go only recognized io.EOF when converting a mid-structure end-of-input into io.ErrUnexpectedEOF:

if err == io.EOF && s.states[len(s.states)-1] != jsonStateTopValue {
    err = io.ErrUnexpectedEOF
}

On Go 1.27 that check is false, so the raw v2 *json.SyntaxError leaks up to cli/inputs.go, which takes the *json.SyntaxError branch (offset one column earlier) and prints the v2 message. The stream option with unterminated input test then fails:

- 	           ^  unexpected EOF
+ 	          ^  unexpected end of JSON input

Fix

Normalize both representations to io.ErrUnexpectedEOF via a small helper. It keeps the original io.EOF case and additionally accepts a *json.SyntaxError whose offset is at the end of the consumed input (se.Offset >= dec.InputOffset()) — that offset condition is what distinguishes a truncation from a genuine mid-stream syntax error (e.g. [1 2], {"a" bad}), which keep their original error untouched. Because the pre-existing io.ErrUnexpectedEOF path in cli/inputs.go already seeks to end-of-input for the caret, the rendered position and message are now identical across Go versions.

One file changed, no new test needed — the existing stream option with unterminated input golden case is the regression test.

Verification

Ran the full suite on both toolchains (GOTOOLCHAIN=local):

Go 1.27.0 (reproduces the bug without the patch):

  • Without fix: --- FAIL: TestCliRun/stream_option_with_unterminated_input
  • With fix: ok github.com/itchyny/gojq/cligo test ./... all green.

Go 1.24.6 (no regression):

  • go test ./...ok github.com/itchyny/gojq and ok github.com/itchyny/gojq/cli.

gofmt -l clean, go vet ./cli/ clean.

Also spot-checked the discriminator directly: truncation inputs ({"a":1,, [1,, {"a":1, [1) all report SyntaxError.Offset == len(input) on Go 1.27 and are normalized; genuine syntax errors ([1 2], {"a" bad}, {"a":@}) have Offset < len and pass through unchanged.

AI assistance disclosure

This change was prepared with an AI coding assistant; the root-cause analysis, fix, and cross-version verification above were all run against real Go 1.24.6 and Go 1.27.0 toolchains, not generated.

….27)

Go 1.27 backs encoding/json with the v2 implementation, which returns a
*json.SyntaxError ("unexpected end of JSON input") from (*Decoder).Token when
the stream is truncated mid-value, where Go 1.26 and earlier returned io.EOF.
The --stream path in cli/stream.go only recognized io.EOF, so on Go 1.27 the
raw v2 error leaked out with a different message and caret column, breaking the
"stream option with unterminated input" test.

Normalize both representations to io.ErrUnexpectedEOF via a small helper that
also accepts a *json.SyntaxError whose offset is at the end of consumed input,
so genuine mid-stream syntax errors are unaffected. Output is now identical on
Go 1.24 through 1.27. Fixes itchyny#301.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

go test fails with golang v1.27

2 participants