Skip to content

Require at least one digit in a number's exponent - #81

Merged
trobro merged 1 commit into
hjson:masterfrom
youdie006:exponent-needs-digits
Sep 1, 2026
Merged

trobro merged 1 commit into
hjson:masterfrom
youdie006:exponent-needs-digits

Conversation

@youdie006

Copy link
Copy Markdown
Contributor

hjson-go cannot read back what it just wrote for a string like 1e.

enc, _ := hjson.Marshal(map[string]interface{}{"k": "1e"})
// enc == "{\n  k: 1e\n}"
var back map[string]interface{}
err := hjson.Unmarshal(enc, &back)
// err == "Internal error"

Same for 1E, 1e+, 1.2e, -1e.

The two halves disagree

The exponent scan in tryParseNumber (parseNumber.go:77) allows the digit loop to match nothing:

if p.ch == 'e' || p.ch == 'E' {
    p.next()
    if p.ch == '-' || p.ch == '+' {
        p.next()
    }
    for p.ch >= '0' && p.ch <= '9' {   // may run zero times
        p.next()
    }
}

That is harmless when the caller reaches strconv.ParseFloat at parseNumber.go:108, which
rejects 1e. But when the destination makes the decoder use json.Number — any
map[string]interface{} or interface{} target — parseNumber.go:105 returns first:

if useJSONNumber {
    return json.Number(string(p.data[0 : end-1])), nil
}

so json.Number("1e") escapes unvalidated and the later json.Marshal fails with
Internal error.

The encoder already gets this right. startsWithNumber (encode.go:137) calls the same
function as tryParseNumber(text, true, false) — with useJSONNumber false it hits ParseFloat,
correctly concludes 1e is not a number, and writes it quoteless. So the two call sites of one
function disagree about whether 1e is a number, and the decoder is the one that is wrong.

The sibling implementations agree

hjson-py   k: 1e    -> '1e'   (str)
hjson-py   k: 1E    -> '1E'   (str)
hjson-py   k: 1.2e  -> '1.2e' (str)
hjson-py   k: 1e+   -> '1e+'  (str)
hjson-py   k: 1e5   -> 100000 (int)

hjson-js behaves the same way. This change moves hjson-go onto that behaviour rather than away
from it.

Change

Four lines at the scan site — an exponent needs at least one digit:

// an exponent needs at least one digit
if p.ch < '0' || p.ch > '9' {
    return 0, errors.New("Invalid number")
}

Nothing about the numeric range is touched, so UseJSONNumber's large-exponent preservation is
unaffected; 1e5 still decodes to 100000.

Verification

A test added next to the others covering the five inputs above plus the 1e5 control. Red with
only parseNumber.go reverted ("1e": Internal error), green with the change. go test ./...
passes before and after; gofmt -l clean.

Found by round-tripping 200,000 generated strings through MarshalUnmarshal; the 106
Internal error occurrences all reduce to this shape, and go to 0 with the change.

Behaviour change: Unmarshal of {k: 1e} into a map goes from returning an error to yielding
the string "1e". That only affects callers relying on the error, and it is what the other
implementations already do.


Disclosure: prepared with AI assistance; I verified the round trip, the hjson-py comparison and the
red/green runs myself.

The exponent loop could match nothing, so 1e scanned as a number. With
UseJSONNumber the token is returned before the ParseFloat check, so the
decoder accepted a value its own encoder had already decided was a
quoteless string.
@trobro

trobro commented Sep 1, 2026

Copy link
Copy Markdown
Member

Looks good, thanks for the PR!

@trobro
trobro merged commit 033fae8 into hjson:master Sep 1, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants