Return clear error for non-finite (NaN/Inf) floats in KServe REST output#4337
Open
exzile wants to merge 3 commits into
Open
Return clear error for non-finite (NaN/Inf) floats in KServe REST output#4337exzile wants to merge 3 commits into
exzile wants to merge 3 commits into
Conversation
The KServe v2 REST output serializer wrote floats via rapidjson Writer::Double() and ignored its return value. rapidjson writes nothing for NaN/Inf (invalid in JSON), so a model output containing non-finite values silently produced malformed JSON. Add an isJsonRepresentable finite-check in both PARSE_OUTPUT_DATA write paths and return JSON_SERIALIZATION_ERROR with a clear message instead. Integer outputs are unaffected. Adds unit tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Apply the same finiteness check used on the KServe path to the TF Serving makeJsonFromPredictResponse: float/double outputs are validated before serialization and a clear JSON_SERIALIZATION_ERROR is returned for NaN/Inf instead of producing malformed JSON. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Author
|
Update: extended the same non-finite (NaN/Inf) guard to the TFS REST serialization path as well (it was originally flagged as a possible follow-up). Both REST paths are now covered, and the description/tests have been updated accordingly — full |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The REST output serializers silently produced malformed JSON when a model output contained non-finite floats (
NaN/±Inf). Floats were written via rapidjsonWriter::Double()while ignoring its return value — rapidjson writes nothing for non-finite values (they are invalid per the JSON spec and the default writer has nokWriteNanAndInfFlag), so the affected array element was dropped, corrupting the response. There was noisfiniteguard anywhere in the REST output path.This now returns a clear
JSON_SERIALIZATION_ERRORwith a descriptive message instead of emitting broken JSON, on both the KServe v2 and the TFS REST paths.Change
isJsonRepresentablehelper (overloaded: integral types always true;float/doubleusestd::isfinite).PARSE_OUTPUT_DATAmacro (val-field and raw-output) to returnJSON_SERIALIZATION_ERRORwithOutput "<name>" contains a non-finite (NaN/Inf) value that cannot be serialized to JSON.makeJsonFromPredictResponse(PredictResponse), validatingfloat/doubleoutputs beforeMakeJsonFromTensorsand returning the same error.Inf,NaN, and the unaffected-integer case on both paths.Validation
Built
ovms_test(Windows, py-off) and ran the serialization suite — all pass:*MakeJsonFromPredictResponse*suite: 85/85 (KFS + TFS, raw + val-field) — no regression.Context
Found while triaging #2337 (a model emitting
-Inf/NaNon GPU). That issue's root cause is upstream/model-side and appears resolved on current OpenVINO; this PR is the independent OVMS-side hardening so that any model producing non-finite values fails cleanly over REST instead of returning corrupt JSON.Relates to #2337.