Skip to content

Make Serde document iteration incremental - #85

Draft
jskoiz wants to merge 1 commit into
mainfrom
c1/issue-80-incremental-serde
Draft

Make Serde document iteration incremental#85
jskoiz wants to merge 1 commit into
mainfrom
c1/issue-80-incremental-serde

Conversation

@jskoiz

@jskoiz jskoiz commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Summary

  • Make the public Deserializer consume DocumentStream one parsed document at a time.
  • Preserve borrowed string/slice lifetimes, zero-based document indexes, deferred errors, empty-stream null behavior, and existing batch Vec<T> APIs.
  • Update the event-backed Serde workpad to describe the current iterator behavior.

Fixes #80

Verification

Local proof:

  • cargo fmt --all --check
  • git diff --check
  • cargo test --locked --lib deserializer_parses_documents_when_they_are_requested
  • CARGO_BUILD_JOBS=1 cargo test --locked --test serde_value_api (149 passed)
  • CARGO_BUILD_JOBS=1 cargo test --locked --test diagnostics (43 passed)
  • CARGO_BUILD_JOBS=1 cargo test --locked --test streaming_api (10 passed)

Hosted/merge state:

  • Hosted checks have not yet been observed.
  • This PR is intentionally draft and has not been merged.

Notes/Risks

  • The explicit batch APIs remain backed by their existing collection path and return Vec<T> as before.
  • Reader-backed Serde iteration still buffers the bounded input bytes first; parsed documents are then produced incrementally.
  • A repository-wide cargo test --locked attempt reached test-binary linking but could not complete because the host filesystem ran out of space (ld: write() failed, errno=28).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2] Make Serde document deserialization incremental

1 participant