Support Json Stream Component - #350
Open
joelwurtz wants to merge 13 commits into
Open
Conversation
…rces Replace hardcoded LazyMap::class routing checks with an is_a() check against a new generic LazyMapInterface marker (extending \ArrayAccess<TKey, TValue>). Any implementer is now treated like array/\stdClass: its property list comes from the other side of the mapping, reads go through offsetGet/offsetExists, and nested objects keep the same source type so laziness propagates recursively. FromTargetMappingExtractor now reuses the actual source class for nested objects instead of hardcoding LazyMap, so alternative implementations (e.g. a streaming JSON decoder) stay themselves all the way down.
Read a single JSON object through a purpose-built lazy decoder instead of materializing it via the fallback stream reader. The decoder exposes the object as a LazyJsonObject (a LazyMapInterface), so the generated mapper reads only the fields it maps, straight from the stream, with nested objects and lists staying lazy — no intermediate array. - JsonBuffer: seekable byte buffer over a string or a (non-seekable) stream, pulling bytes on demand so fields can be revisited out of document order. - JsonParser: position-based scan primitives; structures are skipped by brace matching, leaf values decoded natively via json_decode. - LazyJsonObject / LazyJsonList: lazy, re-iterable object and array nodes. - JsonDecoder: entry point returning the lazy root value. Top-level collections still go through the fallback for now (their single-pass streaming semantics need a discarding buffer).
getCheckExists() and the property-exists condition only recognized 'array' and
\stdClass sources, so a LazyMapInterface (ArrayAccess) source got no existence
gate: an absent key was mapped as null instead of being skipped, which throws on
a non-nullable target property. Both now treat LazyMapInterface sources like an
array, emitting $source->offsetExists('prop') as the gate. Fixes a latent bug
for LazyMap and is required for the streaming JSON decoder's collection path.
…coder Route collection reads through the lazy decoder instead of the fallback: the top-level array/object is exposed as a LazyJsonList/LazyJsonObject and fed to LazyCollection, which maps each element as it is pulled. LazyCollection keeps its role as the map-and-policy layer (buffered re-iteration, single-pass streaming); the decoder is the lazy JSON iteration layer underneath. For the STREAM option, JsonBuffer gains discardBefore(): the streaming LazyJsonList frees each element from the buffer once it has advanced past it, so a large top-level array stays bounded to one element's worth of bytes. Absolute offsets remain stable and accessing a discarded offset throws. Top-level dict-of-objects streaming still holds bytes (its per-entry discarding is a follow-up); single-pass is still enforced by LazyCollection.
…e json_stream_decode with extension or polyfill instead
Add a `json_streamer` option to the bundle, replacing the symfony/json-streamer reader and writer with the AutoMapper ones by decoration, so the Symfony implementations stay as a fallback for the types the AutoMapper does not handle. Both can be made registry aware with `only_registered_mapping`, delegating any unregistered type to the Symfony implementation. The `StreamReaderInterface` and `StreamWriterInterface` are aliased as well, since Symfony only aliases the concrete classes. Resolve the "array-like" decision once during metadata discovery, from a `#[Mapper(arrayLike: ...)]` override or the inferred default, and use it everywhere instead of guessing from the source and target names. This removes the `LazyMapInterface` marker: `LazyMap` now carries the attribute, and the flag is threaded into the mapping extractor, so any `ArrayAccess` class can be read by key. Fixes on the json target: * report a circular reference instead of recursing forever, and increment the context depth so `#[MaxDepth]` keeps working; * stream nullable nested objects and object lists through their json sub-mapper instead of falling back to a buffered `json_encode`; * write a dict as a JSON object keeping its keys, instead of a JSON array; * encode with `JSON_THROW_ON_ERROR`, like the Symfony writer, so an unencodable value fails loudly instead of producing invalid JSON, and stop turning a falsy encoding such as `0` into `null`. `map($object, 'json')` now throws: `json` is a serialization target, only reachable through the JsonStreamer integration. A dedicated API may expose it later.
…with the target
Reading a JSON stream used to map from the decoded document class, with a
dedicated listener teaching the AutoMapper to treat `JsonStream\Document` as an
array-like source, while writing already used the `json` target. The two
directions are now named the same way:
json -> Class reads a JSON stream
Class -> json writes one
`json` is therefore a regular array-like side of a mapping: it is kept as-is by
the metadata registry, inferred as array-like, read by key from the decoded
document, and propagated to the sub-mappers so a nested value keeps `json` as its
own source. The `JsonStreamDocumentListener` becomes useless and is removed, and
the registry aware mode of the reader and the writer both check the `json`
mapping instead of `array` on one side and `json` on the other.
…sion The read path changed the most: a JSON stream is now read through a generated `json` -> class mapper instead of being decoded to an intermediate shape, so deserializing a single object goes from ~101 to ~13.5 µs (~11x faster than the Symfony streamer) and reading a 20k collection from ~3 s to ~0.15-0.25 s (~16-27x faster), while streaming keeps a bounded peak. On the write side the generated `json` mappers brought the per element cost from ~4.0 to ~2.2 µs, and streaming a nested collection now peaks exactly like the Symfony writer. Also document that `json_stream_decode()` is ~6.7x faster with the `json_stream` extension than with the polyfill, since every read number assumes the extension.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace #348
This add support to replace JsonStreamReader and JsonStreamWriter of the Symfony component by an AutoMapper implementation, some differences notable :
About the implementation :
This library is implemented as a PHP extension with a polyfill in pure PHP (extension is x10 to x20 times faster than the pure PHP implementation) see https://github.com/joelwurtz/php-json-stream
If you read correctly it means you should be able to do this :
$json = $this->autoMapper->map($data, 'json');, yes it is possible, but it is disabled by throwing an exception. We don't want to allow that through themapmethod. This should be possible in the future but with a specific interface, however this PR is about supporting JsonStreamReader so it is only internal for now.The whole thing is still expiremental, i don't think the API will change until the next major release, but i still want to have this possibility in case of a big blocker (like always this will be the last choice wanted if we can avoid it we wont break)