Skip to content

decompress: decode legacy frames whose header is split across calls - #4812

Open
kwy404 wants to merge 1 commit into
meta:devfrom
kwy404:fix-legacy-stream-short-first-input
Open

kwy404 wants to merge 1 commit into
meta:devfrom
kwy404:fix-legacy-stream-short-first-input

Conversation

@kwy404

@kwy404 kwy404 commented Oct 1, 2026

Copy link
Copy Markdown

Fixes #4802

Root cause

With legacy support enabled, ZSTD_decompressStream() only recognized a v0.4 to v0.7 frame when a single call started with the magic number and carried at least 5 bytes. Two things broke a smaller first call:

  1. When fewer than 5 bytes arrive, the bytes are buffered in headerBuffer and ZSTD_getFrameHeader_advanced() checks that they match a supported magic number. That check only knew the zstd and skippable magics, so a legacy prefix returned Unknown frame descriptor right away.
  2. On the next call, the legacy detection ran ZSTD_isLegacy() on the current input instead of on the bytes already buffered in headerBuffer, so the magic was never seen there either.

Fix

  • In the early magic check, also accept a prefix of a supported legacy magic.
  • Detect the legacy frame from headerBuffer, replay those buffered bytes to the legacy stream decoder, then pass the rest of the input as before.

When the whole header comes in one call, the decoder gets the same bytes in the same order, so that path is unchanged.

Side effect: with legacy support compiled in, ZSTD_getFrameHeader() given 1 to 4 bytes of a legacy frame now asks for 5 bytes instead of returning prefix_unknown, as it already does for a zstd or skippable prefix. With 5 or more bytes it still returns an error for a legacy frame, as before.

Test

tests/legacy.c: testStreamingAPI() now takes a chunk size and is also run feeding the input one byte at a time, so every frame header is split across calls. The helper also stops calling ZSTD_initDStream() on every iteration (needsInit was never cleared), which would otherwise reset the stream between chunks.

Before the fix (MSVC build, ZSTD_LEGACY_SUPPORT=1):

Testing 8 frames, from format v0.1 upward
Simple API OK
Frame Decoding OK
Streaming API OK
ERROR: ZSTD_decompressStream: Unknown frame descriptor

After the fix:

Testing 8 frames, from format v0.1 upward
Simple API OK
Frame Decoding OK
Streaming API OK
Streaming API OK
OK

The reproducer from the issue now reports consistent for first calls of 1 to 8 bytes. zstreamtest -T30s and fuzzer -T30s (with ZSTD_LEGACY_SUPPORT=5) pass, and tests/legacy.c also passes with legacy support disabled.

ZSTD_decompressStream() only recognized a legacy frame when the current
call started with the full magic number and at least 5 bytes. When the
first call supplied 1 to 4 bytes, the early magic check rejected the
legacy prefix, and on later calls the legacy detection looked at the
current input instead of the bytes already buffered in headerBuffer.

Accept legacy magic prefixes in the early check, detect legacy frames
from headerBuffer, and replay the buffered bytes to the legacy decoder
before the rest of the input.

Fixes meta#4802
@meta-cla meta-cla Bot added the CLA Signed label Oct 1, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ZSTD_decompressStream() rejects a legacy (v0.5–v0.7) frame when the first call supplies fewer than 5 bytes

1 participant