Skip to content

wsflate: accept Go 1.27's BFINAL fixed-Huffman stream tail - #230

Closed
fxfactorial wants to merge 1 commit into
gobwas:masterfrom
fxfactorial:fix/wsflate-go127-flate-tail
Closed

fxfactorial wants to merge 1 commit into
gobwas:masterfrom
fxfactorial:fix/wsflate-go127-flate-tail

Conversation

@fxfactorial

Copy link
Copy Markdown

Fixes #221.

Problem

Building against Go 1.27+ (or gotip after golang/go@388145c) breaks wsflate with:

wsflate: bad compressor: unexpected stream tail: 0x7d0a0300 vs 0x0000ffff

Root cause

Go 1.27 rewrote compress/flate (golang/go@388145c, golang/go#75532). The new encoder
ends a stream with an empty final fixed-Huffman block (BFINAL=1, BTYPE=01,
i.e. 03 00 00 — 3 bytes) instead of the empty non-final stored block
(00 00 00 FF FF — 4 bytes) that previous versions appended as a flush trailer.

cbuf holds back the last 4 bytes, so the held tail is now {0xff, 0xff, 0x03, 0x00}
and the old checkTail() rejected it. Verified byte-for-byte across all levels 0-9 and
various payloads: Go 1.26 -> tail 00 00 FF FF; Go 1.27 -> tail FF FF 03 00.

Both tails are valid per RFC 7692 section 7.2.3.4
("Using a DEFLATE Block with BFINAL Set to 1").

Fix

checkTail() in wsflate/writer.go now accepts both tails:

  • Classic tail (00 00 FF FF): unchanged — the 4 held bytes are dropped (the
    receiver appends them back per RFC 7692 7.2.2).
  • Go 1.27 tail (FF FF 03 00): the 4 held bytes are passed through to the
    destination so the on-wire payload is a complete, self-terminating DEFLATE stream
    (... [00 00 00 FF FF] [03 00]). The receiver then appends 00 00 FF FF per 7.2.2,
    which lands after the BFINAL block and is ignored by the decompressor.
  • Any other tail is still rejected (error now lists both accepted tails).
  • A tailChecked flag keeps checkTail idempotent across Flush() + Close()
    (as Helper.CompressTo does), and Reset() clears it.

The on-wire format for classic-tail compressors is byte-identical to before; only
Go 1.27's tail is newly accepted. The extra 1 byte of payload for the BFINAL tail is
the spec-sanctioned way to keep the stream byte-aligned and minimal.

Tests

  • writer_tail_test.go: toolchain-independent unit tests — a blockCompressor
    drives the exact tail bytes, so the new BFINAL branch is covered even on Go 1.26.
    Covers both accepted tails, the rejected-tail case, and a default round-trip.
  • zz_edge_test.go: round-trips for empty/short/exact-4/big payloads,
    fragmented writes with mid-message Flush() calls, and context-takeover
    (single long-lived writer, carried LZ77 window).

Verified on go1.26.8 and go1.27.0: full module test suite passes on both,
plus -race and go vet clean.

cc @gobwas @cristaloleg — thanks @prattmic for the analysis in #221 (this follows
the RFC 7.2.3.4 BFINAL approach discussed there).

Go 1.27 rewrote compress/flate (golang/go@388145c, golang/go#75532). The
new encoder ends a stream with an empty final fixed-Huffman block
(BFINAL=1, BTYPE=01, i.e. 0x03 0x00 0x00) instead of the 4-byte empty
non-final stored block (0x00 0x00 0xff 0xff) used before. cbuf holds back
the last 4 bytes, so the stream tail is now {0xff,0xff,0x03,0x00} and the
old checkTail() rejected it with 'wsflate: bad compressor: unexpected
stream tail: 0x7d0a0300 vs 0x0000ffff' (gobwas#221).

Both tails are valid per RFC 7692 7.2.3.4. checkTail() now also accepts
the BFINAL-fixed tail, passing its 4 held bytes through to the destination
so the on-wire payload is a complete, self-terminating DEFLATE stream
(the receiver appends 0x00 0x00 0xff 0xff per 7.2.2, which lands after the
BFINAL block and is ignored). The classic tail path is unchanged, and
truly unexpected tails are still rejected. A tailChecked flag keeps
checkTail idempotent across Flush()+Close().

Adds toolchain-independent tests (blockCompressor drives the exact tail
bytes) covering both tails, the rejected-tail case, plus round-trip and
context-takeover/fragmented edge cases. Verified on go1.26.8 and go1.27.0.
@gobwas

gobwas commented Oct 25, 2026

Copy link
Copy Markdown
Owner

Hey @fxfactorial, thanks for your PR! I think for now we're considering #227, which feels to be more in line with the spec and less bound to a concrete compressor implementation behaviour (the stdlib's one): accepting a second known tail keeps us bound to how one particular compressor happens to encode its final block, which is exactly what broke here. #227 treats the message as finished at Flush() and keeps the compressor's Close output off, so no tail list is needed.

@gobwas gobwas closed this Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Possible incoming breakage from upstream golang

2 participants