Repository navigation
kernel: accelerate full LZ4 decoding on C906 - #515
Conversation
Build outputsCompared with the build of main at 2571baa; codex/lz4-c906-scalar is at 1046bfc, so outputs changed since then are listed too. 17 changed, 0 added, 0 removed of 570 outputs.
|
There was a problem hiding this comment.
I compared both loops line by line with LZ4_decompress_generic from LZ4 1.10.0 (full-block, endOnInputSize, noDict), and I found no correctness problem.
- Fast loop: it keeps every upstream bound. The 64-byte entry distance is there. RUN_MASK literals take the safe path within 32 bytes of either end, so the 8-byte wildcopy has more slack than it needs.
ip <= iend - 17guards the 16-byte literal copy. Thematch < lowPrefixcheck comes before every match copy, including thesafe_match_copyentry. - Short-offset copies: offsets below 16 go through
memcpy_using_offset. The register patterns for offsets 1, 2 and 4 are endian-neutral, and theinc32table/dec64tablestep leaves a distance of at least 8 before the wildcopy. - Safe loop: identical to upstream, including the 16-byte shortcut limits (
iend - 16,oend - 32), the exact-end check of the last literals and the bounded tail copy. - Tests: they now reach the near-end tail with longer trailing literals, check in-place overlap in both directions, and put input and output against guard pages.
The remaining comments are a doc precision and a stronger test oracle; see the correction on the first inline comment.
dormancygrace
left a comment
There was a problem hiding this comment.
Review completed at 5ad40fe. No new production-code defect found after checking fast-loop margins, extension bounds, short-offset copies, tail handling and wrapper routing. Added a test-only generic full-decoder oracle, valid-block mutations and parent near-end/malformed regressions; 74,251,011 checks pass in each normal and ASan/UBSan run. Full LZ4/zram patches apply to Linux 7.3 with exact context; platform configuration/C906 flags pass. Existing hardware timings remain attributed to cf09d4a. This PR includes the substantive #501/#503 corrections through #505 and remains open with those dependencies.
dormancygrace
left a comment
There was a problem hiding this comment.
Final review at bc3486a: the complete LZ4 series is now uniquely numbered 0054 (compressor), 0055 (small table), 0056 (scalar paths) and 0057 (full decoder), after the merged RISC-V 0052/0053 patches. Every renamed patch is byte-for-byte identical to its reviewed version, and all test/documentation references are updated. The complete Linux 7.3 patch/Kconfig check, platform build and C906 flags pass. Compressor, small-table and actual patched decoder contracts pass normally and under ASan/UBSan; the full decoder runs 74,251,011 checks in each mode with the generic full-decoder differential oracle. All three review threads remain resolved. The prepared main merge tree exactly matches the validated head; previous hardware measurements retain their original commit attribution.
Adapt the LZ4 1.10.0 full safe decoder for C906. Copy literals and matches in eight-byte scalar words and keep short repeated patterns in a 64-bit register, avoiding the upstream 32-byte copy loop and temporary byte array. Retain bounded tail decoding, overlap support and offset-zero initialization; reject null input and negative sizes.
Only full-block safe decoding changes. The generic implementation remains for partial decoding, nonempty dictionaries and unsafe decoding. Empty-dictionary calls and the first block of safe streaming delegate to the new full safe path. All eight exported APIs are retained. Compression output and the block format are unchanged.
On the NanoKVM C906, the final kernel decoder reduces cycles by 7.35% and retired instructions by 13.67% versus #505 on a fixed corpus of 1,413 four-KiB pages. Six paired comparisons across three rounds and two matched code layouts give 7.21–7.41% fewer cycles; baseline A/A drift is at most 0.19%. BusyBox and libc code pages use 4.95% and 4.91% fewer cycles. Low-compressibility synthetic groups cost up to 2.23% more cycles. These measurements cover kernel block decoding.
Validation on commit cf09d4a:
Review validation at 5ad40fe: 74,251,011 checks per normal/ASan+UBSan decoder run, including 11,173 successful valid/mutated blocks compared byte-for-byte with the retained generic full decoder, tails of 5–11 literals and malformed extension chains. The complete LZ4/zram patch stack applies to Linux 7.3 with exact context.