crypto: skcipher - multi-data-unit dispatch as a template#1023
Open
blktests-ci[bot] wants to merge 4 commits into
Open
crypto: skcipher - multi-data-unit dispatch as a template#1023blktests-ci[bot] wants to merge 4 commits into
blktests-ci[bot] wants to merge 4 commits into
Conversation
added 4 commits
June 30, 2026 09:36
Add a data_unit_size field to struct skcipher_request. When non-zero, the request covers cryptlen / data_unit_size data units that share one starting IV; per-unit IVs are derived from the request IV as a wide data-unit-number counter (the convention also used by blk-crypto for inline encryption). cryptlen must be a positive multiple of data_unit_size. The field is honoured by an skcipher that understands data units -- an instance of the dun(...) template (added next), or a driver that handles a whole multi-DU request natively. A plain skcipher ignores it, so the field is inert for every existing caller; the core en/decrypt path is unchanged. skcipher_request_set_tfm() and the on-stack request initialiser reset it to 0 so a reused request defaults to single-DU. Signed-off-by: Leonid Ravich <lravich@amazon.com>
Add a dun(...) skcipher template that wraps an inner skcipher whose IV is a wide data-unit-number counter (e.g. dun(xts(aes),le)). When the caller sets skcipher_request::data_unit_size, the template splits the request into cryptlen / data_unit_size sub-requests on the inner cipher, walking the IV +1 per unit. Each inner ->encrypt/->decrypt is a direct call, so only the outer dispatch into the crypto API is indirect -- the per-unit work is not. The second template parameter selects the counter endianness: dun(...,le) for a little-endian counter (dm-crypt plain64, blk-crypto inline encryption) and dun(...,be) for a big-endian one (dm-crypt plain64be). Those are the only two ways a per-unit IV relates to its neighbour by a +1 step; IV modes that are not such a counter are simply not wrapped. Like cryptd() and pcrypt(), dun() wraps an inner skcipher and changes only how the request is dispatched -- here, split across data units -- performing no cipher transform of its own. A dun() tfm exists solely for multi-DU dispatch, so a request with data_unit_size 0 is rejected with -EINVAL; a caller wanting plain single-DU encryption uses the inner skcipher. A hardware engine that consumes a whole multi-DU request in one pass registers its own dun(...) at a higher cra_priority and is selected automatically by the existing priority mechanism; no per-algorithm capability flag is needed. The generic template is sync-only (the split loop treats any non-zero inner return as terminal), so it resolves against a sync inner cipher (mask | CRYPTO_ALG_ASYNC); async is left to such native drivers. The inner IV must be a whole number of 64-bit limbs and no wider than 32 bytes: 16 covers xts(...), 32 covers the widest inline-encryption mode (Adiantum). Suggested-by: Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by: Leonid Ravich <lravich@amazon.com>
For every ivsize-16 skcipher, wrap it in both a dun(<inner>,le) and a dun(<inner>,be) instance and cross-check each batched output against an independent N x single-DU reference run directly on the inner tfm (both keyed with one random key, the reference counter walked in the matching endianness), over a deliberately fragmented scatterlist whose entries do not align to the data-unit size. The two must produce byte-identical ciphertext; the batched ciphertext is then round-tripped and the caller IV checked unchanged. Testing both endiannesses exercises the be path independently of any in-tree consumer. Algorithms with no dun wrapper (ivsize != 16) are skipped; a genuine mismatch returns -EBADMSG. Signed-off-by: Leonid Ravich <lravich@amazon.com>
Submit one skcipher request per contiguous bio segment (a single bio_vec) with data_unit_size = cc->sector_size, instead of one request per sector. E.g. the default 512-byte sector with a 4 KiB bio_vec becomes one request of 8 data units; the crypto layer (the dun() template, or a native driver) walks the per-sector IV as a data-unit counter. Because a bio_vec is one contiguous segment, the request uses only the existing inline dmreq->sg_in[0]/sg_out[0] entry -- no per-bio scatterlist allocation, and no regression on small random I/O. crypt_alloc_tfms() wraps the skcipher in dun(<cipher>,<endian>) when crypt_can_batch_dun() holds: an IV mode that is a data-unit counter (its crypt_iv_operations sets dun_endian to the counter endianness -- "le" for plain64, "be" for plain64be; non-counter modes such as lmk/tcw/eboiv leave it NULL and are excluded), single-tfm, non-aead, and sector_size 512 or iv_large_sectors so the per-unit IV step is exactly one. This is the same kind of name rewrite as essiv(), done in the one alloc helper so callers are unchanged. DM_CRYPT selects CRYPTO_DUN and dun() resolves against a sync inner cipher, so wrapping has no acceptable failure that the bare cipher would survive -- there is no fallback; any error propagates. (A config whose only xts provider is async with no generic CRYPTO_XTS would now fail to activate rather than silently run per-sector; generic xts is selected by the dependency chain, so this does not arise in practice.) crypt_convert_block_skcipher() handles both cases in one function: the length is crypt_skcipher_len() -- a whole contiguous segment when batching, else a single sector -- and data_unit_size is set unconditionally (a dun() tfm reads it; a plain skcipher ignores it). It advances the bio iterators itself (as the aead path already does) and reports the bytes processed, so crypt_convert() advances cc_sector / tag_offset uniformly via one helper, no per-case duplication. Verified byte-equivalent to the per-sector path: plain64 and plain64be dm-crypt with dun() produce ciphertext bit-identical to an unpatched kernel over a 256 MB device (xts-aes driving the split). Signed-off-by: Leonid Ravich <lravich@amazon.com>
Author
|
Upstream branch: dc59e4f |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull request for series with
subject: crypto: skcipher - multi-data-unit dispatch as a template
version: 5
url: https://patchwork.kernel.org/project/linux-block/list/?series=1118772