Reject oversized digest_size in generichash_blake2b_final()

[akashchamp]

Summary

nacl.bindings.crypto_generichash.generichash_blake2b_final() aborts the
process with SIGABRT when called on a Blake2State whose digest_size
exceeds crypto_generichash_BYTES_MAX (64).

Blake2State can be constructed directly with an arbitrary digest_size,
bypassing the parameter validation that generichash_blake2b_init()
performs. generichash_blake2b_final() then passes that unchecked
digest_size straight through to libsodium's crypto_generichash_blake2b_final(),
which ultimately calls blake2b_final(). That function calls
sodium_misuse() (which aborts by default) when outlen > BLAKE2B_OUTBYTES,
crashing the interpreter instead of raising a Python exception.

Fixes #964.

Root cause

// src/libsodium/src/libsodium/crypto_generichash/blake2b/ref/blake2b-ref.c
int
blake2b_final(blake2b_state *S, uint8_t *out, uint8_t outlen)
{
    unsigned char buffer[BLAKE2B_OUTBYTES];

    if (!outlen || outlen > BLAKE2B_OUTBYTES) {
        sodium_misuse();   // aborts the process
    }
    ...

generichash_blake2b_final() in src/nacl/bindings/crypto_generichash.py
had no equivalent guard, even though the sibling functions
(generichash_blake2b_salt_personal() and generichash_blake2b_init())
both validate digest_size via the module's _checkparams() helper before
calling into libsodium.

Fix

Add the same digest_size <= crypto_generichash_BYTES_MAX check already
used by _checkparams() to generichash_blake2b_final(), so an
out-of-range digest_size now raises nacl.exceptions.ValueError instead
of aborting the process.

Testing

• Added test_generichash_blake2b_final_rejects_oversized_directly_constructed_state
  in tests/test_generichash.py, which constructs a Blake2State with
  digest_size = crypto_generichash_BYTES_MAX + 1 and asserts
  generichash_blake2b_final() raises ValueError instead of crashing.
• Ran the full test suite: pytest tests/ -n auto --dist=worksteal
  → 4662 passed, 10 skipped (skips are pre-existing/environment-only,
  e.g. "Requires minimal build of libsodium"), unchanged from before this
  change.
• ruff check src/ tests/ and ruff format --check src/ tests/ → clean.
• mypy → Success: no issues found in 49 source files.

Manual verification

Before the fix, the reproducer from the issue crashed the interpreter:

$ python -c "
from nacl.bindings.crypto_generichash import Blake2State, generichash_blake2b_final
generichash_blake2b_final(Blake2State(65))
"
$ echo $?
134   # SIGABRT

After the fix, the same call raises a normal Python exception:

$ python -c "
from nacl.bindings.crypto_generichash import Blake2State, generichash_blake2b_final
generichash_blake2b_final(Blake2State(65))
"
Traceback (most recent call last):
  ...
nacl.exceptions.ValueError: Digest_size greater than 64
$ echo $?
1

I also confirmed the normal generichash_blake2b_init() /
generichash_blake2b_update() / generichash_blake2b_final() path is
unaffected and still matches the one-shot nacl.hash.blake2b() output for
a digest_size=64 hash of b"hello world".

Scope

This is a minimal, targeted fix: one added bounds check plus one
regression test, no unrelated refactors or dependency changes.

🤖 Generated with Claude Code

Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com