fix: segfault in certs codec aborting chalked docker builds

[maximilianhurl]

Problem

Chalked Docker builds have been dying at prep_postexec and silently falling back to unchalked:

#29 10.21 error: pid: 1 - Aborting due to signal: SIGSEGV(11)
#29 ERROR: process "/chalk ... __ prep_postexec" did not complete successfully: exit code: 139
error: docker: retrying without chalk due to: wrapped docker build exited with 1

The retry succeeds and pushes, so CI stays green while the image carries no chalk mark. Downstream that looks like missing build/push records and no deployment correlation — the symptom that led here.

This is not new and not tied to a release: it reproduces on 1.1.4-dev, 1.2.0-dev and 1.2.0. The same repo on the same chalk version produces both outcomes (47 chalked / 69 not), which is what pointed at input-dependent memory corruption rather than a regression.

Cause

prep_postexec subscans / with scan_codecs = ["certs"]. With certs.scan_no_extension defaulting to true, every extensionless file — including a statically linked Go binary — is fed to extract_cert_data(), which falls back to d2i_X509_bio() when the PEM read fails. On arbitrary binary content that occasionally yields a structurally valid but malformed certificate.

src/utils/certs.c:172-174:

EVP_PKEY *pub    = X509_get_pubkey(cert);
int       keynid = EVP_PKEY_base_id(pub);   // pub is NULL -> SEGV

X509_get_pubkey() returns NULL when the public key will not parse. Unchecked.

How it was found

DER-encode a real certificate, flip 1–16 random bytes, drive it through the actual parse loop under ASan+UBSan:

==7==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000
    #0 EVP_PKEY_get_base_id (libcrypto.so.3)
    #1 extract_cert_data certs.c:174

Crashes within 20,000 mutations on the first seed. The same run flagged a second defect at certs.c:126.

Changes

• X509_get_pubkey() NULL check — the crash. The record is treated as "not a certificate" instead of dereferencing.
• BIO_all() made binary-safe — it built its buffer with strndup(), which stops at the first NUL, while total counted every byte read; later indices then ran past the allocation. Now calloc + memcpy with bounded trailing writes.
• SubjectPublicKeyInfo instead of PKCS1 — PKCS1 is RSA-only, so for EC keys the encoder emitted nothing and a NULL landed mid-key_value, truncating both cleanup_key_value() and cstringArrayToSeq() on the Nim side. Every field after Serial was silently lost for EC certificates (39 of 119 in a stock CA bundle).
• Leaks — the X509, EVP_PKEY, BIGNUM and encoder context were never released; cleanup_key_value() freed the strings but not the arrays; subject_short/issuer_short were never freed at all. Also an OPENSSL_free/free mismatch on the serial.
• Unchecked returns — OBJ_nid2ln/OBJ_nid2sn, ASN1_STRING_length of 0 (read data[-1], wrote into a zero-byte allocation), and convert_ASN1TIME, which returned -1 even on success and left its buffer uninitialised on early failure.

Verification

	before	after
Fuzz, 160k mutations × 8 seeds	SEGV on seed 1	clean
LeakSanitizer, 119-cert CA bundle	771,327 bytes / 14,153 allocs	0
Sweep: full alpine prod rootfs + released chalk 1.2.0 binary + 24MB Go binary	—	clean

Built with -fsanitize=address,undefined against system OpenSSL 3 (one local substitution for the crypto/x509.h internal include).

🤖 Generated with Claude Code