Major regression in binary size between 0.12 and 0.13

[cyyynthia]

It appears the init binary has almost doubled in size between Booster 0.12 and Booster 0.13, which has an impact of 4MB on the final binary. For setups without FDE nor LVM that's double the size of the final binary, and ~30% larger for setups with FDE and LVM.

While I guess it's expected as Booster added a lot of new features in the 0.13 release, Booster is larger than mkinitcpio with just the base hook and NVMe modules, and mkinitcpio's very fast with just that hook too. With FDE, Booster is much faster than mkinitcpio and remains smaller, but only by 4MB (and without advanced systemd's features AFAICT; plus I couldn't unlock my drive with my Yubikey). IOW, it's 4MB smaller but with less mature code and more room for failure)

Note that I use a custom script to extract the generated image; this is because Booster 0.13 broke unpack: it can't extract its own archives because of an issue with symlinks (on my system).

# pacman -S booster
# cat /etc/booster.yaml /etc/crypttab
compression: zstd
extra_files: fido2-assert
vconsole: true
enable_fido2: true
enable_lvm: true
# Configuration for encrypted block devices.
# See crypttab(5) for details.
crypt0    	UUID=333d089e-d99b-46e5-bf77-c4e2124c12f6	none      	tpm2-measure-pcr=yes,tpm2-device=auto,fido2-device=auto,x-initrd.attach
# booster build test.img -f; du test.img
16508	test.img
# rm -rf a; booster ls test.img | while read f; do if [[ ! ( "$f" =~ "->" )]]; then mkdir -p "a/"`dirname $f`; booster cat test.img "$f" > "a/$f"; fi; done
# find a -type f -exec du -a {} + | sort -n -r
<See `Booster 0.13-1 (FDE)` below>
# pacman -U https://archive.archlinux.org/packages/c/booster/booster-0.12-2-x86_64.pkg.tar.zst
# booster build test.img -f; du test.img
12444	test.img
# rm -rf a; booster ls test.img | while read f; do if [[ ! ( "$f" =~ "->" )]]; then mkdir -p "a/"`dirname $f`; booster cat test.img "$f" > "a/$f"; fi; done
# find a -type f -exec du -a {} + | sort -n -r
<See `Booster 0.12-2 (FDE) below>

Without FDE:

# pacman -S booster
# cat /etc/booster.yaml /etc/crypttab
compression: zstd
# Configuration for encrypted block devices.
# See crypttab(5) for details.
# booster build test.img -f; du test.img
8116	test.img
# rm -rf a; booster ls test.img | while read f; do if [[ ! ( "$f" =~ "->" )]]; then mkdir -p "a/"`dirname $f`; booster cat test.img "$f" > "a/$f"; fi; done
# find a -type f -exec du -a {} + | sort -n -r
<See `Booster 0.13-1 (No FDE)` below>
# pacman -U https://archive.archlinux.org/packages/c/booster/booster-0.12-2-x86_64.pkg.tar.zst
# booster build test.img -f; du test.img
4312	test.img
# rm -rf a; booster ls test.img | while read f; do if [[ ! ( "$f" =~ "->" )]]; then mkdir -p "a/"`dirname $f`; booster cat test.img "$f" > "a/$f"; fi; done
# find a -type f -exec du -a {} + | sort -n -r
<See `Booster 0.12-2 (No FDE) below>

tech tip: messing with it somehow broke some stuff in /usr/lib/modules at times; pacman -S linux fixes it

Booster 0.13-1 (FDE)

16168	a/init
5784	a/usr/lib/libcrypto.so.3
3788	a/usr/lib/booster/fido2plugin.so
2392	a/usr/bin/lvm
2136	a/usr/lib/libc.so.6
1304	a/usr/lib/libsystemd.so.0.44.0
1224	a/usr/lib/libm.so.6
920	a/usr/lib/libzstd.so.1.5.7
764	a/usr/lib/libbrotlienc.so.1.2.0
600	a/usr/lib/modules/nvme_core.ko
508	a/usr/lib/modules/dm_mod.ko
448	a/usr/lib/libncursesw.so.6.6
400	a/usr/lib/libdevmapper.so.1.02
360	a/usr/lib/libreadline.so.8.3
320	a/usr/lib/libudev.so.1.7.14
296	a/usr/lib/modules/dm_persistent_data.ko
252	a/usr/lib/modules/dm_thin_pool.ko
252	a/usr/lib/libblkid.so.1.1.0
244	a/usr/lib/ld-linux-x86-64.so.2
244	a/lib64/ld-linux-x86-64.so.2
224	a/usr/lib/modules/fat.ko
224	a/usr/lib/libfido2.so.1.17.0
216	a/usr/lib/modules/dm_cache.ko
180	a/usr/lib/libgcc_s.so.1
176	a/usr/lib/modules/nvme.ko
168	a/usr/lib/libz.so.1.3.1.zlib-ng
156	a/usr/lib/modules/aesni_intel.ko
152	a/usr/lib/modules/dm_snapshot.ko
140	a/usr/lib/libbrotlicommon.so.1.2.0
128	a/usr/lib/modules/dm_crypt.ko
124	a/usr/lib/modules/tee.ko
112	a/usr/lib/modules/dm_bufio.ko
92	a/usr/lib/modules/trusted.ko
72	a/usr/lib/libresolv.so.2
72	a/usr/lib/libcbor.so.0.14.0
64	a/usr/lib/modules/dm_mirror.ko
64	a/usr/lib/modules/dm_cache_smq.ko
56	a/usr/lib/modules/encrypted_keys.ko
56	a/usr/lib/libbrotlidec.so.1.2.0
52	a/usr/lib/modules/nvme_auth.ko
52	a/usr/lib/modules/dm_log.ko
52	a/usr/lib/modules/af_alg.ko
48	a/usr/lib/modules/vfat.ko
48	a/usr/lib/modules/dm_region_hash.ko
44	a/usr/lib/modules/dm_bio_prison.ko
32	a/usr/lib/modules/crypto_user.ko
32	a/usr/lib/libdevmapper-event.so.1.02
28	a/usr/lib/modules/algif_hash.ko
28	a/usr/bin/fido2-assert
24	a/usr/lib/modules/nvme_keyring.ko
24	a/usr/lib/modules/algif_skcipher.ko
24	a/usr/lib/modules/aead.ko
16	a/usr/lib/modules/pkcs8_key_parser.ko
16	a/usr/lib/modules/gf128mul.ko
16	a/usr/lib/modules/asn1_encoder.ko
16	a/usr/lib/libaio.so.1.0.2
8	a/etc/booster.init.yaml
4	a/usr/lib/modules/booster.alias
4	a/etc/crypttab
4	a/console/keymap
0	a/etc/initrd-release

Booster 0.12-2 (FDE)

9404	a/init
5784	a/usr/lib/libcrypto.so.3
2392	a/usr/bin/lvm
2136	a/usr/lib/libc.so.6
1304	a/usr/lib/libsystemd.so.0.44.0
1224	a/usr/lib/libm.so.6
920	a/usr/lib/libzstd.so.1.5.7
764	a/usr/lib/libbrotlienc.so.1.2.0
600	a/usr/lib/modules/nvme_core.ko
508	a/usr/lib/modules/dm_mod.ko
448	a/usr/lib/libncursesw.so.6.6
400	a/usr/lib/libdevmapper.so.1.02
360	a/usr/lib/libreadline.so.8.3
320	a/usr/lib/libudev.so.1.7.14
296	a/usr/lib/modules/dm_persistent_data.ko
252	a/usr/lib/modules/dm_thin_pool.ko
252	a/usr/lib/libblkid.so.1.1.0
244	a/usr/lib/ld-linux-x86-64.so.2
244	a/lib64/ld-linux-x86-64.so.2
224	a/usr/lib/modules/fat.ko
224	a/usr/lib/libfido2.so.1.17.0
216	a/usr/lib/modules/dm_cache.ko
180	a/usr/lib/libgcc_s.so.1
176	a/usr/lib/modules/nvme.ko
168	a/usr/lib/libz.so.1.3.1.zlib-ng
156	a/usr/lib/modules/aesni_intel.ko
152	a/usr/lib/modules/dm_snapshot.ko
140	a/usr/lib/libbrotlicommon.so.1.2.0
128	a/usr/lib/modules/dm_crypt.ko
124	a/usr/lib/modules/tee.ko
112	a/usr/lib/modules/dm_bufio.ko
92	a/usr/lib/modules/trusted.ko
72	a/usr/lib/libcbor.so.0.14.0
64	a/usr/lib/modules/dm_mirror.ko
64	a/usr/lib/modules/dm_cache_smq.ko
56	a/usr/lib/modules/encrypted_keys.ko
56	a/usr/lib/libbrotlidec.so.1.2.0
52	a/usr/lib/modules/nvme_auth.ko
52	a/usr/lib/modules/dm_log.ko
52	a/usr/lib/modules/af_alg.ko
48	a/usr/lib/modules/vfat.ko
48	a/usr/lib/modules/dm_region_hash.ko
44	a/usr/lib/modules/dm_bio_prison.ko
32	a/usr/lib/modules/crypto_user.ko
32	a/usr/lib/libdevmapper-event.so.1.02
28	a/usr/lib/modules/algif_hash.ko
28	a/usr/bin/fido2-assert
24	a/usr/lib/modules/nvme_keyring.ko
24	a/usr/lib/modules/algif_skcipher.ko
24	a/usr/lib/modules/aead.ko
16	a/usr/lib/modules/pkcs8_key_parser.ko
16	a/usr/lib/modules/gf128mul.ko
16	a/usr/lib/modules/asn1_encoder.ko
16	a/usr/lib/libaio.so.1.0.2
8	a/etc/booster.init.yaml
4	a/usr/lib/modules/booster.alias
4	a/console/keymap
0	a/etc/initrd-release

Booster 0.13-1 (No FDE)

16168	a/init
2136	a/usr/lib/libc.so.6
600	a/usr/lib/modules/nvme_core.ko
508	a/usr/lib/modules/dm_mod.ko
244	a/usr/lib/ld-linux-x86-64.so.2
244	a/lib64/ld-linux-x86-64.so.2
224	a/usr/lib/modules/fat.ko
176	a/usr/lib/modules/nvme.ko
156	a/usr/lib/modules/aesni_intel.ko
128	a/usr/lib/modules/dm_crypt.ko
124	a/usr/lib/modules/tee.ko
92	a/usr/lib/modules/trusted.ko
72	a/usr/lib/libresolv.so.2
56	a/usr/lib/modules/encrypted_keys.ko
52	a/usr/lib/modules/nvme_auth.ko
52	a/usr/lib/modules/af_alg.ko
48	a/usr/lib/modules/vfat.ko
32	a/usr/lib/modules/crypto_user.ko
28	a/usr/lib/modules/algif_hash.ko
24	a/usr/lib/modules/nvme_keyring.ko
24	a/usr/lib/modules/algif_skcipher.ko
24	a/usr/lib/modules/aead.ko
16	a/usr/lib/modules/pkcs8_key_parser.ko
16	a/usr/lib/modules/gf128mul.ko
16	a/usr/lib/modules/asn1_encoder.ko
8	a/etc/booster.init.yaml
4	a/usr/lib/modules/booster.alias
0	a/etc/initrd-release

Booster 0.12-2 (No FDE)

9404	a/init
600	a/usr/lib/modules/nvme_core.ko
508	a/usr/lib/modules/dm_mod.ko
224	a/usr/lib/modules/fat.ko
176	a/usr/lib/modules/nvme.ko
156	a/usr/lib/modules/aesni_intel.ko
128	a/usr/lib/modules/dm_crypt.ko
124	a/usr/lib/modules/tee.ko
92	a/usr/lib/modules/trusted.ko
56	a/usr/lib/modules/encrypted_keys.ko
52	a/usr/lib/modules/nvme_auth.ko
52	a/usr/lib/modules/af_alg.ko
48	a/usr/lib/modules/vfat.ko
32	a/usr/lib/modules/crypto_user.ko
28	a/usr/lib/modules/algif_hash.ko
24	a/usr/lib/modules/nvme_keyring.ko
24	a/usr/lib/modules/algif_skcipher.ko
24	a/usr/lib/modules/aead.ko
16	a/usr/lib/modules/pkcs8_key_parser.ko
16	a/usr/lib/modules/gf128mul.ko
16	a/usr/lib/modules/asn1_encoder.ko
8	a/etc/booster.init.yaml
4	a/usr/lib/modules/booster.alias
0	a/etc/initrd-release

[pilotstew]

In regards to size increase, there is some work that can be done here. Booster currently bundles several dependencies unconditionally that could be stripped out when not in use, the largest being networking related ssh/clevis. However, we've also as you say added a lot of new features, tpm fixes/hardening have added a lot of code.

Do you have logs or more information on the Yubikey issue? It looks like you are also using tpm. Are you prompted for the yubikey and it's failing or is the tpm unlocking first? Token dispatch order could matter here.

FWIW I was able to recreate your unpack issue. It appears to be related to some recent hardening steps involving symlink path traversal.

[cyyynthia]

Do you have logs or more information on the Yubikey issue?

I'm afraid not, it was too late in the night when it happened and I just rolled back to mkinitcpio. This was on Booster 0.12, I didn't take the time to mess with it again.

It looks like you are also using tpm.

Yeah so my whole setup is TPM2 + 2 YubiKeys as backups. I was in the middle of setting up SB and stuff, so the TPM wasn't enrolled - getting to the hardware key was expected. I would enter my credentials, the key would blink pending touch confirmation, but the drive would not unlock and I was stuck in a loop.

Today was when I decided to give this another try see if 0.13 fixed stuff, hoping to get a leaner UKI but I was surprised to see it barely saved anything - leading to this issue

[pilotstew]

If you decide to give it another shot I'd love to know where the yubikey bug is. At some point I may be able to look into leaning out the image but in the near future I have some more hardening steps I'm working on.

[cyyynthia]

If you decide to give it another shot I'd love to know where the yubikey bug is

The main issue would be getting logs out of the computer, since the disk cannot unlock. Will I need to try enabling an emergency shell from which I can dump logs to the ESP, or is there something in Booster that can do the trick? I'd honestly just consider taking a picture of the screen but I remember Booster printing an absolutely MASSIVE wall of text so perhaps getting all of it would be much better

[pilotstew]

I apologize, I misread your last comment, yubikey was failing on version 12. fido2 handling was rewritten for version 13. One of the many changes added luks token timeout, defualt 30s. This should prevent luks tokens from hard failing the boot. Worst case it should fall back to passphrase. TPM v255+ was also broken in version 12. So in your case I wouldn't consider using the older version.

[cyyynthia]

Makes sense. As my YubiKeys are my 2 backup keys, if the initrd fails to use them to unlock the drive, then there's no passphrase to fall back to; I just can't boot at all (so what I had to do was boot from the live iso and rebuild the initrd with mkinitcpio instead) 😅

Quite secure, but quite annoying when it doesn't work as expected 🫠