Repository navigation
package_core: deduplicate llext-edk headers in core packages - #694
Merged
Merged
Conversation
Finds files identical across a set of directories and mirrors one canonical copy out, deleting the duplicates from the source dirs. Useful for the generated llext-edk headers duplicated across board variants. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
When applying (not --dry-run), repoint each affected -iwithprefixbefore entry in a variant's includes.txt at the mirrored common directory: fully-emptied entries get replaced (and their now-empty directory removed from disk), partially-emptied entries get an extra entry pointing at the common directory alongside the original. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Arguments are now variant names (a leading 'variants/' is ignored), resolved to variants/<name>/llext-edk/include. --common-dir is also a variant name, created at variants/<name>. Each variant's includes.txt is always updated, replacing the old "variants/<board>/" path detection. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When packaging in CI, move the llext-edk headers shared by all the included variants into a 'variants/include' directory via dedupe_includes.py, and add it to the archive. Only variants that actually have an EDK are considered, and the step is skipped when fewer than 2 remain. This modifies the working tree, so it only runs when GITHUB_ACTIONS is set (each packaging job uses a fresh checkout). Since headers may now live outside the variant directory, platform.txt looks up the -imacros files (autoconf.h and zephyr_stdint.h) via the include path instead of a variant-relative path. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
pillo79
changed the title
llext-edk headers in core packagesllext-edk headers in core packages
Built
|
| Artifact | Board | Core | Tests | RAM | Sketches | Warnings | Errors |
|---|---|---|---|---|---|---|---|
✅ zephyr_contrib |
ek_ra8d1
| 1 🏷️ | ✅ |
12.0% |
2 | - | - |
frdm_mcxn947
| 7 🏷️ | ✅ |
53.4% |
2 | - | - | |
frdm_rw612
| 3 🏷️ | ✅ |
76.5% |
2 | - | - | |
✅* zephyr_main |
giga
| 10 🏷️ | ✅* |
45.9% |
86 | 2 | - |
nano33ble
| 6 🏷️ | ✅ |
78.8% |
44 | - | - | |
nano_connect
| 5 🏷️ | ✅* |
68.9% |
38 | 6 | - | |
nano_matter
| 4 🏷️ | ✅ |
|
22 | - | - | |
nicla_sense
| 4 🏷️ | ✅ |
|
18 | - | - | |
nicla_vision
| 11 🏷️ | ✅ |
40.3% |
68 | - | - | |
opta
| 11 🏷️ | ✅ |
42.3% |
76 | - | - | |
portentac33
| 6 🏷️ | ✅ |
|
72 | - | - | |
portentah7
| 10 🏷️ | ✅* |
51.2% |
102 | 8 | - | |
✅ zephyr_unoq |
unoq
| 2 🏷️ | ✅ |
43.5% |
56 | - | - |
ventunoq
| 5 🏷️ | ✅ |
23.7% |
48 | - | - |
Legend
Board Test Status description 🔥 🔥 Test run failed to complete. ❌ 🔴 Test completed with unexpected errors. ⁉️ ⁉️ Test was expected to fail but passed; exception entry is outdated. ✔️* 🚫 Test completed with errors, but all are known/expected. ✅* 🟡 Test completed with some warnings; no errors detected. ✅ 🟢 Test passed successfully, with no warnings or errors. 🌑 🌑 Test was skipped.
Entries like '.../loader/ota/../../libraries/OTA/src' or '.../kernel_service/thread/../' were not normalized, since pathlib keeps '..' components. They never matched the deduplicated files, so no entry for the common directory was added and headers such as ota.h and kernel_shell.h could no longer be found (e.g. in zephyr_contrib). Normalizing the entries also makes the deepest-first sort use the real directory depth. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The sync-zephyr-artifacts tool copies the deduplicated 'variants/include' folder from the downloaded core into the working tree: - ignore it in git, so it is not committed by mistake; - remove it in the sync tool before copying, so stale headers from a previous sync do not stay reachable through includes.txt; - add it to the archive in package_core.sh whenever it exists, so local packages built from a synced tree are not missing headers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Built
|
| Artifact | Board | Core | Tests | RAM | Sketches | Warnings | Errors |
|---|---|---|---|---|---|---|---|
✅ zephyr_contrib |
ek_ra8d1
| 1 🏷️ | ✅ |
12.0% |
2 | - | - |
frdm_mcxn947
| 7 🏷️ | ✅ |
53.4% |
2 | - | - | |
frdm_rw612
| 3 🏷️ | ✅ |
76.5% |
2 | - | - | |
✅* zephyr_main |
giga
| 10 🏷️ | ✅* |
45.9% |
86 | 2 | - |
nano33ble
| 6 🏷️ | ✅ |
78.8% |
44 | - | - | |
nano_connect
| 5 🏷️ | ✅ |
68.9% |
38 | - | - | |
nano_matter
| 4 🏷️ | ✅ |
|
22 | - | - | |
nicla_sense
| 4 🏷️ | ✅ |
|
18 | - | - | |
nicla_vision
| 11 🏷️ | ✅ |
40.4% |
68 | - | - | |
opta
| 11 🏷️ | ✅ |
42.3% |
76 | - | - | |
portentac33
| 6 🏷️ | ✅ |
|
72 | - | - | |
portentah7
| 10 🏷️ | ✅* |
51.2% |
102 | 2 | - | |
✅ zephyr_unoq |
unoq
| 2 🏷️ | ✅ |
43.5% |
56 | - | - |
ventunoq
| 5 🏷️ | ✅ |
23.7% |
48 | - | - |
Legend
Board Test Status description 🔥 🔥 Test run failed to complete. ❌ 🔴 Test completed with unexpected errors. ⁉️ ⁉️ Test was expected to fail but passed; exception entry is outdated. ✔️* 🚫 Test completed with errors, but all are known/expected. ✅* 🟡 Test completed with some warnings; no errors detected. ✅ 🟢 Test passed successfully, with no warnings or errors. 🌑 🌑 Test was skipped.
facchinm
approved these changes
Oct 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every board variant ships its own
llext-edk/includetree, and most of those headers are byte-identical across boards. This PR moves the shared headers into a singlevariants/includedirectory when packaging in CI, which makes each core package much smaller.Changes
extra/dedupe_includes.py(new): takes two or more variant names and finds the files that are byte-identical at the same path in every variant'sllext-edk/include. It moves one copy of each tovariants/<common-dir>and deletes the duplicates. Each variant'sincludes.txtis updated to match:--dry-runonly reports what would be done.extra/package_core.sh: runs the dedup on the included variants that have an EDK, then addsvariants/include/to the archive. It is skipped if fewer than 2 such variants are left. It only runs whenGITHUB_ACTIONSis set, because it changes the working tree in place (each packaging job starts from a fresh checkout).platform.txt: the-imacrosfiles (zephyr/autoconf.h,zephyr/toolchain/zephyr_stdint.h) are now found through the include path instead of a fixed{build.variant.path}/llext-edk/...path.zephyr_stdint.his the same on every board, so it gets moved. Every variant'sincludes.txtalready has the right directories, so this works whether or not the headers were deduplicated.Results (
zephyr_unoq, 2 boards)Testing
zephyr_unoqwithGITHUB_ACTIONS=1and checked that every-iwithprefixbeforepath in the unpacked package resolves.autoconf.hcoming from each variant's own directory.GITHUB_ACTIONS) leaves the working tree untouched.zephyr_mainandzephyr_contribpackages, and boards other than the two Q boards.