Skip to content

examples/sotest: support packaged shared library fixtures#3526

Open
aviralgarg05 wants to merge 6 commits into
apache:masterfrom
aviralgarg05:gsoc/sotest-shared-fixtures-pr3
Open

examples/sotest: support packaged shared library fixtures#3526
aviralgarg05 wants to merge 6 commits into
apache:masterfrom
aviralgarg05:gsoc/sotest-shared-fixtures-pr3

Conversation

@aviralgarg05

@aviralgarg05 aviralgarg05 commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

This patch is another smaller upstream slice being extracted from the ongoing GSoC 2026 Dynamic ELF loading and nxpkg work for NuttX.

An earlier draft PR (#3474) carries the initial nxpkg package lifecycle helper. PR #3519 carries a separate apps/ module-support cleanup slice from the same broader effort. This PR is independent of #3519, but can still be reviewed after it to keep the follow-up series orderly.

Series order for this PR:

The change itself extends the examples/sotest packaging path so the shared-library test fixtures can also be prepared through nxpkg-style package artifacts.

Concretely, it:

  • generates shared-index.json and pkgsotest.nsh from the built modprint and sotest shared objects
  • records the correct target arch / compat metadata and SHA-256 digests for those packaged shared-library fixtures
  • allows sotest to run in either its existing builtin-ROMFS flow or from explicit shared-library paths, with a --mount helper mode for preparing the builtin test mount separately
  • keeps the paired fixture outputs in one grouped make step so parallel builds do not re-enter the generator independently

Impact

This makes the sotest shared-library example usable as a package-style fixture producer for the same Dynamic ELF / nxpkg series.

That is useful for validating the shared-library side of the packaging flow, where the loader should consume installed artifacts rather than only the default builtin test paths.

There is no intended regression for the normal sotest example flow. The existing builtin-ROMFS path is preserved, while the test binary now also accepts explicit shared-library paths when the package-installed copies need to be exercised.

Keeping this work separate from the larger series should make review easier, while still preserving the intended merge order after #3474.

Testing

Host:

  • macOS 26.5
  • Apple Silicon (arm64)
  • xtensa-esp-elf-gcc (crosstool-NG esp-14.2.0_20241119) 14.2.0
  • esptool.py v4.11.0

Focused verification for this PR:

  • clean detached apps worktree with this patch stacked on top of the current #3474 head
  • clean detached companion apache/nuttx worktree at the local boards/xtensa/esp32s3/xiao:sotest: enable nxpkg fixtures commit used for the matching XIAO configuration
  • target: xtensa
  • config: esp32s3-xiao:sotest

Build steps:

./tools/configure.sh -E -a ../apps-stacked esp32s3-xiao:sotest
make olddefconfig
make -j8

Generated fixture artifacts:

bin/shared-index.json
bin/pkgsotest.nsh

Build result:

LD: nuttx
CP: nuttx.hex
MKIMAGE: ESP32-S3 binary
Generated: nuttx.bin

Broader series context:

  • this slice was verified specifically for clean build and fixture generation in the stacked setup above
  • the broader Dynamic ELF / nxpkg series has been validated separately on the Seeed XIAO ESP32S3 Sense during the ongoing bring-up work, but this PR description is intentionally limiting its direct claim to the focused stacked verification above

@acassis

acassis commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

@aviralgarg05 and this one here?

@acassis

acassis commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

@aviralgarg05 what about CMakeLists.txt doesn't it need to be modified as well?

@aviralgarg05

Copy link
Copy Markdown
Contributor Author

Sorry, I missed the matching CMakeLists.txt update. I’ll fix that and then get it ready for the review

@aviralgarg05 aviralgarg05 marked this pull request as ready for review June 21, 2026 14:55
Add the shared-library package index and launch script fixtures used by the packaged sotest flow, covering the install metadata and staged execution path.

Format the helper generator with black so it matches the current apps lint checks.

Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>
@aviralgarg05 aviralgarg05 force-pushed the gsoc/sotest-shared-fixtures-pr3 branch 2 times, most recently from abe0cdf to 450bd87 Compare June 21, 2026 15:05
Add the CMake-side build rules needed for the packaged shared-library sotest flow so it matches the existing Makefile behavior. This wires in the generated symbol table, ROMFS image, and packaged fixture artifacts for the shared-library test path.

Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>
@aviralgarg05 aviralgarg05 force-pushed the gsoc/sotest-shared-fixtures-pr3 branch 2 times, most recently from 0e0f392 to 469d1f8 Compare June 22, 2026 11:27
Comment thread examples/sotest/main/CMakeLists.txt Outdated
Use the correct helper target names in the CMake sotest fixture rules so
that the generated ROMFS assets depend on the shared-library test outputs
in both ELF and non-ELF compiler paths.

Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>
@aviralgarg05 aviralgarg05 force-pushed the gsoc/sotest-shared-fixtures-pr3 branch from 469d1f8 to 76ab80f Compare June 22, 2026 13:31
@github-actions github-actions Bot added Size: L and removed Size: M labels Jun 22, 2026
@aviralgarg05 aviralgarg05 force-pushed the gsoc/sotest-shared-fixtures-pr3 branch from 90ad20d to 7bb6f87 Compare June 22, 2026 13:48
Replace the shell-based CMake fixture generation with Python helpers\nso the generated symtab and ROMFS sources are produced in a\nportable way across the supported build hosts.\n\nThis keeps the sotest packaged shared-library fixture flow aligned\nwith the existing Makefile path while making the generated outputs\nexplicit to CMake.

Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>
@aviralgarg05 aviralgarg05 force-pushed the gsoc/sotest-shared-fixtures-pr3 branch from 7bb6f87 to 49d5f70 Compare June 22, 2026 13:52
@aviralgarg05

Copy link
Copy Markdown
Contributor Author

I checked the failing run, and this one does not seem to be caused by the patch itself. The failure is happening while the CI job tries to pull the shared container image from ghcr.io, before the actual build/test steps start. Since multiple matrix jobs are failing at that stage, this looks more like an infrastructure/network issue than a code issue in this PR.

@acassis Can you pls help me understand if there is something wrong with my code

@acassis

acassis commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

I checked the failing run, and this one does not seem to be caused by the patch itself. The failure is happening while the CI job tries to pull the shared container image from ghcr.io, before the actual build/test steps start. Since multiple matrix jobs are failing at that stage, this looks more like an infrastructure/network issue than a code issue in this PR.

@acassis Can you pls help me understand if there is something wrong with my code

Don't worry, it is a github failure, I just restart the Job

@aviralgarg05 aviralgarg05 force-pushed the gsoc/sotest-shared-fixtures-pr3 branch from 231083e to 5836877 Compare June 25, 2026 05:35
Keep the Python-based fixture generation in the CMake path, but use the shell wrapper in the Makefile path as requested in review.

This keeps the Make-based flow aligned with the existing shell-driven build rules without changing the generated shared-fixture outputs.

Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>
@aviralgarg05 aviralgarg05 force-pushed the gsoc/sotest-shared-fixtures-pr3 branch from 5836877 to f24da1e Compare June 25, 2026 05:37
acassis
acassis previously approved these changes Jun 27, 2026
@acassis acassis requested a review from xiaoxiang781216 June 27, 2026 11:50
Comment thread examples/sotest/main/mk_pkg_fixture_shared.py Outdated

@linguini1 linguini1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading the description and reviewing your code, I'm still uncertain what this PR is doing? Is this to make sotest a loadable application through nxpkg?

This PR includes Python scripts as part of the build process, which I remember was a contentious issue. Is it okay to enforce that the user has Python for this?

@aviralgarg05

Copy link
Copy Markdown
Contributor Author

I pushed an update for this.

The Python generators added by this PR are removed now. The shared fixture helper is now a small C host tool, and the CMake path no longer introduces Python into the build flow.

Also, the goal here is not to make sotest an end-user nxpkg app. The intent is to generate package-style shared-library fixtures so the shared-library loading path can be validated using installed artifacts instead of only the default builtin test layout.

@linguini1

Copy link
Copy Markdown
Contributor

Can you explain what a "package-style fixture producer" is? I'm not sure I really understand, sorry.

Also, please squash your commits.

Replace the Python-based sotest fixture helpers with a small host-side C tool so the generated packaged shared-library fixtures do not depend on Python at build time.\n\nKeep the generated helper and CMake rules formatted to match the current apps lint checks.

Signed-off-by: aviralgarg05 <gargaviral99@gmail.com>
@aviralgarg05 aviralgarg05 force-pushed the gsoc/sotest-shared-fixtures-pr3 branch from 2ca1c72 to 40329b7 Compare July 7, 2026 18:50
@aviralgarg05

Copy link
Copy Markdown
Contributor Author

@linguini1

By “package-style fixture producer”, I mean the small host-side helper that generates the test package metadata and script files for the packaged sotest flow.

In this case it takes the built shared-library artifacts and writes the fixture files used by nxpkg, so the packaged install/run path is exercised as part of the example instead of only the plain builtin layout.

@linguini1

Copy link
Copy Markdown
Contributor

@linguini1

By “package-style fixture producer”, I mean the small host-side helper that generates the test package metadata and script files for the packaged sotest flow.

In this case it takes the built shared-library artifacts and writes the fixture files used by nxpkg, so the packaged install/run path is exercised as part of the example instead of only the plain builtin layout.

What is the packaged sotest flow? Like installing sotest from a package?

@aviralgarg05

Copy link
Copy Markdown
Contributor Author

Yes, exactly.

@linguini1

Copy link
Copy Markdown
Contributor

It doesn't really seem sustainable to have host side tools for every application in order to generate these files. It should probably be a central build tool that works for making any package compatible with nxpkg

@aviralgarg05

Copy link
Copy Markdown
Contributor Author

For this PR, I was trying to keep the scope limited to replacing the Python dependency in the existing example-local fixture path.

I agree the better long-term direction is a shared build helper for nxpkg-compatible fixture generation, instead of per-example host tools. I can keep this PR scoped as-is and we can follow up with that cleanup separately.

@linguini1

Copy link
Copy Markdown
Contributor

For this PR, I was trying to keep the scope limited to replacing the Python dependency in the existing example-local fixture path.

But that doesn't make sense, because this PR is what added the Python dependency? So there was nothing to remove, just a change in your implementation of the patch.

I agree the better long-term direction is a shared build helper for nxpkg-compatible fixture generation, instead of per-example host tools. I can keep this PR scoped as-is and we can follow up with that cleanup separately.

I think it would be better to start your feature correctly to make it easier for you to package more applications later, but if it's easier then it's okay to put this one in using C-based host tools.

Also another thing for later would be to included a SHA hash library instead of rolling your own. But I think it's fine for this PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants