Support python 3.13 and oneAPI 2026.1 by eap · Pull Request #2086 · JCSDA/spack-stack

eap · 2026-08-14T19:32:23Z

September 14, this is updated and ready for review

Apologies up-front for the weighty PR here. I've been working dilligently to get something that builds on the aws-ubuntu system and this is what I've come up with. I've coupled the changes of oneapi and python because they were all interdependent anyways and any single-purpose PR would create a similar mess to be cleaned up by the other connected PR. Putting them all in one place at least concentrates the pain.

While this was developed with oneAPI 2026.1, I suspect it would also work with 2026.0.

Nothing here is a code change to this PR — it is all upstream plumbing that has
to land before the repos/builtin submodule can be bumped and this PR merged.

This PR is blocking #2111

Dependencies

Testing

This has been tested on Ubuntu 2024.04.

Checklist

This PR addresses one issue/problem/enhancement or has a very good reason for not doing so.
These changes have been tested on the affected systems and applications.
All dependency PRs/issues have been resolved and this PR can be merged.
All necessary updates to the documentation (spack-stack wiki) will be made when this PR is merged

rickgrubin-noaa · 2026-08-14T19:38:02Z

@eap perhaps also consider this request?

…ates # Conflicts: # configs/common/packages.yaml

ecFlow 5.18 configures its Python bindings with pybind11. Built against pybind11 3.x, a bare "import ecflow" completes and then aborts during garbage collection at interpreter teardown: munmap_chunk(): invalid pointer -> SIGABRT, exit 134 The work itself succeeds, so this surfaces as scripts and CI steps failing on exit status after doing their job; "uw --version" prints correctly and then dies the same way. The recipe carried this bound as py-pybind11@2.10.3:2 in spack-packages b141387c8d, but bb52369c42 ("Fix for ecflow") dropped the upper bound. That revert reached develop via spack-packages PR JCSDA#82 (spack-stack cd20203), which is why this only appears after merging develop. Verified on aws-ubuntu2404 with unified-dev/gcc-13.3.0: same ecflow 5.18.0 and python 3.13.13, only pybind11 differing -- 3.0.2 exits 134, 2.13.6 exits 0. Full 419-spec env rebuilds and concretizes clean with the bound.

eap · 2026-09-14T21:38:39Z

This is ready for review.

Edit: CI failures were due to known duplicate packages and are fixed.

AlexanderRichert-NOAA · 2026-09-14T23:57:25Z

-      - '@5.11.4'
      - +ui
+      - '@5.18:'
+      - ^py-pybind11@:2


Is this to avoid duplicates, or does the ecflow recipe potentially need to be updated?

I need to update the ecflow recipe.

Here's the fix. JCSDA/spack-packages#88
I'll revert this change in anticipation of the merge.

This can be removed if you update spack-packages to the head of spack-stack-dev

@eap Last request to remove this line 127, then it's good!

eap · 2026-09-15T18:39:51Z

This now works end-to-end on intel and GCC, I was able to simplify a number of old config/common/packages.yaml constraints due to updates in our python build chain.

climbfuji · 2026-09-16T17:59:38Z

@eap I will create a pull request against your branch later today or tomorrow with findings from my neptune-dev template builds.

Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>

climbfuji · 2026-09-16T21:43:24Z

Merge conflict for a80e7c5

climbfuji · 2026-09-17T01:36:02Z

@eap I created eap#2 which does the following:

fix repos/builtin submodule pointer
pin py-numpy to avoid duplicates (2.3.5 and 2.4.6)
move cairo from packages_oneapi.yaml into packages.yaml
py-lis-ncoda-utils (NRL) needs meson with Python 3.12+

Tested: built

neptune-env with oneapi@2026.1.0 and gcc@14.2.1
cylc-env with gcc@14.2.1

Didn't run anything yet, but anything that comes up can be fixed later.

Once that's merged, I'll approve.

…ates

feature/python-updates: bug fixes for building neptune environments with oneapi and gcc

…flow

climbfuji · 2026-09-17T13:11:03Z

@eap Last one! eap#3

Update configs/common/packages.yaml: pin py-bind11 separately from ecflow

climbfuji · 2026-09-17T15:27:56Z

No more duplicate packages! Will approve once the CI tests finish.

climbfuji

Thanks to you @eap and @mathomp4 for all your hard work on this.

climbfuji reviewed Aug 14, 2026

View reviewed changes

Comment thread configs/common/packages.yaml

Comment thread configs/common/packages_oneapi.yaml

eap force-pushed the feature/python-updates branch from 92de0a2 to 7613be7 Compare August 14, 2026 20:15

eap added 4 commits August 14, 2026 23:26

some updates for python 3.12

d7b3e43

Python 3.12 changes

a386b31

More config changes for python 3.13

1899d4f

All the oneapi fixes

ce28283

eap force-pushed the feature/python-updates branch from 7613be7 to ce28283 Compare August 14, 2026 23:28

Merge branch 'develop' into feature/python-updates

a0878a8

eap marked this pull request as ready for review August 18, 2026 16:42

eap added 4 commits August 19, 2026 13:47

Merge branch 'develop' into feature/python-updates

d225520

Merge remote-tracking branch 'origin/develop' into feature/python-upd…

df6db6d

…ates # Conflicts: # configs/common/packages.yaml

Site spack updates with numpy 2.0 overrides and other fixes

18067be

eap requested review from AlexanderRichert-NOAA, AlysonStahl-NOAA and Hang-Lei-NOAA as code owners September 14, 2026 21:21

Merge branch 'develop' into feature/python-updates

840e2a1

eap requested review from climbfuji and mathomp4 September 14, 2026 21:38

integration tests ignore duplicate py-versioneer

0e52543

AlexanderRichert-NOAA reviewed Sep 14, 2026

View reviewed changes

AlexanderRichert-NOAA reviewed Sep 15, 2026

View reviewed changes

Comment thread configs/sites/tier1/acorn/packages_intel-19.1.3.304.yaml Outdated

AlexanderRichert-NOAA reviewed Sep 15, 2026

View reviewed changes

Comment thread configs/sites/tier1/acorn/packages.yaml Outdated

Fix acorn

4971939

mathomp4 mentioned this pull request Sep 15, 2026

[INSTALL]: libtorch/pytorch #2056

Open

Simplify constraints, remove unnecessary constraints

c75190a

AlexanderRichert-NOAA approved these changes Sep 15, 2026

View reviewed changes

climbfuji reviewed Sep 15, 2026

View reviewed changes

Comment thread configs/common/packages.yaml Outdated

Comment thread configs/common/packages_oneapi.yaml Outdated

eap mentioned this pull request Sep 16, 2026

Resolve py-versioneer multi-version issue with upstream patches to dask and pyogrio #2125

Open

eap and others added 2 commits September 16, 2026 14:27

Update configs/common/packages.yaml

76968ca

Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>

Update spack-packages to latest commit

a80e7c5

eap force-pushed the feature/python-updates branch from ec24a61 to a80e7c5 Compare September 16, 2026 20:41

eap requested a review from climbfuji September 16, 2026 20:42

climbfuji reviewed Sep 16, 2026

View reviewed changes

Comment thread configs/common/packages.yaml

climbfuji reviewed Sep 16, 2026

View reviewed changes

Comment thread configs/common/packages_oneapi.yaml Outdated

Bug fixes for building neptune environments with oneapi and gcc

447fdd5

eap and others added 3 commits September 17, 2026 03:01

Merge remote-tracking branch 'origin/develop' into feature/python-upd…

2d7f317

…ates

Merge pull request #2 from climbfuji/feature/python-updates-dom

562120c

feature/python-updates: bug fixes for building neptune environments with oneapi and gcc

Update configs/common/packages.yaml: pin py-bind11 separately from ec…

cfcab56

…flow

Merge pull request #3 from climbfuji/feature/python-updates-dom-2

57c9206

Update configs/common/packages.yaml: pin py-bind11 separately from ecflow

climbfuji reviewed Sep 17, 2026

View reviewed changes

Comment thread configs/common/packages.yaml Outdated

Apply suggestion from @climbfuji

b349fc5

mathomp4 approved these changes Sep 17, 2026

View reviewed changes

climbfuji approved these changes Sep 17, 2026

View reviewed changes

climbfuji enabled auto-merge (squash) September 17, 2026 15:59

climbfuji merged commit 49d0eb9 into JCSDA:develop Sep 17, 2026
6 checks passed

This was referenced Sep 17, 2026

[INSTALL]: (Attempt to) Update Python to 3.13 #2005

Closed

[INSTALL]: py-numpy@2 #1917

Closed

mathomp4 mentioned this pull request Sep 22, 2026

Updates for Python 3.13, HDF5 2, and more #2111

Closed

1 of 9 tasks

Conversation

eap commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependencies

Testing

Checklist

Uh oh!

rickgrubin-noaa commented Aug 14, 2026

Uh oh!

Uh oh!

Uh oh!

eap commented Sep 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexanderRichert-NOAA Sep 14, 2026

Choose a reason for hiding this comment

Uh oh!

eap Sep 15, 2026

Choose a reason for hiding this comment

Uh oh!

eap Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

climbfuji Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

climbfuji Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

eap commented Sep 15, 2026

Uh oh!

Uh oh!

Uh oh!

climbfuji commented Sep 16, 2026

Uh oh!

Uh oh!

Uh oh!

climbfuji commented Sep 16, 2026

Uh oh!

climbfuji commented Sep 17, 2026

Uh oh!

climbfuji commented Sep 17, 2026

Uh oh!

Uh oh!

climbfuji commented Sep 17, 2026

Uh oh!

climbfuji left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

eap commented Aug 14, 2026 •

edited

Loading

eap commented Sep 14, 2026 •

edited

Loading