GH-48172: [Python] Add cp315 to build by paultiq · Pull Request #48191 · apache/arrow

paultiq · 2025-11-20T14:30:12Z

Rationale for this change

Add Python 3.15 to the pyarrow nightly builds, so that issues can be detected early in the development cycles of both cpython & pyarrow.

Note:

~~CFFI and~~ Pandas doesn't have nightly 3.15 wheels, and are accordingly excluded.
Numpy shipped 3.15rc1 in 2.5.2 ~~doesn't ship a 3.15 wheel, but is needed to build. It is building from source.~~

What changes are included in this PR?

Adding cp315/cp315t to the build matrix
Excluding ~~cffi/~~ pandas for 3.15, as the wheels aren't yet available.

2026-08-13 edit

Bump libcst to 1.9.0
Update manylinux image to 2026.08.08-1 (for 3.15rc1)
Add -rc to PYTHON_IMAGE_TAG
Disable pandas tests if Pandas isn't installed:

Are these changes tested?

Yes, tested on a local Linux build using the ninja-release-python preset (not maximal)

Are there any user-facing changes?

No functional changes, other than allowing the user to use Python 3.15.

GitHub Issue: [Python] Add support and wheels for Python 3.15 #48172

Progress Notes (2026-08-15)

Each platform uses different sources for Python builds, each with different release schedules and paths:

manylinux is prebuilt with latest Python builds in /opt/python, so using the latest manylinux build is sufficient. 3.15 wasn't available until beta 1.
musllinux: Uses Alpine system python built from Add Alpine 3.24 and drop 3.22 docker-library/python#1123. Might need to bump ALPINE_LINUX version in arrow/dev/tasks/python-wheels/github.linux.yml.
musllinux free-threaded: uses astral's builds. Edit python-free-threaded-wheel-musllinux-test-imports.dockerfile + -test-unittests.dockerfile: set the python_patch_version, and set build_date in compose.yaml.
macos: Downloads official build from python.org, see ci/scripts/install_python.sh. For early builds, needed to disable --only-binary since at least one dependency needed to be built (no 3.15 wheels available yet)... only macos uses --only-binary
Windows: installs builds using choco, see python-wheel-windows-test-vs2022.dockerfile and python-free-threaded-wheel-windows-test-vs2022.dockerfile

For 3.15.0rc1 builds, some care was needed because certain paths needed 3.15.0 and others needed the full 3.15.0rc1.

…til they ship 3.15 wheels.

github-actions · 2025-11-20T14:30:37Z

⚠️ GitHub issue #48172 has been automatically assigned in GitHub to PR creator.

raulcd · 2025-11-21T08:02:03Z

@github-actions crossbow submit wheel-manylinux-2-28-cp315-cp315-amd64

github-actions · 2025-11-21T08:04:15Z

Revision: 2d2a557

Submitted crossbow builds: ursacomputing/crossbow @ actions-c8d038a747

Task	Status
wheel-manylinux-2-28-cp315-cp315-amd64

paultiq · 2025-11-21T15:16:03Z

The build was destined for failure, because cp315 is not yet in the manylinux build. I'll reproduce locally and see if there's a graceful way to resolve, along with a clearer failure when PYTHON_ROOT is not found.

The build failed oddly, but the earlier issue was that the detection of PYTHON_ROOT failed quietly:

2025-11-21T08:07:23.9363828Z #20 [stage-0 13/16] RUN PYTHON_ROOT=$(find /opt/python -name cp315-cp315) && echo "export PATH=$PYTHON_ROOT/bin:$PATH" >> /etc/profile.d/python.sh
...
2025-11-21T08:08:01.3779653Z -- Found Python3: /bin/python3.11 (found version "3.11.11") found components: Interpreter

A test should be added to verify PYTHON_ROOT is set / not empty and fail fast:

arrow/ci/docker/python-wheel-manylinux.dockerfile

Lines 110 to 111 in 2fb2f79

    
           RUN PYTHON_ROOT=$(find /opt/python -name cp${PYTHON_VERSION/./}-${PYTHON_ABI_TAG}) && \ 
        
               echo "export PATH=$PYTHON_ROOT/bin:\$PATH" >> /etc/profile.d/python.sh

Build Failure

Crossbow failed due to

lto1: fatal error: bytecode stream in file ‘src/arrow/CMakeFiles/arrow_vendored.dir/Unity/unity_1_c.c.o’ generated with GCC compiler older than 10.0

caused by

2025-11-21T08:08:00.9596950Z -- The C compiler identification is GNU 8.5.0
. 2025-11-21T08:08:01.0402969Z -- The CXX compiler identification is GNU 14.2.1

paultiq · 2025-11-21T20:25:31Z

Looking more closely, and considering the history of 3.13 and 3.14:

This (nightlys) needs to wait manylinux adds cp315. The alternative is to add a cpython build step or use one of the other pre-built images, but this seems like more work than other projects are investing at this stage.

Based on history of 3.13/3.14, this would be after b1 / 2026-05-05.

paultiq · 2026-06-10T22:06:50Z

@github-actions crossbow submit wheel-manylinux-2-28-cp315-cp315-amd64

github-actions · 2026-06-10T22:07:20Z

Only contributors can submit requests to this bot. Please ask someone from the community for help with getting the first commit in.
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/27309459895

paultiq · 2026-06-10T22:08:25Z

Retrying/restarting this, now that 3.15b2 is out and available on ManyLinux.

kou · 2026-06-11T01:11:10Z

@github-actions crossbow submit wheel-manylinux-2-28-cp315-cp315-amd64

github-actions · 2026-06-11T01:13:16Z

Revision: 6979e8c

Submitted crossbow builds: ursacomputing/crossbow @ actions-60ca234412

Task	Status
wheel-manylinux-2-28-cp315-cp315-amd64

raulcd · 2026-06-11T07:34:19Z

I forgot this issue existed. I took a stab last week as seen here (only local testing):

[Packaging][Python] Add support for Python 3.15 #50091 (comment)

I added rustc and cargo to the images but libcst wasn't able to be built for Py3.15 due to PyO3 failures (no 3.15 support). Removing temporarily building libcst and the stubs build I had a green pass for tests.

paultiq · 2026-06-11T14:57:28Z

Yup, I see same libcst blocker, cleared by this PR: Instagram/LibCST#1454 (status update/edit: it's now merged, waiting on release, a local test of 1.8.7.dev built successfully)

I got a good build against that PR:

# requirements-wheel-build.txt
libcst @ git+https://github.com/AlexWaygood/LibCST.git@add-315; python_version >= "3.15"

# pyproject.toml
    "libcst>=1.8.6; python_version < '3.15'",
    "libcst; python_version >= '3.15'",  # fork doesn't build with the version

Happy to close in favor of your PR, but otherwise will just wait for that libcst PR to land, then update this to build against libcst main for 3.15 testing

hugovk · 2026-08-10T06:30:16Z

The Python 3.15 release candidate is now out, and Instagram/LibCST#1454 has been merged and released in LibCST v1.9.0: https://github.com/Instagram/LibCST/releases/tag/v1.9.0.

And PyO3 should be ready for 3.15 as well now: https://github.com/PyO3/pyo3/releases/tag/v0.29.0.

paultiq · 2026-08-13T14:19:56Z

Could someone run this build? I bumped the libcst dependency, and it built locally. Thx: @github-actions crossbow submit wheel-manylinux-2-28-cp315-cp315-amd64

rok · 2026-08-13T16:02:05Z

@github-actions crossbow submit wheel-manylinux-2-28-cp315-cp315-amd64

github-actions · 2026-08-13T16:05:36Z

Revision: 9ed3dc3

Submitted crossbow builds: ursacomputing/crossbow @ actions-c6a3c83c52

Task	Status
wheel-manylinux-2-28-cp315-cp315-amd64

rok · 2026-08-13T19:21:03Z

@github-actions crossbow submit wheel-manylinux-2-28-cp315-cp315-amd64

github-actions · 2026-08-13T19:23:15Z

Revision: 5de3f4b

Submitted crossbow builds: ursacomputing/crossbow @ actions-951837cbe1

Task	Status
wheel-manylinux-2-28-cp315-cp315-amd64

rok · 2026-08-13T19:52:03Z

@paultiq

paultiq · 2026-08-13T23:05:25Z

Great!

This is ready for a larger crossbow run, to test other platforms + free threading:
@github-actions crossbow submit *cp315*

After that, it's ready for review. This is just to get 3.15 in the nightly builds.

raulcd · 2026-08-14T08:05:25Z

@github-actions crossbow submit cp315

github-actions · 2026-08-14T08:08:13Z

Revision: 5de3f4b

Submitted crossbow builds: ursacomputing/crossbow @ actions-bf280d6adc

Task	Status
wheel-macos-monterey-cp315-cp315-amd64
wheel-macos-monterey-cp315-cp315-arm64
wheel-macos-monterey-cp315-cp315t-amd64
wheel-macos-monterey-cp315-cp315t-arm64
wheel-manylinux-2-28-cp315-cp315-amd64
wheel-manylinux-2-28-cp315-cp315-arm64
wheel-manylinux-2-28-cp315-cp315t-amd64
wheel-manylinux-2-28-cp315-cp315t-arm64
wheel-musllinux-1-2-cp315-cp315-amd64
wheel-musllinux-1-2-cp315-cp315-arm64
wheel-musllinux-1-2-cp315-cp315t-amd64
wheel-musllinux-1-2-cp315-cp315t-arm64
wheel-windows-cp315-cp315-amd64
wheel-windows-cp315-cp315t-amd64

paultiq · 2026-08-14T12:39:06Z

The musllinux are because docker-library/python#1123 dropped ALPINE_LINUX 3.22 builds... so 3.22 is stuck on 3.15.0b2.

I successfully built/tested locally with 3.23.

Question

Should this be a separate PR to update Alpine Linux version, like GH-47052: [CI][C++] Use Alpine Linux 3.22 instead of 3.18 #47148 (3.18 to 3.22)?
Should we go to 3.24 (latest) or 3.23 (minimum increase needed)

Changes needed:

arrow/.env

Line 54 in e611f48

ALPINE_LINUX=3.22

arrow/dev/tasks/python-wheels/github.linux.yml

Line 35 in e611f48

ALPINE_LINUX: "3.22"

arrow/ci/docker/alpine-linux-3.22-cpp.dockerfile

Line 19 in e611f48

FROM --platform=linux/${arch} alpine:3.22

arrow/ci/docker/alpine-linux-3.22-r.dockerfile

Line 23 in e611f48

FROM --platform=linux/${arch} alpine:3.22

raulcd · 2026-08-14T12:44:09Z

On the attempt I did some weeks ago with the change on the following is enough, we don't require changing the default on .env + R (we can follow that up on a different issue):

arrow/dev/tasks/python-wheels/github.linux.yml

Line 35 in e611f48

ALPINE_LINUX: "3.22"

See:
https://github.com/apache/arrow/pull/50576/changes#diff-58624af974aef2e2980f03f99d2386e78e3fcb1a53d6a4b06612ded49ea8d449

paultiq · 2026-08-14T12:47:50Z

Ok, thx, I'll just commit that minimum. What about the .env change? I didn't trace that through.

raulcd · 2026-08-14T12:49:50Z

What about the .env change?

Not necessary, the .env is the default and with the change proposed it will already override the default.

paultiq · 2026-08-14T12:50:35Z

Thx, and should I keep working on this one / merge in your changes, or do you want to use that other draft?

raulcd · 2026-08-14T12:52:06Z

Thx, and should I keep working on this one / merge in your changes, or do you want to use that other draft?

I am happy to close mine and help you on this one through review, comments if that is ok with you and you want to keep working on it :)
Thanks so much for your work here!

paultiq · 2026-08-14T12:52:45Z

Ok, sounds good! I'll review/build the macos and windows versions next, and then request another rebuild.

raulcd · 2026-08-14T13:12:50Z

minor note for Windows in order to force the Windows wheels to be rebuilt you'll have to update the following:

arrow/.env

Lines 97 to 102 in e611f48

    
           # This must be updated when we update 
        
           # ci/docker/python-*-windows-*.dockerfile or the vcpkg config. 
        
           # This is a workaround for our CI problem that "archery docker build" doesn't 
        
           # use pulled built images in dev/tasks/python-wheels/github.windows.yml. 
        
           PYTHON_WHEEL_WINDOWS_IMAGE_REVISION=2026-07-24 
        
           PYTHON_WHEEL_WINDOWS_TEST_IMAGE_REVISION=2026-07-24

…(maintenance) bump 3.14.5 to 3.14.7

raulcd · 2026-08-14T14:43:30Z

@github-actions crossbow submit wheel-musllinux-1-2-cp315-cp315-amd64

github-actions · 2026-08-14T14:45:54Z

Revision: 521c6ec

Submitted crossbow builds: ursacomputing/crossbow @ actions-ba7d6f12de

Task	Status
wheel-musllinux-1-2-cp315-cp315-amd64

…est 3.14.7

…s. Other platforms don't use --only-binary so built around this issue.

paultiq · 2026-08-15T01:36:55Z

Could you rerun the full cp315 set?

musllinux passed above
OSX is building locally for cp315 and cp315t: added 3.15 to install_python.sh, and added workaround --only-binary=:all: for cp315 (the workaround is to allow builds on cp315 because of one not-yet-available dependency: pyyaml... other platforms don't use --only-binary)
Windows: I think I have it, but haven't been able to verify locally (local docker issues): 7e5f900

rok · 2026-08-15T12:04:53Z

@github-actions crossbow submit cp315

github-actions · 2026-08-15T12:07:33Z

Revision: 7bf40dc

Submitted crossbow builds: ursacomputing/crossbow @ actions-e1c080acb8

Task	Status
wheel-macos-monterey-cp315-cp315-amd64
wheel-macos-monterey-cp315-cp315-arm64
wheel-macos-monterey-cp315-cp315t-amd64
wheel-macos-monterey-cp315-cp315t-arm64
wheel-manylinux-2-28-cp315-cp315-amd64
wheel-manylinux-2-28-cp315-cp315-arm64
wheel-manylinux-2-28-cp315-cp315t-amd64
wheel-manylinux-2-28-cp315-cp315t-arm64
wheel-musllinux-1-2-cp315-cp315-amd64
wheel-musllinux-1-2-cp315-cp315-arm64
wheel-musllinux-1-2-cp315-cp315t-amd64
wheel-musllinux-1-2-cp315-cp315t-arm64
wheel-windows-cp315-cp315-amd64
wheel-windows-cp315-cp315t-amd64

paultiq · 2026-08-16T01:53:51Z

Ok, almost there... now just macos: problem was that the 3.15.0rc1 led to an attempt to download https://www.python.org/ftp/python/3.15.0rc1/python-3.15.0rc1-macos11.pkg when it should be https://www.python.org/ftp/python/3.15.0/python-3.15.0rc1-macos11.pkg (note the truncated version 3.15.0).

Fixed.

Could someone run: @github-actions crossbow submit wheel-macos-monterey-cp315-cp315-arm64

paultiq added 2 commits November 20, 2025 14:10

ci: add cp315/cp315t to builds.

266b12a

ci: add marker for cffi and pandas for < "3.15". this is temporary un…

2d2a557

…til they ship 3.15 wheels.

paultiq requested review from AlenkaF, assignUser, jonkeane, kou, raulcd and rok as code owners November 20, 2025 14:30

github-actions Bot added Component: Python awaiting review Awaiting review labels Nov 20, 2025

Merge remote-tracking branch 'upstream/main' into cp315_nightly

6979e8c

bump libcst to 1.9.0 for Python 3.15

9ed3dc3

paultiq added 2 commits August 13, 2026 12:37

Merge remote-tracking branch 'origin/main' into cp315_nightly

76695d4

chore: bump manylinux image to 08.08 to include 3.15-rc1

9214919

ALPINE_LINUX => 3.23: Python is no longer rebuilt on 3.22

7046bfa

raulcd mentioned this pull request Aug 14, 2026

GH-50091: [Python][Packaging] Add support for Python 3.15 #50576

Closed

Use a version gate, rather than import gate, for 3.15 pandas

6fb35d8

Add Python 3.15.0rc1: bump build_date to 20260807, add 3.15.0rc1 and …

521c6ec

…(maintenance) bump 3.14.5 to 3.14.7

paultiq added 5 commits August 14, 2026 10:58

bump image revision

5b13d85

add 3.15 for windows dockerfiles, and maintenance: move 3.14.5 to lat…

7e5f900

…est 3.14.7

Merge remote-tracking branch 'upstream/main' into cp315_nightly

892f7e5

add 3.15.0rc1 (and bump 3.14.6 to latest: 3.14.7)

bb049d5

disable --only-binary for 3.15 as a temp workaround for missing build…

7bf40dc

…s. Other platforms don't use --only-binary so built around this issue.

Truncate pre-release suffixes

1b9702d

Uh oh!

Conversation

paultiq commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

2026-08-13 edit

Are these changes tested?

Are there any user-facing changes?

Progress Notes (2026-08-15)

Uh oh!

github-actions Bot commented Nov 20, 2025

Uh oh!

raulcd commented Nov 21, 2025

Uh oh!

github-actions Bot commented Nov 21, 2025

Uh oh!

paultiq commented Nov 21, 2025

Build Failure

Uh oh!

paultiq commented Nov 21, 2025

Uh oh!

paultiq commented Jun 10, 2026

Uh oh!

github-actions Bot commented Jun 10, 2026

Uh oh!

paultiq commented Jun 10, 2026

Uh oh!

kou commented Jun 11, 2026

Uh oh!

github-actions Bot commented Jun 11, 2026

Uh oh!

raulcd commented Jun 11, 2026

Uh oh!

paultiq commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hugovk commented Aug 10, 2026

Uh oh!

paultiq commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rok commented Aug 13, 2026

Uh oh!

github-actions Bot commented Aug 13, 2026

Uh oh!

rok commented Aug 13, 2026

Uh oh!

github-actions Bot commented Aug 13, 2026

Uh oh!

rok commented Aug 13, 2026

Uh oh!

paultiq commented Aug 13, 2026

Uh oh!

raulcd commented Aug 14, 2026

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

paultiq commented Aug 14, 2026

Question

Changes needed:

Uh oh!

raulcd commented Aug 14, 2026

Uh oh!

paultiq commented Aug 14, 2026

Uh oh!

raulcd commented Aug 14, 2026

Uh oh!

paultiq commented Aug 14, 2026

Uh oh!

raulcd commented Aug 14, 2026

Uh oh!

paultiq commented Aug 14, 2026

Uh oh!

raulcd commented Aug 14, 2026

Uh oh!

raulcd commented Aug 14, 2026

Uh oh!

github-actions Bot commented Aug 14, 2026

Uh oh!

paultiq commented Nov 20, 2025 •

edited

Loading

paultiq commented Jun 11, 2026 •

edited

Loading

paultiq commented Aug 13, 2026 •

edited

Loading

paultiq commented Aug 15, 2026 •

edited

Loading