Skip to content

Fixed psutil package - #72

Merged
jakub-kocka merged 7 commits into
mainfrom
fix/psutil
Sep 11, 2026
Merged

jakub-kocka merged 7 commits into
mainfrom
fix/psutil

Conversation

@jakub-kocka

@jakub-kocka jakub-kocka commented Aug 31, 2026 •

Copy link
Copy Markdown
Collaborator

Description

  • Prefer official PyPI macOS wheels on CI: pip wheel --only-binary :all: (Intel cryptography still --no-binary for OpenSSL 4), skip delocate except that rebuild, and prune higher macosx_* siblings so a macos-15-* sdist cannot steal pip preference. This is the psutil 7.2.2 SIGABRT path (macosx_15_0 winning over PyPI macosx_10_9 / 11_0).

  • Run native_import_guard.yaml import probes on every test-wheels-install runner (not only ARMv7). Unlisted platform wheels get import <top_level>; *-none-any.whl is skipped. Custom imports remain for cffi / cryptography / psutil / etc.

  • Split the new policy out of _helper_functions.py into wheel_compat_policy.py and native_import_guard.py. Helper stays shared primitives (platform, parsing, exclude, PyPI preflight).

Related

  • Addresses the IDF CI failures with psutil (our build wheel present on the index)

Test pipeline run

Checklist

Before submitting a Pull Request, please ensure the following:

  • 🚨 This PR does not introduce breaking changes.
  • All CI checks (GH Actions) pass.
  • Documentation is updated as needed.
  • Tests are updated or added as necessary.
  • Code is well-commented, especially in complex areas.
  • Git history is clean — commits are squashed to the minimum necessary.

Run native_import_guard.yaml probes on every test-wheels-install runner
so SIGABRT and ABI mismatches fail CI before upload
Keep portable macosx tags on the extra-index
Local macos-15 builds were winning over official PyPI wheels (for example psutil 7.2.2 SIGABRT)
@jakub-kocka jakub-kocka self-assigned this Aug 31, 2026
@github-actions

github-actions Bot commented Aug 31, 2026 •

Copy link
Copy Markdown
Warnings
⚠️ Please consider squashing your 7 commits (simplifying branch history).
Messages
📖 This PR seems to be quite large (total lines of code: 2421), you might consider splitting it into smaller PRs

👋 Hello jakub-kocka, we appreciate your contribution to this project!


Click to see more instructions ...


This automated output is generated by the PR linter DangerJS, which checks if your Pull Request meets the project's requirements and helps you fix potential issues.

DangerJS is triggered with each push event to a Pull Request and modify the contents of this comment.

Please consider the following:
- Danger mainly focuses on the PR structure and formatting and can't understand the meaning behind your code or changes.
- Danger is not a substitute for human code reviews; it's still important to request a code review from your colleagues.
- Resolve all warnings (⚠️ ) before requesting a review from human reviewers - they will appreciate it.
- Addressing info messages (📖) is strongly recommended; they're less critical but valuable.
- To manually retry these Danger checks, please navigate to the Actions tab and re-run last Danger workflow.

Review and merge process you can expect ...


We do welcome contributions in the form of bug reports, feature requests and pull requests.

1. An internal issue has been created for the PR, we assign it to the relevant engineer.
2. They review the PR and either approve it or ask you for changes or clarifications.
3. Once the GitHub PR is approved we do the final review, collect approvals from core owners and make sure all the automated tests are passing.
- At this point we may do some adjustments to the proposed change, or extend it by adding tests or documentation.
4. If the change is approved and passes the tests it is merged into the default branch.

Generated by 🚫 dangerJS against 8494667

@jakub-kocka
jakub-kocka marked this pull request as draft August 31, 2026 13:27
@jakub-kocka
jakub-kocka requested a lite review from Copilot August 31, 2026 13:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts wheel-build and wheel-repair policy to avoid CI-produced wheels “stealing” pip’s preference over official PyPI wheels (notably on macOS for psutil), and expands post-install native import probing to run consistently across platforms via a more flexible YAML-driven configuration.

Changes:

  • Add wheel_compat_policy.py to centralize macOS/manylinux tag pruning and PyPI wheel mirroring policy, and wire it into build/repair flows.
  • Introduce native_import_guard.py + updated native_import_guard.yaml to support platform/python/version filters, default probing for unlisted platform wheels, and skipping *-none-any.whl.
  • Update tests and tooling (test_*, scripts/run_native_import_probe.py, docs) to use the new policy/modules.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
wheel_compat_policy.py New module implementing macOS --only-binary policy, PyPI mirroring, and wheel pruning logic.
native_import_guard.py New loader/resolver for YAML-driven native import probes (filters + defaults).
native_import_guard.yaml Expanded configuration format (top-level options + per-package filters + psutil probe).
test_wheels_install.py Switch native import probing to YAML resolver with default probing for unlisted wheels.
test_native_import_guard.py New unit tests for YAML loading and statement resolution behavior.
test_build_wheels.py Rewire tests to new policy module and add coverage for new behaviors.
build_wheels.py Use wheel_compat_policy for pip wheel invocation and mirror success logic.
build_wheels_from_file.py Use wheel_compat_policy for pip wheel invocation and mirror success logic.
repair_wheels.py Apply macOS pruning + delocate-skip policy; keep manylinux pruning behavior via new module.
scripts/run_native_import_probe.py Use native_import_guard loader instead of in-script YAML parsing.
_helper_functions.py Remove migrated policy/guard logic and add psutil to relevant skip lists.
README.md Document new native import guard semantics and macOS wheel-preference mitigation.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread wheel_compat_policy.py Outdated
Comment thread wheel_compat_policy.py
Comment thread native_import_guard.py Outdated
@jakub-kocka
jakub-kocka force-pushed the fix/psutil branch 3 times, most recently from 5abb1b4 to 8e0be3a Compare September 1, 2026 11:04
@jakub-kocka
jakub-kocka marked this pull request as ready for review September 1, 2026 11:05
@jakub-kocka
jakub-kocka requested a lite review from Copilot September 1, 2026 11:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

test_build_wheels.py:445

  • Test name test_only_binary_args_off_linux is misleading: it patches platform.system() to "Linux" and verifies behavior off macOS. Renaming helps clarify intent.

This issue also appears on line 469 of the same file.

    @patch("wheel_compat_policy.platform.system", return_value="Linux")
    def test_only_binary_args_off_linux(self, _sys: object) -> None:
        self.assertEqual(get_macos_only_binary_args(), [])

test_build_wheels.py:471

  • Test name test_skip_delocate_off_linux is misleading: it patches platform.system() to "Linux" and verifies behavior off macOS. Renaming improves readability.
    @patch("wheel_compat_policy.platform.system", return_value="Linux")
    def test_skip_delocate_off_linux(self, _sys: object) -> None:
        self.assertFalse(should_skip_macos_delocate_for_pypi_mirror("psutil-7.2.2-cp36-abi3-macosx_10_9_x86_64.whl"))

Comment thread scripts/run_native_import_probe.py

@dobairoland dobairoland left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fhrbata fhrbata left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW LGTM :) Thank you

@jakub-kocka
jakub-kocka force-pushed the fix/psutil branch 3 times, most recently from fb36ac9 to d4ba42c Compare September 2, 2026 12:36
@jakub-kocka
jakub-kocka force-pushed the fix/psutil branch 2 times, most recently from 909fc5a to 4e88805 Compare September 9, 2026 09:17
jakub-kocka and others added 2 commits September 10, 2026 14:58
Sourcing linux_armv7_docker_prepare.sh turned errexit off in the workflow
shell: bash clears errexit inside the command substitution of an assignment,
so the "$(set +o)" snapshot restored "set +o errexit". Failures in
linux_arm.sh then looked like success.

That hid an apt failure on the bullseye armhf images, where python3-dev has
no installable version (they carry libpython3.9-stdlib from
bullseye-security, which is gone from archive.debian.org, while main only
offers the matching -dev at the original point release). Setup aborted
before installing Rust and the dbus dev packages, so bcrypt and dbus-python
failed to build half an hour later.

Wheels build against the interpreter on PATH, so the distro headers are now
best-effort. macos_openssl4_intel.sh had the same errexit snapshot bug.

Co-authored-by: Cursor <cursoragent@cursor.com>
@jakub-kocka

Copy link
Copy Markdown
Collaborator Author

Thank you, @dobairoland, @fhrbata, for the review. I have added a few more commits since then, but the core logic is untouched.

It was a few minor patches to pass the build pipeline and the "resurrection" of EOL Debian 11 builds (ARMv7 Legacy)

So since the build pipeline has passed (https://github.com/espressif/idf-python-wheels/actions/runs/34575708821) and the wheels are uploaded to our index, I am merging this.

I believe the next step is to unlock the psutil package in the Constraint files and test it with the IDF pipeline - this MR will follow up.

@jakub-kocka
jakub-kocka merged commit 2902844 into main Sep 11, 2026
226 of 227 checks passed
@jakub-kocka
jakub-kocka deleted the fix/psutil branch September 11, 2026 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants