Query Windows physical core count using ctypes and native Win32 API

[nrusch]

Overview

During the course of synchronizing some internal patches with the 3.4.0 release, I found that rez invocations were noticeably slower than they are using our current 3.3.0-based version. Some basic investigation showed that this is almost entirely the result of the new PowerShell-based CPU count query added in #2108.

In short, this method is extremely slow compared to the previous WMIC-based solution.

This PR introduces a new default code path for querying the physical core count that uses ctypes and the Win32 API to query the system directly.

Performance comparison

To verify the difference, I extracted the current PowerShell- and WMIC-based methods for querying CPU count into a standalone file and ran them through the timeit module.

Here's what I get, running in Python 3.11 on Windows 11 Pro 25H2 (I still have WMIC installed):

PowerShell

1 loop, best of 5: 1.34 sec per loop

Extremely slow, and it performs even worse in our work environment (averaging 2.5+ seconds per call).

WMIC

5 loops, best of 5: 53 msec per loop

An easy order of magnitude faster, even though it still involves a process invocation.

This PR (native Win32 wrapped in ctypes)

10000 loops, best of 5: 33.6 usec per loop

Let me know what you think.

[codecov]

Codecov Report

❌ Patch coverage is 69.44444% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.31%. Comparing base (73a094d) to head (749f73d).

Files with missing lines	Patch %	Lines
src/rez/utils/platform_.py	69.44%	6 Missing and 5 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2187      +/-   ##
==========================================
+ Coverage   61.30%   61.31%   +0.01%     
==========================================
  Files         164      164              
  Lines       20572    20607      +35     
  Branches     3575     3581       +6     
==========================================
+ Hits        12611    12635      +24     
- Misses       7089     7095       +6     
- Partials      872      877       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[maxnbk]

Very cool, thanks for this solution @nrusch . Might be tough to review since I doubt many of us have CTypes API experience, but if we can't figure out any problems, I think we will probably have to / want to merge this either way.

One thing I am wondering, or was wondering after I merged the previous powershell workaround for the deprecated WMIC methods is:

1. Is there any reason to, or not to, memoize the calls? It's hard for me to imagine situations where the underlying system could somehow change available core counts during a running rez process, other than cgroup-managed kubernetes-style pod/cluster setups, and even that, I don't know if that works that way precisely. (Though I also am not totally sure if there are any routes in rez that involve these calls being made more than once?)
2. The other windows-based (and I think all the others too) do some form of try/except. With your new method, if it threw an exception anywhere in that function, the other fallback options wouldn't even fire because it would raise an exception and break, defaulting back to undetected -> 1 core, unless I'm misreading something?

[nrusch]

Hey @maxnbk , thanks for taking a look. Regarding your questions:

1. Is there any reason to, or not to, memoize the calls?

I had a similar thought as well, and I can't imagine why one wouldn't want to do this. Even if there are some ways to change the apparent core count of a system while processes are running (which seems extremely niche), I'm hard-pressed to imagine why rez would care...

I'm happy to add this if you think it's a reasonable step.

2. The other windows-based (and I think all the others too) do some form of try/except. With your new method, if it threw an exception anywhere in that function, the other fallback options wouldn't even fire because it would raise an exception and break, defaulting back to undetected -> 1 core, unless I'm misreading something?

You are not misreading. Win32 is a C API, so it cannot raise exceptions, and thus there are no exception semantics to propagate to Python.

However, I don't disagree with your overall instinct for caution here... I suspect it's possible there could be some temporary/edge-case conditions under which ctypes itself could raise an exception (separate from any of the Win32 interaction), so I'm happy to add in exception handling.

On a tangentially-related note, given that the method is already fairly long, I was thinking about establishing a new module for Windows-specific utilities, and then that method could import and call the function from there. I've also thought of adding ctypes implementations for some other system interaction patterns in rez, and that would likely end up being a natural home for any Windows-specific ones.

Any thoughts on that idea, or what the most logical module location/identity might be?

[Copilot]

Pull request overview

Adds fast native Windows physical-core detection while retaining existing fallbacks.

Changes:

• Queries Win32 processor topology through ctypes.
• Prioritizes native detection before PowerShell and WMIC.
• Expands Windows core-count tests.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
src/rez/utils/platform_.py	Implements native core detection and fallback ordering.
src/rez/tests/test_utils.py	Tests fallback selection and helper consistency.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[nrusch]

@maxnbk just to follow up on this particular question:

Is there any reason to, or not to, memoize the calls?

It's worth noting that the value is ultimately cached at a "higher" level by the Platform.physical_cores accessor.

[JeanChristopheMorinPerso]

LGTM, thanks for contributing this change!