Make sure gpu4pyscf allocator is never used - #99
Conversation
There was a problem hiding this comment.
Pull request overview
This PR aims to ensure CuPy uses the PyTorch caching allocator (not GPU4PySCF’s allocator) by installing the allocator configuration in the shared PySCF backend import path, preventing the GPU4PySCF allocator from “escaping” via direct backend imports.
Changes:
- Install
use_torch_mempool_in_cupy()duringskala.pyscf.backendGPU backend initialization (after importinggpu4pyscf). - Remove allocator installation side effects from
skala.gpu4pyscf.__init__. - Update the GPU4PySCF gradient test to import the allocator utilities from
skala.utils, and refine the allocator module docstring.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/test_gpu4pyscf_gradients.py | Switches allocator import path to skala.utils to validate allocator state. |
| src/skala/utils/torch_allocator.py | Docstring clarification for the CuPy↔PyTorch allocator adapter. |
| src/skala/pyscf/backend.py | Adds allocator installation so the backend import path consistently overrides GPU4PySCF’s allocator. |
| src/skala/gpu4pyscf/init.py | Removes allocator installation side effects from GPU4PySCF integration module init. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
src/skala/pyscf/backend.py:56
- The ImportError raised on allocator-configuration failure loses the underlying RuntimeError message, which makes it harder to diagnose stream-mismatch or CuPy allocator issues from the surfaced exception text. Consider including the original error message in the ImportError string (you already preserve the traceback via
from e).
except RuntimeError as e:
raise ImportError(
"Failed to configure CuPy to use the PyTorch allocator."
) from e
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
src/skala/conftest.py:24
- The module-level docstring explains that doctest collection is skipped for
skala.gpu4pyscfon CPU-only hosts, but this file now also skipsutils/torch_allocator.py. Please update the docstring to reflect thatskala.utils.torch_allocatoris also excluded (it imports CuPy / CUDA-dependent code and can fail collection similarly).
if not torch.cuda.is_available():
collect_ignore_glob.append("gpu4pyscf/*.py")
collect_ignore_glob.append("utils/torch_allocator.py")
There was a theoretical way the gpu4pyscf allocator could escape by someone directly importing backend.
This does not work now, as backend sets the torch allocator and as a skala package it is imported after the gpu4pyscf everywhere, so only the torch allocator will be used. Tiny fix on #98