Skip to content

Make sure gpu4pyscf allocator is never used - #99

Open
JensWehner wants to merge 6 commits into
mainfrom
use_own_use_torch_allocator
Open

Make sure gpu4pyscf allocator is never used#99
JensWehner wants to merge 6 commits into
mainfrom
use_own_use_torch_allocator

Conversation

@JensWehner

@JensWehner JensWehner commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

There was a theoretical way the gpu4pyscf allocator could escape by someone directly importing backend.

This does not work now, as backend sets the torch allocator and as a skala package it is imported after the gpu4pyscf everywhere, so only the torch allocator will be used. Tiny fix on #98

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to ensure CuPy uses the PyTorch caching allocator (not GPU4PySCF’s allocator) by installing the allocator configuration in the shared PySCF backend import path, preventing the GPU4PySCF allocator from “escaping” via direct backend imports.

Changes:

  • Install use_torch_mempool_in_cupy() during skala.pyscf.backend GPU backend initialization (after importing gpu4pyscf).
  • Remove allocator installation side effects from skala.gpu4pyscf.__init__.
  • Update the GPU4PySCF gradient test to import the allocator utilities from skala.utils, and refine the allocator module docstring.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
tests/test_gpu4pyscf_gradients.py Switches allocator import path to skala.utils to validate allocator state.
src/skala/utils/torch_allocator.py Docstring clarification for the CuPy↔PyTorch allocator adapter.
src/skala/pyscf/backend.py Adds allocator installation so the backend import path consistently overrides GPU4PySCF’s allocator.
src/skala/gpu4pyscf/init.py Removes allocator installation side effects from GPU4PySCF integration module init.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/skala/pyscf/backend.py Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
awvwgk
awvwgk previously approved these changes Jul 30, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/skala/pyscf/backend.py:56

  • The ImportError raised on allocator-configuration failure loses the underlying RuntimeError message, which makes it harder to diagnose stream-mismatch or CuPy allocator issues from the surfaced exception text. Consider including the original error message in the ImportError string (you already preserve the traceback via from e).
        except RuntimeError as e:
            raise ImportError(
                "Failed to configure CuPy to use the PyTorch allocator."
            ) from e

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

src/skala/conftest.py:24

  • The module-level docstring explains that doctest collection is skipped for skala.gpu4pyscf on CPU-only hosts, but this file now also skips utils/torch_allocator.py. Please update the docstring to reflect that skala.utils.torch_allocator is also excluded (it imports CuPy / CUDA-dependent code and can fail collection similarly).
if not torch.cuda.is_available():
    collect_ignore_glob.append("gpu4pyscf/*.py")
    collect_ignore_glob.append("utils/torch_allocator.py")

Comment thread src/skala/conftest.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants