Accelerate scverse/scanpy steps with rapids-singlecell

[Zethson]

Description of feature

Heyy,

the pipeline currently uses scanpy for lots of preprocessing steps including the generation of embeddings & clustering.

I suggest that we integrate github.com/scverse/rapids-singlecell to optionally GPU accelerate these steps as soon as I've implemented a nf-core module for it.

Any thoughts, concerns, or helpful pointers?

CC @gburnett-nvidia @DCGenomics

[fasterius]

Interesting! I haven't used rapids-singlecell myself, but reading the documentation it looks worth investigating. How do you envision implementing it in the modules? A requirement from our side would be that the user has control over whether to run on CPU or GPU through e.g. a value input of each module (which could be globally set at pipeline execution), so there would need to be some sort of check whether to use sc or rsc throughout each module. Maybe you could provide an example for one of the modules?

[Zethson]

Still TBD how I'll eventually implement it (I'm currently trying to figure out the best ways) but here's the complex answer:

1. In the beginning, I'd try to potentially be inspired by nf-core/scdownstream which exposes GPU related parameters to run scvi-tools algorithms on either the CPU or GPU. Here, we'd have the additional complexity that we'd need to select the right module. I'd do that based on the corresponding parameter (like say --gpu) values.
2. In the future, scanpy will delegate to the GPU automatically via RSC if available and desired. This is a long term project but actively tackled by @Intron7