Pr501 nufft final#571
Open
aaadelmann wants to merge 3 commits into
Open
Conversation
Port the NUFFT portion of the PR501 split onto current master while keeping master's FFT, CMake, Kokkos, CUDA architecture, and test infrastructure as the authoritative baseline. The new transform support adds: - NUFFTransform as an IPPL FFT transform type - native Type-1 and Type-2 NUFFT implementation headers - exponential-of-semicircle kernel support - NUFFT correction and utility helpers - aggregate inclusion through FFT/Transform/Transform.h Add NUFFT unit coverage: - NUFFT basic Type-1 and Type-2 correctness tests - native no-upsampling and upsampling coverage - tolerance sweep checks against direct DFT reference values - optional FINUFFT comparison paths when ENABLE_FINUFFT is available Adapt the imported tests to the current master particle API: - use ParticleSpatialLayout<T, Dim, mesh_type, ExecSpace> - make the test Bunch attribute storage execution-space aware - preserve current master particle layout semantics rather than restoring the older PR501 layout shortcuts Add optional FINUFFT/CUFINUFFT CMake plumbing: - IPPL_ENABLE_FINUFFT defaults OFF - IPPL_ENABLE_CUFINUFFT defaults OFF and requires CUDA plus FINUFFT - IPPL_FINUFFT_USE_DUCC0 defaults ON for fetched FINUFFT builds - fetched FINUFFT is pinned to v2.5.0 unless FINUFFT_VERSION is provided - fetched FINUFFT builds static and disables its examples/tests - IPPL links FINUFFT targets only when FINUFFT support is explicitly enabled - unit tests can receive additional link libraries through add_ippl_test() Keep this PR finite: - native NUFFT is built and tested without requiring FINUFFT - FINUFFT support is optional dependency plumbing only - cuFINUFFT is guarded but not exercised locally - no PIF integration is included here Validation performed on macOS/OpenMP: - configured Release OpenMP build with IPPL_ENABLE_FFT=ON and IPPL_ENABLE_FINUFFT=OFF - built all targets successfully - ran FFT/NUFFT subset: FFT, NUFFT, NUFFTAccuracy, TestFFTCC, TestFFTRC all passed - ran the full OpenMP test suite before the final CMake-only cleanup: 72/72 tests passed - git diff --check passed Notes: - FINUFFT-enabled configuration was tested with fetched FINUFFT v2.5.0. Configure succeeds with DUCC0. Fetched FINUFFT is configured with FINUFFT_USE_DUCC0=ON by default through IPPL_FINUFFT_USE_DUCC0. DUCC0 is FINUFFT's bundled FFT backend alternative to FFTW. This keeps IPPL's optional FINUFFT path self-contained and avoids relying on a system FFTW installation whose CMake package may expose incomplete or incompatible imported targets. The option remains user-configurable: -DIPPL_FINUFFT_USE_DUCC0=OFF With DUCC0 disabled, FINUFFT falls back to its FFTW path. Users can then provide a working external FFTW setup or ask FINUFFT to download FFTW via its own FINUFFT_FFTW_LIBRARIES=DOWNLOAD option.
Keep the NUFFT/cuFINUFFT implementation aligned with the original PR501 staging model while making the backend semantics explicit. The original PR501 NUFFT implementation is preserved: - FINUFFT/cuFINUFFT dispatch still goes through FinufftTraits<T> - cuFINUFFT still maps to cuFloatComplex/cuDoubleComplex and cufinufft plans - field, particle position, and particle charge staging still uses the PR501 Kokkos temporary-buffer model - CopyFieldToTempFunctor, CopyParticlesToTempFunctor, CopyFieldFromTempFunctor, and CopyParticlesFromTempFunctor remain the data movement mechanism - setpts/execute/destroy continue to use the original trait wrappers Add explicit cuFINUFFT scope guards: - reject cuFINUFFT execution when Comm->size() != 1 - report clearly that cuFINUFFT has no MPI decomposition - reject cuFINUFFT execution for non-CUDA-backed IPPL/Kokkos memory spaces - document that host-only SERIAL fields are not supported because this PR does not add a host-to-CUDA staging backend Tighten CMake behavior around cuFINUFFT: - require CUDA in IPPL_PLATFORMS when IPPL_ENABLE_CUFINUFFT=ON - make the error explain that cuFINUFFT is a single-rank GPU backend, not a serial-host backend - propagate CUDA architectures through master's Kokkos-based helper before enabling fetched cuFINUFFT - preserve the PR501 rationale for forcing fetched FINUFFT/cuFINUFFT static: cuFINUFFT CUDA RDC fatbin registration can fail when device code is wrapped in a shared library Update NUFFT tests for the clarified backend scope: - skip FINUFFT/cuFINUFFT backend tests for non-3D instantiations, since the imported backend path is currently 3D-only - skip cuFINUFFT backend tests when running with more than one MPI rank - keep the tests adapted to current master's ParticleSpatialLayout API This keeps the PR501 technical implementation intact while making the intended contract explicit: native IPPL NUFFT is the distributed path; cuFINUFFT is only a single-rank CUDA-backed GPU backend. Validation: - rebuilt local OpenMP / IPPL_ENABLE_FINUFFT=OFF tree successfully - ctest -R 'NUFFT|FFT' passed: FFT, NUFFT, NUFFTAccuracy, TestFFTCC, TestFFTRC - git diff --check passed CUDA/cuFINUFFT execution was not run locally because this machine is not a CUDA build environment.
Introduce test/FFT/TestGaussianNUFFT.cpp as an integration-style NUFFT
driver for correctness-aware scaling runs.
The new executable exercises FFT<NUFFTransform> directly on a 3D periodic
Gaussian particle/source setup. It supports command-line control over the
mode-grid size, global particle count, iteration count, transform type,
backend, and native spread/gather method:
TestGaussianNUFFT <nx> <ny> <nz> <num_particles> <iterations>
<type> <backend> [method]
The test includes an extended Doxygen description of the mathematical
workload. Type-1 validates selected low-frequency Fourier modes against
direct particle sums, while Type-2 uses a sparse analytic Fourier series so
particle values can be checked without a full O(M N^3) direct transform.
This gives a meaningful relative-error check while keeping validation outside
the timed region.
The driver records separate timers for total runtime, initialization, warmup,
and measured transform iterations. It also preserves the existing internal
NativeNUFFT timers, making it useful for rank/node scaling and backend
comparisons.
Register a small default CTest case:
TestGaussianNUFFT 16 16 16 1024 2 1 native tiled
The registered test is intentionally small enough for normal integration test
runs, while larger scaling cases can be launched manually with the same
executable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR ports the NUFFT part of the original PR501 work onto current
master.The PR adds:
FFT<NUFFTransform, ...>The native IPPL NUFFT path is the distributed-capable path. cuFINUFFT is only exposed for single-rank CUDA-backed execution because cuFINUFFT itself has no MPI decomposition.
What Is Included
Native NUFFT
Adds the native IPPL NUFFT implementation:
src/FFT/NUFFT/Correction.hsrc/FFT/NUFFT/ESKernel.hsrc/FFT/NUFFT/NUFFTUtilities.hsrc/FFT/NUFFT/NativeNUFFT.hsrc/FFT/Transform/NUFFT.hsrc/FFT/Transform/NUFFT.hppThe implementation supports Type 1 and Type 2 NUFFT through the existing IPPL FFT interface:
Type 1:
fft->transform(R, Q, field);spreads particle values
Qat particle positionsRonto the Fourier-mode field.Type 2 uses the same entry point and interpolates from the field back to particle values depending on the transform type selected at construction.
FINUFFT / cuFINUFFT Plumbing
Adds CMake options:
IPPL_ENABLE_FINUFFTenables the optional external FINUFFT backend.IPPL_ENABLE_CUFINUFFTenables the optional CUDA cuFINUFFT backend and requires CUDA inIPPL_PLATFORMS.IPPL_FINUFFT_USE_DUCC0defaults toONfor fetched FINUFFT builds. DUCC0 is FINUFFT's bundled FFT backend alternative to FFTW. This keeps fetched FINUFFT builds self-contained and avoids relying on a possibly inconsistent system FFTW CMake package.cuFINUFFT Scope
cuFINUFFT is explicitly treated as:
The code now rejects unsupported cuFINUFFT use cases clearly:
Comm->size() != 1SERIALfieldsThis PR does not add a host-to-CUDA staging backend. Therefore, cuFINUFFT is not intended for a host-only serial IPPL build.
The intended message is:
Tests
Added:
unit_tests/FFT/NUFFT.cppunit_tests/FFT/NUFFTAccuracy.cppunit_tests/FFT/NUFFTTestUtils.hCoverage includes:
Validation
Validated locally on macOS with OpenMP and FINUFFT disabled:
cmake --build build-pr501-nufft-final-openmp -j 8 ctest --test-dir build-pr501-nufft-final-openmp -R 'NUFFT|FFT' --output-on-failureResult:
Also checked:
Result: clean.
A previous full local OpenMP run with FINUFFT disabled passed all tests:
Notes
The core NUFFT implementation is intentionally kept close to the original PR501 NUFFT implementation.
Intentional deviations from PR501 are limited to:
masterparticle layout APImasterCMake/CUDA architecture handling authoritativeOut Of Scope
This PR does not include: