Isolate compression and decompression at src/fz.cu

[brunoortega]

Hello guys, I'm a HPC masters student currently working with FWI, GPU compression and wavelet transform.

I've seen your paper and chose to use FZ-GPU as one of the compressors in my research, I'd like to integrate it inside the FWI code and there are two moments when I'll need to call the tool (compress on forward propagation and decompress at adjoint / backward propagation).

With intent to use FZ-GPU as a submodule and to improve reproducibility, can I send a PR to add "fzgpu_compress" and a "fzgpu_decompress" functions at src/fz.cu?

Best regards,

Bruno.

[brunoortega]

Hi, new update!

I did isolate into init, compress decompress, verify and free functions. The current problem is that as far as i know the variables deviceStartPosition and deviceBitFlagArr are modified by compressionFusedKernel and are needed for decompressionFusedKernel, so if we work with programs that need to compress and decompress in different orders, we'll need to store these two variables.

For example, this pipeline will crash:

compress(A)
compress(B)
compress(C)
decompress(C)
decompress(B)
decompress(A)

But if we save the two variables along with the compressed data, it's possible (and the compressed data size will increase a little bit), so:

compress(A) // save A_compressed, A_deviceStartPosition and A_deviceBitFlagArr
compress(B) // save B_compressed, B_deviceStartPosition and B_deviceBitFlagArr
compress(C) // save C_compressed, C_deviceStartPosition and C_deviceBitFlagArr
decompress(C) // restore C_compressed, C_deviceStartPosition and C_deviceBitFlagArr
decompress(B) // restore B_compressed, B_deviceStartPosition and B_deviceBitFlagArr
decompress(A) // restore A_compressed, A_deviceStartPosition and A_deviceBitFlagArr

I'll send a PR just so you can see how I did it, currently it's just a prototype so my apologies for the mess / bad code, didn't clean and organize it yet :D

In the txt file there are the results that I've got here with the datasets from the script (fz_isolated is the adapted code). I'm using a laptop with NVIDIA RTX 4050 + Intel(R) Core(TM) 7 240H.

tests.txt

[dingwentao]

Hi @brunoortega ,

Thanks for your attention to FZ-GPU compression and your willingness to integrate it into your FWI application. I would appreciate it if you could further modify the README file to make future users aware of this improvement (isolating the compression and decompression functions), which makes the code more flexible. I believe the crash was due to memory management.

We have several questions regarding your application and why you chose FZ-GPU, and we would appreciate it if you could help us understand the suitable use cases for FZ-GPU:

1. Why did you choose FZ-GPU over other SZ-family GPU compressors (I saw you're also posting PR to cuSZp)?
2. Do you have any specific requirements for compression ratio, speed, and quality?
3. Are you familiar with wavelet-based compressors such as SPERR? It doesn't have GPU compression yet—would you be interested if we provided you with one (which we are currently developing)?

Looking forward to your response and any potential collaboration.

Best,
Dingwen

[brunoortega]

Hi @dingwentao, thanks for your attention and reply!

I'll dedicate some time to organize the code and change some things, and then modify the README. I'm thinking that those specific parameters (deviceStartPosition and deviceBitFlagArr) can be saved with the compressed data as metadata, assuming that their combined size is something near 0.4% or 0.5% of the original data size.

Before I answer your questions specifically, I need to say that my advisor has a Full-Waveform Inversion library (Simwave) and my research is to integrate GPU compression in the program's pipeline. It already has a checkpointing system and I figured that just writing "compress" before checkpoint saving and "decompress" at restore would be quite simple.
So I've put some effort to look for what could I do to make it better and I found that Unicamp (Brazil) has a very interesting prefetching system called GPUZIP (https://doi.org/10.1177/10943420251340794) and they also have an compression API to select different compressors and parameters in a simpler way, so I started to search for compressors and techniques to improve compressibility (such as wavelet transforms).
With help from a teacher from LNCC (Brazil), I've learned a little bit about wavelet transforms, ROM and POD/DMD and chose to implement a wavelet transform (CDF 9/7) + compression pipeline. I'm looking for paralellizing WaveRange's (https://doi.org/10.1007/s12650-021-00813-8) wavelet transform with CUDA and / or OpenMP.

Also, my Master's research thesis title is In-Situ Data Compression for GPU-Accelerated Full Waveform Inversion, so it narrows my compressor's options to only GPU-based ones.

1- I found FZ-GPU bitshuffle kernel to be very interesting, it has a (quite) constant throughput for different error bounds and It has better compression ratios compared to cuSZp3 (or VGC). Maybe I should also try cuSZ-i? Anyway, I've selected the following compressors to test in the FWI pipeline:

cuSZp3 (or VGC)
FZ-GPU
cuZFP
nvCOMP Bitcomp
MGARD-X

Those are great GPU compressors and maybe I'll be missing some other ones, but the priority is not to test every compressor (at least, not in my master's).
The PR to cuSZp was to help to integrate it as a git submodule, so it can inherit the CUDA architecture from the main CMakeLists.txt and don't get trouble compiling for a lot of different unnecessary architectures.

2- I'm about to run complete tests using Simwave with lossy compression (currently cuSZp3, Bitcomp and cuZFP are working fine on it), and the most important metrics are how faster FWI converges, how the lossy compression influences on the error from objective function, just like in Navjot Kukreja's: https://doi.org/10.5194/gmd-15-3815-2022. So I don't know for sure but I'll investigate if PSNR, MSE and other metrics can relate to its convergence speed. But my guess is that those lossy compressors throughput work fine and compression ratio and quality would have a bigger impact on FWI performance.

3- I'm not hands-on familiar but I've read a lot of papers and SPERR would be an amazing option if it had full GPU integration! I would love to work on it and collaborate as much as possible!

Again, thanks for replying and for the opportunity!

Att.,

Bruno.

[brunoortega]

By the way, I'd also like to ask for your advice.

Since I'm still at the beginning of my research career, I'm trying to build a solid background in these topics (GPU computing and data compression). Are there any books, papers, courses, or topics that you would recommend studying to gain a deeper understanding of the field?

I'd really appreciate any recommendations. Thank you!