CMake: skip unnecessary re-hashing of coeffs tarball - #354
Conversation
|
@BenjaminTJohnson I just saw you commented on the issue (#353 (comment)) just as I opened this PR. I'm happy to reorganize a bit so we have a single |
|
@BenjaminTJohnson Let me know if this update to the PR matches what you had in mind! |
srherbener
left a comment
srherbener
left a comment
There was a problem hiding this comment.
Thanks for fixing this! It will be nice to get the cmake runtime reduction.
I tested this on my MacBook (OrbStack) and it seems to work just fine, and it definitely speeds up the ecbuild/cmake step on runs after the initial run when starting with a blank slate (ie the run that does need to download and unpack the CRTM tarball.
Before this change the re-runs of ecbuild/cmake typically took ~85s and now they are taking ~40s. Roughly 2x speed up which is a fantastic result!
|
I just experienced the result of this for the first time. Bliss! Thank you 🎉 |
|
(but when will I make coffee?) |
Brings in v3.1.5 (PR #364: netCDF SpcCoeff sibling-load fix + test_NLTE_Verification) and the CMake download/untar refactors (PR #354, #356). Conflict resolution: - CMakeLists.txt, VERSION.cmake, README.md: keep the 3.2.0 version strings; add the v3.1.5 line to the README release history. - SpcCoeff_netCDF_IO.f90: keep this branch's version. develop's fix is a subset of it (same sibling load, minus the "no sibling found" INFORMATION messages). - test/CMakeLists.txt: keep the fix_REL-3.2.0.0 tarball/checksum on top of develop's combined download+untar block; keep the netCDF-only coefficient staging list and add the iasi616_metop-b SpcCoeff/NLTECoeff/TauCoeff set the new test needs; port the testinput_no_siblings staging block to the netCDF filenames since the 3.2.0 tarball ships no .bin coefficients.
4 participants
Description
This PR adds logic so that the coeff tarball is only hashed and verified when the tarball will be untarred.
In more detail, the coeffs tarball is only untarred when
${CRTM_COEFFS_PATH}/${CRTM_COEFFS_BRANCH}does not exist. Verifying the tarball when the directory does exist is wasting time (and ~30+ seconds of it). So, this PR adds the same directory as guard for the acquire/verification step.Issue(s) addressed
Resolves #353
Dependencies
Impact
Checklist