[Storehouse] 013 - Add util to verify checkpoint file

[zhangchiqing]

• Add a util checkpoint-verify-hash to verify the hash value of each node in the checkpoint file.
• Compare to the previous implementation MTrie.AValidTrie() which also verify the hash value of each node, this util had the following advantages:

1. Much less memory usage. The previous implementation has to load the entire checkpoint into memory before starting verification work. But this util doesn't load the checkpoint, instead, it iterates over each node in DFS manner, since the checkpoint file is saved in DFS manner, it guarantees the iteration callback can be called with the node and their children. The memory usage is O(n), n as the max depth of the trie.
2. Concurrency. The previous implementation is single threaded. This util has --n-worker to process verify n subtrie concurrently.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 36f8713c-591f-457d-ba25-4d0398877aa1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch leo/payloadless-util-checkpoint-verify-hash

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

[codecov-commenter]

Codecov Report

❌ Patch coverage is 58.18815% with 120 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
ledger/complete/wal/checkpoint_verifier.go	62.54%	57 Missing and 43 partials ⚠️
cmd/util/cmd/checkpoint-verify-hash/cmd.go	0.00%	19 Missing ⚠️
cmd/util/cmd/root.go	0.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!