[Storehouse] 010 Util convert v6 -> v7 checkpoint by zhangchiqing · Pull Request #8583 · onflow/flow-go

zhangchiqing · 2026-06-18T02:53:32Z

To bootstrap a payloadless EN, a v7 root checkpoint file is needed. This PR adds a util to convert a v6 root checkpoint into a v7 root checkpoint.

It adds two ways to convert:

original way (less code, but use more memory), it loads the entire v6 checkpoint into memory, and then convert it into v7
steam way (more new code, but much less memory), it iterate each node in the v6 checkpoint file, if it's a leaf node, convert it into v7 format.

coderabbitai · 2026-06-18T02:53:40Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 66cd94fa-8b6f-41e3-af58-ddaac9c90cee

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch leo/payloadless-util-convert-checkpoint

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

janezpodhostnik · 2026-07-24T12:11:57Z

+) error {
+	err := convertCheckpointV6ToV7Stream(inputDir, inputFileName, outputDir, outputFileName, logger, nWorker)
+	if err != nil {
+		cleanupErr := deleteCheckpointFiles(outputDir, outputFileName)


Re-running the conversion deletes the previously converted output. The "V7 output already exists" rejection from convertCheckpointV6ToV7Stream reaches this cleanup, and deleteCheckpointFiles removes every <outputFile>* file.

janezpodhostnik · 2026-07-24T12:11:57Z

+		streamFiles := filePaths(dir, streamName, subtrieLevel)
+		require.Equal(t, len(nonStreamFiles), len(streamFiles))
+		for i, nf := range nonStreamFiles {
+			require.NoError(t, compareFiles(nf, streamFiles[i]),


compareFiles has a bug! It opens file1 twice (checkpoint_v6_test.go:451), so it can never fail. Fix the helper to os.Open(file2).

janezpodhostnik · 2026-07-24T12:11:57Z

+	if err != nil {
+		return 0, fmt.Errorf("could not read subtrie footer: %w", err)
+	}
+	if embeddedSum != expectedSum {


This only compares two stored checksums; the bytes actually read are never CRC-verified, so input corruption is copied into a V7 file that then carries a fresh valid checksum. The non-stream path verifies content CRC via readCheckpointSubTrie. Same applies to the top-trie path below.

janezpodhostnik · 2026-07-24T12:11:57Z

+	checksums := make([]uint32, subtrieCount)
+	for k := 0; k < subtrieCount; k++ {
+		r := <-results
+		if r.err != nil {


Returning on the first error leaves the other workers running. Their deferred Close renames temp files to final part files, possibly after the wrapper's deleteCheckpointFiles has run, leaving stray output that contradicts the godoc and blocks retries. Drain all subtrieCount results, then return the first error, aggregate the errors or/and build a stopping mechanism into the workers.

janezpodhostnik · 2026-07-24T12:11:58Z

+		go func() {
+			for i := range jobs {
+				sum, err := convertSubTrieFileV6ToV7Stream(
+					inputDir, inputFileName, outputDir, outputFileName, i, subtrieChecksums[i], logger)


subtrieChecksums is sized from the header while this loop hard-codes subtrieCount, so a checksum-valid header declaring fewer than 16 subtries panics in a worker goroutine. You could validate len(subtrieChecksums) == subtrieCount up front.

janezpodhostnik · 2026-07-24T12:15:02Z

-		flagNWorker,
-	)
+	var err error
+	if flagStream {


stream could be a boolean input into ConvertCheckpointV6ToV7 so that you don't need to expose 2 methods with identical inputs.

github-actions · 2026-08-18T01:55:45Z

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

github-actions · 2026-08-18T01:55:45Z

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

codecov-commenter · 2026-08-18T02:00:31Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…l V7 conversion cleanup at bootstrap, logger and godoc fixes, and tests for the leaf-hash flag and V7 subtrie paths

Co-authored-by: zhangchiqing <811374+zhangchiqing@users.noreply.github.com>

zhangchiqing marked this pull request as ready for review June 18, 2026 03:04

zhangchiqing requested a review from a team as a code owner June 18, 2026 03:04

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch 3 times, most recently from ce39b24 to 287bfbb Compare June 18, 2026 23:34

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from 287bfbb to a9c6645 Compare June 19, 2026 00:01

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from a9c6645 to ca39b1a Compare July 2, 2026 19:11

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from ca39b1a to 4d5c8bf Compare July 13, 2026 17:26

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from 4d5c8bf to 91eb0a7 Compare July 14, 2026 19:56

janezpodhostnik reviewed Jul 24, 2026

View reviewed changes

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch 3 times, most recently from 91eb0a7 to 1c62fec Compare July 31, 2026 04:08

zhangchiqing requested a review from AlexHentschel August 17, 2026 16:33

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from b82aac5 to 4528bb1 Compare August 18, 2026 01:55

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from 4528bb1 to 7d6436a Compare August 18, 2026 15:43

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from 7d6436a to 2708c79 Compare August 18, 2026 17:25

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from 2708c79 to bb62e79 Compare August 18, 2026 18:18

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from bb62e79 to 2adac61 Compare August 18, 2026 20:29

Copilot started work on behalf of zhangchiqing August 18, 2026 21:21 View session

Copilot finished work on behalf of zhangchiqing August 18, 2026 21:30

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from e91fb18 to 3ec8d0a Compare August 24, 2026 14:01

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from 3ec8d0a to 5863488 Compare August 25, 2026 02:47

This comment has been minimized.

Sign in to view

zhangchiqing and others added 6 commits August 25, 2026 16:00

add convert_stream

155d6b1

remove temp files

cc33b9e

remove verify leaf hash flag

47f5c66

fix lint

9dd0904

address review comments: version-aware latest-checkpoint path, partia…

f0470ca

…l V7 conversion cleanup at bootstrap, logger and godoc fixes, and tests for the leaf-hash flag and V7 subtrie paths

fix tidy by applying go fix loop rewrites

f03b735

Co-authored-by: zhangchiqing <811374+zhangchiqing@users.noreply.github.com>

zhangchiqing force-pushed the leo/payloadless-util-convert-checkpoint branch from 5863488 to f03b735 Compare August 25, 2026 23:00

Navigation Menu

Uh oh!

[Storehouse] 010 Util convert v6 -> v7 checkpoint - #8583

[Storehouse] 010 Util convert v6 -> v7 checkpoint#8583
zhangchiqing wants to merge 6 commits into
leo/payloadless-remote-ledger-servicefrom
leo/payloadless-util-convert-checkpoint

zhangchiqing commented Jun 18, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Jun 18, 2026 •

edited

Loading

Review skipped

Uh oh!

janezpodhostnik Jul 24, 2026

Uh oh!

janezpodhostnik Jul 24, 2026

Uh oh!

janezpodhostnik Jul 24, 2026

Uh oh!

janezpodhostnik Jul 24, 2026

Uh oh!

janezpodhostnik Jul 24, 2026

Uh oh!

janezpodhostnik Jul 24, 2026

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

github-actions Bot commented Aug 18, 2026

Uh oh!

codecov-commenter commented Aug 18, 2026 •

edited

Loading

Uh oh!

This comment has been minimized.

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

zhangchiqing commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Jun 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

janezpodhostnik Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

janezpodhostnik Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

janezpodhostnik Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

janezpodhostnik Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

janezpodhostnik Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

janezpodhostnik Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 18, 2026

Dependency Review

Scanned Files

Uh oh!

github-actions Bot commented Aug 18, 2026

Dependency Review

Scanned Files

Uh oh!

codecov-commenter commented Aug 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

This comment has been minimized.

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zhangchiqing commented Jun 18, 2026 •

edited

Loading

coderabbitai Bot commented Jun 18, 2026 •

edited

Loading

codecov-commenter commented Aug 18, 2026 •

edited

Loading