Repository navigation
Make object storage provider-neutral for B2 - #698
Volyarosh94 wants to merge 3 commits into
Conversation
OBJECT_STORE_* env names, with R2_* as the fallback until the droplets move. Adds scripts/object_store_check and a runbook section for the move.
This comment has been minimized.
This comment has been minimized.
|
Claude finished @Volyarosh94's task in 10m 30s —— View job Code review: Make object storage provider-neutral for B2
SummaryThis is a clean, thoroughly-tested rename/generalization of Worth a look before merging:
Minor, non-blocking:
Verified against repo conventions (
Things worth calling out as careful work:
Testing: Per the PR description, |
What
services/object_store.pyreplacesservices/r2_client.py. It readsOBJECT_STORE_*(endpoint, region, key id, secret, bucket) and falls back to theR2_*names while none of those is set, so the droplets keep working until their env moves.s3.<region>.backblazeb2.comendpoint. Non-AWS endpoints use path-style requests and send checksums only where an operation requires one; botocore's newer default CRC32 trailers were rejected by B2 until mid-2025.scripts/object_store_check.pywrites, reads, lists, multipart-uploads and deletes test objects, and prints PASS/FAIL per step..env.exampleand the architecture doc name the store neutrally.Calls I made
Any
OBJECT_STORE_*set means only those names are read. An incomplete set turns the store off with one error naming what's missing, instead of mixing B2 and R2 settings.OBJECT_STORE_BUCKEThas no default.A bare endpoint host gets
https://.The alert
r2_replication_failedis nowobject_store_replication_failed, and the two "r2 disabled" status strings follow. Nothing in the repo keys on them; alert rules outside it might.The final pass after the switch is
rclone copy --update, notrclone sync, which would delete what the app wrote to B2 since the switch.The runbook has rclone read its credentials with
read -rs, so no key lands in the shell history, and uses a separate temporary B2 key for the copy.The test suite ignores any store env in the shell, so a local run can't write to a real bucket.
delete_objectsstill sends a CRC32 header (botocore requires one there). B2 documents support for it; the live check covers it.The runbook copies through B2's S3 API rather than rclone's native b2 backend, so
rclone checkcompares MD5 instead of sizes only.The runbook raises Backblaze's daily caps first. Copying this bucket with the free-tier caps hit the Class B cap (rclone reads each upload back unless told not to) and then the storage cap.
Tested:
just checkpasses up toweb, whose dashboard lint fails with the same 54 problems as main. The rest ofwebpasses.scripts.object_store_checkagainst the new B2 bucket: provider b2, 7/7 PASS.