Add more granular collection step checkpoints - #517

Merged

ppinchuk merged 37 commits into

mainfrom

collection-step-checkpoints

Aug 5, 2026

Commits on Aug 4, 2026

func now uses dict
ppinchuk
committed
b1e229b
Set jur website from info
ppinchuk
committed
57e6c28
Update logger info
ppinchuk
committed
7c1b573
New func input
ppinchuk
committed
d43b1d7
Update variable name and minor logic
ppinchuk
committed
91bdf75
Add `_store_docs_as_needed`
ppinchuk
committed
d506832
Update logic
ppinchuk
committed
429fcff
New `persist_documents` logic
ppinchuk
committed
a0be749
Steps are now a dynamically compted property
ppinchuk
committed
7070f05
Fix docstring
ppinchuk
committed
2377f1b
Update function
ppinchuk
committed
a58b23a

Commits on Aug 5, 2026

Fix bug
ppinchuk
committed
cfa9a92
Add prop
ppinchuk
committed
aa4b9a0
Add `write_collection_shard_no_fail`
ppinchuk
committed
ef77762
Remove method
ppinchuk
committed
a07a2d3
Simplified call
ppinchuk
committed
f7e2ce8
Add imports
ppinchuk
committed
06daf2d
Relative to set later in the pipeline
ppinchuk
committed
00f943e
Fix calls
ppinchuk
committed
a3f4e3a
FIx imports and add class
ppinchuk
committed
8e3db03
Update docstring
ppinchuk
committed
575128f
Fix recursion bug
ppinchuk
committed
1283eb3
Collection now persists
ppinchuk
committed
563a9a0
Don't force specifying jurisdiction name
ppinchuk
committed
781d0c3
Add more info to top-level manifest
ppinchuk
committed
3478ef1
FIx bug
ppinchuk
committed
3ef0a05
Get from steps from existing doc
ppinchuk
committed
cc6a6ba
Add test
ppinchuk
committed
e2fc5d7
Fix tests
ppinchuk
committed
28a0868
Add test
ppinchuk
committed
8370d86
Fix tests
ppinchuk
committed
54e128b
Add tests
ppinchuk
committed
64c90a2
Move methods
ppinchuk
committed
60c74ca
Fix edge case
ppinchuk
committed
4997f59
Remove debug logging
ppinchuk
committed
753ac4b
Add logging statement
ppinchuk
committed
2939be8
Fix eager extract regression
ppinchuk
committed
c5b27ce