Add v0.4 adaptive large-book scheduler safeguards

[xichia]

Summary

This PR adds the v0.4 adaptive-large-book scheduler work for DeepReader’s paragraph-summary service.

It focuses on making large-book summarization more resilient and governable while keeping risky scheduler behavior unchanged.

Completed scope

• Added dynamic provider-call cap planning.
• Wired dynamic provider-call caps using actual packed batch count.
• Added hard-ceiling/source-label consolidation for provider-call caps.
• Added large-book lane tuning to pin adaptive RPM during large-book runs.
• Added counts-only cleanup_plan classification for failed/skipped artifact lines.
• Extracted per-record artifact validation into artifact_validator.py.
• Updated validation documentation for the v0.4 branch.

Validation

Offline/static/unit validation only.

cd /Users/ianchia/deepreader/services/paragraph-summary-service
python3 -m pytest -q
154 passed, 1 warning

The warning is the pre-existing Starlette/httpx deprecation warning from fastapi/testclient.py and is unrelated to this branch.

Review result

Final RepoAware review found:

• No blockers.
• No required code changes.
• No secret exposure.
• No live-provider assumptions.
• No service-start requirement.
• Stats-shape changes are additive only.
• cleanup_plan remains counts-only and does not expose record IDs, stable IDs, or source text.

Rollout note

Dynamic provider-call caps are opt-in/default-off. For very low batch counts, the default safety_factor=2.0 can be below the theoretical worst-case retry/rate-limit claim consumption for a single heavily rate-limited batch. Operators should raise safety_factor or margin, or enable the setting mainly alongside large-book mode.

Deferred / intentionally out of scope

• No live Gemini/provider validation.
• No OpenStax run.
• No service startup.
• No executable cleanup modes.
• No batch_controller implementation.
• No risky rewrites to _process_batch, worker requeue logic, QuotaLane, or claim_provider_call.