feat(Notebooks): Add backtest-funding use-case notebook - #37
Conversation
Percent-format .py notebook implementing the backtest-funding outline spec from #34. Runs end-to-end on the DEMO-KEY May 2025 preview slice; .ipynb and HTML are generated by CI. Registers the notebook in the README and preview-notebooks.json. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HVxnJZYNRifP6pgJ5D13bz
szemyd
left a comment
There was a problem hiding this comment.
Quant review of notebooks/backtest-funding.py
Verdict: REQUEST CHANGES — posted as a comment review because GitHub blocks formal change-requests from the PR author's own identity. Treat the blocking items as merge-blockers; written so a follow-up agent can implement directly.
Reviewed the .py source against the executed .ipynb outputs. Much to like: settlement-print accrual instead of per-bar smearing (93 prints detected — matches 31 days × 3), funding folded into the return leg so turnover costs aren't double-counted, realised-only framing, and the constant-long cumulative-funding chart (−0.43%/month) is exactly the right exhibit. The problems are in the details of when/what is accrued and in the choice of demonstration vehicle. Details inline; summary below.
Blocking
- Funding print timing is off by one bar (inline at the accrual cell): the print at bar t is charged to
position[t], which is entered five minutes after the settlement; the position that actually held into the print isposition[t−1]. Related: the amount exchanged should be the last rate observed before the settlement (bar t−1's value) if the feed is a running/predicted rate — currently the bar-t value is used, unverified. Small in-window impact for this slow signal, but this notebook's stated purpose is exact settlement accounting, and the pattern will be copied to faster strategies. Fix the indexing and add a cross-check cell against Binance's published funding history for 2–3 prints. - The three-way comparison fails to demonstrate the thesis (inline at the scenario cell): funding contributes −0.036% to the momentum signal (Sharpe 0.958 → 0.947; curves indistinguishable) because a direction-neutral signal nets funding out by construction, while the flat-5bps row (−70%) destroys the chart scale. Run the comparison on the constant long (−0.43%/month, ~5%/yr — the number that actually makes the case) or a slow long-biased signal, and decouple/re-label the cost scenario.
Important
- The fade-funding bonus chart strips out the funding leg — the strategy's own raison d'être — by backtesting
fretinstead of the already-computedfret_with_funding(inline; one-line fix). Also note it's ~constantly short in this window (90% positive prints), so the curve is mostly inverse-BTC, not signal evidence. funding_settlement_mask's fallback would massively over-accrue on continuously-updating feeds, despite the docstring claiming robustness to exactly that case (inline; add a guard or fix the docstring). Latent on Binance, live on any venue with a different settlement schedule.
Minor
- Section numbering jumps
## 3 → ## 6(template remnant). basisandopen_interestare fetched as "optional context" but only surface as two medians in the coverage table — either show them (e.g. funding vs basis co-movement panel, relevant to the crowding story in the siblingrisk-positioningnotebook) or cut the fetches.funding_contribution_pctis recomputed identically inside the scenario loop (3×) and zeroed by a string match on the label — hoist it out of the loop.- Chart 1's step plot draws the forward-filled 5m series (
where="post") — fine, but with a continuously updating feed it is not the step function the caption describes; plotting prints only (already scattered) plus a thin raw line would be more truthful.
Suggested acceptance criteria
- Accrual indexing charges the position held into each print, with the settled-amount convention stated and verified against at least a few exchange-published prints.
- The main comparison makes the funding effect visible (constant-long or slow signal), with the momentum insensitivity kept at most as a secondary observation.
- Fade-funding diagnostic includes the funding leg.
- Re-executed
.ipynb/HTML committed by CI so outputs match the revised source.
Generated by Claude Code
- Accrue funding to the position held INTO each settlement (the pre-settlement bar) using the last rate observed before the settlement, per the catalogue's "last rate in interval" continuously-updated feed — no longer charged to the position entered after the print. Add a cadence cross-check (settlements ~8h apart, ~3/day) and state the convention; note a Binance-history reconciliation as recommended external follow-up. - Restructure the headline comparison to demonstrate the thesis: price-only vs price+funding on a constant long (funding binds, ~-0.9%/month) and a 1h momentum signal (direction-neutral, funding nets out). Drop the turnover-cost scenario that dominated the chart; costs are backtest-slippage's topic. - Fix the fade-funding diagnostic to include its own funding leg (plot price-only vs price+funding — the gap is the carry) and note it is ~constantly short this month, so price-only is mostly inverse-BTC. - Guard funding_settlement_mask's fallback: raise on a continuously-updating feed instead of over-accruing, exclude row 0, and correct the docstring. - Cut the unused basis/open_interest fetches; hoist the funding contribution out of the loop; label it an additive approximation; renumber sections. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HVxnJZYNRifP6pgJ5D13bz
|
Thanks — all addressed in the latest commit (CI green on real data). Point by point: Blocking
Important
Minor: cut the unused Generated by Claude Code |
szemyd
left a comment
There was a problem hiding this comment.
Quant re-review (round 2) of notebooks/backtest-funding.py @ 0a22e73
Verdict: no remaining merge-blockers in the notebook. One required follow-up outside the code (stale PR description) and one inline nit. Posted as a comment review (GitHub blocks formal states from the PR author's identity).
All four round-1 items are fixed; I verified the accrual logic by hand and the results against the re-executed .ipynb:
- Settlement timing — resolved correctly.
funding_leg[t] = funding_rate_ff[t] · 1[bar t+1 is a settlement]charges the position entered at close of bar t — exactly the position held into the settlement at the start of bar t+1 — priced at the last rate observed before the settlement, justified by the quoted catalogue semantics ("last funding rate observed in the interval"). The convention is stated in the markdown, and a cadence cross-check prints (median gap 8.0h). The remaining external-audit caveat (reconcile 2–3 prints against Binance's published history) is disclosed rather than silently skipped — acceptable for a notebook whose environment has no external data access. - Demonstration vehicle — resolved. The comparison now runs constant long and 1h momentum, price-only vs price+funding: constant long 10.91% → 10.44% (funding −0.43%/month, the number that makes the thesis) and momentum 2.47% → 2.42% with the netting-out explicitly narrated as a finding, not hidden. The −70% flat-cost strawman is gone entirely (costs correctly delegated to
backtest-slippage.py). The additive-vs-compounded caveat onfunding_contribution_pctis printed. - Fade-funding carry — resolved. Both curves plotted (price-only grey vs price+funding green), the gap labelled as the carry, and the "~constant short in this window" caveat added.
- Fallback guard — resolved. Change-detection now excludes row 0, and raises with a clear message when the change count implies a continuously-updated feed (>3× expected 8h windows); the docstring no longer over-promises.
Also done from round 1's minors: unused basis/OI fetches removed, section numbering fixed, Chart 1 redrawn as thin raw line + settled prints, funding-contribution computation hoisted.
Remaining items
- [Required] Update the PR description. It still says the notebook "yields exactly 93 prints", describes a "price + funding + cost" three-way structure that no longer exists, and quotes −0.43% for the old accrual (now −0.4277% under the corrected convention, and the headline comparison is the constant long, not the momentum signal). The description should match what will be merged.
- [Nit, inline] One sentence explaining
Settlements detected 92vsExpected 93(first in-window print has no pre-window rate; correctly excluded).
Generated by Claude Code
Explain the 92-vs-93 settlement count: the first in-window settlement has no pre-window rate observation, so its settled_rate is NaN and it is dropped (the position holding into it was established before the window) — detected = expected - 1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HVxnJZYNRifP6pgJ5D13bz
|
Round-2 items done:
Generated by Claude Code |
Summary
Adds
notebooks/backtest-funding.py, implementing the backtest-funding outline from #34 (use case: Improve your backtest).Isolates how realised funding changes strategy PnL and shows where it binds.
DEMO-KEY.backtest-slippage.py.run_position_backtest..ipynband HTML are generated by CI. Adds one README row and onepreview-notebooks.jsonslug (sibling PRs touch these too — whichever merges last needs a trivial rebase).Part of splitting #35 into one PR per notebook. Implements #34.
🤖 Generated with Claude Code