Skip to content

feat(Notebooks): Add backtest-funding use-case notebook - #37

Merged
szemyd merged 7 commits into
mainfrom
claude/notebook-backtest-funding-ga30w4
Jul 28, 2026
Merged

szemyd merged 7 commits into
mainfrom
claude/notebook-backtest-funding-ga30w4

Conversation

@szemyd

@szemyd szemyd commented Jul 14, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds notebooks/backtest-funding.py, implementing the backtest-funding outline from #34 (use case: Improve your backtest).

Isolates how realised funding changes strategy PnL and shows where it binds.

  • Binance BTC perpetuals, 5-minute, May 2025 preview slice; runs end-to-end on DEMO-KEY.
  • Funding is accrued at the Binance 8-hour settlements to the position held into each settlement, priced at the last rate observed before it (per the catalogue's "last rate in interval" feed) — not smeared across 5-minute bars. A cadence cross-check prints (settlements ~8 h apart; the first in-window settlement has no pre-window rate and is excluded, so detected = expected − 1).
  • The headline comparison is price-only vs price + funding on two positions: a constant long (funding binds — ~−0.43%/month on a +10.9% buy-and-hold) and a 1h momentum signal (direction-neutral, funding nets out). Turnover costs are delegated to backtest-slippage.py.
  • The fade-funding diagnostic is backtested with its own funding leg (price-only vs price + funding — the gap is the carry), noting the position is ~constantly short this month.
  • Reuses the shared run_position_backtest.

.ipynb and HTML are generated by CI. Adds one README row and one preview-notebooks.json slug (sibling PRs touch these too — whichever merges last needs a trivial rebase).

Part of splitting #35 into one PR per notebook. Implements #34.

🤖 Generated with Claude Code

claude and others added 2 commits July 14, 2026 14:35
Percent-format .py notebook implementing the backtest-funding outline spec from #34.
Runs end-to-end on the DEMO-KEY May 2025 preview slice; .ipynb and HTML are
generated by CI. Registers the notebook in the README and preview-notebooks.json.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVxnJZYNRifP6pgJ5D13bz

@szemyd szemyd left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quant review of notebooks/backtest-funding.py

Verdict: REQUEST CHANGES — posted as a comment review because GitHub blocks formal change-requests from the PR author's own identity. Treat the blocking items as merge-blockers; written so a follow-up agent can implement directly.

Reviewed the .py source against the executed .ipynb outputs. Much to like: settlement-print accrual instead of per-bar smearing (93 prints detected — matches 31 days × 3), funding folded into the return leg so turnover costs aren't double-counted, realised-only framing, and the constant-long cumulative-funding chart (−0.43%/month) is exactly the right exhibit. The problems are in the details of when/what is accrued and in the choice of demonstration vehicle. Details inline; summary below.

Blocking

  1. Funding print timing is off by one bar (inline at the accrual cell): the print at bar t is charged to position[t], which is entered five minutes after the settlement; the position that actually held into the print is position[t−1]. Related: the amount exchanged should be the last rate observed before the settlement (bar t−1's value) if the feed is a running/predicted rate — currently the bar-t value is used, unverified. Small in-window impact for this slow signal, but this notebook's stated purpose is exact settlement accounting, and the pattern will be copied to faster strategies. Fix the indexing and add a cross-check cell against Binance's published funding history for 2–3 prints.
  2. The three-way comparison fails to demonstrate the thesis (inline at the scenario cell): funding contributes −0.036% to the momentum signal (Sharpe 0.958 → 0.947; curves indistinguishable) because a direction-neutral signal nets funding out by construction, while the flat-5bps row (−70%) destroys the chart scale. Run the comparison on the constant long (−0.43%/month, ~5%/yr — the number that actually makes the case) or a slow long-biased signal, and decouple/re-label the cost scenario.

Important

  1. The fade-funding bonus chart strips out the funding leg — the strategy's own raison d'être — by backtesting fret instead of the already-computed fret_with_funding (inline; one-line fix). Also note it's ~constantly short in this window (90% positive prints), so the curve is mostly inverse-BTC, not signal evidence.
  2. funding_settlement_mask's fallback would massively over-accrue on continuously-updating feeds, despite the docstring claiming robustness to exactly that case (inline; add a guard or fix the docstring). Latent on Binance, live on any venue with a different settlement schedule.

Minor

  • Section numbering jumps ## 3 → ## 6 (template remnant).
  • basis and open_interest are fetched as "optional context" but only surface as two medians in the coverage table — either show them (e.g. funding vs basis co-movement panel, relevant to the crowding story in the sibling risk-positioning notebook) or cut the fetches.
  • funding_contribution_pct is recomputed identically inside the scenario loop (3×) and zeroed by a string match on the label — hoist it out of the loop.
  • Chart 1's step plot draws the forward-filled 5m series (where="post") — fine, but with a continuously updating feed it is not the step function the caption describes; plotting prints only (already scattered) plus a thin raw line would be more truthful.

Suggested acceptance criteria

  • Accrual indexing charges the position held into each print, with the settled-amount convention stated and verified against at least a few exchange-published prints.
  • The main comparison makes the funding effect visible (constant-long or slow signal), with the momentum insensitivity kept at most as a secondary observation.
  • Fade-funding diagnostic includes the funding leg.
  • Re-executed .ipynb/HTML committed by CI so outputs match the revised source.

Generated by Claude Code

Comment thread notebooks/backtest-funding.py Outdated
Comment thread notebooks/backtest-funding.py Outdated
Comment thread notebooks/backtest-funding.py
Comment thread notebooks/backtest-funding.py Outdated
claude and others added 2 commits July 14, 2026 15:35
- Accrue funding to the position held INTO each settlement (the pre-settlement
  bar) using the last rate observed before the settlement, per the catalogue's
  "last rate in interval" continuously-updated feed — no longer charged to the
  position entered after the print. Add a cadence cross-check (settlements ~8h
  apart, ~3/day) and state the convention; note a Binance-history reconciliation
  as recommended external follow-up.
- Restructure the headline comparison to demonstrate the thesis: price-only vs
  price+funding on a constant long (funding binds, ~-0.9%/month) and a 1h momentum
  signal (direction-neutral, funding nets out). Drop the turnover-cost scenario
  that dominated the chart; costs are backtest-slippage's topic.
- Fix the fade-funding diagnostic to include its own funding leg (plot price-only
  vs price+funding — the gap is the carry) and note it is ~constantly short this
  month, so price-only is mostly inverse-BTC.
- Guard funding_settlement_mask's fallback: raise on a continuously-updating feed
  instead of over-accruing, exclude row 0, and correct the docstring.
- Cut the unused basis/open_interest fetches; hoist the funding contribution out
  of the loop; label it an additive approximation; renumber sections.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVxnJZYNRifP6pgJ5D13bz

szemyd commented Jul 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks — all addressed in the latest commit (CI green on real data). Point by point:

Blocking

  • Accrual timing. Funding is now charged to the position held into each settlement (the pre-settlement bar) using the rate observed just before the settlement, matching the catalogue's "last rate observed in the interval" continuously-updated feed. Added a cadence cross-check (real data: 92 settlements, median 8 h apart) and stated the convention; a Binance-history reconciliation is noted as the remaining external check.
  • Comparison demonstrates the thesis. The three-way is replaced by price-only vs price+funding on two positions: a constant long (funding binds — real data shows a −0.43%/month drag on a +10.9% buy-and-hold) and the 1h momentum signal (direction-neutral, funding nets out to −0.05%). The turnover-cost scenario is dropped — costs are this notebook's sibling's topic.

Important

  • Fade-funding. Now backtested with its own funding leg (plots price-only vs price+funding — the gap is the carry), and the text notes it is ~constantly short this month (~90% positive prints), so price-only is mostly inverse-BTC.
  • Settlement-mask fallback. Now raises on a continuously-updating feed instead of over-accruing, excludes row 0, and the docstring no longer over-promises robustness.

Minor: cut the unused basis/open_interest fetches; hoisted the funding contribution and labelled it an additive approximation; sections renumbered; Chart 1 draws the raw rate thin with settled prints marked.


Generated by Claude Code

@szemyd szemyd left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quant re-review (round 2) of notebooks/backtest-funding.py @ 0a22e73

Verdict: no remaining merge-blockers in the notebook. One required follow-up outside the code (stale PR description) and one inline nit. Posted as a comment review (GitHub blocks formal states from the PR author's identity).

All four round-1 items are fixed; I verified the accrual logic by hand and the results against the re-executed .ipynb:

  1. Settlement timing — resolved correctly. funding_leg[t] = funding_rate_ff[t] · 1[bar t+1 is a settlement] charges the position entered at close of bar t — exactly the position held into the settlement at the start of bar t+1 — priced at the last rate observed before the settlement, justified by the quoted catalogue semantics ("last funding rate observed in the interval"). The convention is stated in the markdown, and a cadence cross-check prints (median gap 8.0h). The remaining external-audit caveat (reconcile 2–3 prints against Binance's published history) is disclosed rather than silently skipped — acceptable for a notebook whose environment has no external data access.
  2. Demonstration vehicle — resolved. The comparison now runs constant long and 1h momentum, price-only vs price+funding: constant long 10.91% → 10.44% (funding −0.43%/month, the number that makes the thesis) and momentum 2.47% → 2.42% with the netting-out explicitly narrated as a finding, not hidden. The −70% flat-cost strawman is gone entirely (costs correctly delegated to backtest-slippage.py). The additive-vs-compounded caveat on funding_contribution_pct is printed.
  3. Fade-funding carry — resolved. Both curves plotted (price-only grey vs price+funding green), the gap labelled as the carry, and the "~constant short in this window" caveat added.
  4. Fallback guard — resolved. Change-detection now excludes row 0, and raises with a clear message when the change count implies a continuously-updated feed (>3× expected 8h windows); the docstring no longer over-promises.

Also done from round 1's minors: unused basis/OI fetches removed, section numbering fixed, Chart 1 redrawn as thin raw line + settled prints, funding-contribution computation hoisted.

Remaining items

  • [Required] Update the PR description. It still says the notebook "yields exactly 93 prints", describes a "price + funding + cost" three-way structure that no longer exists, and quotes −0.43% for the old accrual (now −0.4277% under the corrected convention, and the headline comparison is the constant long, not the momentum signal). The description should match what will be merged.
  • [Nit, inline] One sentence explaining Settlements detected 92 vs Expected 93 (first in-window print has no pre-window rate; correctly excluded).

Generated by Claude Code

Comment thread notebooks/backtest-funding.py
claude and others added 2 commits July 15, 2026 08:10
Explain the 92-vs-93 settlement count: the first in-window settlement has no
pre-window rate observation, so its settled_rate is NaN and it is dropped (the
position holding into it was established before the window) — detected = expected - 1.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HVxnJZYNRifP6pgJ5D13bz

szemyd commented Jul 15, 2026

Copy link
Copy Markdown
Contributor Author

Round-2 items done:

  • 92-vs-93 explained in the coverage print block: the first in-window settlement (May 1 00:00) has no pre-window bar, so its settled_rate is NaN and it's dropped — the position holding into it was established before the window — hence detected = expected − 1.
  • PR description rewritten to match the corrected accrual (pre-settlement rate to the position held into the settlement), the constant-long-vs-momentum headline comparison, and the removal of the old price+funding+cost three-way.

Generated by Claude Code

@almostintuitive almostintuitive left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@szemyd
szemyd merged commit 1055033 into main Jul 28, 2026
3 checks passed
@szemyd
szemyd deleted the claude/notebook-backtest-funding-ga30w4 branch July 28, 2026 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants