Fix N+1 query performance issues in stats and list endpoints - #1161
Merged
Conversation
cast/stats and character/stats were firing 5,137 individual SQL queries per request (~8s) due to nested lazy-loaded ORM traversal. Replace with selectinload chains that load all required data in ~8 bulk queries, and a single upfront set-query for revision-scoped cuts rather than one check per line part. Also fixes character list N+1 (per-character character_group and mic_allocation lazy loads) via selectinload on CharacterController, and removes Act.lines / Scene.lines from their schemas — both relationships loaded ~950 ORM rows per act/scene on every list request but were never read by either frontend client. Adds 11 behavioural count-verification tests for cast/stats and character/stats to document and guard the exact counting semantics. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Client V3 Test Results23 tests 23 ✅ 0s ⏱️ Results for commit 3c22e32. |
Client Test Results128 tests 128 ✅ 0s ⏱️ Results for commit 3c22e32. |
|
Python Test Results 1 files 1 suites 2m 10s ⏱️ Results for commit 3c22e32. |
Playwright E2E Results (chromium)164 tests 164 ✅ 1m 39s ⏱️ Results for commit 3c22e32. |
Playwright E2E Results (firefox)164 tests 164 ✅ 1m 44s ⏱️ Results for commit 3c22e32. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cast/statsandcharacter/statswere firing 5,137 individual SQL queries per request (~8 seconds each), blocking Tornado's event loop for the duration and queueing all other requests behind them. Root cause: a Python loop overrevision.line_associationswith lazy-loaded access at every level (association → line → line_parts → cuts per-part → character per-part → group.characters per-group). Fixed by replacingsession.get(ScriptRevision, ...)with aselect(...).options(selectinload(...))chain that loads all required data in ~8 bulk queries, plus a single upfront set-query for revision-scoped cut IDs replacing per-part lazy loads.GET /show/characterhad an N+1 per character forcharacter_groupsandmic_allocationsduring schema serialisation. Fixed by addingselectinloadoptions to the character query inCharacterController.GET /show/actandGET /show/scenewere loading ~950 script line ORM objects per act/scene viaActSchema/SceneSchemawithinclude_relationships = True. Thelinesrelationship is confirmed unused by both frontend clients (neitherActnorSceneTypeScript interface includes alinesfield). Fixed by addingexclude = ("lines",)to both schemas.cast/statsandcharacter/statscovering: dialogue lines counted, characters without cast excluded, cut lines excluded, stage directions excluded, character group expansion, and multi-act/scene attribution.Test plan
pytest test/controllers/api/v1/show/test_cast.py test/controllers/api/v1/show/test_characters.py -v— all 26 tests should passpytestsuite — all 662 tests should pass🤖 Generated with Claude Code