Repository navigation
[BUG] /books and /shelf page time out repeatedly and might be cause of high memory usage #290
Description
2 pages that repeatedly stall on me. Taking discussion from https://discord.com/channels/1376676460647022752/1554508748276375582 and open GH issue as requested.
/authors responds in a timely manner, even /unmapped Comes back reasonably quickly.
/books and /shelf however do not populate or respond even if I am patient or walk away.
/ (main library tab) does respond quickly once Chaptarr gets past initial loading delay. Similar delay to readarr back in the day on first return of home page on browse. I Just did a human count on browsing from a closed session and got a count of 31 (Which is awfully close to potential 32nd time out on something, or a total coincidence) Home page load times probably deserves it's own report but adding as a portion of this specific /book and /shelf report as context. Can split if requested.
I found this thread when I was looking for high memory utilization as well. I normally run at a couple gig of memory But I've had it drain my server before and I have not yet added the memory cap parameter to this docker. It shot up to 20gig earlier today.
/authors responds in ~25 seconds. Test method is going directly to that url rather than / and then clicking authors.
direct jump to http://ip/books starts returning and goes to /books but is at pulsing circle. Has matching log entry of [Warn] BookController: [API-PERFORMANCE] Taking general book path without an author/book filter.
Same result twice, pulsing circle, the ~29 seconds by human count for /books to return but I never get data there, same log warning.
sqlite not postgres, unraid bypassing fuse for appdata and book library
with /shelf I gave up at a 2 minute count. memory usage definitely jumps when I try those particular pages.
Version
0.9.965.0
.NET Core
Yes (10.0.12)
Docker
Yes
Database
Sqlite 3.50.4
Database Migration
107
AppData Directory
/config
Startup Directory
/app
Mode
chaptarr.trace.txt
chaptarr.debug.txt
Cons
Took discord suggestion from Golden_Dragons and switched from sqlite to postgress, same issue after pgloader import
Dropped chaptarr dbs and recreated and re-imported. Same results
also
chaptarr.debug (1).txt
message (1).txt
message.txt
spotted thread pool starvation each time
[Warn] Microsoft.AspNetCore.Server.Kestrel: As of "10/04/2026 17:20:56 +00:00", the heartbeat has been running for "00:00:06.0139203" which is longer than "00:00:01". This could be caused by thread pool starvation.
Activity
bhoffman20 commentedon Oct 4, 2026
Checked against
develop(e67374f). Your logs show the cause./shelf, and/booksin Table or Overview view, both callGET /api/v1/book?mediaType=…with no author or book filter. That request loads the whole library into memory before it filters anything:GetBooksForDisplaycallsGetAllBooks()(L2234). That reads all 643,135 book rows, for both media types, and only then filters to the requested type in memory.GetAllMonitoredEditions()(L2267). That runsSELECT * FROM Editions WHERE Monitored = trueacross the whole library, wide JSON columns included (Chapters,Images,Links), before narrowing to the books it needs. YourOutOfMemoryExceptionis thrown there.This fails the same way on SQLite and Postgres, so switching databases and re-importing wouldn't have changed it. The thread-pool-starvation warnings come from the same memory pressure.
Related to #170 (item 3): same endpoint, but on a library about 8× smaller it was slow rather than out of memory. The
/api/v1/authortimes in your log (50 s, then 143 s) fit #170 item 1.edsilberberg commentedon Oct 4, 2026
Updated environmental info:
Version
0.9.965.0
.NET Core
Yes (10.0.12)
Docker
Yes
Database
Postgresql 18.6
Database Migration
107
AppData Directory
/config
Startup Directory
/app
Mode
Console
Uptime
01:29:05
edsilberberg commentedon Oct 4, 2026
db log entries taht may support report 170
EDT [25406] ERROR: canceling statement due to user request
EDT [25406] STATEMENT: SELECT "Editions".*
FROM "Editions" WHERE ("Editions"."Monitored" = $1)
EDT [25406] ERROR: canceling statement due to user request
EDT [25406] STATEMENT: SELECT "Editions".*
FROM "Editions" WHERE ("Editions"."Monitored" = $1)
bhoffman20 commentedon Oct 4, 2026
Two possible fixes, from smallest to largest:
1. Make the general/api/v1/bookpath load only what it needs (backend only). This should stop the crash.- Filter by media type in SQL instead of callingGetAllBooks()and filtering in memory.- ReplaceGetAllMonitoredEditions()with a query for the monitored editions of the books being returned. Leave outChapters, which list mapping never reads and which is the largest column inEditions(129 of 219 MB in the #170 table). ReadOverviewonly wheninclude=overviewis passed.FindByBookalready chunks the IDs, but it also joins Book and Author on every edition, so this path needs its own leaner query.- Load series links once:GetAllBooks()loads them, andMapToResourceloads them again.This roughly halves the rows read and drops the largest column. It still returns about 320k books for one media type in a single JSON response, which the server buffers in full and the browser holds in Redux. So it stops the crash but leaves the page slow.2. Page
/shelfand the Table/Overview views on the server (frontend and API). This fixes the slowness too./book/pagedand/book/idsendpoints that Posters view already uses for infinite scroll. Custom filters would need server-side support as well, or they'll keep loading the full list./shelfshows one row per author, and it fetches every book only to work out which authors to list and what goes in each row. It could build the rows from the author list and load each author's books as the row scrolls into view. The existing?authorId=path is the heavy one, so that load needs a lean variant.Fix 1 is small and safe to ship first. Fix 2 is the real fix for libraries this size.
bhoffman20 commentedon Oct 4, 2026
Reproduced locally on a synthetic SQLite library the same size as yours (643,599 books, half audiobook and half eBook), with the .NET heap capped at 4 GB:
develop/api/v1/book?mediaType=audiobook&include=author(Books page)OutOfMemoryExceptioninGetAllMonitoredEditions(), same as your logs/api/v1/book?mediaType=audiobook(Shelf page)With fix 1 the server survives, but only just. The browser then runs out of memory parsing the 583 MB response on Shelf. So fix 1 on its own moves the failure from the server to the browser, and fix 2 (server-side paging) is needed for these pages to work at this size.
Correction to my earlier comment: the server doesn't buffer the response.
BufferingMiddlewarebuffers request bodies only. The memory goes into loading the rows and building the resources.bhoffman20 commentedon Oct 4, 2026
Workaround for
/booksuntil there's a fix, tested on the same 643k-book repro: switch to Posters view and pick one of the built-in filters (All, Monitored, Unmonitored, Exists, Missing, Wanted). That combination loads books a page at a time through/api/v1/book/pagedinstead of the full list, and the page loads fine.Table and Overview views, and custom filters, still load everything.
/shelfhas no equivalent yet.bhoffman20 commentedon Oct 5, 2026
Draft PRs for this, tested together on the 643k-book repro:
/api/v1/booklist from running the server out of memory (filters by media type in SQL, skips edition chapters).With all three,
/books(every view, built-in filters) and/shelfload normally and the server stayed under 1.3 GB. Custom filters on/booksstill load the full list.edsilberberg commentedon Oct 7, 2026
25k files and 96k titles much better after filter, app stayed stable, new scrolling interim method is stable, but end user scroll pause is a smidge invasive feeling.