Repository navigation
One Trakt fake, checked against recorded Trakt behavior - #218
Merged
Merged
Conversation
The fake lives in packages/core/test/trakt-fake: the account, its read bodies, the write endpoints, OAuth sessions, the rate limiter and every route the app calls, as MSW handlers. Tests can run it in process, and serve.ts puts the same handlers behind a small node:http server for the smoke flows. It follows recordings of real Trakt made on a throwaway account: refresh rejections answer 400 invalid_grant, a pending device poll answers 400 with an empty body, 404 and 401 bodies are plain text, pages default to 100 rows and cap at 250, watchlist writes move only their own type's activity and answer with the list's item count, a season mark splits added from updated and echoes a missing season's whole entry, and removing a movie removes every play of it. Behavior that cannot be recorded follows Trakt's documentation: 7-day tokens, the device token statuses, and one authed POST per second and 500 GETs per 5 minutes on the fake's clock. recordings.json merges the recorded read shapes, the earlier behavior transcripts and the new recordings. conformance.test.ts drives the fake through scenarios.ts and compares statuses, content types, body shapes and write semantics with them, and fails when a recorded read answers 404.
The tests that wrote their own MSW Trakt responses now read the fake's default account, check what the app sent through the fake's request log, and get failures from its one-shot helpers: fail, hold and drop. Bodies that are deliberately off-spec stay as decode tests over a stubbed fetch. The fake's own internals tests, the request journal, the in-process server wrapper and the third fake inside read-budget.integration are gone. That file's cold-sync cases that the fake can express moved into scenario-budget; the ones about watched rows the fake does not produce (a restarted show, more plays than aired episodes, an unstamped row, a broken progress body) are unit tests in read-budget.test.ts. A short fake.test.ts covers what the fake does from Trakt's documentation: token lifetime, the device token statuses and the rate limits. The fake now stamps each play with its own watched_at, can peek at a read without spending a request, and resets to either Multiple Plays setting. scenario-budget waits out a rate-limit pause the way a person would. A heavy library's first sync sends more than the 500 GETs Trakt allows in five minutes, so with the limit enforced it pauses before it settles; that scenario was already marked over budget. TRAKT_API_BASE is no longer exported from oauth.ts and client.ts, since no test imports it. The persisted cache shape witness is reseeded for that; no persisted shape changed.
The smoke jobs start the fake with `node packages/core/test/trakt-fake/serve.ts`, which Node 22 runs directly by stripping the types. It answers on 127.0.0.1:8787 as before and keeps the two controls the flows and the iOS re-run use, POST /__approve and POST /__reset. Its artwork points at three placeholder images it serves itself, so screenshots still show posters without reaching Trakt's image host. The old server, its fault, scale and journal modules and its seed are deleted. `pnpm fake:trakt` replaces `pnpm mock:trakt`, and the README describes the one fake and how it is checked.
oxlint's no-restricted-imports now rejects `msw` and `msw/*` everywhere except packages/core/test/trakt-fake, so a test cannot grow its own hand-written Trakt responses again; it has to go through the fake.
For a show you have never played, the Calendar and On the way list only premieres. A premiere used to mean episode 1 of a season. It now means an episode Trakt types as series_premiere or season_premiere, which Trakt sends on every calendar row (seen on a real account). A season numbered on from the last one, such as an anime whose second season starts at episode 62, now counts. Episode 1 is still the rule when a row carries no type. Calendar entries keep the type, so the persisted cache buster is bumped.
The fake now models Trakt's "restart show": a show can carry a reset date, its watched row sends reset_at, and progress counts only plays after it. A show spec can also override two watched-row fields the play counter cannot derive, a missing last_watched_at and a different aired_episodes, so the cases about unusual watched rows read the fake instead of a stub fetch. This also restores the case lost when read-budget.integration was folded: 20 recently watched shows that are caught up, ahead of 80 with episodes left. The cold sync still spends all 60 progress reads on shows with episodes left; it fails if the budget is cut before caught-up shows are filtered out. The broken progress body case moved to the tolerant decoding tests, which are the only ones that stub fetch with off-spec bodies.
…igration weakened
…ones Real Trakt stamps each activity with the time of the write, cut to the second. The fake used to push every stamp at least 1 ms past the previous one; no test depended on that.
A new device code used to replace the previous one, so polling an earlier code answered 404. Each code now stays valid until its own expires_in runs out on the fake's clock, and approving or denying acts on the latest code, the one on screen. fake.test.ts drops the case that only checked the fault helper answered what it was told.
The core test tsconfig turns on erasableSyntaxOnly, since serve.ts and the modules it imports run under Node's type stripping. The scenario recorder's constructor no longer uses parameter properties, its pass-through call method is gone, and a stale doc comment is removed. The file's header no longer claims the scenarios were run against real Trakt as written.
The fake's own README now describes its modules, the rules for tests that use it (only the tolerant decoding tests stub fetch, with off-spec bodies), how conformance works, and the provenance of each group in recordings.json: the date, the throwaway account and the paced, guarded recorder. The repository README points to it.
Pull request footprint
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Cue had three Trakt fakes: the socket server in
scripts/mock-trakt, an in-processrespond()transport built on it, and a third one insideread-budget.integration.test.ts. On top of those, 11 core test files wrote their own Trakt responses with MSW. They disagreed with each other and with Trakt. For example, a refresh rejection was a 401 where Trakt answers 400, a pending device poll had a JSON body where Trakt sends none, and rate limits were never enforced, so a pacing regression would pass. Only GET 200 shapes were checked against a recording, and a 404 on a recorded endpoint was skipped silently.This PR replaces all of that with one fake in
packages/core/test/trakt-fake. It follows recordings of real Trakt made on a throwaway account, and a conformance test fails whenever the fake disagrees with them.What the fake does now
{"error":"invalid_grant"}(a fault profile){"error":"invalid_grant","error_description":"session not found"}Unauthorizedas plain text{}interval: 1, verification URL on the fakeinterval: 6,https://auth.trakt.tv/activate{"error":"authorization_pending"}{"error": ...}text/plainbodyX-Ratelimitand sendsRetry-Afteradded{shows,movies}, 4not_foundlistsnot_foundlists;list{updated_at,item_count}watchlisted_atandlists.updated_atshows.watchlisted_at, a movie writemovies.watchlisted_at, both movewatchlist.updated_at;lists.updated_atstaysnot_found.seasonsalways emptyupdatedreplaces the earlier play's datedeleted.movies: 2with two plays)watched_atin one writeepisode_typeextended=fullhttp://URLs to the fakeexpires_in(600 s)reset_at, and progress counts only plays after itThe default account now also has a hidden show, a dropped show, a show watched only through its specials, a long runner numbered from its specials, a show numbered across seasons, a thinly catalogued show, a show whose progress snapshot predates its newest episode, and a played show whose next season drops all at once, so the calendar's grouped row shows up in the smoke screenshots.
sized(...)still builds an account of a given size for the request budget, andemptyAccount()gives a test an account to fill with its own shows. A show can also override two watched-row fields the play counter cannot derive: a missinglast_watched_atand a differentaired_episodes.What was deleted
scripts/mock-trakt:server.mjs,seed.mjs,faults.mjs,scale.mjs,journal.mjsandgenres.mjs, with the__faultcontrol, the seed and fault profiles, the authorization-code route and the CORS headers.packages/core/test/harness:mock-server.ts,mock-trakt.test.ts,mock-behavior.test.ts,journal.test.ts,trakt-limits.ts,trakt-real-shapes.jsonandtrakt-transcripts.json.packages/core/test/data/_msw.tsandread-budget.integration.test.ts. Its cold-sync cases are now inscenario-budget.test.tsx. The cases about unusual watched rows (a restarted show, more plays than aired episodes, a row with no watched date) are inread-budget.test.tsand go through the fake. So does the case of 20 recently watched shows that are caught up, ahead of 80 with episodes left. It fails if the progress budget is cut before caught-up shows are filtered out. The broken progress body is a decode test.trakt-endpoints.test.ts. Its deliberately broken bodies moved totrakt-tolerant-decoding.test.ts. That is now the only test that stubsfetch, and the fake's README says so.read-retry.test.tsxgets its 503 from the fake's fault helper.Kept on purpose:
serve.tsonly. A simulator cannot load Trakt-style paths from a local fake, and the smoke screenshots need posters.routes.test.ts. Its compile-time list of every read and write the app has fails to build when a new one is added, and the test fails when the fake has no route for it.How conformance works
recordings.jsonholds two kinds of recording, both with no account values:conformance.test.tsdrives the fake through the same reads and throughscenarios.ts, on a clock spaced as the recording's request guard was. It fails on any path, type, class, status or header Trakt was never seen sending, on any 404 from a recorded read, and on any difference from a transcript. What cannot be recorded (token lifetime, the device statuses, the rate limits) follows docs.trakt.tv and developer.trakt.tv, and is tested infake.test.ts.Provenance is in the fake's README, per group in
recordings.json. All of it came from a throwaway account through a paced, guarded recorder:scenarios.ts.None of the scenarios in
scenarios.tshas been run against real Trakt as it stands, and this PR made no Trakt requests.Mutation checks I ran, each reverted afterwards:
routes.test.tsboth fail.<instead of!==, ranking an unstamped row first, and ignoringreset_ateach fail one of the budget selection tests.scenario-budget. A shortened entry list fails the caught-up entry count. A rotated token the fake does not know fails the rotated-token check.erasableSyntaxOnlyrejects a parameter property.Transports
startTraktFake()wrapssetupServerwithonUnhandledRequest: "error", andfake.fetchusesgetResponsefor code that takes a fetch.serve.tsis a smallnode:httpadapter around the same handlers. It runs with plainnodethrough Node 22's type stripping, answers on 127.0.0.1:8787, and keepsPOST /__approveandPOST /__reset. CI starts it in place ofserver.mjs, andpnpm fake:traktreplacespnpm mock:trakt.Lint gate
oxlint's
no-restricted-importsrejectsmswandmsw/*outsidepackages/core/test/trakt-fake. I checked that it flags an MSW import in another test file.App changes the recordings forced
invalid_grantcode, not its status, so Trakt's real 400 still ends the session.episode_type(series_premiere,season_premiere), falling back to episode 1 when a row has no type. A season numbered on from the last one, such as an anime whose second season starts at episode 62, now counts. I mutation-checked the filter both ways. Calendar entries carry the type, so the persisted cache buster is bumped.TRAKT_API_BASEis no longer exported fromoauth.tsandclient.ts, since only tests imported it.Rate limits: product issues to fix next, not in this PR
With Trakt's documented limit of 500 GETs per 5 minutes enforced, the heavy library in
scenario-budgetgoes past it in all three launch scenarios (counts from the review):last_activitiestwice.scenario-budgetnow waits out each pause the way a person would, instead of failing to settle.Net lines
packages/core/src)packages/*/test,__tests__), including the fakescripts/mock-trakt,ci.yml,knip.json,.oxlintrc.json,package.json,biome.jsonc,tsconfig.test.json)Test lines grow because the fake's seed, bodies and writes moved from
scripts/mock-traktintopackages/core/test/trakt-fake.Harness size, before and after:
scripts/mock-trakt,packages/core/test/harnessand_msw.tsbefore;packages/core/test/trakt-fakewithout its README, andpackages/core/test/harnessafter): 6,421 lines before, 5,642 after.read-budget.test.ts): about 9,250 lines before, about 7,600 after.