Improve load test quality and expand scenario coverage #165

Summary

TournamentAPI.LoadTests has 4 tests in 3 classes. All of them send the same anonymous read query (GetAllWithBracketAndMatches). Two cover rate limiting and two cover latency. There are no writes, no auth flows, no steps and no runtime thresholds, and one test appears to exercise the wrong limiter. This issue tracks fixing the existing tests and adding the missing coverage.

Problems in the current tests

1. ConcurrencyLimiter_Should_LimitConcurrentRequests most likely tests the token bucket, not the concurrency limiter.
   • The Task.Delay(1000) runs after the response returns, so it does not hold a concurrency permit. Requests are in flight for milliseconds and never approach the limit of 100.
   • The 429s come from the per-IP token bucket (100 tokens, 50 per minute), which LoadTestWebAppFactory still enables.
   • The delay also counts toward measured latency.
2. Test classes run in parallel. There is no xunit.runner.json, [Collection] or DisableTestParallelization, so the classes compete for CPU and make the p95 assertions flaky.
3. Responses carry no status code or payload size. Response.Ok() and Response.Fail() are called bare, so Ok.StatusCodes is empty and DataTransfer is zero. Status-mix and payload thresholds cannot work until this changes.
4. A 200 with the wrong body counts as success. Nothing checks that edges is non-empty, and HasErrors is the only failure signal.
5. There are no steps or thresholds. Everything is a scenario-level Assert after the run, so nothing is reported per business operation.
6. Api_ShouldDegradeGracefully_AsLoadIncreases asserts one p95 aggregated over three phases (5, 15 and 20 copies), so the top phase is diluted by the easier ones.
7. Smaller issues:
   • TestClient deserializes the full JSON inside the timed path.
   • TestClient._lastResponse is a shared mutable field, and its HttpResponseMessage is never disposed.
   • The 429 handling is copy-pasted between two tests.
   • azure-sql-edge:latest is unpinned, and Azure SQL Edge may be retired. Worth checking.
   • The seed data is small (8 users, 17 tournaments), so N+1 and pagination costs will not show. TournamentAPI.Benchmarks/BenchmarkDatabaseSeeder.cs already has Small, Medium and Large sizes.
   • The tests run in-process on TestServer, so numbers are relative, not absolute. This may deserve an ADR.

Requirements

HttpClient reuse

Already met for the current read-only tests: each test creates one client and closes over it. Two things break once auth is added:

• SetAuthToken mutates the shared DefaultRequestHeaders. Use one client per virtual user, created once in WithInit and kept in ScenarioInstanceData.
• WebApplicationFactory clients handle cookies by default, so the refreshToken cookie would be shared across every virtual user. Set HandleCookies = false.

Setup outside the measured path

• Log in once per user in WithInit and reuse the tokens. Identity password hashing is too expensive to run per iteration.
• Write scenarios need tournaments, users and slots created up front, with ids derived from the invocation number.
• Mutation tests dirty the class-shared DB, so give them their own fixture or reseed in init.

Steps named after business operations

Use Step.Run("list_tournaments", context, ...), view_tournament_details, join_tournament, login and so on. Scenario names should describe the workload, for example browse_tournaments.

Threshold types (NBomber 6.4.1)

Runtime thresholds are checked once per reporting interval (5s by default). For the 2 to 5 second rate-limit tests, assert result.Thresholds.All(t => !t.IsFailed) after the run, plus explicit asserts.

Proposed scenarios, in priority order

1. Browse mix (list, by-id, participants, name filter, sort) as concurrent weighted scenarios. This is the baseline the others compare against.
2. Expensive shapes:
   • Owner and participant history (wonTournaments, playedTournaments, wonMatches), which is the known N-per-user gap.
   • Standings on the round-robin branch.
   • The excessive-page-size query, as a payload guard that should be rejected quickly.
3. Join contention on "Almost Full Tournament", which has exactly one slot. Expect one success, the rest domain rejections and no 500s. This exercises the optimistic concurrency path.
4. Auth flow: login, me, refresh and logout, with per-user clients.
5. Concurrent match play or correction (Timestamp and version tokens), plus bracket generation at larger participant counts for both strategies.
6. Spike (a burst on top of normal load), then soak (10 to 15 minutes, watching for leaks and connection-pool exhaustion), then breakpoint (ramp until the error budget breaks).
7. A corrected concurrency-limiter test with the token bucket off and permits actually held. Optionally add limiter recovery after the token bucket refills.
8. Medium and Large dataset variants of 1 and 2.

Suggested structure

• A shared GraphQLStep helper that runs one operation and returns a Response with status code, byte size and a body check, so every step gets the status-mix and payload thresholds.
• A shared thresholds builder, so error budget, tail latency and step health are defined once.
• Feature-named classes such as TournamentBrowseLoadTests and TournamentJoinLoadTests, tagged [Trait("Category","Load")] so they can be excluded from normal runs.

Tasks

• Serialize the load test classes (collection or assembly-level setting)
• Fix the concurrency-limiter test (separate factory with the token bucket off, permits held)
• Add GraphQLStep helper (status code, size, body check) and the thresholds builder
• Convert existing tests to named steps with thresholds for all five types
• Split the ramp test so each phase is checked on its own
• Add per-user client setup and login in WithInit
• Add scenarios 1 to 6 above, one at a time
• Add Medium and Large dataset seeding (reuse or move BenchmarkDatabaseSeeder)
• Decide whether the in-process vs Kestrel choice needs an ADR

Definition of done

• dotnet build succeeds, and the unit and integration tests still pass.
• Each new scenario has a named step per business operation and thresholds for the applicable types.
• The load tests do not run in parallel with each other.