perf(gateway): release idle World Sessions from blocking workers

docs/accept-policy.md

@@ -21,5 +21,9 @@ a non-stream socket, cannot apply to a `TcpListener` created by `bind`. `EPERM`
 refusal for one connection and also retries. `EMFILE` reports descriptor exhaustion and must leave
 existing sessions alive while capacity recovers.
 
-A shared non-waiting capacity limit keeps accepted sockets out of Tokio's unbounded blocking-task
-queue. Holding the permit for the connection's work makes the limit cover the resource it protects.
+Logon handshakes use a non-waiting permit limit before entering Tokio's blocking executor.
+World sockets use asynchronous tasks and a separate accepted-connection limit,
+`LYRACORE_MAX_WORLD_CONNECTIONS`. Each World Session submits at most one game action at a time
+through the bounded worker pool shared with logon. Relay encoding has its own bounded pool.
+Permits cover submitted and running work and return on completion or unwinding. Idle World Sessions
+and authenticated queue waits hold no worker permit.

docs/architecture.md

@@ -311,58 +311,62 @@ gateway's environment. Omit one and you get a **working-looking single-database
 | `LYRACORE_AOI` | AOI-scoped subscriptions | **on** (`=0` disables) | — |
 | `LYRACORE_QUEST_LOG` | quest-log descriptor fields | **on** | — |
 | `LYRACORE_MAX_SESSIONS` / `LYRACORE_ADMIT_CONCURRENCY` | Established World Sessions / queued admissions per tick, per Gateway process. Divide the intended realm-wide limit by the Gateway count. | Unset or `0` = unlimited | Malformed values stop serving startup before Coordinator connection or listener binding. Surrounding whitespace is allowed; signs, separators, empty values and overflow are refused. Provision mode does not use these limits. |
-| `LYRACORE_MAX_BLOCKING_THREADS` | tokio blocking-pool cap — **the real ceiling on concurrent players** (see below) | `512` (tokio's own default) | malformed **or zero** falls back to `512`; the resolved value is logged at startup |
+| `LYRACORE_MAX_BLOCKING_THREADS` | Blocking game actions, logon handshakes and Relay encoding | `512` | Malformed or zero falls back to `512`; startup logs the resolved value. |
+| `LYRACORE_MAX_WORLD_CONNECTIONS` | All accepted World sockets, including pre-auth and queued clients | `16384` | Malformed or zero falls back to `16384`; excess sockets close at accept. |
 | `LYRACORE_LOAD_SAMPLE_SECS` | shard load sampling cadence | `30` | — |
 | `LYRACORE_METRICS_DB_IDS` | `<shard>=<hex-identity-prefix>` map for `/v1/metrics` | `""` | warns loudly at startup; occupancy unmeasured |
 | `LYRACORE_WRITER_TRACE` | per-session writer black-box ring | off | — |
 | `LYRACORE_TRANSFER_ABORT_AFTER` | crash-injection harness (aborts a named transfer step) | unset | an unknown step name logs an error and nothing fires |
 | `LYRACORE_PROFILE_SECS` | `--features dhat-heap` builds only | `120` | — |
 
-Budget three client socket descriptors per World Session: reader, writer and ownership-loss
-shutdown handle. Add the database sockets for its subscribed Shards and reserve headroom for
-listeners, handshakes and Coordinator connections when setting `LYRACORE_MAX_SESSIONS`.
+Each World Session uses one client socket descriptor. Add the shared Coordinator and call-pipe
+sockets, listeners and logon handshakes when setting descriptor limits.
 
-Two non-`LYRACORE_` variables matter: `RUST_LOG` (consumed by `env_logger` in `gateway/src/main.rs`)
-and `MALLOC_ARENA_MAX` (glibc, not the binary — worth ~4× RSS per connection).
+`RUST_LOG` controls logging through `env_logger`. `MALLOC_ARENA_MAX` controls glibc allocation arenas.
 
-### `LYRACORE_MAX_BLOCKING_THREADS`, and the two limits above it (#451)
+### World Session concurrency and limits
 
-Both listeners hand admitted sockets to `spawn_blocking` because the `wow_login_messages` and
-`wow_world_messages` codecs use blocking `std::io`. A **World Session holds its blocking thread for
-the session's entire life**. So the blocking-pool cap is the seat count shared between live World
-Sessions and in-flight logon handshakes. It was Tokio's inherited default of 512 until #451,
-configured by nothing and reported by nothing.
+World socket reads, writes and authenticated login-queue waits use asynchronous tasks. An idle
+World Session holds no blocking worker. Each session submits one game action at a time to a bounded
+pool shared with logon handshakes. Store interfaces remain synchronous, including Durable Request
+completion waits. The pool bounds those active waits independently of connected player count.
 
-That mattered because a full pool does not refuse: `spawn_blocking` **queues**. Before #410, a
-socket could be accepted and then wait without a bound for a blocking thread, retaining its file
-descriptor and sitting ahead of later logins. Both listeners now share a non-waiting capacity with
-the same size as the pool. A connection that cannot take a permit is closed before its task is
-submitted, and each task holds its permit through normal return or unwinding. This bounds submitted
-plus running connection tasks at the configured pool size. Measured 2026-08-07 on an 8-core/15 GB
-container, 600 clients offered:
+Relay encoding has a separate permit pool, capped at eight workers and one quarter of
+`LYRACORE_MAX_BLOCKING_THREADS`, with a minimum of one. Both pools use Tokio's blocking executor.
+This reserves encoding capacity when game actions await database completion. A one-thread executor
+still serializes both kinds of work. No capacity measurement for this implementation is recorded yet.
 
-| `max_blocking_threads` | seated | failed |
-|---|---|---|
-| 512 (tokio default) | 477 | 123 |
-| 4096 | 535 | 65 |
+The World listener bounds accepted sockets independently through `LYRACORE_MAX_WORLD_CONNECTIONS`.
+`LYRACORE_MAX_SESSIONS` controls admitted players and the authenticated queue. Neither value is a
+measured player capacity. Logon still refuses a new handshake when it cannot obtain a worker permit.
+
+Pre-auth has a total deadline of ten seconds for logon and fifteen seconds for World. The World budget
+includes socket reads, writes and the session lookup in one asynchronous timeout, then ends before the
+authenticated queue. Queue writes have their own timeout. A queue seat returns on cancellation,
+including when admission races with cancellation.
 
-477 rather than a clean 512 because logon handshakes draw from the same pool. Raising it is worth
-about 58 sessions on that host. (The next ceiling this section used to name — per-player
-connection setup — is gone with the per-player connections themselves, #483.)
+Each World Session has a non-waiting outbound queue capped at 2,048 entries and a 4 MiB reservation
+budget. Raw frames reserve their wire size; each typed frame and deferred Relay job reserves 4 KiB until
+encoding. Compression stays on encoding workers. Each encoded projection and combined write batch is
+also limited to 4 MiB.
+These are queue and encoded-output limits, not an exact bound on arbitrary captured Rust objects.
+A queue entry has ten seconds to finish encoding and writing. Exceeding a limit closes the session;
+critical packets are not silently discarded. Movement keeps its existing coalescing and shedding.
 
-Each admitted socket also starts one absolute pre-auth I/O deadline at accept: 10 seconds for
-logon and 15 seconds for world. Blocking reads and writes receive only the remaining time. An
-async watchdog closes the socket at the same instant even when its task has not started. Successful
-proof cancels the watchdog and clears both socket timeouts. World does this before the login queue,
-so an authenticated queue wait does not spend the pre-auth budget.
+One writer owns each session's header cipher and preserves FIFO order and contiguous entry batches.
+It combines frames into writes of about 64 KiB.
+A supervisor retains cleanup ownership when its caller is cancelled. Accepted game actions finish
+before logout cleanup, so a delayed login cannot recreate a departed session after cleanup. A clean
+read EOF drains direct replies from accepted actions within the outbound age budget before returning
+the admission seat. Cleanup invalidates queued viewer Relays, as it does during Transfer.
 
 Two related startup behaviours have no environment variable, because there is no case for turning
 them off:
 
 - **`RLIMIT_NOFILE` soft is raised to hard** before the runtime is built (`gateway/src/fd_limit.rs`).
   A stock Docker container is soft 1024 / hard 524288, and #447 died at ~200 sessions with
-  `Too many open files (os error 24)` against 512× of unclaimed headroom. A live session costs 3–5
-  descriptors. Failure logs and continues — it never blocks startup.
+  `Too many open files (os error 24)` against 512× of unclaimed headroom. World Sessions now use one
+  client descriptor each. Failure logs and continues — it never blocks startup.
 - **A transient `accept(2)` errno costs one connection, not the realm** (`gateway/src/accept.rs`).
   `EMFILE`, `ENFILE`, `ECONNABORTED`, `EINTR`, `ENOBUFS`, `ENOMEM` and every unenumerated errno are
   logged and skipped, with a backoff that settles at one attempt per second under sustained

gateway/src/accept.rs

@@ -26,6 +26,19 @@ impl BlockingTaskCapacity {
             .map(|permit| BlockingTaskPermit { _permit: permit })
     }
 
+    pub(crate) async fn run<F, T>(&self, work: F) -> anyhow::Result<T>
+    where
+        F: FnOnce() -> T + Send + 'static,
+        T: Send + 'static,
+    {
+        let permit = self.permits.clone().acquire_owned().await?;
+        Ok(tokio::task::spawn_blocking(move || {
+            let _permit = permit;
+            work()
+        })
+        .await?)
+    }
+
     #[cfg(test)]
     pub(crate) fn available(&self) -> usize {
         self.permits.available_permits()

gateway/src/config.rs

@@ -48,11 +48,27 @@ impl GatewayConfig {
             module_name: get("LYRACORE_DATABASE", "lyracore"),
             coordinator_token: std::env::var("LYRACORE_COORDINATOR_TOKEN").ok(),
             gateway_id: get("LYRACORE_GATEWAY_ID", &default_gateway_id),
-            blocking_task_capacity: BlockingTaskCapacity::new(max_blocking_threads()),
+            blocking_task_capacity: BlockingTaskCapacity::new(
+                max_blocking_threads()
+                    .saturating_sub(encoding_workers())
+                    .max(1),
+            ),
         }
     }
 }
 
+pub(crate) fn encoding_workers() -> usize {
+    (max_blocking_threads() / 4).clamp(1, 8)
+}
+
+pub(crate) fn world_connection_limit() -> usize {
+    std::env::var("LYRACORE_MAX_WORLD_CONNECTIONS")
+        .ok()
+        .and_then(|value| value.parse::<usize>().ok())
+        .filter(|limit| *limit > 0 && *limit <= tokio::sync::Semaphore::MAX_PERMITS)
+        .unwrap_or(16_384)
+}
+
 pub const INSTANCE_BUCKETS: u64 = 8;
 
 #[derive(Clone, Debug, PartialEq)]
@@ -431,7 +447,7 @@ pub fn max_blocking_threads() -> usize {
 pub fn max_blocking_threads_from_env(raw: Option<&str>) -> usize {
     raw.map(str::trim)
         .and_then(|v| v.parse::<usize>().ok())
-        .filter(|&n| n > 0)
+        .filter(|&n| n > 0 && n <= tokio::sync::Semaphore::MAX_PERMITS)
         .unwrap_or(DEFAULT_MAX_BLOCKING_THREADS)
 }
 

gateway/src/main.rs

@@ -32,9 +32,8 @@ fn main() -> Result<()> {
     let max_blocking_threads = config::max_blocking_threads();
     log::info!(
         "tokio blocking pool: max_blocking_threads={max_blocking_threads} \
-         (LYRACORE_MAX_BLOCKING_THREADS; default {}). One world session holds one thread for its \
-         entire life, shared with in-flight logon handshakes. Excess connection tasks are rejected \
-         before the blocking-task queue.",
+         (LYRACORE_MAX_BLOCKING_THREADS; default {}). World Session socket I/O is asynchronous. \
+         Game actions, Relay encoding and logon handshakes use bounded blocking work.",
         config::DEFAULT_MAX_BLOCKING_THREADS
     );