perf(runtime): cache local message targets with directory entries

[ReubenBond]

Problem

The fast-path destination cache from #10064 improved hosted-client throughput but coupled long-lived grain references to independently managed receiver state. That allowed stale routes and introduced the unbounded client mapping reported in #10238, leading to the revert in #10243.

Solution

Use bounded grain-directory cache entries as shared handles for local activation routing:

• Retain a shared weak handle on grain references so evicted entries, addresses, activations, silos, and service graphs remain collectible.
• Cache only exact local activation targets; remote, external-client, response, forwarding, and connection-balancing paths keep their established routing behavior.
• Coordinate address updates with target binding so an obsolete asynchronous bind cannot resurrect a stale activation.
• Invalidate targets on replacement, removal, eviction, expiry, clear, and disposal, and refresh LRU/TTL state on live handle hits.
• Add deterministic fixed-concurrency measurements with failure detection, latency percentiles, allocation/Gen0, CPU, and contention reporting.

Results

Exact-source comparisons used baseline ef3e57d313830a330b178fdef857a26e0391548d and the final local-only runtime at c774bd0bb18d5afb46446ccf78a8e5b322cac238 (current head adds tests and disposal cleanup without changing the measured steady-state path).

Hosted-client throughput improved by approximately 6–9% at concurrency 16/100/250/500 in the final symmetric pair, with lower sampled latency and unchanged steady-state allocation. Silo-to-silo throughput remained within approximately ±0.5% above concurrency 16 after remote handle retention was removed. The stable one-silo external-client control remained within about 1%.

The retained-handle primitive measured 25.45 ns versus 34.67 ns for the tuple-cache lookup and remained allocation-free. A shared entry plus weak handle allocates 128 B versus 80 B for the original tuple entry, adding 48 B per live cache entry; construction cost amortizes after roughly 28 warm local sends on this machine.

Microsoft Reviewers: Open in CodeFlow

[Copilot]

Copilot review overview

Review tier: Lite
Findings: None

What changed in this PR

This PR reintroduces a message-target fast path for Orleans runtime messaging by caching resolved send/receive targets on bounded grain-directory cache entries, ensuring invalidation/eviction/expiry naturally tears down cached routes and avoiding the unbounded client mapping regression from the earlier approach.

Changes:

• Introduces GrainDirectoryCacheEntry as an LRU-backed shared “route handle” which can also hold a cached message target (activation or connection-group entry) with address validation.
• Integrates directory-entry based fast-path selection and capture into MessageCenter routing and GrainReference handling (including interface casts), plus supporting locator/cache plumbing.
• Adds runtime tests and new deterministic benchmark scenarios (plus a small cache microbenchmark) to validate behavior and aid repeatable perf comparisons.

File	Description
test/​Orleans.Runtime.Tests/​SharedEntryMessageTargetFastPathTests.cs	New coverage for binding targets to shared directory entries (local activation, remote connection group, invalidation, casting, and exclusions).
test/​Orleans.Runtime.Tests/​Directories/​GrainDirectoryCacheFactoryTests.cs	Expanded tests validating entry handle updates, invalidation paths, eviction/expiry behavior, and message-target lifecycle semantics.
test/​Benchmarks/​Program.cs	Adds CLI entry points for deterministic ping benchmark scenarios.
test/​Benchmarks/​Ping/​AdaptivePingBenchmark.cs	Implements deterministic fixed-concurrency scenario/matrix runner and result formatting.
test/​Benchmarks/​Ping/​AdaptiveConcurrencyLoadGenerator.cs	Adds RunFixedConcurrencyAsync for repeated fixed-concurrency measurements.
test/​Benchmarks/​Caching/​GrainDirectoryCacheBenchmark.cs	New microbenchmark comparing tuple cache vs shared-entry cache access patterns.
src/​Orleans.Runtime/​Placement/​PlacementService.cs	Exposes placement strategy check needed to gate directory-entry capture.
src/​Orleans.Runtime/​Messaging/​MessageCenter.cs	Core fast-path integration: selects/captures directory entries, caches activations/connections on entries, and avoids resurrecting stale targets.
src/​Orleans.Runtime/​GrainDirectory/​LruGrainDirectoryCache.cs	Makes cache expose entries via IGrainDirectoryCacheEntrySource and hooks update/removal to clear/invalidate cached targets.
src/​Orleans.Runtime/​GrainDirectory/​LocalGrainDirectory.cs	Adds internal helper for retrieving a matching cache entry handle for a given grain + silo.
src/​Orleans.Runtime/​GrainDirectory/​IGrainDirectoryCache.cs	Adds IGrainDirectoryCacheEntrySource for retrieving shared entry handles.
src/​Orleans.Runtime/​GrainDirectory/​GrainLocator.cs	Adds dispatcher to retrieve cache entry handles across locator implementations.
src/​Orleans.Runtime/​GrainDirectory/​GrainDirectoryCacheEntry.cs	New shared entry type storing address/version plus an optional cached message target with invalidation semantics.
src/​Orleans.Runtime/​GrainDirectory/​DhtGrainLocator.cs	Plumbs cache entry handle retrieval through DHT locator.
src/​Orleans.Runtime/​GrainDirectory/​CachedGrainLocator.cs	Plumbs cache entry handle retrieval through cached locator.
src/​Orleans.Runtime/​Core/​InsideRuntimeClient.cs	Passes the target GrainReference into message addressing to enable cache entry selection/capture.
src/​Orleans.Core/​Runtime/​GrainReferenceRuntime.cs	Preserves the shared entry handle when casting between compatible grain interfaces.
src/​Orleans.Core/​Networking/​ConnectionManager.cs	Exposes/stabilizes a connection-group entry type for caching and supports safe detachment on removal.
src/​Orleans.Core/​Caching/​ConcurrentLruCache.cs	Adds extensibility hooks and retrieval helpers needed by the directory cache to manage entry lifecycle.
src/​Orleans.Core.Abstractions/​Runtime/​GrainReference.cs	Adds per-reference volatile MessageTargetCache storage and a compare-exchange clear helper.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

Code coverage

78.56% line coverage - 99,671 / 126,871 lines

Coverage details

• Source files: 1,663
• Scope: every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS
• Sources: maintained repository code under src/ loaded by the complete CI test scope
• Commit: b32774fa57
• View the test run and merged coverage artifact
• View the coverage reporting run

[Copilot]

Copilot review overview

Review tier: Lite
Findings: None

[Copilot]

Copilot review overview

🔵 Needs a closer look

Review tier: Lite
Findings: 1 · 1

New issues introduced by this change (2)

Severity	Finding
	test/​Benchmarks/​Ping/​AdaptivePingBenchmark.cs — The benchmark output labels these values as generic “latency”, but the load generator samples only…
	test/​Orleans.Runtime.Tests/​Directories/​GrainDirectoryCacheFactoryTests.cs — These GC-based assertions can be flaky under CI jitter/background threads because they only attempt…

Suppressed comments (3)

test/Benchmarks/Ping/AdaptivePingBenchmark.cs:373

• The deterministic results table header says “Latency P50/P95/P99”, but the values reported are first-request latency per work block (not per-request latency). Clarifying the column label will prevent misreading benchmark results.

        Console.WriteLine("| Scenario | Concurrency | P50 throughput | P95 throughput | P99 throughput | Latency P50/P95/P99 (us) | B/request | Gen0/M requests | CPU | Contentions/M requests |");
        Console.WriteLine("|----------|------------:|---------------:|---------------:|---------------:|-------------------------:|----------:|----------------:|----:|-----------------------:|");

test/Orleans.Runtime.Tests/Directories/GrainDirectoryCacheFactoryTests.cs:856

• These GC-based assertions can be flaky under CI jitter/background threads because they only attempt collection 3 times and use the default GC.Collect overload. Consider using forced, compacting collections and a few more attempts to reduce intermittent failures.

        for (var attempt = 0; attempt < 3; attempt++)
        {
            GC.Collect();
            GC.WaitForPendingFinalizers();
            GC.Collect();

test/Orleans.Runtime.Tests/Directories/GrainDirectoryCacheFactoryTests.cs:887

• CollectGarbage() uses 3 default GC.Collect calls, which may be insufficient for deterministic weak-reference assertions on busy CI machines. Using forced, compacting collections can make these retention tests more reliable.

        for (var attempt = 0; attempt < 3; attempt++)
        {
            GC.Collect();
            GC.WaitForPendingFinalizers();
            GC.Collect();

[Copilot]

Copilot review overview

🔵 Needs a closer look

Review tier: Lite
Findings: None

Issues resolved since last review (2)

Severity	Finding
	test/​Orleans.Runtime.Tests/​Directories/​GrainDirectoryCacheFactoryTests.cs — These GC-based assertions can be flaky under CI jitter/background threads because they only attempt… View resolved comment
	test/​Benchmarks/​Ping/​AdaptivePingBenchmark.cs — The benchmark output labels these values as generic “latency”, but the load generator samples only… View resolved comment