feat(event-sourcing): add journaled state log consistency provider

docs/site/src/content/docs/grains/event-sourcing/event-sourcing-configuration.md

@@ -1,7 +1,7 @@
 ---
 title: Event sourcing configuration
 description: Configure JournaledGrain log consistency and storage in Orleans.
-ms.date: 08/02/2026
+ms.date: 08/23/2026
 ms.topic: how-to
 ---
 
@@ -20,6 +20,7 @@ Available registration methods are:
 - <xref:Orleans.Hosting.StateStorageSiloBuilderExtensions.AddStateStorageBasedLogConsistencyProvider*>
 - <xref:Orleans.Hosting.LogStorageSiloBuilderExtensions.AddLogStorageBasedLogConsistencyProvider*>
 - <xref:Orleans.Hosting.CustomStorageSiloBuilderExtensions.AddCustomStorageBasedLogConsistencyProvider*>
+- <xref:Orleans.Hosting.JournaledStateSiloBuilderExtensions.AddJournaledStateBasedLogConsistencyProvider*>
 
 Each also has an `AsDefault` form. If a default log-consistency provider and default grain storage provider are registered, provider attributes can be omitted.
 
@@ -35,6 +36,16 @@ Custom storage doesn't use <xref:Orleans.Storage.IGrainStorage>. The grain imple
 
 :::code language="csharp" source="../../snippets/compiled/EventSourcing/EventSourcingSnippets.cs" id="custom_storage_grain":::
 
+## Journaled-state provider
+
+The journaled-state provider stores the event log in the same Orleans journal as the activation's other durable states. One write atomically publishes the captured event log, write marker, and auxiliary durable state. Snapshot replacement failures leave the previous journal generation published and retain the captured changes for retry.
+
+This provider runs on a single turn-serialized grain activation. Grain types configured for reentrancy, selective interleaving, always-interleaved methods, or stateless-worker placement are rejected during activation.
+
+The provider's persisted write marker supports a versioned length-prefixed encoding, so every valid `ClusterId`, including identifiers containing commas and punctuation, has an exact identity. Existing comma-token markers remain readable and continue using that representation while all cluster identifiers are delimiter-safe, preserving rolling upgrades with previous Orleans versions.
+
+Complete the Orleans rolling upgrade before configuring a comma-containing `ClusterId`. The first write from that cluster upgrades the marker to the versioned representation, which previous Orleans versions don't understand. After that upgrade, rollback requires restoring a journal generation written with the legacy marker format.
+
 ## Multi-cluster responsibility
 
 Custom storage owns the write-topology rules needed by a multi-cluster deployment. The `primaryCluster` registration argument is retained by the provider but doesn't restrict submissions, configure Orleans multi-cluster networking, replicate storage, or provide failover. Enforce any single-writer or regional-write rule in the application and storage implementation.

docs/site/src/content/docs/grains/journaling/runtime-behavior.md

@@ -1,7 +1,7 @@
 ---
 title: Journaling runtime behavior and consistency
 description: Understand Orleans Journaling activation, write, recovery, compaction, concurrency, and failure semantics.
-ms.date: 08/21/2026
+ms.date: 08/23/2026
 ms.topic: conceptual
 ---
 
@@ -49,17 +49,22 @@ Design commands to tolerate retries at the application boundary. Use operation i
 
 A normal append failure leaves encoded pending entries available for a later write attempt.
 
-A compaction write has two storage stages: it first appends committed pending entries, then publishes a snapshot replacement. The append can succeed before the replacement fails. In that outcome, <xref:Orleans.Journaling.DurableGrain.WriteStateAsync*> faults even though the state mutation is durable in the append history. Treat every failed write as an uncertain application outcome and retry commands using an operation identifier or another idempotency mechanism.
+A snapshot failure leaves the previously published journal unchanged and keeps the captured in-memory changes available for a later write attempt. Commands added while the replacement is awaiting storage remain outside the captured snapshot and are persisted by a later operation.
+
+An optimistic-concurrency conflict identifies a competing journal generation. The manager recovers that winning generation and discards the losing activation's uncommitted in-memory changes before reporting the conflict.
+
+Storage acknowledgement can still have an uncertain network outcome. Treat a failed write as an uncertain application outcome and retry commands using an operation identifier or another idempotency mechanism.
 
 Recovery exceptions fault activation or queued work rather than replacing or truncating stored data. Restore the required format/codec registration or repair the backing data before retrying activation.
 
 ## Compaction
 
 Each provider reports when its journal crosses a configured storage threshold. The next <xref:Orleans.Journaling.DurableGrain.WriteStateAsync*>:
 
-1. Persists any already-buffered append data.
-1. Builds a snapshot containing the state directory and every active durable state.
-1. Atomically replaces the published journal with the snapshot.
+1. Captures the pending journal prefix and builds a snapshot containing the state directory and every active durable state.
+1. Atomically replaces the published journal with the complete snapshot.
+1. Consumes the captured pending prefix after storage acknowledges the replacement.
+1. Leaves commands added during the replacement pending for the next write.
 1. Clears the compaction request after storage acknowledges the replacement.
 
 Compaction bounds replay work and storage growth according to provider thresholds. Snapshot size still scales with the complete durable state owned by the grain, so capacity tests must include hot and large grain identities.

src/Orleans.EventSourcing/Common/StringEncodedWriteVector.cs

@@ -1,52 +1,157 @@
-namespace Orleans.EventSourcing.Common
+using System.Globalization;
+using System.Text;
+
+namespace Orleans.EventSourcing.Common;
+
+/// <summary>
+/// Encodes a set of replica write bits in a string.
+/// </summary>
+/// <remarks>
+/// New values use the versioned format <c>v1:</c> followed by UTF-16 length-prefixed replica identifiers.
+/// This representation supports every valid cluster identifier without delimiter restrictions. Legacy values
+/// containing comma-prefixed tokens remain readable. Updates remain in the legacy format while every identifier
+/// is representable by it, preserving rolling-upgrade compatibility. A comma-containing identifier upgrades the
+/// value to the current format.
+/// In legacy values, commas are interpreted as token delimiters because the previous format did not escape them.
+/// Malformed or unsupported versioned values throw <see cref="FormatException"/>.
+/// </remarks>
+public static class StringEncodedWriteVector
 {
-    public static class StringEncodedWriteVector
+    private const string CurrentFormatPrefix = "v1:";
+
+    /// <summary>
+    /// Gets one of the bits in <paramref name="writeVector"/>.
+    /// </summary>
+    /// <param name="writeVector">The write vector.</param>
+    /// <param name="Replica">The replica whose bit is returned.</param>
+    /// <returns><see langword="true"/> when the replica's bit is set.</returns>
+    public static bool GetBit(string writeVector, string Replica)
     {
+        ArgumentNullException.ThrowIfNull(writeVector);
+        ArgumentException.ThrowIfNullOrEmpty(Replica);
+        return Decode(writeVector, out _).Contains(Replica, StringComparer.Ordinal);
+    }
 
-        // BitVector of replicas is implemented as a set of replica strings encoded within a string
-        // The bitvector is represented as the set of replica ids whose bit is 1
-        // This set is written as a string that contains the replica ids preceded by a comma each
-        //
-        // Assuming our replicas are named A, B, and BB, then
-        // ""     represents    {}        represents 000 
-        // ",A"   represents    {A}       represents 100 
-        // ",A,B" represents    {A,B}     represents 110 
-        // ",BB,A,B" represents {A,B,BB}  represents 111 
-
-        /// <summary>
-        /// Gets one of the bits in writeVector
-        /// </summary>
-        /// <param name="writeVector">The write vector which we want get the bit from</param>
-        /// <param name="Replica">The replica for which we want to look up the bit</param>
-        /// <returns></returns>
-        public static bool GetBit(string writeVector, string Replica)
-        {
-            var pos = writeVector.IndexOf(Replica);
-            return pos != -1 && writeVector[pos - 1] == ',';
-        }
-
-        /// <summary>
-        /// toggle one of the bits in writeVector and return the new value.
-        /// </summary>
-        /// <param name="writeVector">The write vector in which we want to flip the bit</param>
-        /// <param name="Replica">The replica for which we want to flip the bit</param>
-        /// <returns>the state of the bit after flipping it</returns>
-        public static bool FlipBit(ref string writeVector, string Replica)
-        {
-            var pos = writeVector.IndexOf(Replica);
-            if (pos != -1 && writeVector[pos - 1] == ',')
+    /// <summary>
+    /// Toggles one of the bits in <paramref name="writeVector"/>.
+    /// </summary>
+    /// <param name="writeVector">The write vector.</param>
+    /// <param name="Replica">The replica whose bit is toggled.</param>
+    /// <returns>The bit value after it is toggled.</returns>
+    public static bool FlipBit(ref string writeVector, string Replica)
+    {
+        ArgumentNullException.ThrowIfNull(writeVector);
+        ArgumentException.ThrowIfNullOrEmpty(Replica);
+
+        var replicas = Decode(writeVector, out var isLegacy);
+        var removed = false;
+        for (var index = replicas.Count - 1; index >= 0; index--)
+        {
+            if (string.Equals(replicas[index], Replica, StringComparison.Ordinal))
             {
-                var pos2 = writeVector.IndexOf(',', pos + 1);
-                if (pos2 == -1)
-                    pos2 = writeVector.Length;
-                writeVector = writeVector.Remove(pos - 1, pos2 - pos + 1);
-                return false;
+                replicas.RemoveAt(index);
+                removed = true;
             }
-            else
+        }
+
+        if (!removed)
+        {
+            replicas.Add(Replica);
+        }
+
+        writeVector = isLegacy && replicas.All(static replica => !replica.Contains(','))
+            ? EncodeLegacy(replicas)
+            : EncodeCurrent(replicas);
+        return !removed;
+    }
+
+    private static List<string> Decode(string writeVector, out bool isLegacy)
+    {
+        if (writeVector.Length == 0)
+        {
+            isLegacy = true;
+            return [];
+        }
+
+        if (!writeVector.StartsWith(CurrentFormatPrefix, StringComparison.Ordinal))
+        {
+            isLegacy = true;
+            return DecodeLegacy(writeVector);
+        }
+
+        isLegacy = false;
+        var result = new List<string>();
+        var position = CurrentFormatPrefix.Length;
+        if (position == writeVector.Length)
+        {
+            throw new FormatException("The versioned write vector does not contain any replica identifiers.");
+        }
+
+        while (position < writeVector.Length)
+        {
+            var separator = writeVector.IndexOf(':', position);
+            if (separator < 0
+                || separator == position
+                || !int.TryParse(
+                    writeVector.AsSpan(position, separator - position),
+                    NumberStyles.None,
+                    CultureInfo.InvariantCulture,
+                    out var length)
+                || length <= 0
+                || length > writeVector.Length - separator - 1)
             {
-                writeVector = string.Format(",{0}{1}", Replica, writeVector);
-                return true;
+                throw new FormatException("The write vector contains an invalid length-prefixed replica identifier.");
             }
+
+            position = separator + 1;
+            result.Add(writeVector.Substring(position, length));
+            position += length;
         }
+
+        return result;
+    }
+
+    private static List<string> DecodeLegacy(string writeVector)
+    {
+        if (writeVector[0] != ',')
+        {
+            throw new FormatException("The write vector has an unsupported format.");
+        }
+
+        var tokens = writeVector[1..].Split(',');
+        if (tokens.Any(static token => token.Length == 0))
+        {
+            throw new FormatException("The legacy write vector contains an empty replica identifier.");
+        }
+
+        return [.. tokens];
+    }
+
+    private static string EncodeCurrent(List<string> replicas)
+    {
+        if (replicas.Count == 0)
+        {
+            return string.Empty;
+        }
+
+        var builder = new StringBuilder(CurrentFormatPrefix);
+        foreach (var replica in replicas)
+        {
+            builder.Append(replica.Length.ToString(CultureInfo.InvariantCulture));
+            builder.Append(':');
+            builder.Append(replica);
+        }
+
+        return builder.ToString();
+    }
+
+    private static string EncodeLegacy(List<string> replicas)
+    {
+        if (replicas.Count == 0)
+        {
+            return string.Empty;
+        }
+
+        return string.Concat(",", string.Join(',', replicas));
     }
 }

src/Orleans.EventSourcing/Hosting/JournaledStateSiloBuilderExtensions.cs

@@ -0,0 +1,57 @@
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.DependencyInjection.Extensions;
+using Orleans.EventSourcing;
+using Orleans.EventSourcing.JournaledState;
+using Orleans.Journaling;
+using Orleans.Providers;
+using Orleans.Runtime;
+
+#nullable disable
+#pragma warning disable ORLEANSEXP005
+namespace Orleans.Hosting;
+
+/// <summary>
+/// Extensions for configuring journaled-state log consistency.
+/// </summary>
+public static class JournaledStateSiloBuilderExtensions
+{
+    /// <summary>
+    /// Adds a journaled-state log consistency provider as the default consistency provider.
+    /// </summary>
+    /// <param name="builder">The silo builder.</param>
+    /// <returns>The silo builder.</returns>
+    public static ISiloBuilder AddJournaledStateBasedLogConsistencyProviderAsDefault(this ISiloBuilder builder)
+    {
+        ArgumentNullException.ThrowIfNull(builder);
+
+        return builder.AddJournaledStateBasedLogConsistencyProvider(ProviderConstants.DEFAULT_STORAGE_PROVIDER_NAME);
+    }
+
+    /// <summary>
+    /// Adds a journaled-state log consistency provider.
+    /// </summary>
+    /// <param name="builder">The silo builder.</param>
+    /// <param name="name">The provider name.</param>
+    /// <returns>The silo builder.</returns>
+    public static ISiloBuilder AddJournaledStateBasedLogConsistencyProvider(this ISiloBuilder builder, string name = "JournaledState")
+    {
+        ArgumentNullException.ThrowIfNull(builder);
+        ArgumentException.ThrowIfNullOrWhiteSpace(name);
+
+        builder.AddJournalStorage();
+        return builder.ConfigureServices(services => services.AddJournaledStateBasedLogConsistencyProvider(name));
+    }
+
+    internal static IServiceCollection AddJournaledStateBasedLogConsistencyProvider(this IServiceCollection services, string name)
+    {
+        ArgumentNullException.ThrowIfNull(services);
+        ArgumentException.ThrowIfNullOrWhiteSpace(name);
+
+        services.AddLogConsistencyProtocolServicesFactory();
+        services.TryAddSingleton<ILogViewAdaptorFactory>(
+            serviceProvider => serviceProvider.GetKeyedService<ILogViewAdaptorFactory>(ProviderConstants.DEFAULT_STORAGE_PROVIDER_NAME)!);
+        return services.AddKeyedSingleton<ILogViewAdaptorFactory, LogConsistencyProvider>(name);
+    }
+}
+
+#pragma warning restore ORLEANSEXP005

src/Orleans.EventSourcing/JournaledState/LogConsistencyProvider.cs

@@ -0,0 +1,29 @@
+using Orleans.Storage;
+#nullable disable
+namespace Orleans.EventSourcing.JournaledState;
+
+/// <summary>
+/// A log-consistency provider that stores event-sourcing events in the host grain's journaled state.
+/// </summary>
+public sealed class LogConsistencyProvider : ILogViewAdaptorFactory
+{
+    /// <inheritdoc/>
+    public bool UsesStorageProvider => false;
+
+    /// <inheritdoc/>
+    public ILogViewAdaptor<TView, TEntry> MakeLogViewAdaptor<TView, TEntry>(
+        ILogViewAdaptorHost<TView, TEntry> hostGrain,
+        TView initialState,
+        string grainTypeName,
+        IGrainStorage grainStorage,
+        ILogConsistencyProtocolServices services)
+        where TView : class, new()
+        where TEntry : class
+    {
+        ArgumentNullException.ThrowIfNull(hostGrain);
+        ArgumentNullException.ThrowIfNull(initialState);
+        ArgumentNullException.ThrowIfNull(services);
+
+        return new LogViewAdaptor<TView, TEntry>(hostGrain, initialState, services);
+    }
+}