Skip to content

Time-started subscriptions on EF Core can skip events (append-time vs commit-order) #4

Open

Description

@fw2568

Summary

On the EF Core store, a time-started subscription (StartingAtAsync → FindPositionAtAsync) can silently skip events. It resolves a start position from a wall-clock time, but CreatedAt (append time) and CommitSequence ($all / commit order) are deliberately decoupled, so the computed boundary is not a clean cut and events appended after the requested time can fall behind it.

This is already called out as a limitation in docs/reading-and-subscriptions.md ("Start by position where every event counts"); this issue tracks whether to close the gap so time-based starts are complete, rather than best-effort.

Where

FindPositionAtAsync in src/Dbosoft.EventStore.EfCore/EfCoreEventStore.cs:

var before = await ctx.Events.AsNoTracking()
    .Where(e => e.Domain == domain && e.CreatedAt < timestamp && e.CommitSequence != null)
    .OrderByDescending(e => e.CommitSequence)
    .Select(e => e.CommitSequence)
    .FirstOrDefaultAsync(ct);
return before is { } sequence ? GlobalPosition.FromSequence(sequence) : GlobalPosition.Start;

It returns the highest CommitSequence among rows with CreatedAt < timestamp; the subscription then reads CommitSequence > that value.

Why it skips

CommitSequence is taken as the last step before commit, so an append enlisted in a long-running caller transaction is inserted early (CreatedAt) but committed late (high CommitSequence). The two orders diverge by design.

Scenario (one domain):

  • Event X: appended 10:00 (before T), transaction commits 10:05 → CommitSequence = 100.
  • Event Z: appended 10:05 (after T), commits immediately → CommitSequence = 50.

FindPositionAtAsync(10:03):

  • Rows with CreatedAt < 10:03: only X. Max CommitSequence = 100 → start reading at > 100.
  • Z has CommitSequence = 50 < 100, so Z is never delivered — even though Z was appended at 10:05, i.e. after the requested start time.

A consumer asking for "everything from 10:03 onward" silently loses Z.

Impact

  • Only affects time-based starts (StartingAtAsync). Position-based starts (GlobalPosition) are unaffected and already recommended for completeness.
  • Severity depends on how long append-to-commit windows get in practice (enlisted message handlers that call other systems are the realistic worst case).

Possible resolution

Mirror the Azure Tables store's safety window: for time-based starts, don't resolve a boundary newer than the longest expected open-transaction window, so a not-yet-committed lower sibling is never passed over. Cost is delivery latency. Alternatively, keep it as a documented best-effort and lean harder on the "use position, not time" guidance.

No code change proposed here — this is a design decision to track.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions