Replace Redis queue LIST with unique ZSET pending work

[danielgerlag]

Describe the change
RedisQueueProvider now stores pending work in a Redis sorted set (ZSET) instead of a LIST. Uniqueness is ZADD NX (at most one pending member per id). FIFO is the sort score. Dequeue is atomic Lua ZRANGE + ZREM (a ZPOPMIN equivalent). IQueueProvider is unchanged; this is Redis-provider only.

This is a single coherent Redis queue algorithm. It replaces the LIST + LINSERT/RPUSH/LREM path (including the atomic Lua LIST enqueue in #1446). Do not land both.

Issue #1417 / QueueConsumer._secondPasses is not addressed here.

Describe your implementation or design
Choice: same key names + in-place LIST→ZSET migrate on Start() (option B). No new {prefix}-*-z keys, no config flag, no dual-read/dual-write.

Key	Type	Role
{prefix}-workflows	ZSET	pending workflow ids
{prefix}-events	ZSET	pending event ids
{prefix}-index	ZSET	pending index ids

• Enqueue: one Lua script, ZADD KEYS[1] NX <Redis TIME µs> id. Re-queue while pending is a no-op and keeps the existing score/position. After pop, the same id may be ZADD again with a new score.
• Score: Redis TIME (seconds * 1e6 + microseconds), not client clocks. No app-node clock skew. Same-microsecond ties are lexicographic by member. Single-key script, so Redis Cluster safe (an INCR seq key in the same EVAL would need hash tags).
• Dequeue: Lua ZRANGE 0 0 + ZREM. Equivalent to ZPOPMIN without a Redis 5+ floor.
• Cutover: Start() runs a one-time Lua migrate when the key is still a LIST: LRANGE → DEL → ZADD NX with scores 1..n (first occurrence kept). Migrated items dequeue before later TIME-scored enqueues.

Rejected: new key names (silent work-split during rolling upgrade) and a long dual-read window.

Tests
RedisQueueProviderFixture (existing Squadron Redis collection; WORKFLOWCORE_REDIS can point at a running instance):

• Concurrent QueueWork of a missing id leaves exactly one pending member
• Re-QueueWork while pending is a no-op and preserves FIFO position
• Distinct ids dequeue in enqueue order, then null
• Dequeue then re-enqueue of the same id succeeds (new score / tail position)
• Concurrent dequeue returns each unique id once and drains the ZSET
• Start() migrates a legacy LIST (dedupes, keeps FIFO, new enqueues go behind)
• Start() leaves an existing ZSET untouched

Local: 10/10 passed against Redis 7.0.15 (WORKFLOWCORE_REDIS=localhost:6379, dotnet test … --filter FullyQualifiedName~RedisQueueProviderFixture).

CI (02672eb): Redis-Tests passed. Unit-Tests, Mongo/MySQL/PostgreSQL/SQLServer/Oracle/Azure/Elasticsearch, and AppVeyor passed. Integration-Tests failed on RetrySagaWithUserTaskScenario (Dictionary enumeration race in GetOpenUserActions while polling). That suite does not use Redis; the same test passed on another TFM in the same job. Unrelated flake — rerun Integration-Tests; no ZSET change needed.

Breaking change
Yes — Redis value type on the existing queue keys changes from LIST to ZSET. See Compatibility.

Compatibility

• LIST vs ZSET on the same key: Redis cannot store both types on {prefix}-workflows|events|index. A ZADD against a leftover LIST (or LPUSH/LPOP against a ZSET) is WRONGTYPE.
• Rolling upgrades with mixed old/new providers: Not supported. Old processes speak LIST; new processes speak ZSET. Mixed processes on the same keys will WRONGTYPE or ignore each other's pending work.
• What operators need to do: Coordinated cutover. Stop every host that uses UseRedisQueues, deploy this build, start hosts. Start() migrates leftover LIST items in place (first occurrence kept). A flush is not required unless you choose to drop in-flight work (DEL those three keys) instead of migrating.
• Optional flush: only if you want an empty queue rather than migrated pending ids.
• Score semantics: Redis server TIME in microseconds. Not client DateTime. No app-node clock skew. Equal scores (same µs) order lexicographically by id. After ZPOPMIN-equivalent pop, a later ZADD of the same id gets a new (later) score.
• Re-QueueWork while pending: ZADD NX no-op — existing score and position are preserved.
• After dequeue: the same id may be queued again with a new score (goes to the tail).
• Redis version floor: Lua + ZSET (Redis 2.6+). Native ZPOPMIN needs Redis 5+; this provider does not call it, so Redis 5+ is not required. The previous LIST path already needed Lua (2.6+).
• IQueueProvider / other providers: unchanged.

Additional context
Conflicts with #1446 (LIST Lua uniqueness). Prefer this ZSET implementation if both are considered.

Queue key names are unchanged: {prefix}-workflows, {prefix}-events, {prefix}-index.