Repository navigation

0.2 architecture: class-based backends, partitioned queues, activity handlers - #16

Merged

tcdent merged 52 commits into

mainfrom

claude/kafka-backend-exploration-elVzO

Mar 31, 2026

Commits on Mar 27, 2026
- Add Kafka stream backend architecture with split KV/stream protocols
  claude
  committed
  b8d1ee3
- Simplify to single-backend architecture with ops layer wired to all callers
  claude
  committed
  44f570d
- Add queue commit/nack semantics and retry support for Kafka resilience
  claude
  committed
  51afd25
- Concurrent task execution per worker and Kafka consumer heartbeat
  claude
  committed
  a5a6584
- Revert "Concurrent task execution per worker and Kafka consumer heartbeat"
  claude
  committed
  29685ca
- Add worker_id to Kafka client IDs for observability
  claude
  committed
  de16450
- Route activity system through ops layer with Kafka activity topic
  claude
  committed
  8af3122
- Split backends into packages with domain-specific modules
  claude
  committed
  7099247
- Go full async and rename backend methods to descriptive names
  claude
  committed
  35a6a2a
- Add Kafka integration tests and CI workflow
  claude
  committed
  84423a9
- Fix CI: use correct Kafka image tag, override addopts, fix readiness check
  claude
  committed
  a4d87cf
- Switch to apache/kafka:3.9.2 for CI Kafka service
  claude
  committed
  659f9fe
- CI: disable fail-fast, add verbose test output for debugging
  claude
  committed
  969f6f6
- Fix Kafka consumer hangs: per-topic group IDs, retry loop, faster heartbeat
  claude
  committed
  f763709
- Use manual partition assignment instead of consumer groups
  claude
  committed
  48c3e2d
- Fix force_metadata_update — use partition discovery retry loop
  claude
  committed
  b50156c
- Remove consumer group_id to avoid group coordinator hangs
  claude
  committed
  584994c
- CI: capture Kafka test output in job summary on failure
  claude
  committed
  66082e7
- CI: post Kafka test output as PR comment on failure
  claude
  committed
  918363a
- Use subscribe() with per-topic group IDs, set rebalance delay to 0
  claude
  committed
  896ecdc
- Use admin metadata for partition discovery in manual assignment
  claude
  committed
  a861c9a
- CI: use curl for PR comment instead of github-script
  claude
  committed
  1a96fed
- CI: upload test output as artifact instead of PR comment
  claude
  committed
  67922c9
- Add debug prints to queue_pop and test_push_and_pop
  claude
  committed
  86fde70
- Update uv.lock after adding kafka extra dependency
  claude
  committed
  686f3d2
- CI: emit test output as warning annotations for API access
  claude
  committed
  985d483
- Better debug output: print consumer state on timeout, filter annotations
  claude
  committed
  5f6384a
- CI: filter annotations to only show failures and debug output
  claude
  committed
  ce54be7
- CI: re-trigger after transient Docker pull failure
  claude
  committed
  e7fb281
- CI: retry after transient failures
  claude
  committed
  3f792b1
- CI: use apache/kafka:latest to avoid Docker pull issues with pinned tag
  claude
  committed
  47c0324
- CI: switch to confluentinc/cp-kafka:7.7.1 for reliable Docker pulls
  claude
  committed
  d0aa733
- CI: use docker run instead of service containers for Kafka
  claude
  committed
  92ab91c
- Add docker-compose.kafka.yml, clean up debug prints, use docker run in CI
  claude
  committed
  e4b3946
- Fix queue_pop message buffer and produce() key type handling
  
  tcdent
  and
  claude
  committed
  424d41c
- Kafka consumer groups, full async, producer-side topic creation
  
  tcdent
  and
  claude
  committed
  cabf769
- Class-based backend architecture, eliminate ops passthrough layer
  
  tcdent
  and
  claude
  committed
  a9fdbdd
- Flatten backend modules, remove dead code, clean up noise
  
  tcdent
  and
  claude
  committed
  b2e28c5
Commits on Mar 28, 2026
- Kafka headers, stateless activity backend, schedule backend, pool supervision
  
  tcdent
  and
  claude
  committed
  a3c3f6d
- Extract activity from backends into producer/consumer pattern
  
  tcdent
  and
  claude
  committed
  17f9819
- Typed worker messages, Task as pure data, multiprocessing IPC
  
  tcdent
  and
  claude
  committed
  4259527
- Partitioned Redis queues with scan-based fair dequeue
  
  tcdent
  and
  claude
  committed
  2c7ef1f
Commits on Mar 29, 2026
- Remove pubsub, inline dequeue, queue.complete, activity over IPC
  
  tcdent
  and
  claude
  committed
  b1ef5d5
- Schedule backend, session cleanup, dead code removal, resiliency tests
  
  tcdent
  and
  claude
  committed
  b382ef3
- Kafka state: raise NotImplementedError, drop in-memory KV/counter caches
  
  tcdent
  and
  claude
  committed
  4f9ba7c
Commits on Mar 30, 2026
- Fix KafkaStateBackend instantiation (no longer takes backend arg)
  
  tcdent
  and
  claude
  committed
  68e1b94
- Restore docstrings stripped during refactor
  
  tcdent
  and
  claude
  committed
  042e81e
- Update Kafka integration tests and fix queue interface mismatch
  
  tcdent
  and
  claude
  committed
  a068772
- Fix Kafka CI: add OFFSETS_TOPIC_REPLICATION_FACTOR, test timeout
  
  tcdent
  and
  claude
  committed
  78d8f79
- Remove --timeout flag (pytest-timeout not installed)
  
  tcdent
  and
  claude
  committed
  d1a5ccf
- Queue fairness benchmark: partition-level metrics, fix stale APIs
  
  tcdent
  and
  claude
  committed
  084f8ed
Commits on Mar 31, 2026
- Update README for 0.2 API changes and Kafka backend
  
  tcdent
  and
  claude
  committed
  5c0f7a5

Commits

Commits on Mar 27, 2026

Commits on Mar 28, 2026

Commits on Mar 29, 2026

Commits on Mar 30, 2026

Commits on Mar 31, 2026