chore: improve Ecto projections documentation

[yordis]

Note

Adds a new guide comparing built-in vs external Ecto projections, significantly expands Ecto projections architecture/consistency docs, and fixes how-to links.

• Explanations:
  • New guide: guides/explanations/built-in-vs-external-projections.md detailing differences between built-in and external Ecto projections (compatibility, concurrency removal, batching, telemetry, migration, performance, future plans).
  • Ecto Projections doc expanded: guides/explanations/ecto-projections.md
    • Added sections on watermark vs subscription checkpoint, transaction semantics, after-update callbacks.
    • Introduced detailed consistency guarantees (:eventual vs :strong) with behaviors, trade-offs, and batching interaction.
    • Clarified batching workflow and why concurrency isn’t supported; expanded design decisions.
• How-to:
  • Fixed links to explanation guide in guides/howtos/building-read-models-with-ecto.md (.html → .md).

^{Written by Cursor Bugbot for commit 96abd9f. This will update automatically on new commits. Configure here.}

[coderabbitai]

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

This pull request adds comprehensive documentation on Ecto projections, including a new guide explaining built-in versus external projection packages, expanded explanations of consistency guarantees, watermarking concepts, and transaction semantics. Internal documentation links are corrected from HTML to Markdown format. No code or public API changes are introduced.

Changes

Cohort / File(s)	Summary
New Projection Comparison Guide guides/explanations/built-in-vs-external-projections.md	New documentation file detailing differences between built-in Ecto projections and the external commanded-ecto-projections package, including compatibility, feature parity, breaking changes (concurrency option removal), migration guidance, and performance comparisons.
Expanded Ecto Projections Documentation guides/explanations/ecto-projections.md	Expanded sections covering projection watermark vs. event store checkpoint, consistency guarantees (eventual and strong), transaction semantics, monitoring implications, batch processing trade-offs, and error handling. Content reorganized for prominence and consistency concepts emphasized throughout.
Documentation Link Updates guides/howtos/building-read-models-with-ecto.md	Corrected two internal documentation links from HTML to Markdown format (../explanations/ecto-projections.html → ../explanations/ecto-projections.md).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

• Specific areas for review:
  • Verify technical accuracy of consistency guarantee definitions and their behavioral implications
  • Ensure watermark vs. checkpoint explanation aligns with actual implementation semantics
  • Confirm batch processing trade-off descriptions are current and accurate
  • Cross-check that migration guidance in the new comparison guide accurately reflects the breaking change
  • Validate that explanations in both new and expanded documents are consistent and non-contradictory

Possibly related PRs

• feat: add batch support to Ecto projections #30: Main PR documentation updates directly reference and document batch-processing features and option changes (concurrency removal), providing implementation context for these documentation additions.

Poem

🐰 A rabbit hops through docs so bright,
With projections mapped and watermarks right,
Consistency tales now boldly told,
Batch processing wisdom, worth its weight in gold! 📚✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'chore: improve Ecto projections documentation' directly relates to the changeset, which adds new documentation, expands existing guides, and fixes links—all documentation improvements.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description check	✅ Passed	The pull request description accurately describes the changeset, detailing new documentation files, expansions to existing guides, and link fixes across multiple files.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch yordis/chore-2

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (4)

guides/explanations/built-in-vs-external-projections.md (3)

9-9: Simplify redundant phrasing.

Remove the redundant "originally" from this sentence.

-The `commanded-ecto-projections` package was originally created as an external library to provide Ecto integration for Commanded.
+The `commanded-ecto-projections` package was created as an external library to provide Ecto integration for Commanded.

---

206-224: Convert emphasis blocks to proper Markdown structure.

Lines 211–215 and 217–224 use **bold** text but serve as subsection headers. Convert them to proper Markdown heading syntax (e.g., ###).

-**What didn't change:**
+### What Didn't Change

- Core projection API
- Module and function names
- Callback signatures
- Database schema

-**What did change:**
+### What Did Change

- Concurrency option removed (replaced with batch_size)

---

267-272: Soften performance comparison claims.

Lines 267–272 present specific throughput numbers for batch processing (5,000–10,000 events/second). These are valuable but should include disclaimers that actual performance depends on event complexity, database hardware, and workload.

 **Built-in with batch processing** (safe)
 ```elixir
 use Commanded.Projections.Ecto, batch_size: 100
-# ~5,000-10,000 events/second (safe and faster)
+# ~5,000-10,000 events/second (typical; varies by event complexity and database)

</blockquote></details>
<details>
<summary>guides/explanations/ecto-projections.md (1)</summary><blockquote>

`223-383`: **Consistency Guarantees section is comprehensive but lengthy.**

The new section on `:eventual` vs. `:strong` consistency is technically sound and provides good examples. However, the section is substantial (~161 lines). Consider whether it could be shortened by moving detailed error handling examples to a separate how-to guide, keeping this section focused on the core concepts and trade-offs.



Potential candidates for external how-to:
- Lines 332–339: Error handling specifics (detailed retry logic)
- Lines 344–362: Combining with batch processing (could reference batch processing guide)

This would make the explanation more scannable while preserving the core decision-making content here.

</blockquote></details>

</blockquote></details>

<details>
<summary>📜 Review details</summary>

**Configuration used**: CodeRabbit UI

**Review profile**: CHILL

**Plan**: Pro

<details>
<summary>📥 Commits</summary>

Reviewing files that changed from the base of the PR and between 50ec4128f9d684bd33820b527e2320f7f788caff and 96abd9fb30336b1e9c850b6b8cee41c6e528c8a3.

</details>

<details>
<summary>📒 Files selected for processing (3)</summary>

* `guides/explanations/built-in-vs-external-projections.md` (1 hunks)
* `guides/explanations/ecto-projections.md` (4 hunks)
* `guides/howtos/building-read-models-with-ecto.md` (2 hunks)

</details>

<details>
<summary>🧰 Additional context used</summary>

<details>
<summary>🪛 LanguageTool</summary>

<details>
<summary>guides/explanations/built-in-vs-external-projections.md</summary>

[style] ~9-~9: This phrase is redundant. Consider writing “created”.
Context: ...commanded-ecto-projections` package was originally created as an external library to provide Ecto ...

(ORIGINALLY_CREATED)

</details>

</details>
<details>
<summary>🪛 markdownlint-cli2 (0.18.1)</summary>

<details>
<summary>guides/explanations/built-in-vs-external-projections.md</summary>

206-206: Emphasis used instead of a heading

(MD036, no-emphasis-as-heading)

---

216-216: Emphasis used instead of a heading

(MD036, no-emphasis-as-heading)

---

229-229: Emphasis used instead of a heading

(MD036, no-emphasis-as-heading)

---

267-267: Emphasis used instead of a heading

(MD036, no-emphasis-as-heading)

---

290-290: Emphasis used instead of a heading

(MD036, no-emphasis-as-heading)

</details>

</details>

</details>

<details>
<summary>🔇 Additional comments (7)</summary><blockquote>

<details>
<summary>guides/explanations/built-in-vs-external-projections.md (1)</summary><blockquote>

`67-116`: **Ensure concurrency data loss explanation is technically sound.**

The concurrency/watermark data loss scenario (lines 84–96) is a critical design justification. Verify this explanation is accurate and that the timeline correctly illustrates why concurrent watermark updates cause event loss.



Context from the explanation:
- Multiple concurrent workers can process events out of order
- Watermark jumps from 3 → 5 (bypassing 4)
- Event #4 arrives later but is skipped because `4 < 5`
- Claim: "Event #4 is permanently lost"

Confirm that:
1. This is indeed how the current idempotency mechanism works
2. This scenario is unavoidable with concurrent workers and watermark-based tracking
3. The batch processing alternative truly prevents this scenario

</blockquote></details>
<details>
<summary>guides/explanations/ecto-projections.md (4)</summary><blockquote>

`42-120`: **Watermark vs. Checkpoint section is well-structured and valuable.**

The new section clearly explains a common source of confusion. The examples are practical and the monitoring implications are actionable. This addition is excellent documentation.

---

`298-298`: **Verify cross-reference link accuracy.**

Line 298 references module documentation: `For usage examples, see module documentation for Commanded.Projections.Ecto`. Confirm that the module's documentation is available and covers consistency usage examples.

---

`382-382`: **Verify cross-reference link accuracy.**

Line 382 links to the how-to guide: `[Building Read Models How-To](../howtos/building-read-models-with-ecto.md)`. Confirm this file exists and is updated (as per PR summary, it was updated with link fixes).

---

`1-413`: Based on verification of both files, the documentation is **internally consistent** across the key topics:

1. **Concurrency explanations**: Both files explain the watermark-based idempotency problem identically—concurrent workers processing events out of order cause the watermark to skip past intermediate events. The only difference is presentation format (timeline vs. table).

2. **Why regular handlers support concurrency**: Both files use identical wording explaining that regular event handlers rely on checkpoint mechanisms and `partition_by/2` partitioning rather than watermark idempotency.

3. **Batch processing as solution**: Both files consistently recommend batch processing as the alternative to concurrency, with aligned explanations of benefits.

4. **Terminology**: "Watermark-based idempotency" is used consistently across both files.

The minor differences in presentation (built-in file emphasizes business benefits like "10-50x faster", while ecto-projections file emphasizes technical details like "fsync reduction") are stylistic variations, not inconsistencies.

</blockquote></details>
<details>
<summary>guides/howtos/building-read-models-with-ecto.md (2)</summary><blockquote>

`3-3`: **Link updates correctly target Markdown guides.**

Both link updates (line 3 and line 365) correctly change from `.html` to `.md` to point to the expanded `guides/explanations/ecto-projections.md` guide. This ensures navigation consistency throughout the documentation.





Also applies to: 365-365

---

`1-365`: **File under review is clean, but related guides need `.html` → `.md` updates.**

The file `building-read-models-with-ecto.md` contains no remaining `.html` links and correctly uses `.md` for internal references.

However, related guides still reference this file and others using `.html`:

- `guides/howtos/ecto-projections-getting-started.md` (line 75): Update `building-read-models-with-ecto.html` and `../explanations/ecto-projections.html` to `.md`
- `guides/explanations/read-model-projections.md` (line 7): Update `../howtos/ecto-projections-getting-started.html`, `../howtos/building-read-models-with-ecto.html`, and `ecto-projections.html` to `.md`

</blockquote></details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->

[coderabbitai]

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Verify that referenced guide exists.

Line 5 links to ../howtos/migrating-from-commanded-ecto-projections.md. Confirm this file exists in the repository, as it's referenced as part of the PR objectives but not provided in the review files.

---

🏁 Script executed:

#!/bin/bash
# Verify migration guide exists
fd -t f "migrating-from-commanded-ecto-projections.md"

Length of output: 0

---

🏁 Script executed:

#!/bin/bash
# Check if guides/howtos directory exists and list its contents
echo "=== Checking guides/howtos directory ==="
ls -la guides/howtos/ 2>&1

echo -e "\n=== Searching for migration-related files in guides ==="
find guides -type f -name "*migration*" -o -name "*migrat*" 2>/dev/null

echo -e "\n=== Searching for files containing 'commanded' or 'ecto' in name ==="
find guides -type f \( -name "*commanded*" -o -name "*ecto*" \) 2>/dev/null

Length of output: 828

---

Remove broken link or create the referenced migration guide.

The file guides/howtos/migrating-from-commanded-ecto-projections.md does not exist in the repository. The link in line 5 will be broken. Either remove the "See also" reference or ensure the migration guide file is added to the PR.

🤖 Prompt for AI Agents

In guides/explanations/built-in-vs-external-projections.md around line 5 the
"See also" link points to a non-existent file
(guides/howtos/migrating-from-commanded-ecto-projections.md) which will produce
a broken link; either remove the entire "See also" reference line or add the
missing migration guide at that exact path
(guides/howtos/migrating-from-commanded-ecto-projections.md) with appropriate
content and frontmatter, and if you add the file ensure the link target and
filename exactly match the referenced path and update any relative path if the
file is placed elsewhere.