Skip to content

feat: Add MongoDB Collection Comparison Support #108

Description

@ReginaldErzoah

Summary

Add direct MongoDB collection comparison workflows.


Motivation

MongoDB data is organized as collections instead of relational tables.

Dift should support comparing MongoDB collections in a way that feels natural to NoSQL users.

This enables:

  • collection-to-collection validation
  • production vs staging document comparison
  • NoSQL regression testing
  • application data quality checks

Proposed Improvements

  • Support collection-level comparison
  • Compare old and new MongoDB collections
  • Use a user-defined document key for matching
  • Preserve existing Dift comparison behavior where possible

Suggested Files

Potential implementation areas:

dift/io/mongodb_reader.py
dift/core/comparator.py
dift/core/schema_diff.py
dift/core/row_diff.py
tests/test_mongodb_reader.py
tests/test_mongodb_integration.py
docs/connectors/mongodb.md

Suggested Tasks

  • Define MongoDB collection URI format
  • Load collections into comparable tabular structure
  • Support document key matching
  • Add tests for collection comparisons
  • Add examples
  • Update documentation

How to Test

Run:

pytest
ruff check .

Run targeted tests:

pytest tests/test_mongodb_reader.py
pytest tests/test_mongodb_integration.py

Manual validation example:

dift mongodb://localhost:27017/shop.customers_old \
     mongodb://localhost:27017/shop.customers_new \
     --key customer_id

Verify:

  • collections compare successfully
  • added and removed documents are detected
  • key-based matching works
  • reports remain valid

Documentation Impact

Update:

docs/connectors/mongodb.md
docs/examples.md
docs/connectors/index.md

Documentation should include:

  • collection comparison examples
  • document key usage
  • supported URI format
  • expected output behavior

Acceptance Criteria

  • MongoDB collections can be compared directly
  • Key-based document matching works
  • Existing comparison reports work with MongoDB data
  • Tests pass
  • Documentation updated

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions