feat: Add JSON Schema Inference for NoSQL Documents

[ReginaldErzoah]

Summary

Add JSON schema inference support for NoSQL and document-style datasets.

---

Motivation

NoSQL datasets often have flexible schemas.

Dift should be able to infer document structure and detect schema drift across collections or JSON-like datasets.

This improves support for:

• MongoDB
• nested JSON
• semi-structured data
• schema drift detection
• document validation workflows

---

Proposed Improvements

• Infer schema from JSON-like documents
• Detect field presence changes
• Detect type changes across documents
• Support nested fields
• Integrate inferred schemas with existing schema comparison

---

Suggested Files

Potential implementation areas:

dift/io/mongodb_reader.py
dift/core/schema_diff.py
dift/utils/
tests/test_json_schema_inference.py
docs/connectors/mongodb.md
docs/developer/architecture.md

---

Suggested Tasks

• Add JSON schema inference utility
• Infer field names and types
• Support nested fields
• Detect optional and missing fields
• Add tests for mixed document shapes
• Update documentation

---

How to Test

Run:

pytest
ruff check .

Run targeted tests:

pytest tests/test_json_schema_inference.py

Manual validation:

dift nested_old.json nested_new.json --key id

Verify:

• schema inference identifies fields correctly
• type changes are detected
• missing fields are handled safely
• schema reports remain valid

---

Documentation Impact

Update:

docs/connectors/mongodb.md
docs/developer/architecture.md
docs/examples.md

Documentation should include:

• how JSON schema inference works
• nested schema behavior
• supported document structures
• limitations and caveats

---

Acceptance Criteria

• JSON schema inference works for document-style data
• Nested fields are supported
• Schema drift is detected correctly
• Tests pass
• Documentation updated