Fix MongoDB 6.0+ Compatibility and Add Comprehensive Test Coverage - #265
Merged
comtihon merged 1 commit intoDec 16, 2025
Merged
Conversation
samwar
force-pushed
the
fix-auto-detect-protocol-bson-bug
branch
7 times, most recently
from
December 1, 2025 22:03
ba7296d to
6a9cfc9
Compare
Contributor
Author
|
@comtihon - I found a few errors with the driver when I started to use it with MongoDB 6. Please take a look at this and merge when ready. |
samwar
marked this pull request as ready for review
December 1, 2025 22:11
This was referenced Dec 1, 2025
samwar
marked this pull request as draft
December 2, 2025 02:44
samwar
force-pushed
the
fix-auto-detect-protocol-bson-bug
branch
5 times, most recently
from
December 2, 2025 04:03
8427baf to
41fd368
Compare
Owner
|
hi @samwar , thank you very much for the contribution. |
Contributor
Author
|
@comtihon yeah this is a small work in progress. i'll get everything fixed up and ready and then bring it out of draft status. |
samwar
force-pushed
the
fix-auto-detect-protocol-bson-bug
branch
from
December 3, 2025 06:22
845d7f7 to
4655ed9
Compare
samwar
changed the title
This commit fixes critical BSON document formatting issues that prevented
the mongodb-erlang driver from working with MongoDB 6.0+, adds comprehensive
test coverage including property-based testing, and establishes authentication
testing infrastructure.
Problem:
MongoDB 6.0+ requires proper BSON document formatting and the OP_MSG protocol.
The driver had multiple issues with incorrect tuple formats instead of proper
BSON documents, causing crashes during connection initialization, command
execution, and index operations.
Core BSON Formatting Fixes (7 critical fixes):
* mc_worker_pid_info.erl: Fixed protocol detection hello command format
- Changed from bson:fields({hello, 1}) to bson:document([{hello, 1}])
* mongo_protocol.erl: Fixed OP_MSG command construction
- Added conditional handling for command_doc field (tuple vs list)
- Properly adds $db field using bson:append or list concatenation
- Fixed OP_MSG error response handling for authorization errors (code 13)
* mc_connection_man.erl: Fixed read preference handling
- Wrapped read preference in bson:document() for proper BSON format
* mc_worker_logic.erl: Fixed index creation
- Changed from tuple to list format: {<<"name">>, ...} → [{<<"name">>, ...}]
- Wrapped with bson:document() before merging
* mc_monitor.erl: Fixed server monitoring commands
- Fixed isMaster/hello commands to use bson:document() format
- Ensures topology management works correctly
* mc_worker.erl: Fixed write concern handling
- Added type checking to handle both tuple and list formats for command_doc
* mc_worker_api.erl: Fixed index API for op_msg protocol
- Wrapped IndexSpec in array for createIndexes command
- Changed return value to 'ok' on success for consistency
Additional Protocol Fixes:
* Fixed negative batchsize handling (singleBatch field) in OP_MSG protocol
* Added automatic tuple-to-map normalization for MongoDB 6.0+ compatibility
* Proper handling of maps with nested maps in query_to_op_msg_cmd
* Convert auth_source to binary if provided as atom
* Exported normalize_map_values/1 for testing
Test Coverage (73 tests total, +26 new):
New Test Suites:
* protocol_detection_SUITE (8 tests) - Protocol auto-detection and selection
* bson_format_SUITE (7 tests) - BSON document operations and validation
* error_handling_SUITE (6 tests) - Connection failures and error scenarios
* property_SUITE (5 properties, 240 cases) - Property-based testing with PropEr
- BSON round-trip, append associativity, ping idempotency, insert/find
* query_conversion_SUITE (7 tests) - Query to OP_MSG conversion validation
* batchsize_behavior_SUITE (8 tests) - Integration tests for batchsize behavior
* auth_SUITE (5 tests) - Authentication success, failure, and operations
Enhanced Existing Tests:
* Added authentication fallback to mc_worker_api_SUITE, mongo_api_SUITE,
switch_db_SUITE for Jenkins compatibility
* Fixed tuple_normalization_SUITE by exporting normalize_map_values/1
* Updated bson_format_SUITE to reflect that maps are now accepted
Authentication Testing Infrastructure:
Scripts:
* scripts/start_mongo_auth.sh - Starts MongoDB with auth on port 27021
* scripts/start_mongo_single_node.sh - Single node on port 27017
* scripts/start_mongo_cluster.sh - Replica set on ports 27018-27020
Docker Support:
* docker-compose-auth.yml - Local auth testing on port 27021
* test/mongo-init-auth.js - Creates test users (admin, testuser)
Port Allocation:
* 27017: Single node MongoDB (most tests)
* 27018-27020: Replica set cluster (3 nodes)
* 27021: Authentication-only MongoDB instance
CI/CD Improvements:
* Updated GitHub Actions test matrix (6 combinations):
- Erlang 24 & 25: MongoDB 5.0, 6.0 (4 combinations)
- Erlang 26: MongoDB 6.0, 7.0 (2 combinations)
* Fixed netcat package for Debian Bookworm (netcat → netcat-openbsd)
* Added artifact uploads for replica set logs and CT logs on failure
* Updated coverage workflow to match test matrix
Debug and Error Handling:
* Added try/catch blocks for better error reporting in critical paths
* Filtered hello commands from debug logging to reduce log spam
* Added validation checks before BSON operations
* Improved error messages for debugging BSON conversion issues
Test Results:
✅ All 73 tests passing (68 existing + 5 new auth tests)
✅ 240 property-based test cases across 5 properties
✅ 3 tests skipped (srv_connect_SUITE - requires MongoDB Atlas)
Compatibility:
✅ MongoDB 5.0.2, 6.0.16, 7.0.12
✅ Erlang/OTP 24, 25, 26
✅ OP_MSG protocol (auto-detected) + Legacy protocol (backward compatible)
Breaking Changes: None
Impact:
The driver now successfully connects to and communicates with MongoDB 6.0+
servers, properly detects and uses the OP_MSG protocol, handles authentication
correctly, maintains backward compatibility with MongoDB 5.0, and has
comprehensive test coverage with property-based testing.
samwar
force-pushed
the
fix-auto-detect-protocol-bson-bug
branch
from
December 3, 2025 18:21
4655ed9 to
79d042d
Compare
samwar
marked this pull request as ready for review
December 3, 2025 18:30
Contributor
Author
|
@comtihon this is good to go. |
Contributor
Author
|
@comtihon - are you able to merge this PR? |
Owner
|
thank you very much for your contribution! |
Contributor
Author
|
@comtihon you need to make a new tag and release now :) |
Owner
|
Just did it. Thank you very much again and wish you happy holidays! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix MongoDB 6.0+ Compatibility and Add Comprehensive Test Coverage
Overview
This PR fixes critical BSON document formatting issues preventing the mongodb-erlang driver from working with MongoDB 6.0+, adds comprehensive test coverage including property-based testing, and establishes authentication testing infrastructure.
Problem
MongoDB 6.0+ requires proper BSON document formatting and the OP_MSG protocol. The driver had multiple issues with incorrect tuple formats instead of proper BSON documents, causing crashes during connection initialization, command execution, and index operations.
Solution
Core Fixes (7 critical BSON formatting fixes)
Additional Improvements
Test Coverage (73 tests, +26 new)
New Test Suites:
protocol_detection_SUITE(8 tests) - Protocol auto-detectionbson_format_SUITE(7 tests) - BSON document operationserror_handling_SUITE(6 tests) - Connection and error scenariosproperty_SUITE(5 properties, 240 cases) - Property-based testing with PropErquery_conversion_SUITE(7 tests) - Query to OP_MSG conversionbatchsize_behavior_SUITE(8 tests) - Integration tests for batchsizeauth_SUITE(5 tests) - Authentication scenariosEnhanced Existing Tests:
Authentication Testing Infrastructure
Scripts:
start_mongo_auth.sh- Auth MongoDB on port 27021start_mongo_single_node.sh- Single node on port 27017start_mongo_cluster.sh- Replica set on ports 27018-27020Docker:
docker-compose-auth.yml- Local auth testingmongo-init-auth.js- Test user creationCI/CD Updates
Test Results
✅ All 73 tests passing (68 existing + 5 new auth tests + 240 property test cases)
✅ 3 tests skipped (srv_connect_SUITE - requires MongoDB Atlas)
Compatibility
Protocols: OP_MSG (auto-detected) + Legacy (backward compatible)
Breaking Changes
None. All changes maintain backward compatibility.
Impact
The driver now: