feat(rabbitmq): enable core plugins, production config, and Erlang VM tuning - #58
Conversation
… tuning Enable eight tier-1 RabbitMQ plugins required by the cognitive architecture and production operations: consistent_hash_exchange, stream, stream_management, shovel, shovel_management, event_exchange, tracing, and top. Add rabbitmq.conf with production-tuned defaults: absolute memory watermark (512 MiB) with allocated calculation strategy for accurate container reporting, disk free limit, TCP keepalives with Nagle disabled, 128 channel max, 15 s stats emission interval, classic queue default, stream listener on port 5552, and guest loopback restriction. Configure Erlang VM scheduler optimizations via RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS: disable speculative busy-waiting (+sbwt/+sbwtdcpu/+sbwtdio none) and bind scheduler threads to CPU topology (+stbt ts) to reduce context switching. Wire stream protocol port 5552 through docker-compose and .env.example, add container mem_limit/cpus knobs, mount rabbitmq.conf read-only, and register rabbitmq-plugins manage.sh command.
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the RabbitMQ setup by integrating essential plugins, applying production-grade configuration, and optimizing the underlying Erlang VM. These changes aim to improve the broker's performance, observability, and operational capabilities, laying the groundwork for more robust and efficient message processing within the system. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request enhances the RabbitMQ service by enabling several useful plugins, adding a production-ready configuration, and providing options for Erlang VM tuning and container resource limits. The changes are well-structured and include corresponding updates to management scripts.
I have one specific suggestion in scripts/lib/rabbitmq.sh to improve the new rabbitmq-plugins command to fully support management capabilities as described in the help text.
Additionally, a critical point to consider is the base image rabbitmq:4.2-management-alpine used in docker/rabbitmq/Dockerfile. This image tag does not appear to be valid on the official Docker Hub repository for RabbitMQ, which could cause the build to fail. Please double-check that the base image and tag are correct.
| cmd_rabbitmq_plugins() { | ||
| ensure_env | ||
| ensure_rabbitmq_service | ||
| rabbitmq_exec rabbitmq-plugins list "$@" |
There was a problem hiding this comment.
The current implementation hardcodes the list subcommand, which prevents using other rabbitmq-plugins commands like enable or disable. This contradicts the help text in manage.sh which states "List or manage RabbitMQ plugins." For example, running manage.sh rabbitmq-plugins enable my_plugin would incorrectly execute rabbitmq-plugins list enable my_plugin.
To align with the documented behavior and the test plan (which requires manage.sh rabbitmq-plugins to default to list), I suggest checking if arguments are provided. If not, default to list; otherwise, pass all arguments through to rabbitmq-plugins.
| rabbitmq_exec rabbitmq-plugins list "$@" | |
| if [[ $# -eq 0 ]]; then | |
| rabbitmq_exec rabbitmq-plugins list | |
| else | |
| rabbitmq_exec rabbitmq-plugins "$@" | |
| fi |
There was a problem hiding this comment.
Pull request overview
Adds production-oriented RabbitMQ defaults to the stack: enabling core operational/plugins in the RabbitMQ image, mounting a tuned rabbitmq.conf, exposing the stream protocol port, and extending manage.sh with a RabbitMQ plugins helper command.
Changes:
- Enable a set of RabbitMQ plugins in the RabbitMQ Docker image build.
- Add and mount
rabbitmq/rabbitmq.conf, plus expose/configure the stream listener on port 5552. - Add
manage.sh rabbitmq-pluginscommand and introduce RabbitMQ container resource/VM tuning knobs in compose +.env.example.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
scripts/manage.sh |
Adds a new rabbitmq-plugins command to the CLI dispatcher and help text. |
scripts/lib/rabbitmq.sh |
Implements cmd_rabbitmq_plugins helper and adds a stream port env default. |
scripts/lib/maintenance.sh |
Adjusts how the pgBadger command is passed to bash -lc. |
rabbitmq/rabbitmq.conf |
Introduces a production-tuned RabbitMQ configuration (memory/disk/network/monitoring/streams). |
docker/rabbitmq/Dockerfile |
Enables required RabbitMQ plugins at build time (offline). |
docker-compose.yml |
Mounts rabbitmq.conf, exposes stream port 5552, adds resource limit knobs. |
.env.example |
Documents new RabbitMQ stream ports, resource limits, and Erlang VM args. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # +sbwtdcpu none — disable dirty-CPU scheduler busy-waiting | ||
| # +sbwtdio none — disable dirty-IO scheduler busy-waiting | ||
| # +stbt ts — bind scheduler threads to topology (reduces context switches) | ||
| RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS=+sbwt none +sbwtdcpu none +sbwtdio none +stbt ts |
There was a problem hiding this comment.
RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS is set to an unquoted value containing spaces. Since scripts/lib/common.sh sources .env with set -a; source, this line will break shell parsing (and can cause manage.sh to fail) unless the value is quoted or spaces are escaped.
| RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS=+sbwt none +sbwtdcpu none +sbwtdio none +stbt ts | |
| RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS="+sbwt none +sbwtdcpu none +sbwtdio none +stbt ts" |
| cmd_rabbitmq_plugins() { | ||
| ensure_env | ||
| ensure_rabbitmq_service | ||
| rabbitmq_exec rabbitmq-plugins list "$@" |
There was a problem hiding this comment.
cmd_rabbitmq_plugins always runs rabbitmq-plugins list and then appends user-supplied args. This prevents using other subcommands (enable/disable/etc.) and also breaks common usage like manage.sh rabbitmq-plugins list (it becomes rabbitmq-plugins list list). Consider passing through to rabbitmq-plugins directly, or defaulting to list only when no args are provided.
| rabbitmq_exec rabbitmq-plugins list "$@" | |
| if [[ $# -eq 0 ]]; then | |
| rabbitmq_exec rabbitmq-plugins list | |
| else | |
| rabbitmq_exec rabbitmq-plugins "$@" | |
| fi |
| local cmd=(pgbadger --quiet --format csv --jobs "$jobs" --outfile "$output") | ||
| [[ -n $since ]] && cmd+=(--begin "$since") | ||
| cmd+=(/var/lib/postgresql/data/log/postgresql-*.csv) | ||
| compose_exec bash -lc "${cmd[@]}" | ||
| compose_exec bash -lc "${cmd[*]}" | ||
| echo "pgBadger report written to ${output}" >&2 |
There was a problem hiding this comment.
compose_exec bash -lc "${cmd[*]}" builds a shell command string from user-controlled values (--since, --output) without robust escaping. This can break when values contain spaces/shell metacharacters and can lead to shell injection inside the container. Prefer executing pgbadger without bash -lc (if possible), or escape each argument (e.g., via printf %q) / pass args via bash -lc '... "$@"' to avoid interpolation.
| rabbitmq-plugins) | ||
| cmd_rabbitmq_plugins "$@" | ||
| ;; |
There was a problem hiding this comment.
A new rabbitmq-plugins manage.sh command is introduced here but isn’t covered by the existing tests/test_manage.py workflow suite that exercises other manage.sh commands. Adding a test that runs manage.sh rabbitmq-plugins under the rabbitmq profile (and asserts expected plugin names show up) would prevent regressions (and would have caught issues like list list).
CI sources .env directly, so unquoted values with spaces like "+sbwt none" cause the shell to interpret "none" as a command. Wrap the RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS value in double quotes so both `source .env` and Docker Compose handle it correctly.
Summary
consistent_hash_exchange,stream,stream_management,shovel,shovel_management,event_exchange,tracing,toprabbitmq/rabbitmq.confwith production-tuned defaults (absolute memory watermark, TCP keepalives, Nagle disabled, 15s stats interval, stream listener on 5552, guest loopback restriction)rabbitmq-pluginsmanage.sh commandTest plan
docker compose build rabbitmqsucceeds with plugins enableddocker compose --profile rabbitmq up -dstarts cleanly./scripts/manage.sh rabbitmq-pluginslists all 8 new plugins as enabled./scripts/manage.sh rabbitmq-overviewshows stream listener on 5552rabbitmq.confvalues loadedpython -m pytest -k full_workflowpasses with rabbitmq profile active