Publish, Store, Consume with Streams​

When a publishing application sends messages to a RabbitMQ stream using the stream protocol, the client library batches them together into a publishing frame. RabbitMQ receives these messages and aggregates them, often combining messages from multiple publishers into a single unit of storage, a "chunk". This chunk of messages is then stored durably on the file system.

When a consumer subscribes to a stream using the stream protocol, RabbitMQ dispatches messages by sending them in sequential chunks over the network. The number of messages packed into each chunk - the chunk size - is a critical factor for throughput; large chunks containing hundreds of messages each lead to high message delivery rates for consumers.

The ingress rate is the major factor in the chunk size: the higher the ingress rate, the greater the chunk size. Streams have been designed for high throughput since the beginning.

The Problem With Small Chunks​

What happens when the ingress rate is low? Chunks will contain just a few messages each—or even a single message in the worst case. This eliminates the benefits of batching: each frame delivers only a few messages to the consumer. While chunk dispatching is optimized, it still requires several system calls (reading the file system and writing to the socket).

It is hard to give definitive numbers, but let's say a stream consumer can read several million messages per second from a stream with a chunk size of 300. The rate can drop to about 200,000 messages per second with a chunk size of 15. The ratio is what matters here, not the absolute numbers.

Luckily there are ways to make things better with small chunks.

Optimization For Small Chunks: Read Ahead​

Streams tend to have consistent structure: if a stream contains small chunks at one point, it's likely composed mostly of small chunks. So why not try to read several chunks ahead if a chunk we just read is small? Dispatching several chunks still requires multiple frames, but we only need to read from the file system once, saving some costly system calls.

The read-ahead limit is 4,096 bytes, so not all small-chunk streams will benefit from this optimization. Still, this rather simple idea improves the delivery rates dramatically for targeted streams. Even better, RabbitMQ uses this technique for all protocols, not only the stream protocol but also AMQP.

Performance Results​

We used a 3-node cluster and a VM to run tests with PerfTest and Stream PerfTest. All VMs were m7i.4xlarge AWS instances. We created streams and filled them with 3 million messages, varying the message size and chunk size for each stream. We then consumed all messages using PerfTest (AMQP 0.9.1) and Stream PerfTest (stream protocol). We ran tests against RabbitMQ 4.1.4 and 4.2.0-beta.3.

We filled the streams with a tool written for the occasion to have the expected chunk sizes. Here are the performance tool commands to consume the messages:

# PerfTest
java -jar perf-test.jar --producers 0 --consumers 1 --predeclared --qos 200 \
    --stream-consumer-offset first --cmessages 3000000 --queue stream
# Stream PerfTest
java -jar stream-perf-test.jar --producers 0 --consumers 1 --offset first \
    --initial-credits 50 --no-latency --cmessages 3000000 --streams stream

12-byte messages, AMQP 0.9.1

Even if a 12-byte message is tiny, we see the read-ahead approach works well: 10 times faster for a stream with 1-message chunks. Rate is still twice as high with a 384-message-per-chunk stream.

12-byte messages, stream protocol

We achieved almost a 10x increase for a 1-message-per-chunk stream with the stream protocol (16,000 vs 134,000 msg/s). Read-ahead performs better until reaching 128 messages per chunk.

The results for the other message sizes are consistent, we will discuss them at the end.

48-byte Messages, AMQP 0.9.1

48-byte messages, stream protocol

256-byte Messages, AMQP 0.9.1

256-byte messages, stream protocol

512-byte Messages, AMQP 0.9.1

512-byte messages, stream protocol

1024-byte Messages, AMQP 0.9.1

1024-byte messages, stream protocol

Result Analysis​

The consumption rate improves as expected with the stream protocol: it is higher for streams with small chunks but remains the same when a chunk reaches the read-ahead limit (4,096 bytes). This is particularly noticeable with 1,024-byte messages, where the rate improved for chunk sizes of 1 and 2 (less than the read-ahead limit) but stays roughly the same for chunk sizes of 4 and 6 (reaching or exceeding the limit, so read-ahead does not kick in).

The trend is similar with AMQP 0.9.1, but we still observe improved results after a chunk exceeds the read-ahead limit. With AMQP 0.9.1 (and other non-stream protocols, like AMQP 1.0), RabbitMQ uses an iterator-like approach to dispatch messages. This approach was already using a form of read-ahead, but it's now more aggressive and better adapts to the size of chunks. This explains the good results with AMQP 0.9.1 that we should also get with other non-stream protocols.

Last Details​

The read-ahead optimization is enabled by default but can be disabled globally by setting the stream.read_ahead configuration entry to false.

Large chunks are still dispatched to consumers in an optimized manner: the chunk header is read in memory but the chunk data (the messages) are sent through the socket in a zero-copy fashion.

Conclusion​

The read-ahead optimization in RabbitMQ 4.2 proves that sometimes looking ahead pays off—delivering up to 10x better performance for streams with small chunks across all protocols. Who knew that being a little greedy when reading from disk could make your consumers so much happier?

In our benchmarks, combining SQL filters with Bloom filters achieved filtering rates of more than 4 million messages per second — in highly selective scenarios with high ingress rates. This means only the messages your consumers actually care about leave the broker, greatly reducing network traffic and client-side processing overhead.

Motivation​

High-throughput event streams often deliver large volumes of data to consumers, much of which may not be relevant to them. In real systems there may be tens of thousands of subjects (event types, tenants, regions, SKUs, etc.), making a dedicated stream per subject impractical or unscalable.

RabbitMQ Streams address this with broker-side filtering.

Bloom filters skip entire chunks that don’t contain values of interest, while SQL Filter Expressions evaluate precise per-message predicates so only matching messages cross the network. This reduces network traffic, lowers client CPU and memory use, and keeps application code simpler.

Demand for broker-side filtering is longstanding - Kafka users have requested it for years (see KAFKA-6020) — but Kafka still lacks this capability. RabbitMQ’s Bloom + SQL filtering makes selective consumption practical at scale today.

Let’s walk through a hands-on example.

Running the Sample App​

To run this example in your environment:

1. Start RabbitMQ with a single scheduler thread:

docker run -it --rm --name rabbitmq -p 5672:5672 -e ERL_AFLAGS="+S 1" rabbitmq:4.2.0-beta.3

2. From the root directory of the sample application, run:

mvn clean compile exec:java

The sample application uses the RabbitMQ AMQP 1.0 Java Client, as SQL filter expressions are a feature of AMQP 1.0.

Publishing Events​

Consider a typical e-commerce platform generating a continuous stream of customer events:

• product.search
• product.view
• cart.add
• cart.remove
• order.created
• and many others

Our sample application publishes 10 million such events to a stream, with order.created events occurring every 100,000 messages — representing just 0.001% of the total volume.

Each message includes a Bloom filter annotation set to its event type, enabling efficient chunk-level filtering:

publisher
    .message(body.getBytes(StandardCharsets.UTF_8))
    .priority(priority)
    // set the Bloom filter value
    .annotation("x-stream-filter-value", eventType)
    .subject(eventType)
    .creationTime(creationTime)
    // set application properties, e.g. region, price, or premium_customer
    .property("region", region);

Defining Your Filter​

Suppose you only want to process high-value orders that satisfy all of the following:

• Event type is order.created
• Order was created within the last hour
• Order originates from regions AMER, EMEA, or APJ
• And at least one of:
  • Priority > 4
  • Price ≥ 99.99
  • Premium customer

In our demonstration, only 10 messages out of 10 million meet these criteria — a highly selective filter scenario common in real-world applications.

Traditional approaches would require consuming all 10 million messages and filtering client-side, resulting in significant network overhead and wasted resources.

SQL filter expressions solve this elegantly by performing all filtering broker-side:

String SQL =
     "properties.subject = 'order.created' AND " +
     "properties.creation_time > UTC() - 3600000 AND " +
     "region IN ('AMER', 'EMEA', 'APJ') AND " +
     "(header.priority > 4 OR price >= 99.99 OR premium_customer = TRUE)";

The consumer implementation becomes straightforward:

ConsumerBuilder.StreamOptions builder = connection.consumerBuilder()
    .queue(STREAM_NAME)
    .stream()
    .offset(FIRST);

if (useBloomFilter) {
    // Stage 1: Bloom filter - quickly skip chunks without order.created events
    builder = builder.filterValues("order.created");
}

Consumer consumer = builder
    // Stage 2: SQL filter - precise broker-side per-message filtering
    .filter()
        .sql(SQL)
    .stream()
    .builder()
    .messageHandler((ctx, msg) -> {
        System.out.printf("  [%s] Received: %s\n",
            consumerType, new String(msg.body(), StandardCharsets.UTF_8));
        latch.countDown();
        ctx.accept();
    })
    .build();

Performance Results​

SQL Filtering Only​

Received 10 messages in 24.71 seconds using SQL filter only
Broker-side filtering rate: 404,645 messages/second

The consumer receives exactly 10 messages that match the criteria. All filtering occurs on the broker, processing over 400k messages per second while transmitting only the relevant data over the network.

Bloom + SQL Filtering​

Received 10 messages in 2.05 seconds using Bloom + SQL filters
Broker-side filtering rate: 4,868,549 messages/second

By combining both filtering stages, performance improves by an order of magnitude. The Bloom filter (Stage 1) eliminates entire chunks that don't contain order.created events before they're read from disk, while the SQL filter (Stage 2) applies precise business logic to the remaining messages.

Note that actual Bloom filtering performance depends on the number of messages per chunk, which varies with message ingress rates.

Learn More​

Check out our new Stream Filtering guide for best practices, examples, and configuration tips.

We have performed a number of benchmarks, showing significant performance improvements in many metadata operations. A comparison table can be found below.

Khepri Feature Flag now stable​

The khepri_db feature flag has now been upgraded to Stable, meaning it will now be enabled when running the command rabbitmqctl enable_feature_flag all, which should be done after every successful version upgrade.

Starting with version 4.2, all RabbitMQ clusters will be strongly recommended to adopt Khepri by enabling the khepri_db feature flag. This feature flag will likely become mandatory for upgrading from 4.2 onwards.

While the final decision depends on the community feedback, we expect that starting with RabbitMQ 4.3, the khepri_db feature flag will graduate to be Required.

Feature Flag Subsystem​

The RabbitMQ feature flag subsystem was recently improved by introducing a new category of feature flags known as Soft Required. If a feature flag is Soft Required starting from version N, it is automatically enabled once all RabbitMQ nodes are upgraded to version N of RabbitMQ. This is a change from the previous behavior of Required, where a feature flag that became required in version N of RabbitMQ must be enabled before upgrading to version N.

It remains best practice to enable feature flags as soon as they become Stable, generally immediately after a successful upgrade by running the command rabbitmqctl enable_feature_flag all. Nonetheless, we view the introduction of Soft Required feature flags as an improvement in user experience, as any required feature flags not already enabled will be automatically enabled when required.

Khepri Performance Improvements​

The benchmarks below were performed on a 3 node cluster running on Kubernetes

1000 queues, each with 100 bindings​

1000 Vhosts​

100,000 Classic Queues​

10,000 Quorum Queues​

1,000 Streams​

It's been a while since we wrote about how to migrate from Classic Mirrored Queues (CMQ) to Quorum Queues (QQ). In case you already don't know, CMQs are deprecated since RabbitMQ 3.9, and were removed in RabbitMQ 4.0. Today we are going to cover Quorum Queues in more detailed, comparing them to CMQs, and high level migration stategies to abandon CMQs for good, in favour of QQs, or a combination of QQ and Classic Queues (CQ). Note that CQs are still well supported, it's their mirroring feature what was deprecated and removed.

Why Quorum Queues?

Mirrored Classic Queues, a.k.a. Classic Mirrored Queues, a.k.a. CMQs, were introduced to add in-cluster data replication to classic queues (CQs). The CQs were originally designed as a non-replicated queue type back in 2006, during the first year of RabbitMQ development.

Then around 2009, the mirroring feature added the ability to replicate data to other nodes. The algorithm to "mirror" data was homegrown, and it was not very resilient to network partitions. To make matters worse, the behaviour in failure scenarios of CMQs was unpredictable in some cases, and generally quite hard to reason about. There were even cases where CMQs could lose messages.

To address all these issues, and to supercharge RabbitMQ with a reliable, safe and replicated queue type, Quorum Queues were designed and incorporated to RabbitMQ.

Quorum Queues are replicated queue type from the ground up, based on Raft consensus algorithm. Quorum Queues are designed with data safety as top priority. All QQs have a leader and some followers; locagically, leader and followers are distributed among RabbitMQ nodes. When a QQ leader receives a message, it records this operation in a write-ahead log (WAL), then stores the message on disk on a local Raft log and in parallel issues replication commands to the followers, and awaits for a confirmation from a majority before sending a confirmation back to the client.

One key difference with CMQs is that the replication part is done in parallel, while CMQs use a chain replication algorithm. Another important difference between the two: quorum queues pass a stricter version of the Jepsen test, while CMQs fail to pass even a less demanding original version.

In the event of a leader failure, the up-to-date followers start a voting process, and elect a new QQ leader. The resulting leader will resume operations on the queue. You can learn more details about Raft consensus algorithm in GitHub.

With those characteristics, QQs address the main issues of CMQs: data safety and predictability of failure scenarios. On top of that, quorum queues offer nearly a feature parity and better throughput for many workloads.

However, QQs are not a good fit for certain use cases present in messaging systems. For example: temporary queues used in e.g. RPC (request-reply) communication.

It doesn't make sense to use a quroum queue for very short lived transient data, since data safety will not be a priority for such use cases. For example, if your applications use a fire-and-forget approach to publish messages, and/or your consumer applications don't use manual acknowledgement. For such use cases, Classic Queues without mirroring are an excelent fit.

Migration from CMQs to QQs

Given the different nature of both queue types, including at the storage level, it is not possible to turn an existing classic queue into a quorum queue "in place".

However, a Blue-Green Deployment can be used for migrating. This is a strategy where applications are migrated from an existing cluster with mirrored classic queues, called the "blue" cluster, to a new cluster, the "green" one, which will use quorum queues for all classic queues that were replicated in the blue cluster.

RabbitMQ has tooling to facilitate the migration. `rabbitmqadmin v2, an HTTP API-based CLI tool built by the RabbitMQ Core Team at Broadcom, supports a number of commands that simplify the migration.

Before jumping into the details, let's take a look at how an existing RabbitMQ 3.13.x cluster that uses mirrored classic queues can be migrated to a new 4.1.x cluster that will use quorum queues for all replicated queues.

High Level Migration Plan​

The Blue-Green upgrade strategy has some requirements that must be met prior to the upgrade:

• RabbitMQ Queue Federation must be enabled
• The Blue (original) cluster must be reachable from the Green (new) one for queue federation to work
• The Green (new) RabbitMQ version must run a 4.x version, ideally latest version available
• All stable feature flags must be enabled in Green (new) cluster

Once the prerequisites are met, the migration plan consists of the following steps:

1. Export the definitions from the Blue cluster using rabbitmqadmin v2 and apply a few transformations to make sure that the definitions do not include any keys that the new cluster does not support (namely, the CMQ mirroring policies)
2. Import definitions into the Green cluster
3. Configure queue federation between the two clusters
4. Migrate consumers to the Green cluster
5. Migrate producers to the Green cluster
6. Monitor the state of the Green cluster
7. Shutdown the original cluster
8. Remove a number of temporary migration policies from Green

Now, let's dive into the details.

Considerations for Applications​

Quorum Queues support all features of durable mirrored classic queues. That means that most applications can be moved to the new cluster without any changes. For the small percentage of applications that explicitly specifies a queue type, a special rabbitmq.conf setting can be used to relax a key queue property equivalence check performed by RabbitMQ nodes when a client tries to declare a queue.

The feature comparison matrix covers how Quorum Queues are different from Classic Queues.

This is a good moment to reconsider whether certain apps need to use replicated queues at all. For example, an application that creates and binds a temporary queue to a fanout exchange, process information for some time and deletes its queue after disconnection, will not benefit from what a replicated queue type — quorum queues — have to offer, because the queue is very short lived, specific to a particular client, and the app is capable of re-declaring its topology.

If some queues do not need to be replicated, remove the classic queue mirroring-related keys from their policies before proceeding. The keys are

• "ha-mode"
• "ha-params"
• "ha-promote-on-shutdown"
• "ha-promote-on-failure"
• "ha-sync-mode"
• "ha-sync-batch-size"

Such non-replicated classic queues will be migrated as classic queues to the Green cluster, avoiding the use of quorum queues where they are not necessary.

Detailed Migration Plan​

This plan has been tested with RabbitMQ Blue 3.13.7 and RabbitMQ Green 4.1.2. Both TLS, mTLS and plain TCP has been tested. Both the CLI rabbitmqadmin v2 and RabbitMQ work without issues in all the tested setups.

Export Definitions From the Blue Cluster​

In this step, we will backup all rabbitmq objects, excluding messages, to a JSON file. All usernames, vhosts, queues, permissions, bindings, policies, parameters, exchanges, everything, will be exported to a JSON file. It is possible to apply transformations to exclude certain data. This transformations feature will greatly help with the CMQ to QQ migration.

The following command exports a definitions file of RabbitMQ and transforms the result by removing the deprecated CMQ policy keys and optional queue arguments, replacing previously mirrored classic queues with quorum queues.

Any queue that had a mirroring policy applied to it will be automatically transformed into a Quorum Queue in the definitions (backup) file. If as a result of this transformations, a policy becomes empty (beecause it only had CMQ keys), it will also be deleted, because it is not possible to import empty policies in RabbitMQ.

RabbitMQ definitions are shared among all cluster nodes, therefore, it is only necessary to run this command in one node.

# Export definitions from the original cluster into a file and applies two transformations
# to remove all traces of classic mirrored queues
rabbitmqadmin --host blue definitions export --file blue.json -t prepare_for_quorum_queue_migration,drop_empty_policies

Import definitions​

In this step, we will "restore" the definitions file from the previous step (with the CMQs transformed into QQ) into the new "green" cluster. This command does not allow for much configurability, it's a very straightforward step:

# Import definitions into the new (Green) cluster
rabbitmqadmin --host green definitions import --file ./blue.json

Configure a Federation Upstream in Green​

In this step, we will configure queue federation. These steps are critically important because it's the queue federation links that will be transferring any existing messages from the original cluster to the new ones, which allows applications to be moved to the new cluster at a later stage.

Take some time to read through and prepare the commands in advance.

Create federation upstreams​

For each vhost, create a federation upstream in Green:

rabbitmqadmin --host green federation declare_upstream_for_queues --name cmq-qq-migration --uri 'amqp://<federation-user>:<federation-password>@blue.rabbit'

Where <federation-user> is an existing user in Blue RabbitMQ. It is advisable to create a dedicated user for federation.

Where <federation-password> is the federation user password used to authenticate in Blue RabbitMQ.

Create Override Policies​

This step creates "override" policies to enable queue federation for all queues that are already matched by a policy. Note that the override policy is a rabbitmqadmin v2 concept, not a RabbitMQ HTTP API, so this operation can be performed on a 3.13.x node.

In this step, we are configuring RabbitMQ to use federated queues. Federated queues transfer messages from the upstream cluster, when local consumers request messages and local queues are empty. This will allow you to move your consumer applications to Green without disruption to your operations.

For each policy, create a policy override to utilise the federation upstream:

rabbitmqadmin --host green policies list
# => ┌──────┬─────────┬───────────────────────┬──────────┬──────────┬──────────────────┐
# => │ name │ vhost   │ pattern               │ apply_to │ priority │ definition       │
# => ├──────┼─────────┼───────────────────────┼──────────┼──────────┼──────────────────┤
# => │ ha   │ finance │ (?:^po$)|(?:.*\.dlx$) │ all      │ 0        │ max-length: 100  │
# => │      │         │                       │          │          │ queue-version: 2 │
# => │      │         │                       │          │          │                  │
# => └──────┴─────────┴───────────────────────┴──────────┴──────────┴──────────────────┘

rabbitmqadmin --host green policies declare_override --name ha --definition '{"federation-upstream": "cmq-qq-migration"}'

Next, create a blanket/catch-all policy for any queues not matched by an existing policy:

rabbitmqadmin --host green policies declare_blanket --name cmq-qq-migration_blanket --apply-to queues --definition '{"federation-upstream": "cmq-qq-migration"}'

This will ensure that all queues are federated between the Blue and Green clusters.

Verify That Federation is Functional​

Once the policies are created, the federation plugin in the Green cluster will create links (connections) to the upstream (Blue). The presence of links ensures that federated queues are ready to move messages from the upstream (Blue) into the Green cluster when the consuming applications are migrated from Blue to Green.

rabbitmqadmin --host green federation list_all_links
# => ┌──────────────┬─────────┬──────────┬─────────────────────┬─────────┬───────┬──────────────────┬──────────────────────────────────┐
# => │ node         │ vhost   │ id       │ uri                 │ status  │ type  │ upstream         │ consumer_tag                     │
# => ├──────────────┼─────────┼──────────┼─────────────────────┼─────────┼───────┼──────────────────┼──────────────────────────────────┤
# => │ rabbit@green │ finance │ 8f0f0d8a │ amqps://blue.rabbit │ running │ queue │ cmq-qq-migration │ federation-link-cmq-qq-migration │
# => ├──────────────┼─────────┼──────────┼─────────────────────┼─────────┼───────┼──────────────────┼──────────────────────────────────┤
# => │ rabbit@green │ finance │ 01efa5b5 │ amqps://blue.rabbit │ running │ queue │ cmq-qq-migration │ federation-link-cmq-qq-migration │
# => ├──────────────┼─────────┼──────────┼─────────────────────┼─────────┼───────┼──────────────────┼──────────────────────────────────┤
# => │ rabbit@green │ finance │ 950faf11 │ amqps://blue.rabbit │ running │ queue │ cmq-qq-migration │ federation-link-cmq-qq-migration │
# => └──────────────┴─────────┴──────────┴─────────────────────┴─────────┴───────┴──────────────────┴──────────────────────────────────┘

Prepare to Move Consumers​

Both RabbitMQ clusters (Blue and Green) are ready to support a migration. The first step of the app migration is moving consumer applications to the new RabbitMQ cluster (Green).

How the applications are deployed to the Green cluster depends entirely on how you deploy and run RabbitMQ and the applications. There is no urgency or rush to complete this step. Ensure that your consumer applications are working as expected before moving to the next step.

Prepare to Move Producers​

This is the last step to complete the migration! Moving the producers consists of stopping the producer apps, letting the consumers drain the queues in Blue via queue federation, and starting the producer apps in Green.

There is a special scenario if you can't afford to have your producer apps stopped until the consumers drain your queues. This can be the case if your usage consists of very long backlog queues and slow consumers. In this special case, in addition to setting up queue fedration, consider declaring one or more shovels for moving messages from Blue to Green, before redeploying producer apps into the Green cluster.

Clean Up After Migrating​

After confirming that the migration completed successfully and that your apps are working without issues, proceed to cleanup the policies declared for the needs of the migration, as well as the federation upstream in the Green cluster.

Delete Temporary Policies​

Override policies created earlier in the process will all be prefixed with override.. In this guide, the name of the blanket, catch-all policy is cmq-qq-migration_blanket:

rabbitmqadmin --host green policies list
# => ┌──────────────────────────┬─────────┬───────────────────────┬──────────┬──────────┬─────────────────────────────────────────┐
# => │ name                     │ vhost   │ pattern               │ apply_to │ priority │ definition                              │
# => ├──────────────────────────┼─────────┼───────────────────────┼──────────┼──────────┼─────────────────────────────────────────┤
# => │ cmq-qq-migration_blanket │ finance │ .*                    │ queues   │ -21      │ federation-upstream: "cmq-qq-migration" │
# => │                          │         │                       │          │          │                                         │
# => ├──────────────────────────┼─────────┼───────────────────────┼──────────┼──────────┼─────────────────────────────────────────┤
# => │ ha                       │ finance │ (?:^po$)|(?:.*\.dlx$) │ all      │ 0        │ max-length: 100                         │
# => │                          │         │                       │          │          │ queue-version: 2                        │
# => │                          │         │                       │          │          │                                         │
# => ├──────────────────────────┼─────────┼───────────────────────┼──────────┼──────────┼─────────────────────────────────────────┤
# => │ overrides.ha             │ finance │ (?:^po$)|(?:.*\.dlx$) │ all      │ 100      │ federation-upstream: "cmq-qq-migration" │
# => │                          │         │                       │          │          │ max-length: 100                         │
# => │                          │         │                       │          │          │ queue-version: 2                        │
# => │                          │         │                       │          │          │                                         │
# => └──────────────────────────┴─────────┴───────────────────────┴──────────┴──────────┴─────────────────────────────────────────┘

rabbitmqadmin --host green policies delete --name cmq-qq-migration_blanket
rabbitmqadmin --host green policies delete --name overrides.ha

Next, delete the federation upstream that was used for migrating messages:

rabbitmqadmin --host green federation delete_upstream --name cmq-qq-migration

Testing the Migration in a Development Environment​

This migration can seem daunting at first. Team RabbitMQ has done extensive testing of different scenarios and different queue feature combination, however, nothing would give more confidence than trying this out yourselves in your own environment!

For that reason, it is highly recommended to test all the migration commands and plan in a development or staging environment. Sometimes it is challenging to make your own application simulate load as in production. For such cases, RabbitMQ Perf Test is a great alternative to simulate workloads and test specific RabbitMQ features.

For example, to simulate one producer publishing to exchange inc at a rate of 30 msg/s with routing key bills, you could use the following perf-test command:

java -jar perf-test.jar \
    -y 0 -x 1 -c 10 -p -qq \
    --rate 30 -e inc \
    --routing-key 'bills' \
    --uri "amqp://myuser:mypass@blue.rabbit.example.com"

All perf-test options are documented in Perf Test documentation and in its help command:

java -jar perf-test.jar --help

Post-Migration Checklist​

After completing the migration, the following list can help to double check the success of the migration. Feel free to add any additional items that make sense in your environment.

• All consumer apps are connected to Green (new) RabbitMQ cluster
• All producer applications are connected to Green (new) RabbitMQ cluster
• Temporary policies are not present
• Federation upstream is not present
• RabbitMQ does not have any alarm in effect
• All metrics are within their typical ranges

Migration pitfalls​

TLS​

rabbitmqadmin v2 supports TLS-enabled connections. However, its requires the trusted CAs to be part of the system trusted CAs.

How to add a trusted CA to your system keychain varies from one operating system to another. The following links are not a complete reference, but convenience for most common systems:

• MacOS: Add certificates to keychain
• Fedora: Using Shared System Certificates
• RHEL 9: Using shared system certificates
• Ubuntu: Install a root CA certificate in the trust store

Once the CAs are added to the trusted certificate list at the OS level, their bundle file in the PEM format can be used together with rabbitmqadmin. For example:

rabbitmqadmin --use-tls --tls-ca-cert-file /path/to/your/chained_ca_certificate.pem queues list

If you have RabbitMQ configured to use mutual peer verification (mTLS), rabbitmqadmin will also need to be provided with a client certificate and key pair.

For example:

rabbitmqadmin --use-tls --tls-ca-cert-file /path/to/your/chained_ca_certificate.pem \
    --tls-cert-file /path/to/your/client_certificate.pem \
    --tls-key-file /path/to/your/client_key.pem \
    queues list

Alternatively, since adding all TLS options to each command can be cumbersome, it is possible to configure all TLS options, hostname, port and credentials in a rabbitmqadmin configuration file. The configuration file for rabbitmqadmin is TOML format, and it accepts some options, in addition to a "node" alias. For example:

[blue]
hostname = "blue.rabbit"
tls = true
ca_certificate_bundle_path = "/path/to/your/chained_ca_certificate.pem"
client_certificate_file_path = "/path/to/your/client_certificate.pem"
client_private_key_file_path = "/path/to/your/client_key.pem"
port = 32795

[green]
hostname = "green.rabbit"
port = 32811
tls = true
ca_certificate_bundle_path = "/path/to/your/chained_ca_certificate.pem"
client_certificate_file_path = "/path/to/your/client_certificate.pem"
client_private_key_file_path = "/path/to/your/client_key.pem"

Use the above configuration with the --config and --node. For example rabbitmqadmin --config rabbitmqadmin.toml --node blue queues list.

Conclusion

Quorum Queues (QQ) are the go-to queue type for data safety in RabbitMQ. They provide predictable failover behaviour and higher throughput than Classic Mirrored Queues (CMQ). Quorum Queues keep receiving updates, performance improvements and bug fixes, whilst Classic Mirrored Queues are in life support in 3.13 and completely removed in RabbitMQ 4. Classic Queues (without mirroring) are still a valid and supported queuee type. Classic Queues are still a good fit for certain uses cases, for example: RPC pattern. Or any use case that does not require high availability.

With the new generation of rabbitmqadmin, it is now easier than every to migrate from mirrored classic queues to quorum queues, and at the same time upgrade to the latest supported RabbitMQ release series.

The existing repos, ppa1.rabbitmq.com and ppa2.rabbitmq.com, will remain in operation until Nov 1, 2025 but not receive any new updates.

Our Debian (Apt) Repositories are Moving​

Our Debian (apt) repositories are moving from ppa*.rabbitmq.com to deb*.rabbitmq.com. The new repositories are live and covered in the Debian doc guide.

How to Migrate​

To add the new repositories and their signing key, use the following scripts.

#!/bin/sh

sudo apt-get install curl gnupg apt-transport-https -y

## Team RabbitMQ's signing key
curl -1sLf "https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA" | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null

## Add apt repositories maintained by Team RabbitMQ
sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
## Modern Erlang/OTP releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-erlang/ubuntu/noble noble main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-erlang/ubuntu/noble noble main

## Latest RabbitMQ releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-server/ubuntu/noble noble main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-server/ubuntu/noble noble main
EOF

## Update package indices
sudo apt-get update -y

## Install Erlang packages
sudo apt-get install -y erlang-base \
                        erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
                        erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
                        erlang-runtime-tools erlang-snmp erlang-ssl \
                        erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl

## Install rabbitmq-server and its dependencies
sudo apt-get install rabbitmq-server -y --fix-missing

#!/bin/sh

sudo apt-get install curl gnupg apt-transport-https -y

## Team RabbitMQ's signing key
curl -1sLf "https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA" | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null

## Add apt repositories maintained by Team RabbitMQ
sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
## Modern Erlang/OTP releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-erlang/ubuntu/jammy jammy main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-erlang/ubuntu/jammy jammy main

## Latest RabbitMQ releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-server/ubuntu/jammy jammy main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-server/ubuntu/jammy jammy main
EOF

## Update package indices
sudo apt-get update -y

## Install Erlang packages
sudo apt-get install -y erlang-base \
                        erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
                        erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
                        erlang-runtime-tools erlang-snmp erlang-ssl \
                        erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl

## Install rabbitmq-server and its dependencies
sudo apt-get install rabbitmq-server -y --fix-missing

#!/bin/sh

sudo apt-get install curl gnupg apt-transport-https -y

## Team RabbitMQ's signing key
curl -1sLf "https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA" | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null

## Add apt repositories maintained by Team RabbitMQ
sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
## Modern Erlang/OTP releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-erlang/ubuntu/focal focal main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-erlang/ubuntu/focal focal main

## Latest RabbitMQ releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-server/ubuntu/focal focal main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-server/ubuntu/focal focal main
EOF

## Update package indices
sudo apt-get update -y

## Install Erlang packages
sudo apt-get install -y erlang-base \
                        erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
                        erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
                        erlang-runtime-tools erlang-snmp erlang-ssl \
                        erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl

## Install rabbitmq-server and its dependencies
sudo apt-get install rabbitmq-server -y --fix-missing

#!/bin/sh

sudo apt-get install curl gnupg apt-transport-https -y

## Team RabbitMQ's signing key
curl -1sLf "https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA" | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null

## Add apt repositories maintained by Team RabbitMQ
sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
## Modern Erlang/OTP releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-erlang/debian/bookworm bookworm main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-erlang/debian/bookworm bookworm main

## Latest RabbitMQ releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-server/debian/bookworm bookworm main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-server/debian/bookworm bookworm main
EOF

## Update package indices
sudo apt-get update -y

## Install Erlang packages
sudo apt-get install -y erlang-base \
                        erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
                        erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
                        erlang-runtime-tools erlang-snmp erlang-ssl \
                        erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl

## Install rabbitmq-server and its dependencies
sudo apt-get install rabbitmq-server -y --fix-missing

#!/bin/sh

sudo apt-get install curl gnupg apt-transport-https -y

## Team RabbitMQ's signing key
curl -1sLf "https://keys.openpgp.org/vks/v1/by-fingerprint/0A9AF2115F4687BD29803A206B73A36E6026DFCA" | sudo gpg --dearmor | sudo tee /usr/share/keyrings/com.rabbitmq.team.gpg > /dev/null

## Add apt repositories maintained by Team RabbitMQ
sudo tee /etc/apt/sources.list.d/rabbitmq.list <<EOF
## Modern Erlang/OTP releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-erlang/debian/bullseye bullseye main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-erlang/debian/bullseye bullseye main

## Latest RabbitMQ releases
##
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb1.rabbitmq.com/rabbitmq-server/debian/bullseye bullseye main
deb [arch=amd64 signed-by=/usr/share/keyrings/com.rabbitmq.team.gpg] https://deb2.rabbitmq.com/rabbitmq-server/debian/bullseye bullseye main
EOF

## Update package indices
sudo apt-get update -y

## Install Erlang packages
sudo apt-get install -y erlang-base \
                        erlang-asn1 erlang-crypto erlang-eldap erlang-ftp erlang-inets \
                        erlang-mnesia erlang-os-mon erlang-parsetools erlang-public-key \
                        erlang-runtime-tools erlang-snmp erlang-ssl \
                        erlang-syntax-tools erlang-tftp erlang-tools erlang-xmerl

## Install rabbitmq-server and its dependencies
sudo apt-get install rabbitmq-server -y --fix-missing

Changes In the New Repositories​

The new repositories, deb1.rabbitmq.com and deb2.rabbitmq.com, are signed using our team's standard signing key instead of of the keys used by their upstream Cloudsmith repositories.

For the distributions that are sufficiently old, the new repositories provide packages going back to RabbitMQ 3.10.x and Erlang versions going back to 24 or 25. Newer distributions, such as Ubuntu 24.04, start with RabbitMQ 3.12.x and Erlang 26. Note that even the 3.12.x series are entirely out of support by now. We strongly encourage everyone to move to the latest RabbitMQ 4.x series.

If you use apt pinning, note that the Origin field of the new repo has changed to RabbitMQ.

Existing Repository Shutdown Schedule​

The existing repos, ppa1.rabbitmq.com and ppa2.rabbitmq.com, will remain in operation until Nov 1, 2025 but will not receive any new updates. After Nov 1, 2025, these repositories will be shut down.

Please do not put off the migration, it is a matter of updating an apt repository definition file: the signing key used by the new repos has been recommended for installing by our docs for years.

Thank you, Cloudsmith​

Cloudsmith.io has been powering our apt and dnf (yum, RPM) repositories directly or as a mirror upstream since 2016. For nearly a decade, Cloudsmith have offered us a reliable service and an API that's a pleasure to automate with.

The fix reverts the pull request 393 that introduced a dangerous bug where the library skipped chunk delivery when the channel's maximum capacity was reached. In practical terms, message dispatch to the application would stop.

The bug was triggered when the consumer was experiencing a near peak delivery pressure for some time, or when the consumer was consistently slow to process the deliveries.

Affected versions​

The bug affects the following versions: 1.5.5, 1.5.6 and 1.5.7.

We strongly recommend updating the client to 1.5.8 as soon as possible.

Team RabbitMQ's Erlang Packages Do Not Include SSH​

Team RabbitMQ produces a zero dependency Erlang RPM that does not include the SSH library since it is not used. Our Debian packages are split into multiple fine-grained components, and the RabbitMQ installation guide skips SSH library installation.

Patched Versions Are Available​

Team RabbitMQ's RPM repositories and Debian repositories were updated to include Erlang 27.3.3, 26.2.5.11 and 25.3.2.20.

For aarch64 (64-bit ARM) RPM packages, see rabbitmq/erlang-rpm releases.

For aarch64 (64-bit ARM) Debian packages of Erlang 26.2.5.11, see this Launchpad repository.

RabbitMQ community Docker image was also upgraded to Erlang 27.3.3 and 26.2.5.11 last week.

This feature enables any browser-based application to communicate with RabbitMQ using AMQP 1.0, paving the way for a wide range of efficient browser-based business messaging scenarios.

What is WebSocket?​

WebSocket, defined in RFC 6455, is a simple protocol consisting of two parts:

Client                                   Server
  |                                         |
  |================ Part I =================|
  |---- WebSocket Handshake Request ------->|
  |       GET /some/path HTTP/1.1           |
  |       Upgrade: websocket                |
  |       Connection: Upgrade               |
  |       Sec-WebSocket-Protocol: amqp      |
  |                                         |
  |<-- WebSocket Handshake Response --------|
  |       HTTP/1.1 101 Switching Protocols  |
  |       Upgrade: websocket                |
  |       Connection: Upgrade               |
  |       Sec-WebSocket-Protocol: amqp      |
  |                                         |
  |================ Part II ================|
  |<========= WebSocket Connection ========>|
  |        Full-duplex communication        |
  |                                         |
  |<---------- Binary Data [AMQP] --------->|
  |<---------- Binary Data [AMQP] --------->|
  |<---------- Binary Data [AMQP] --------->|
  |                 ...                     |
  |                                         |
  |----------- Close Frame ---------------->|
  |<---------- Close Frame -----------------|

The first part is the handshake, consisting of an HTTP request and response. The second part is the data transfer: a single TCP connection remains open between client and server, allowing either side to push binary data to the other at any time.

In this example, the binary data consists of AMQP frames because the client proposed Sec-WebSocket-Protocol: amqp to the server, and the server accepted it. The Sec-WebSocket-Protocol header specifies the application-level protocol layered over WebSocket.

The protocol stack for this communication looks like this:

+-------------------+
|      AMQP         |  Application Layer
+-------------------+
|    WebSocket      |
+-------------------+
|       TCP         |  Transport Layer
+-------------------+
|       IP          |  Network Layer
+-------------------+
|     Ethernet      |  Link Layer
+-------------------+

The WebSocket protocol itself does not dictate the application-layer protocol, allowing for flexibility. It could support any protocol, including STOMP and MQTT.

Why WebSocket?​

WebSocket provides the following advantages:

1. Browser Compatibility: It allows browser-based applications to use application-layer protocols other than HTTP. For security reasons, browsers restrict JavaScript from opening raw TCP connections to communicate using protocols like AMQP or MQTT. WebSocket protects against malicious JavaScript running in a browser, therefore enabling secure application-layer communication over the WebSocket layer.
2. Firewall traversal: WebSocket facilitates communication in environments with restrictive firewall rules. For example, while port 443 (https) may be allowed, port 5671 for amqps might be blocked. In such cases, a secure WebSocket (wss) connection on port 443 can be used to communicate over AMQP.

Modern browsers universally support WebSocket, making it a practical choice for web-based applications.

WebSocket in RabbitMQ​

RabbitMQ has long supported STOMP over WebSocket and MQTT over WebSocket through the rabbitmq_web_stomp and rabbitmq_web_mqtt plugins, respectively.

VMware Tanzu RabbitMQ 4.1 introduces the new rabbitmq_web_amqp plugin, compliant with the AMQP WebSocket Binding Committee Specification 01. This plugin operates similarly to the existing WebSocket plugins, starting a listener to manage WebSocket protocol aspects.

Previously, browser-based applications connected to RabbitMQ using MQTT or STOMP over WebSocket, often due to the lack of AMQP support. AMQP over WebSocket brings the following benefits:

• Feature Richness: Unlike MQTT and STOMP which are designed for simplicity, AMQP is a protocol designed for business messaging supporting more advanced features.
• Efficiency: AMQP is binary and efficient, whereas STOMP is text-oriented.

This makes AMQP over WebSocket an excellent choice for efficient, feature-rich browser-based business messaging.

WebSocket Relay​

Before this plugin, a workaround involved using a WebSocket relay. A relay would accept WebSocket connections from clients and open separate TCP connections to RabbitMQ. While functional, this approach introduced downsides such as:

1. Additional latency from the extra network hop.
2. Increased resource usage to maintain two TCP connections per client.
3. Operational overhead for deploying and monitoring the relay.
4. An additional potential point of failure.

The new plugin eliminates these issues by providing direct AMQP over WebSocket support.

Example​

RabbitMQ includes plugins rabbitmq_web_stomp_examples and rabbitmq_web_mqtt_examples with basic examples called "echo" and "bunny". Similarly, VMware Tanzu RabbitMQ 4.1 introduces rabbitmq_web_amqp_examples.

The AMQP over WebSocket "bunny" example works as follows:

1. The rabbitmq_web_amqp_examples plugin creates a stream named amq.web_amqp_examples.bunny.
2. When visiting http://localhost:15670/web-amqp-examples/bunny.html, the plugin serves files bunny.html, bunny.png, and rhea.js. The bunny.html file displays the bunny.png. Additionally, the bunny.html file contains JavaScript code which creates an AMQP over WebSocket connection from the browser to RabbitMQ:

// default AMQP over WebSocket port and path in RabbitMQ
var url = "ws://" + location.hostname + ":15678/ws"
var ws = client.websocket_connect(WebSocket);
var connection = client.connect({
    "connection_details":ws(url, ["amqp"]),
    // Setting username without password causes rhea to use SASL mechanism ANONYMOUS.
    "username": "ignored",
});

This code snippet uses the rhea.js file. rhea is an open source AMQP 1.0 TypeScript/JavaScript library. This single file can be created by running the following commands in the root directory of the rhea repository:

npm install
make browserify

Not only did the browser open an AMQP over WebSocket connection, but it also created a session and links to/from the pre-declared stream:

var address = "/queues/" + stream
client.on("connection_open", function (context) {
    sender = context.connection.open_sender(address);
    // If we open a new brower tab, we want to see the existing drawing.
    const filter = {'my-filter': amqp_types.wrap_described("first", "rabbitmq:stream-offset-spec")};
    context.connection.open_receiver({source:{address: address, filter: filter}});
});

3. If you open a second browser tab at the same URL, it also creates an AMQP over WebSocket connection publishing to and consuming from the same stream.
4. When you draw on either of the two browser tabs, the other tab sees the live drawing because both tabs consume from the same stream:

Clients​

As an alternative to rhea, you can use the RabbitMQ AMQP 1.0 JavaScript library. As explained in the Native AMQP 1.0 blog post, RabbitMQ libraries include RabbitMQ specific features such as declaring exchanges, queues, and bindings.

Wrapping up​

VMware Tanzu RabbitMQ 4.1 enables browsers to communicate with RabbitMQ using AMQP 1.0.

Please note: the new AMQP over WebSocket plugin is a closed-source feature available exclusively in the commercial VMware Tanzu RabbitMQ offering. It is not part of the open source RabbitMQ distribution.

While this blog post illustrated the feature with a playful example involving a bunny, AMQP 1.0 is an efficient protocol built for business messaging. Think of enterprise tools like Salesforce, Workday, or Jira - all running in the browser and benefiting from real-time messaging.

See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.

Highlights​

Some key improvements in this release are listed below.

Quorum Queue Throughput and Parallelism Improvements​

Quorum queue log reads are now offloaded to channels (sessions, connections).

In practical terms this means improved consumer throughput, lower interference of publishers on queue delivery rate to consumers, and improved CPU core utilization by each quorum queue (assuming there are enough cores available to the node).

Initial Support for AMQP 1.0 Filter Expressions​

Support for the properties and application-properties filters of AMQP Filter Expressions Version 1.0 Working Draft 09.

As described in the AMQP 1.0 Filter Expressions blog post, this feature enables multiple concurrent clients each consuming only a subset of messages from a stream while maintaining message order.

Feature Flags Quality of Life Improvements​

Graduated (mandatory) feature flags several minors ago has proven that they could use some user experience improvements. For example, certain required feature flags will now be enabled on node boot when all nodes in the cluster support them.

See core server changes below as well as the GitHub project dedicated to feature flags improvements for the complete list of related changes.

rabbitmqadmin v2​

rabbitmqadmin v2 is a major revision of the original CLI client for the RabbitMQ HTTP API.

It supports a much broader set of operations, including health checks, operations on federation upstreams, shovels, transformations of exported definitions, (some) Tanzu RabbitMQ HTTP API endpoints, --long-option and subcommand inference in interactive mode, and more.

Breaking Changes and Compatibility Notes​

Initial AMQP 0-9-1 Maximum Frame Size​

Before a client connection can negotiate a maximum frame size (frame_max), it must authenticate successfully. Before the authenticated phase, a special lower frame_max value is used.

With this release, the value was increased from the original 4096 bytes to 8192 to accommodate larger JWT tokens.

Clients that do override frame_max now must use values of 8192 bytes or greater. We recommend using the default server value of 131072: do not override the frame_max key in rabbitmq.conf and do not set it in the application code.

amqplib is a popular client library that has been using a low frame_max default of 4096. Its users must upgrade to a compatible version (starting with 0.10.7) or explicitly use a higher frame_max.

MQTT​

• The default MQTT Maximum Packet Size changed from 256 MiB to 16 MiB.

  This default can be overridden by configuring mqtt.max_packet_size_authenticated. Note that this value must not be greater than max_message_size (which also defaults to 16 MiB).

etcd Peer Discovery​

The following rabbitmq.conf settings are unsupported:

• cluster_formation.etcd.ssl_options.fail_if_no_peer_cert
• cluster_formation.etcd.ssl_options.dh
• cluster_formation.etcd.ssl_options.dhfile

Erlang/OTP Compatibility Notes​

This release requires Erlang 26.2 and supports Erlang 27.x.

Provisioning Latest Erlang Releases explains what package repositories and tools can be used to provision latest patch versions of Erlang 26.x and 27.x.

Release Artifacts​

Artifacts for preview releases are distributed via GitHub releases:

• In main repository, rabbitmq/rabbitmq-server
• In the development builds repository, rabbitmq/server-packages

There is a 4.1.0 preview version of the community RabbitMQ image.

Upgrading to 4.1.0​

Documentation guides on upgrades​

See the Upgrading guide for documentation on upgrades and GitHub releases for release notes of individual releases.

This release series supports upgrades from 4.0.x and 3.13.x.

Blue/Green Deployment-style upgrades are available for migrations from RabbitMQ 3.12.x series.

New Required Feature Flags​

None. The required feature flag set is the same as in 4.0.x.

Mixed version cluster compatibility​

RabbitMQ 4.1.0 nodes can run alongside 4.0.x nodes. 4.1.x-specific features can only be made available when all nodes in the cluster upgrade to 4.1.0 or a later patch release in the new series.

While operating in mixed version mode, some aspects of the system may not behave as expected. The list of known behavior changes will be covered in future updates. Once all nodes are upgraded to 4.1.0, these irregularities will go away.

Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended periods of time (no more than a few hours).

Recommended Post-upgrade Procedures​

This version does not require any additional post-upgrade procedures compared to other versions.

Release Artifacts​

Release artifacts can be obtained on GitHub as well as RPM, Debian package repositories.

Community Support Now Only Covers the 4.1.x Series​

With the release of RabbitMQ 4.1.0, this series is no longer covered by community support.

Future 4.0.x releases will only be available to paying customers via the Broadcom customer portal.

All non-paying users must upgrade to 4.1.0 in order to be covered by community support from the core team.

Release Artifacts​

Release artifacts can be obtained on GitHub as well as RPM, Debian package repositories.

Community Support Now Only Covers the 4.1.x Series​

With the release of RabbitMQ 4.1.0, this series is no longer covered by community support.

Future 4.0.x releases will only be available to paying customers via the Broadcom customer portal.

All non-paying users must upgrade to 4.1.0 in order to be covered by community support from the core team.

Upgrade Guidance​

If upgrading from a version prior to 4.0, please consult the 4.0 release notes.

There are at least 4 notable changes:

1. Lower and more stable memory usage for quorum queues
2. Much better performance when consuming a long quorum queue
3. Better performance for Websocket connections
4. Lower memory usage and/or higher throughput for TCP connections

Quorum Queues: Lower Memory Usage​

Quorum queues in RabbitMQ 4.1 should use less memory in many situations. As you may be aware, in the past quorum queues had a sawtooth-like memory usage pattern. They would fill up an in-memory buffer (cache) for recent Raft operations and once full, the buffer was emptied and then filled up again.

In RabbitMQ 4.1, these entries are deleted much more frequently, leading to a more stable memory usage under many conditions. Here's the memory usage of a cluster initially running 4.0 and then upgraded to 4.1:

The exact details of the workload are not super important, since this difference should be visible for many different workloads, but for completeness, here they are:

• there were 10 quorum queues
• all messages were 1kb in size
• each queue received 500 messages per second from a single publisher (so 5000 messages per second total for all queues)
• each queue had a single consumer (a vast majority of messages was consumed within 10 milliseconds since being published)
• the queues were effectively empty, since all messages were promptly consumed

It's worth remembering that such a low and stable memory usage cannot be expected in all conditions. For example, quorum queues keep metadata about messages in the queue in memory and therefore, if you have many messages in the queues (messages are not immediately consumed), this metadata will consume memory. There are also other factors and in-memory structures that will grow based on the workload. Nevertheless, memory usage should be lower and less spiky in many common situations.

Quorum Queues: Offloading Disk Reads​

Let's consider a completely different workload - one where messages accumulate in the queues and then consumers need to catch up to empty the queues. Historically, quorum queues could get overwhelmed by an influx of consumers, especially if the messages were large and consumers requested a lot of them (either they had a large prefetch buffer or there were a lot of consumers, or both). In this scenario, the queue could get so busy reading the older messages from disk (to dispatch them to the consumers) that publishers had to wait quite a bit to have their messages accepted by the queue.

In RabbitMQ 4.1, such disk reads are offloaded to the AMQP 0.9.1 channel or AMQP 1.0 session processes (based on the protocol used). The queue has much less work to do and can keep serving the publishers.

Let's take a look at the differences in publishing and consumption rates between 4.0 and 4.1:

Here's what's going on in this graph:

1. We have two clusters running, 4.0 (green lines) and 4.1 (yellow)
2. Both clusters receive ~6000 messages per second, each message is 20kb
3. Initially, there are no consumers; hence, the consumption rate is zero
4. After some time, consumers start and try to consume messages
5. In each environment, there are are now 10 consumers, each with a 300 message prefetch buffer
6. 4.0 environment is overwhelmed - the publishing rate drops to just ~100 messages per second
7. Meanwhile, 4.1 environment continues serving the publishers with no noticeable impact
8. Additionally, the consumption rate is almost double in the 4.1 environment
9. Once the backlog of messages is consumed, both environments can handle ~7000 messages per second in and out

Not only are the publishers not throttled, but the consumers are also able to consume the messages much faster!

Better Performance of WebSocket Connections​

To serve HTTP connections, RabbitMQ uses a popular Erlang HTTP server called Cowboy (developed by Loïc Hoguin long before he joined the RabbitMQ team). RabbitMQ 4.1 upgrades Cowboy to version 2.13.0, which significantly improves WebSocket performance for all systems that rely on Cowboy, including RabbitMQ. Upgrading to RabbitMQ 4.1 should therefore be particularly beneficial to anyone using AMQP, MQTT or STOMP over WebSocket.

TCP Buffer Auto-Tuning​

A key improvement described in Cowboy 2.13.0 release blog post is the dynamic TCP buffer auto-tuning. For WebSocket connections, these improvements in Cowboy automatically benefit RabbitMQ users, since Cowboy handles HTTP connections to RabbitMQ.

In RabbitMQ 4.1, we incorporated the same TCP buffer auto-tuning mechanism into the AMQP listener, which is a completely separate code path and does not use Cowboy (since Cowboy is an HTTP server). Thanks to this work, RabbitMQ should use less memory for AMQP 0.9.1 and 1.0 connections, without a noticeable performance penalty. The amount of memory saved depends on your current buffer size and the number of connections, but in our tests it saved a few hundred megabytes of memory in a system with a few thousand connections.

It's worth pointing out that the buffer discussed in this paragraph is a user-space buffer and should not be confused with recbuf / sndbuf buffers which are kernel buffers. These can be statically configured and if they are not, they are auto-tuned by the Linux kernel (the behaviour on other operating systems may differ).

The value of tcp_listen_options.buffer, which used to control the size of the now auto-tuned buffer, is ignored.

What Is Peer Discovery?​

Say you want to have a 3-node RabbitMQ cluster - you start 3 instances of RabbitMQ but then what? You can manually tell two of them to join the third one using rabbitmqctl join_cluster command and voilà, you have a 3-node cluster.

Most users would prefer this process to be automated however. That's where peer discovery comes in. There is a handful of peer discovery plugins available in RabbitMQ for different situations. The simplest one is called classic peer discovery and allows you to just put the hostnames of the nodes in the configuration file, so that RabbitMQ automatically initiates the cluster formation with them upon startup.

However, based on how you deploy RabbitMQ, the hostnames may not be known upfront. Even if they are, you need a different configuration file for each cluster, which may be inconvenient if you want a quick way to spin up new clusters for testing environments for example.

In such cases, you can use other peer discovery plugins, which allow nodes to register with some external systems such as Consul or etcd and query these systems for a list of registered nodes. This way you don't need to know the hostnames upfront - the nodes discover each other automatically.

Kubernetes Peer Discovery before RabbitMQ 4.1​

Before RabbitMQ 4.1, rabbitmq_peer_discovery_k8s performed the peer discovery by querying the Kubernetes API server for a list of endpoints behind a service (Kubernetes automatically registers pods of a given StatefulSet as endpoints). However, there were a few issues with this approach:

1. some users reported that occasionally, cluster formation would fail and the pods would form multiple separate clusters; we never received enough data to diagnose this issue and it never occurred in our testing (we tried thousands of times...)
2. it required permissions to query the Kubernetes API; not a big deal, but it was unnecessary and some security-conscious users were asking why we needed this
3. it was a convoluted way of asking a question, we already know the answer to...

Kubernetes Peer Discovery in RabbitMQ 4.1​

When deploying RabbitMQ to Kubernetes, you should always use a StatefulSet. All pods that belong to a StatefulSet are named consistently with the name of the StatefulSet, followed by a hyphen and an ordinal index. The ordinal index start is configurable, but is almost always 0, so let's just assume it is 0. Given that, a 3-node cluster deployed to Kubernetes will always have nodes with suffixes -0, -1 and -2. There's no need to query the Kubernetes API to know this!

The new plugin doesn't perform any Kubernetes API queries. It just assumes that a pod with -0 suffix will exist and treats it as the "seed" node. All other nodes will join the cluster by joining the -0 node. If the -0 node is not up, other nodes will wait forever for it to come up (they will never form a cluster without the -0 node). Remember that peer discovery only happens when a node starts for the first time, so "waiting forever for node -0" only applies to the first time you deploy a given cluster.

Advanced Configuration​

For the vast majority of users, this upgrade should be completely transparent. First of all, since peer discovery is only performed when a node starts for the first time, if you upgrade an existing cluster, peer discovery changes won't affect you.

Second, the new plugin accepts, but ignores, all configuration options of the old plugin. You will see some warnings in the logs about deprecated options being used, but you can safely ignore them.

If the default configuration doesn't work for you, there are two settings you can use:

1. If you are using an ordinal start other than 0 (and seriously, why would you?!), you should configure the plugin by setting cluster_formation.k8s.ordinal_start = N where N is the ordinal start. When set, all nodes will try to join the -N node, rather than the -0 node.

2. Additionally, you can set cluster_formation.k8s.seed_node = rabbit@seed-node-hostname to just say what the seed node is. We don't expect this setting to ever be needed, but it's there if you really need it.

What If I'm Using The Cluster Operator?​

Cluster Operator is the recommended way of deploying RabbitMQ to Kubernetes, so if you are using it - great. You should be able to continue using it with no changes. You will see the aforementioned warnings in the logs, because the Cluster Operator allows deploying different RabbitMQ versions, not just 4.1. Therefore, for the time being, it will continue setting values required by the old version of rabbitmq_peer_discovery_k8s in the configuration file. Such a configuration works for both 4.1 and older versions. At some point in the future, Cluster Operator will drop support for RabbitMQ versions older than 4.1 and we'll remove these settings from the ConfigMap declared by the Cluster Operator.

RabbitMQ 4.0.7 is a new patch release in the 4.0.x series.

This series is currently covered by community support.

Release Artifacts​

Release artifacts can be obtained on GitHub as well as RPM, Debian package repositories.

Upgrade Guidance​

If upgrading from a version prior to 4.0, please consult the 4.0 release notes.

RabbitMQ 4.0.6 is a new patch release in the 4.0.x series.

This series is currently covered by community support.

Release Artifacts​

Release artifacts can be obtained on GitHub as well as RPM, Debian package repositories.

Upgrade Guidance​

If upgrading from a version prior to 4.0, please consult the 4.0 release notes.

For publicly available open source releases, see the 4.0.x series.

Release Artifacts

Release artifacts for the 3.13.x series can be obtained via the Broadcom customer portal.

Upgrade Guidance

If upgrading from a version prior to 3.13.9, please consult the 3.13.0 release notes.

This edition includes additional feature such as Warm Standby Replication, intra-cluster traffic compression and a FIPS-enabled Erlang runtime.

Tanzu RabbitMQ is available on Kubernetes, as an OCI image, and an OVA image.

This release is based on open source RabbitMQ 3.13.x and includes all the latest backports in open source RabbitMQ 3.13.8.

Release Artifacts

Release artifacts for the 3.13.x series can be obtained via the Broadcom customer portal:

• Artifacts
• Tanzu RabbitMQ on Kubernetes

Upgrade Guidance

If upgrading from a version prior to 3.13.9, please first consult the open source RabbitMQ 3.13.0 release notes.

Let's start by clarifying what we mean by storing messages in memory, since it is not a precise statement. If interpreted as "RabbitMQ uses memory to process messages", this phrase is certainly true. When client applications send data to RabbitMQ, that data appears in memory buffers first (that's true for all network-based software). It is also true, that RabbitMQ might cache some messages in memory, for example to improve performance (again, true for virtually all software that needs to serve data).

However, I keep hearing this phrase used to express concerns about message durability - if you power off the server, your messages will be lost! In this context, modern RabbitMQ versions almost never store messages in memory.

We'll go through different queue types and discuss how the messages are processed, stored and when they are confirmed to the publishers. Publisher confirmations are critical here - RabbitMQ doesn't offer guarantees for messages that were not confirmed. If you haven't received the confirmation, you can't even know whether the message reached RabbitMQ (the network connection could have failed for example). With that in mind, let's go through the different queue types.

Classic Queues​

Classic queues is the oldest kind of queues in RabbitMQ and the main source of "RabbitMQ stores messages in memory" misconception. RabbitMQ was first released in 2007. Disks were significantly slower back then and therefore classic queues were designed to try to avoid writing messages to disk. There was a whole bunch of settings to configure when RabbitMQ should write messages to disk (a process called "paging") so it would not keep messages in memory indefinitely but indeed, you could say RabbitMQ stored messages in memory back then. However, that was a long time ago.

In RabbitMQ 3.6, released in 2015, the "lazy mode" was introduced. A queue configured to be lazy, always stored messages on disk and didn't keep them in memory at all. This means that "RabbitMQ stores messages in memory" was not true 10 years ago. It'd still do this by default, but it was completely optional.

Lazy mode was removed in RabbitMQ in 3.12, released in 2023, but the default (and the only available) behaviour changed and is similar to the lazy mode, although not exactly the same. Therefore, for over a year now, classic queues don't store messages in memory and can't even be configured to do so. The myth is almost entirely false at this point.

Almost entirely? So here's how classic queues work right now: they accumulate incoming messages in a small in-memory buffer and write them to disk as a batch, as soon as the in-memory buffer is full. Since we don't know if more messages will come, there are additional triggers that flush that buffer, including a flush after a certain number of batched messages (even if they are too small to use the whole buffer) and after a certain number of operations on that batch is performed. Additionally, the queue monitors how quickly messages are consumed and makes decisions based on that (if there are fast consumers, more messages will be cached in memory). Finally, there's the last-resort trigger which will flush the buffer every 200ms. So the absolute maximum of how long a message can be stored in memory is 200ms but in practice, I've never seen this happen. Publishers usually receive confirms within a few milliseconds and they are sent only after the message was written to disk.

But I See No Disk Activity When I Use a Classic Queue!​

Indeed, it's entirely possible to publish messages to classic queues and see virtually no disk writes nor disk reads. How is that possible? It's an optimisation for a very specific, but relatively common case. As explained above, messages can be briefly kept in memory but if there are active consumers that are waiting for messages (their prefetch buffer is not full), messages that reach the queue are dispatched to the consumers immediately, without waiting for their batch to be written to disk. A message that gets acknowledged by the consumer before this message's batch is written to disk, will not be written to disk at all, because it simply doesn't need to be. Queues don't store acknowledged messages, so if the message is acknowledged before it is written, it doesn't get written. If it's acknowledged after it was written, it's deleted from the queue (the actual removal from disk will happen later, asynchronously, but it is considered deleted immediately).

It's worth mentioning that classic queues have two separate storage mechanisms. Messages below 4kb (configurable through queue_index_embed_msgs_below) are stored in a per-queue message store and messages above that threshold are stored in a per-vhost message store. The optimisation mentioned above only works for messages that would be stored in the per-queue message store.

So here it is, in modern RabbitMQ versions, classic queues store messages in memory for a very short period of time (milliseconds) and no more than 200ms for sure. They may not write messages to disk at all, if the messages are small, and consumed quickly enough, but this is just a performance optimisation. I'll leave it up to you to decide if that qualifies as "RabbitMQ stores messages in memory", but I think a more accurate statement would be "when a message is delivered to a classic queue, RabbitMQ writes messages to disk with a short delay". But yes, that means that for a brief moment, they are only in memory.

Surely, Transient Messages Are Stored in Memory?​

No. Again, things were different in the past but as of RabbitMQ 4.0, the only difference between persistent and transient messages is when RabbitMQ sends back the publisher confirm. The messages are stored the same way as described above.

For persistent messages, the confirm is sent when either of these two events takes place:

1. The message is written to disk
2. The message is delivered and acknowledged by a consumer (if that happens before it's written to disk)

For transient messages, the confirm is sent as soon as the message reaches the queue and enters the in-memory buffer. Since the message is transient, the guarantees are lax: the queue received the message, the publisher can move on.

What About fsync?​

fsync is a low-level file system operation that should ensure that messages are really written to disk. There are multiple layers of I/O buffers between a user-space process such as RabbitMQ and actual hardware, including operating system buffers and internal disk buffers. Performing a write without performing fsync doesn't guarantee that the data will survive a sudden power loss. Unfortunately, fsync is a relatively slow operation, so any I/O intensive software has to decide if, and when, to call it. While classic queues call fsync in some cases (for example, when RabbitMQ stops gracefully), fsync is not performed before publisher confirms are sent. Therefore, even durable messages that a publisher received a confirmation for, can technically be lost if the server crashes. If you need stronger guarantees, you can use quorum queues.

Quorum Queues​

From the initial release in RabbitMQ 3.8 (released in 2019), quorum queues always stored messages on disk. While the initial versions had an additional in-memory cache for messages, it was removed in RabbitMQ 3.10.

The situation is therefore simple: if the publisher received a confirmation, this means the message had already been written to disk and fsync-ed on the quorum of nodes (in the most common scenario of a 3-node cluster, that means it was written and fsync-ed on at least 2 nodes).

Since RabbitMQ doesn't offer any guarantees for messages that have not been confirmed to publishers, we could pretty much stop here. However, for the sake of completeness, I'll mention that some messages are technically in memory:

1. The queue process has a mailbox (an Erlang/OTP concept) where requests to the queue process (such as enqueue/dequeue operations) arrive for processing. The quorum queue process receives messages from the mailbox and processes them as a batch. When there's a lot of requests, these operations may accumulate in the mailbox and therefore, assuming there are enqueue operations there, at this point, some messages are only in-memory. However, this generally means RabbitMQ is at least briefly overloaded and either way, these operations are usually processed within a few milliseconds. Plus, these messages are not confirmed yet.
2. Quorum queues rely on the Raft protocol and our Raft implementations stores the most recent Raft operations in memory. For enqueue operations this means that the message is in memory as well. However, at this point the message is already written to disk and fsync-ed or it hasn't been confirmed yet.

Streams​

With Streams, the situation is even simpler than with quorum queues: Streams never supported keeping messages in memory, period. The main difference between queues and stream in general, is that streams can be read multiple times and therefore consuming a message doesn't remove that message from a stream. There's no point in storing messages only in memory, if we need to be able to deliver them to consumers multiple times, potentially long after they were published.

Streams do not perform fsync, since they were optimised for high message throughput.

For completeness, just like quorum queues (and any other Erlang process), the stream process has a mailbox where requests to the stream process arrive. There's therefore a moment where the messages are stored in memory for a short time. Once again though, these are messages that have not been confirmed yet and they rarely stay in memory for more than a few milliseconds.

MQTT QoS 0 Queues​

RabbitMQ 3.12 introduced Native MQTT support, and as part of that work, a new queue type was introduced, specifically for MQTT QoS 0 consumers (you can't explicitly declare a queue of this type, you have to create an MQTT QoS 0 subscription). Since QoS 0 basically means best-effort but no guarantees, QoS 0 messages are not written to disk at all and are delivered directly to the consumers that are present. Effectively, there's no queue at all (beyond the Erlang mailbox). Messages received from the publisher are immediately delivered to the consumers and removed from memory.

Does this qualify as storing messages in memory? I'd say it doesn't - the messages are in memory initially, simply because that's how computers work, and are removed from memory as soon as they are delivered to the consumers. We don't really store them in memory - we just process them and never write them to disk in this case. You can disagree and say that this is exactly what "storing messages in memory means" but even then - this only applies to MQTT QoS 0 usage and the messages will generally stay in memory for no more than a fraction of second.

Message Metadata​

So far I focused on message bodies, since that's what people usually mean when talking about storing messages in memory. However, RabbitMQ also needs to keep track of the messages that are currently present in the queues. For example, when a queues has a defined x-max-length limit, RabbitMQ needs to keep track of the total size of all the messages in the queue, so when it delivers a message, it keeps the message size (but not the message body itself) in memory, to just subtract it quickly from the total size of the queue, once the consumer acknowledges the messages.

This kind of metadata is stored differently by different queue type, but even when stored in memory, it will consume significantly less memory than the message bodies would and doesn't change any guarantees about the message durability.

Here's how we store the metadata for different queue types:

• Classic queues
  • for messages stored in the per-queue message store, no data is stored in memory
  • for messages stored in the per-vhost message store, there's some metadata in memory
• Quorum queues
  • metadata is stored in memory (at least 32 bytes per message, sometimes a bit more, for example when message TTL is used)
• Streams
  • no message metadata is stored in memory

This basically means that for messages under 4KB stored in classic queues, as well as for streams, regardless of how many messages there are in the queue/stream, the memory usage is constant. You will run out disk before you run out of memory (you should configure retention/length limits to avoid running out disk, but that's a different story).

Here's an illustration highlighting the difference between the two classic queue storage mechanisms. In this test, I first published 1 million messages of 4000 bytes each, then deleted the queue and published 1 million messages of 4100 bytes each. As you can see, the memory usage was stable in the first phase (small fluctuations notwithstanding), but when publishing larger messages, we can see the memory usage grows as well. This is because 4100 bytes is above the threshold, so these messages are stored in the per-vhost message store and the per-vhost message store keeps some metadata in memory. A million 4KB messages would have taken up 4GB of memory to store, while the actual usage is still below 400MB.

Summary​

Here's the summary of the key points.

The flexibility provided by RabbitMQ, with support for multiple protocols, queue types and other configurations (eg. single node, vs a cluster with queue replication), combined with 18 years of history and evolution, means that almost any "RabbitMQ does/doesn't do X" statement is incorrect or at least imprecise. They should almost always be quantified with a specific version and configuration.

Going back to the title of this post, I think it's fair to say that "RabbitMQ doesn't store messages in memory" is much closer to the truth, than the opposite claim, which still circulates in discussions involving RabbitMQ. Regardless of the queue type, there is no configuration in which publishing, say, 1GB of messages to RabbitMQ with no connected consumers, would lead to 1GB of memory being used to store these messages. Most importantly, if you want high data safety guarantees, quorum queues are available and store data safely by default. If you publish a message to a quorum queue and receive the confirmation, it'd take a disastrous event for RabbitMQ to lose it (and if you want to protect messages from disastrous events, you might be interested in the commercial Warm Standby Replication plugin).

If you don't need such data safety guarantees, you don't have to pay the intrinsic overhead of data safety. Just use the right tool for the job.

The Erlang Port Mapper Daemon (epmd) is a built-in component that helps Erlang-based applications (including RabbitMQ) discover each other’s distribution ports. Together with DNS for hostname resolution, epmd is a piece of infrastructure RabbitMQ nodes rely on for clustering, inter-node communication and CLI tools connectivity.

While epm is very limited in scope, its exposure to the public Internet often means that Erlang distribution ports are also exposed. This creates a potential security risk: if attackers find these distribution ports, they'd be one secret value away from being able to run CLI commands against the node (or cluster).

Recent scans have revealed over 85,000 instances of publicly accessible epmd, with roughly half associated with RabbitMQ servers.

Fortunately, all it usually takes to mitigate this risk is limiting network access to a range of ports. epmd and inter-node communication can also be limited to local network interfaces, in particular for single node clusters used for running tests.

Read the full article on the Erlang Ecosystem Foundation blog.

RabbitMQ 4.0.5 is a new patch release in the 4.0.x series.

This series is currently covered by community support.

Release Artifacts​

Release artifacts can be obtained on GitHub as well as RPM, Debian package repositories.

Upgrade Guidance​

If upgrading from a version prior to 4.0, please consult the 4.0 release notes.

This feature enables RabbitMQ to support multiple concurrent clients, each consuming only a specific subset of messages while preserving message order. Additionally, it minimizes network traffic between RabbitMQ and its clients by dispatching only the messages that match the clients' interests.

In this blog post, we’ll explore what AMQP filter expressions are and walk through a simple Java example of how to use them.

Specification​

As outlined in the Native AMQP 1.0 blog post, one of AMQP 1.0's strengths is its extensibility, supported by numerous extension specifications. RabbitMQ 4.1 takes advantage of the extension specification AMQP Filter Expressions Version 1.0 Working Draft 09.

This specification defines AMQP type definitions for message filter expressions. Filter expressions are predicates evaluated against a message, returning either true or false. If a predicate evaluates to true, the broker dispatches the message to the consumer.

RabbitMQ 4.1 implements a subset of this specification, including:

• § 4.2.4 properties filter: Applies to the immutable properties section of the message.
• § 4.2.5 application-properties filter: Applies to the immutable application-properties section of the message.

Example​

Imagine each message carries metadata specifying a particular color. Different consumers can subscribe to the same stream, filtering messages to receive only those matching the color they are interested in.

The first consumer receives all green messages. The second consumer receives all purple messages. The third consumer receives all blue messages.

docker run -it --rm --name rabbitmq \
    -p 5672:5672 -p 15672:15672 \
    rabbitmq:4.1-rc-management

mvn clean compile exec:java

publisher sent message 0 with color green
publisher sent message 1 with color blue
publisher sent message 2 with color purple
publisher sent message 3 with color purple
publisher sent message 4 with color green
publisher sent message 5 with color green
consumer (filter green) received message 0
consumer (filter green) received message 4
consumer (filter green) received message 5
consumer (filter purple) received message 2
consumer (filter purple) received message 3
consumer (filter blue) received message 1
consumer (filter &s:e) received message 1
consumer (filter &s:e) received message 2
consumer (filter &s:e) received message 3

AMQP filter expressions enable multiple clients to concurrently consume specific subsets of messages from the same stream while preserving message order. This feature also minimizes network traffic between RabbitMQ and its clients by dispatching only the messages that match each client’s interests.

Stream Filtering Comparison​

The AMQP filter expressions feature described in this blog post should not be confused with the Bloom filter-based stream filtering introduced in RabbitMQ 3.13.

Both features serve the same purpose: filtering messages from a stream. However, their implementations differ, resulting in distinct characteristics:

Both features can be used together when consuming via AMQP 1.0.

Summary​

RabbitMQ 4.1 addresses the challenge of enabling multiple consumers on a single queue/stream while ensuring certain messages (e.g., those with the same subject or ID) are always processed by the same consumer, preserving in-order processing.

Although this feature is not available for classic queues or quorum queues, AMQP filter expressions allow consumers to filter messages when consuming from a stream. Since streams are immutable logs, total message order is maintained.