RocketMQ Broadcast Consumption: Mechanics, Limitations, and Practical Usage

RocketMQ provides two distinct consumption models: clustering and broadcasting, each suited to different architectural requirements.

Core Behavioral Differences

  • Clustering mode: Messages from a topic are distributed across consumers within the same group—each message is processed by exactly one instance. This enables horizontal scaling and fault tolerance.

  • Broadcasting mode: Every message is delivered to all active consumers in the group, regardless of instance count. This ensures uniform state propagation or fan-out notifications.

Key Implementation Distinctions

1. No Automatic Retry Topic Subscription

In DefaultMQPushConsumerImpl.copySubscription(), clustering consumers automatically subscribe to a retry topic (%RETRY%<group>), enabling failed-message redelivery. Broadcasting consumers skip this step entirely:

switch (this.defaultMQPushConsumer.getMessageModel()) {
    case BROADCASTING:
        break; // No retry topic registration
    case CLUSTERING:
        final String retryTopic = MixAll.getRetryTopic(this.defaultMQPushConsumer.getConsumerGroup());
        SubscriptionData retryData = FilterAPI.buildSubscriptionData(retryTopic, SubscriptionData.SUB_ALL);
        this.rebalanceImpl.getSubscriptionInner().put(retryTopic, retryData);
        break;
}

As a result, broadcast consumers do not support message retries.

2. Local Offset Storage Only

Offset persistence differs fundamentally:

switch (this.defaultMQPushConsumer.getMessageModel()) {
    case BROADCASTING:
        this.offsetStore = new LocalFileOffsetStore(this.mQClientFactory, this.defaultMQPushConsumer.getConsumerGroup());
        break;
    case CLUSTERING:
        this.offsetStore = new RemoteBrokerOffsetStore(this.mQClientFactory, this.defaultMQPushConsumer.getConsumerGroup());
        break;
}

Broadcast offsets are written to disk under $HOME/.rocketmq_offsets/{clientId}/{group}/offsets.json. There is no broker-side offset coordination—each consumer independently tracks its own progress per queue.

3. Full Queue Assignment in Rebalancing

In RebalanceImpl.rebalanceByTopic(), broadcasting bypasses queue allocation logic:

case BROADCASTING: {
    Set<MessageQueue> allQueues = this.topicSubscribeInfoTable.get(topic);
    if (allQueues != null) {
        this.updateProcessQueueTableInRebalance(topic, allQueues, isOrder);
    }
    break;
}

Every consumer receives all message queues for the subscribed topic—no partitioning or hashing occurs.

4. Incompatibility with Ordered Delivery

Ordered consumption relies on queue-level locking via lockMQPeriodically(), scheduled only in clustering mode:

if (MessageModel.CLUSTERING.equals(consumerImpl.messageModel())) {
    this.scheduledExecutorService.scheduleAtFixedRate(
        () -> ConsumeMessageOrderlyService.this.lockMQPeriodically(),
        1000, ProcessQueue.REBALANCE_LOCK_INTERVAL, TimeUnit.MILLISECONDS);
}

Since broadcast consumers never acquire locks, they cannot guaratnee message ordering, even when using MessageListenerOrderly.

5. No Failed-Message Recovery Path

In ConsumeMessageConcurrentlyService.processConsumeResult(), failure handling diverges sharply:

case BROADCASTING:
    for (int i = ackIndex + 1; i < consumeRequest.getMsgs().size(); i++) {
        MessageExt msg = consumeRequest.getMsgs().get(i);
        log.warn("BROADCASTING: consume failed, discarding {}", msg.getMsgId());
    }
    break;
case CLUSTERING:
    // Sends failed messages back to broker via CONSUMER_SEND_MSG_BACK
    // Retries up to maxReconsumeTimes
    break;

Broadcast failures result in silent discard—not retry, not dead-letter routing.

Minimal Working Example

public class BroadcastConsumerDemo {
    public static void main(String[] args) throws Exception {
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("driver-alert-group");
        consumer.setNamesrvAddr("127.0.0.1:9876");
        consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET);
        
        // Critical: enable broadcasting
        consumer.setMessageModel(MessageModel.BROADCASTING);
        
        consumer.subscribe("dispatch-notifications", "*");
        
        consumer.registerMessageListener((msgs, context) -> {
            msgs.forEach(msg -> {
                String payload = new String(msg.getBody());
                System.out.println("Received: " + payload);
                // Forward to driver's TCP channel, update local cache, etc.
            });
            return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
        });
        
        consumer.start();
        System.out.println("Broadcast consumer running...");
    }
}

Real-World Use Case: Driver Dispatch Notifications

A ride-hailing platform uses broadcasting to notify all dispatch services simultaneously about a new order. Each service:

  • Maintains an in-memory map of active drivers → TCP channels.
  • Receives every dispatch event.
  • Checks locally whether the target driver is connected.
  • If found, pushes the notification over the existing connection.

Network resilience is handled externally: drivers poll periodically, and services implement exponential backoff for unacknowledged pushes.

Feature Compatibility Summary

Feature Clustering Broadcasting
Ordered consumption
Offset reset
Message retry
Offset storage Broker Local file

Tags: rocketmq messaging broadcast-consumption distributed-systems

Posted on Sat, 12 Sep 2026 16:11:48 +0000 by Amman-DJ