RocketMQ Installation and Configuration Guide

Download

Download the binary distribution. For source compilation, Maven is required, and JDK should be pre-installed.

Installation Steps

unzip rocketmq-all-4.7.0-bin-release.zip
mv rocketmq-all-4.7.0-bin-release rocketmq

# Navigate to bin directory and start the NameServer
nohup sh mqnamesrv &

# View startup logs in the home directory
tail -f ~/logs/rocketmqlogs/namesrv.log

# Start the broker
nohup sh mqbroker -n localhost:9876 &

If the processes don't appear visible with the jps command, RocketMQ by default configures very large virtual memory. You'll need to modify the default JVM settings.

Edit runbroker.sh and adjust the JVM parameters:

# Original settings
JAVA_OPT="${JAVA_OPT} -server -Xms8g -Xmx8g -Xmn4g"

# Modified for lower resource usage
JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn256m"

Edit runserver.sh and apply similar memory adjustments:

JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"

Testing Message Production and Consumption

Open two terminal windows and navigate to the RocketMQ bin directory in each.

Producer (message sender):

# Set the NameServer address environment variable
export NAMESRV_ADDR=localhost:9876

# Send messages using the bundled example
sh tools.sh org.apache.rocketmq.example.quickstart.Producer

Consumer (message receiver):

# Set the NameServer address environment variable
export NAMESRV_ADDR=localhost:9876

# Receive messages
sh tools.sh org.apache.rocketmq.example.quickstart.Consumer

Shutdown Procedures

# Stop the NameServer
sh mqshutdown namesrv

# Stop the broker
sh mqshutdown broker

Cluster Installation

Role Definitions

  • Producer: Message sender
  • Consumer: Message receiver
  • Broker: Message storage and transmission component
  • NameServer: Broker management service
  • Topic: Message classification identifier; one producer can send to multiple topics; one consumer can subscribe to multiple topics
  • Message Queue: Partitions of a Topic enabling parallel message production and consumption

Cluster Characteristics

NameServer nodes are essentially stateless and can be deployed in a cluster. There's no information synchronization between nodes.

Broker deployment is more complex. Each Broker has a Master and can have multiple Slaves. However, each Slave can only correspond to one Master. The Master-Slave relationship is defined by sharing the same brokerName with different brokerId values (0 for Master, non-zero for Slave). Multiple Masters can also be deployed.

Each Broker establishes persistent connections with all NameServer cluster nodes and periodically registers Topic information.

Producers randomly select one NameServer node to establish a persistent connection, periodically fetch Topic routing information, and maintain persistent connections with the Master providing Topic services. Producers are completely stateless and can be deployed in clusters.

Consumers similarly connect to a randomly selected NameServer node, fetch Topic routing information, and establish connections with both Master and Slave brokers. Consumers can subscribe from either Master or Slave, depending on Broker configuration.

Cluster Deployment Modes

Single-Mode Deployment

This mode poses significant risk since service becomes unavailable when the Broker restarts or fails. Not recommended for production environments, but suitable for local testing.

Multi-Master Mode

A cluster consists entirely of Masters without Slaves (e.g., 2 or 3 Masters):

Advantages: Simple configuration. A single Master's downtime or restart has no impact on applications. With RAID10 disks, messages remain safe even if the machine cannot be recovered (async flush may lose some messages, sync flush loses none). Maximum performance is achieved.

Disadvantages: During machine downtime, undelivered messages cannot be subscribed until the machine recovers, affecting message timeliness.

Multi-Master Multi-Slave Mode (Async Replication)

Each Master has one Slave, creating multiple Master-Slave pairs. HA uses async replication with minimal delay between primary and backup (milliseconds):

Advantages: Even with disk damage, message loss is minimal and message timeliness is unaffected. When a Master fails, consumers continue consuming from the Slave transparently without manual intervention. Performance is nearly identical to multi-Master mode.

Disadvantages: Some messages may be lost if Master fails and disk is damaged.

Multi-Master Multi-Slave Mode (Sync Dual-Write)

Each Master has one Slave with HA using synchronous dual-write. Success is only returned when both primary and backup write successfully:

Advantages: Neither data nor service has a single point of failure. Messages experience no delay during Master failure, providing high availability and data durability.

Disadvantages: Performance is slightly lower than async replication (approximately 10% lower) with higher response time per message. Currently, the backup cannot automatically become primary after primary failure.

Cluster Setup Configuration

This guide implements high availability using 2M-2S (synchronous dual-write) architecture:

Node IP Address Roles Architecture Ports
1 192.168.100.120 nameserver, brokerserver Master1, Slave2 9876, 10911, 11011
2 192.168.100.121 nameserver, brokerserver Master2, Slave1 9876, 10911, 11011

Disable firewalls on both servers or configure the relevant port exceptions.

Configure one server first, then duplicate and modify settings for the second server.

Environment Variable Configuration

vim /etc/profile
# RocketMQ environment configuration
export ROCKETMQ_HOME=/opt/rocketmq
export PATH=$PATH:$ROCKETMQ_HOME/bin

# Apply the changes
source /etc/profile

Create Message Storage Directories

# Navigate to RocketMQ directory
mkdir -p store/commitlog store/consumequeue store/index

Broker Configuration Files

Navigate to /rocketmq/conf to access pre-configured cluster templates. Use configurations from the 2m-2s-sync directory.

Master1 Configuration (192.168.100.120)

Edit broker-a.properties:

# Cluster name
brokerClusterName=rocketmq-cluster

# Broker name (must differ across configuration files)
brokerName=broker-a

# 0 = Master, non-zero = Slave
brokerId=0

# NameServer addresses (semicolon-separated)
namesrvAddr=192.168.100.120:9876;192.168.100.121:9876

# Default queue count for auto-created topics
defaultTopicQueueNums=4

# Allow automatic topic creation (enable for dev, disable for prod)
autoCreateTopicEnable=true

# Allow automatic subscription group creation
autoCreateSubscriptionGroup=true

# Broker listening port
listenPort=10911

# File cleanup time (4 AM)
deleteWhen=04

# File retention period (hours)
fileReservedTime=120

# CommitLog file size (1GB default)
mappedFileSizeCommitLog=1073741824

# ConsumeQueue file entries (300,000 default)
mappedFileSizeConsumeQueue=300000

# Disk space usage threshold
diskMaxUsedSpaceRatio=88

# Storage root directory
storePathRootDir=/rocketmq/store

# CommitLog storage path
storePathCommitLog=/rocketmq/store/commitlog

# ConsumeQueue storage path
storePathConsumeQueue=/rocketmq/store/consumequeue

# Message index storage path
storePathIndex=/rocketmq/store/index

# Checkpoint file path
storeCheckpoint=/rocketmq/store/checkpoint

# Abort file path
abortFile=/rocketmq/store/abort

# Maximum message size
maxMessageSize=65536

# Broker role: ASYNC_MASTER, SYNC_MASTER, or SLAVE
brokerRole=SYNC_MASTER

# Flush type: ASYNC_FLUSH or SYNC_FLUSH
flushDiskType=SYNC_FLUSH
Slave2 Configuration (192.168.100.120)

Edit broker-b-s.properties:

# Cluster name
brokerClusterName=rocketmq-cluster

# Broker name
brokerName=broker-b

# Slave identifier
brokerId=1

# NameServer addresses
namesrvAddr=192.168.100.120:9876;192.168.100.121:9876

# Default queue count
defaultTopicQueueNums=4

autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true

# Different listening port for slave
listenPort=11011

deleteWhen=04
fileReservedTime=120
mappedFileSizeCommitLog=1073741824
mappedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88

# Separate storage paths for slave
storePathRootDir=/rocketmq/store_slave
storePathCommitLog=/rocketmq/store_slave/commitlog
storePathConsumeQueue=/rocketmq/store_slave/consumequeue
storePathIndex=/rocketmq/store_slave/index
storeCheckpoint=/rocketmq/store_slave/checkpoint
abortFile=/rocketmq/store_slave/abort

maxMessageSize=65536

# SLAVE role
brokerRole=SLAVE

# Async flush for slave
flushDiskType=ASYNC_FLUSH
Master2 Configuration (192.168.100.121)

Edit broker-b.properties:

# Cluster name
brokerClusterName=rocketmq-cluster

# Broker name
brokerName=broker-b

# Master identifier
brokerId=0

# NameServer addresses
namesrvAddr=192.168.100.120:9876;192.168.100.121:9876

defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true

listenPort=10911

deleteWhen=04
fileReservedTime=120
mappedFileSizeCommitLog=1073741824
mappedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88

storePathRootDir=/rocketmq/store
storePathCommitLog=/rocketmq/store/commitlog
storePathConsumeQueue=/rocketmq/store/consumequeue
storePathIndex=/rocketmq/store/index
storeCheckpoint=/rocketmq/store/checkpoint
abortFile=/rocketmq/store/abort

maxMessageSize=65536

# SYNC_MASTER for HA
brokerRole=SYNC_MASTER
flushDiskType=SYNC_FLUSH
Slave1 Configuration (192.168.100.121)

Edit broker-a-s.properties:

# Cluster name
brokerClusterName=rocketmq-cluster

# Broker name
brokerName=broker-a

# Slave identifier
brokerId=1

# NameServer addresses
namesrvAddr=192.168.100.120:9876;192.168.100.121:9876

defaultTopicQueueNums=4
autoCreateTopicEnable=true
autoCreateSubscriptionGroup=true

# Slave port
listenPort=11011

deleteWhen=04
fileReservedTime=120
mappedFileSizeCommitLog=1073741824
mappedFileSizeConsumeQueue=300000
diskMaxUsedSpaceRatio=88

storePathRootDir=/rocketmq/store_slave
storePathCommitLog=/rocketmq/store_slave/commitlog
storePathConsumeQueue=/rocketmq/store_slave/consumequeue
storePathIndex=/rocketmq/store_slave/index
storeCheckpoint=/rocketmq/store_slave/checkpoint
abortFile=/rocketmq/store_slave/abort

maxMessageSize=65536

# SLAVE role
brokerRole=SLAVE

# Async flush
flushDiskType=ASYNC_FLUSH

Service Startup Procedures

Starting NameServer Cluster

# Start NameServer on both servers
cd /rocketmq/bin
nohup sh mqnamesrv &

Starting Master and Slave Brokers

On 192.168.100.120:

cd /rocketmq/bin

# Start Master1
nohup sh mqbroker -c /rocketmq/conf/2m-2s-sync/broker-a.properties &

# Start Slave2
nohup sh mqbroker -c /rocketmq/conf/2m-2s-sync/broker-b-s.properties &

On 192.168.100.121:

cd /rocketmq/bin

# Start Master2
nohup sh mqbroker -c /rocketmq/conf/2m-2s-sync/broker-b.properties &

# Start Slave1
nohup sh mqbroker -c /rocketmq/conf/2m-2s-sync/broker-a-s.properties &

Verifying Process Status

Use jps to verify process status. If Slave fails to start with error "Lock failed, MQ already started", ensure Master and Slave do not share the same storePath. Create separate storage directories before restarting.

mqadmin Management Commands

Execute commands from the RocketMQ bin directory: ./mqadmin {command} {args}

Topic Management Commands

Command Description Options Details
updateTopic Create or update Topic -b Broker address (ip:port)
-c Cluster name
-n NameServer address
-p Permissions (W=2, R=4, WR=6)
-r Readable queue count
-w Writable queue count
-t Topic name
deleteTopic Delete Topic -c Cluster name
-n NameServer address
-t Topic name
topicList List Topics -c Show cluster and subscription info
-n NameServer address
topicRoute View Topic routing -t Topic name
-n NameServer address
topicStatus View Topic queue offsets -t Topic name
-n NameServer address
topicClusterList View Topic clusters -t Topic name
-n NameServer address
updateTopicPerm Update Topic permissions -t Topic name
-b Broker address
-p Permissions
-c Cluster name
statsAll View Topic statistics -a Show active topics only
-t Specific topic
-n NameServer address

Cluster Management Commands

Command Description Options Details
clusterList View cluster information -m Show detailed metrics
-i Print interval (seconds)
-n NameServer address
clusterRT Test broker response time -a Message count per test
-s Message size (bytes)
-c Target cluster
-p Formatted output
-i Send interval

Broker Management Commands

Command Description Options Details
updateBrokerConfig Update Broker configuration -b Broker address
-c Cluster name
-k Configuration key
-v Configuration value
brokerStatus View Broker statistics -b Broker address
-n NameServer address
brokerConsumeStats View consumer statistics -b Broker address
-l Diff threshold
-o Ordered topic flag
getBrokerConfig Get Broker configuration -b Broker address
cleanExpiredCQ Clean expired queues -b Broker address
-c Cluster name
cleanUnusedTopic Clean unused topics -b Broker address
-c Cluster name
sendMsgStatus Test message sending -b Broker name
-s Message size
-c Send count

Message Query Commands

Command Description Options Details
queryMsgById Query by message ID -i Message ID
-n NameServer address
queryMsgByKey Query by message key -k Message key
-t Topic name
queryMsgByOffset Query by offset -b Broker name
-i Queue ID
-o Offset value
-t Topic name
queryMsgByUniqueKey Query by unique key -i Unique message ID
-g Consumer group
-d Client ID
sendMessage Send a test message -t Topic name
-p Message body
-k Message keys
-c Message tags
-b Broker name
-i Queue ID
consumeMessage Consume and view messages -t Topic name
-o Start offset
-g Consumer group
-s Start timestamp
-d End timestamp
-c Message count
resetOffsetByTime Reset consumer offset -g Consumer group
-t Topic name
-s Target timestamp
-f Force reset

Consumer Management Commands

Command Description Options Details
consumerProgress View consumer progress -g Consumer group
-s Show client IPs
consumerStatus View consumer status -g Consumer group
-i Client ID
-s Include jstack
getConsumerStatus Get consumption offset -g Consumer group
-t Topic name
-i Client IP
updateSubGroup Update subscription -g Consumer group name
-s Enable consumption
-m Start from min offset
-d Broadcast mode
-q Retry queue count
-r Max retry count
deleteSubGroup Delete subscription -g Consumer group

Connection Management Commands

Command Description Options Details
consumerConnection Query consumer connections -g Consumer group
-n NameServer address
producerConnection Query producer connecctions -g Producer group
-t Topic name

NameServer Management Commands

Command Description Options Details
getNamesrvConfig Get NameServer config -n NameServer address
updateNamesrvConfig Update NameServer config -n NameServer address
-k Config key
-v Config value

Other Commands

Command Description Options Details
startMonitoring Start monitoring service -n NameServer address

Important Notes:

  • Almost all commands require -n to specify the NameServer address
  • Use -h on any command to display help information
  • When both -b (Broker address) and -c (cluster name) are specified, commands execute on the specific Broker

Cluster Monitoring Console

Deploy the RocketMQ Console for visual cluster management:

  1. Clone the external tools repository: git clone https://github.com/apache/rocketmq-externals.git
  2. Navigate to rocketmq-console
  3. Configure the NameServer addresses in src/main/resources/application.properties: ``` rocketmq.config.namesrvAddr=192.168.100.120:9876;192.168.100.121:9876
  4. Build and run: ``` mvn clean package -Dmaven.test.skip=true java -jar target/rocketmq-console-ng-1.0.0.jar
  5. Access the console at http://localhost:8080

Docker Deployment

Create Required Directories

# Create directory structure
mkdir -p /opt/rocketmq/{broker/{conf,logs,store},namesrv/{logs,store},store}

# Create broker configuration
cat > /opt/rocketmq/broker/conf/broker.conf << 'EOF'
namesrvAddr=YOUR_NAMESRV_IP:9876
brokerClusterName=docker-rocketmq
brokerName=broker-a
brokerId=0
deleteWhen=04
fileReservedTime=48
brokerRole=ASYNC_MASTER
flushDiskType=ASYNC_FLUSH
brokerIP1=YOUR_BROKER_IP
listenPort=10911
enablePropertyFilter=true
autoCreateTopicEnable=true
EOF

Pull Docker Images

# Pull RocketMQ images
docker pull rocketmqinc/rocketmq:latest

# Pull console image
docker pull pangliang/rocketmq-console-ng:latest

Start Containers

NameServer Container:

docker run -d \
--restart=always \
--name rmqnamesrv \
-p 9876:9876 \
-v /opt/rocketmq/namesrv/logs:/root/logs \
-v /opt/rocketmq/namesrv/store:/root/store \
-e "MAX_POSSIBLE_HEAP=100000000" \
rocketmqinc/rocketmq \
sh mqnamesrv

Broker Container:

docker run -d \
--restart=always \
--name rmqbroker \
--link rmqnamesrv:namesrv \
-p 10911:10911 \
-p 10909:10909 \
-v /opt/rocketmq/broker/logs:/root/logs \
-v /opt/rocketmq/broker/store:/root/store \
-v /opt/rocketmq/broker/conf/broker.conf:/opt/rocketmq-4.4.0/conf/broker.conf \
-e "NAMESRV_ADDR=namesrv:9876" \
-e "MAX_POSSIBLE_HEAP=200000000" \
rocketmqinc/rocketmq \
sh mqbroker -c /opt/rocketmq-4.4.0/conf/broker.conf

Console Container:

docker run -d \
--restart=always \
--name rmqadmin \
-e "JAVA_OPTS=-Drocketmq.namesrv.addr=YOUR_IP:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false" \
-p 8082:8080 \
pangliang/rocketmq-console-ng

Access the console at http://YOUR_IP:8082

Tags: rocketmq apache-rocketmq messaging-system message-queue distributed-messaging

Posted on Mon, 28 Sep 2026 16:41:13 +0000 by rocket

Hot Tags

©2026 Freaks City