Redis Replication Overview

Similar to distributed systems like Kafka, RocketMQ, MySQL, and Zookeeper, Redis supports a clustered architecture consisting of primary and secondary nodes. In this hierarchy, the primary node is referred to as the master, and secondary nodes are called replicas (formerly slaves). Replicas utilize a replication mechanism to automatically synchronize data from the master node.

Configuring Replication

Setting up replication in Redis is straightforward, requiring only a single line in the configuration file. For instance, in a topology with one master and multiple replicas, if the master resides at IP 192.168.10.5, you would add the following directive to the redis.conf file of every replica node:

replicaof 192.168.10.5 6379

Upon startup, the replica automatically connects to the master and initiates the data synchronization process. If the master changes—for example, if the original master fails and a new one is elected—this configuration is dynamically updated.

Alternatively, you can specify the master during server startup using command-line arguments:

redis-server --replicaof 192.168.10.5 6379

It is also possible to reconfigure a running instance to become a replica of another node without restarting. This is achieved by executing the REPLICAOF command directly in the CLI:

redis> REPLICAOF 192.168.10.5 6379

This allows for chained replication, where a replica acts as a master to other nodes. To verify the status of the cluster, use:

redis> INFO replication

Replicas are read-only by default. Attempting to write data to a replica will result in an error:

(error) READONLY You can't write against a read only replica.

Data written to the master is asynchronously propagated to the replicas.

Promoting a Replica to Master

To promote a replica to a standalone master (effectively removing it from the replication chain), you can use the following command:

redis> REPLICAOF NO ONE

After executing this, the node stops synchronizing data and behaves as an independent master.

Replication Internals

Redis replication operates in two distinct modes: Full Synchronization and Partial Synchronization. Full sync occurs when a replica connects to a master for the first time and requires a complete dataset transfer. Partial sync is used when a replica reconnects after a brief disconnection or outage, needing only the missing data.

1. Connection Phase

1. When a replica starts (or issues the REPLICAOF command), it stores the master's host and IP information locally.
2. An internal cron task replicationCron runs every second to check for new master connections.

Once a master is identified, the replica establishes a connection. Upon success, a dedicated file event handler is created to manage the replication workflow. The replica sends periodic PING commands to signal its liveness to the master. Once connected, the process moves to data synchronization.

2. Data Synchronization Phase

For a new replica, a full synchronization is required. The master executes the BGSAVE command to generate an RDB snapshot in the background and transmits this file to the replica.

Handling Existing Data: If the replica already contains data, it must purge its old datasets before loading the new RDB file.

Handling Writes During Snapshot: While the RDB file is being generated, the master buffers all incoming write commands in memory. Once the replica finishes loading the RDB, the master sends these buffered write commands to the replica to bring it up to date. This logic parallels how commands are handled during an AOF rewrite.

After the initial full sync, the master and replica are consistent. Subsequent updates are handled via continuous command propagation.

3. Command Propagation Phase

The master asynchronously streams write commands to the connected replicas. In summary, the process uses an RDB file for the initial state and a continuous stream of commands for ongoing updates.

Note on Read/Write Splitting: Unlike SQL databases, Redis rarely requires read/write splitting for performance due to its extremely high thorughput. However, replication lag is inherent, similar to MySQL, and can only be mitigated by network optimization.

4. Partial Replication

If a replica disconnects temporarily, performing a full resynchronization would be inefficient. Instead, Redis uses incremental replication. The replica tracks its progress using the master_repl_offset value found in the INFO replication output:

master_repl_offset:7324556

This offset allows the replica to request only the data missed during the disconnection.

5. Diskless Replication

Available since version 2.8.18, Redis supports diskless replication, configured via:

repl-diskless-sync yes

This feature minimizes disk I/O on the master by creating the RDB file and streaming it directly to replicas over the network socket without writing to the master's disk. This is particularly useful when the master has slow disk storage but high network bandwidth.

Limitations of Replication

While master-slave replication addresses data backup and read distribution, it does not provide high availability. In a single-master topology, if the master node fails, the service becomes unavailable due to the single point of failure. Manually promoting a replica to master and reconfiguring other nodes is time-consuming and leads to significant downtime. This limitation necessitates the use of Redis Sentinel or Cluster for automatic failover.