Redis cluster addresses storage limitations by distributing data across multiple master-slave node groups. Unlike sentinel mode, where all nodes store complete datasets, cluster mode partitions data to reduce individual node pressure.

Data Partitioning Strategies

Hash Modulo Approach

This method applies a hash function to keys and uses modulo operation to determine shard placement. With N shards numbered 0 to N-1:

shard_index = hash(key) % N

While simple and efficient, this approach requires extensive data migration when cluster size changes, as the modulo result changes for all existing keys.

Consistent Hashing

Consistant hashing maps nodes to positions on a logical ring. Keys are placed based on thier hash values relative to node positions:

• New nodes occupy segments on the ring
• Only data between adjacent nodes needs migration
• Significantly reduces redistribution overhead

However, this method can lead to uneven data distribution, with some nodes handling disproportionately more data than others.

Hash Slot Mechanism

Redis cluster employs 16384 hash slots for optimal balance between migration cost and distribution uniformity:

slot_id = crc16(key) % 16384

Slots are distributed among nodes:

• Node 0: slots 0-5461 (5462 slots)
• Node 1: slots 5462-10923 (5462 slots)
• Node 2: slots 10924-16383 (5460 slots)

When adding nodes, slots can be redistributed flexibly:

• Node 0: slots 0-4095 (4096 slots)
• Node 1: slots 5462-9557 (4096 slots)
• Node 2: slots 10924-15019 (4096 slots)
• Node 3: slots 4096-5461, 9558-10923, 15019-16383 (4096 slots)

Redis automatically handles slot reassignment and data migration during scaling operations.

Cluster Deployment

Configuration Generation

for port in $(seq 1 9); do
    mkdir -p redis${port}/
    cat > redis${port}/redis.conf << EOF
port 6379
bind 0.0.0.0
protected-mode no
appendonly yes
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.30.0.10${port}
cluster-announce-port 6379
cluster-announce-bus-port 16379
EOF
done

for port in $(seq 10 11); do
    mkdir -p redis${port}/
    cat > redis${port}/redis.conf << EOF
port 6379
bind 0.0.0.0
protected-mode no
appendonly yes
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
cluster-announce-ip 172.30.0.1${port}
cluster-announce-port 6379
cluster-announce-bus-port 16379
EOF
done

Cluster Initialization

redis-cli --cluster create \
172.30.0.101:6379 172.30.0.102:6379 172.30.0.103:6379 \
172.30.0.104:6379 172.30.0.105:6379 172.30.0.106:6379 \
172.30.0.107:6379 172.30.0.108:6379 172.30.0.109:6379 \
--cluster-replicas 2

Client Connection

# Enable cluster redirection
redis-cli -p 6379 -c

# View cluster topology
cluster nodes

Fault Detection and Recovery

Nodes exchange heartbeat packets periodically. When node A cannot reach node B:

1. A marks B as PFAIL (potentially failed)
2. A uses Gossip protocol to verify B's status with other nodes
3. If majority confirm failure, B is marked FAIL
4. Slave nodes initiate election process

Election follows Raft algorithm:

sleep_time = 500ms + random(0-500ms) + rank_offset * 1000ms

The node with shortest sleep time becomes the new master.

Scaling Operations

Adding Nodes

# Add new master node
redis-cli --cluster add-node 172.30.0.110:6379 172.30.0.101:6379

# Redistribute slots
redis-cli --cluster reshard 172.30.0.101:6379

# Add slave node
redis-cli --cluster add-node 172.30.0.111:6379 172.30.0.101:6379 --cluster-slave

Removing Nodes

# Remove slave node
redis-cli --cluster del-node 172.30.0.101:6379 [slave_node_id]

# Redistribute slots from node to be removed
redis-cli --cluster reshard 172.30.0.101:6379

# Remove master node
redis-cli --cluster del-node 172.30.0.101:6379 [master_node_id]