Redis High Availability

High availability in web servers refers to the time during which a service remains accessible, typically measured by uptime percentages (99.9%, 99.99%, 99.999%). In the Redis context, high availability encompasses more than just service availability—it also involves data capacity scaling, data durability, and rapid disaster recovery mechanisms.

Redis implements high availability through four primary technologies: persistence, replication, Sentinel, and Cluster. Each serves a distinct purpose and addresses different concerns.

• Persistence: The most basic high availability mechanism, primarily serving as a backup strategy. It ensures data survives process crashes by storing information on disk.
• Replication: The foundation for Redis high availability, upon which Sentinel and Cluster are built. Replication provides data redundancy across multiple servers, load balancing for read operations, and basic failover capabilities. Limitations include: no automatic failover, no write load distribution, and storage capacity limited by a single node.
• Sentinel: Builds on replication to deliver automated failover. Limitations include: no write load distribution and storage capacity limited by a single node.
• Cluster: Addresses write load distribution and storage capacity constraints, providing a comprehensive high availability solution.

Redis Persistence

Persistence protects against data loss when Redis processes terminate unexpectedly. By regularly saving data from memory to disk, Redis can recover state on restart. Additionally, persistence files enable disaster recovery through remote backups.

Persistence Methods

Redis provides two persistence approaches:

1. RDB Persistence: Captures point-in-time snapshots of the in-memory database to disk at configured intervals.
2. AOF Persistence (Append Only File): Records every write operation as an append-only log, similar to MySQL's binary logging.

AOF offers better real-time durability since it minimizes data loss during unexpected shutdowns. While AOF is the preferred approach for most production environments, RDB remains valuable for specific use cases.

RDB Persistence

RDB persistence generates a binary compressed snapshot of the dataset at specified intervals. The resulting files use the .rdb extension and enable fast data recovery on restart.

Triggering Conditions

RDB snapshots can be triggered manually or automatically.

Manual Trigger

Both save and bgsave commands generate RDB files, but they differ significantly in behavior:

• save blocks the Redis server until the RDB file is complete. During this period, the server cannot process any client requests.
• bgsave spawns a child process to handle snapshot creation while the parent process continues serving clients. Only the initial fork() operation blocks the server.

The save command is largely deprecated in production environments due to its blocking behavior.

Automatic Trigger

Automatic RDB snapshots use the save directive in the configuration file:

# save m n
# Triggers bgsave when m seconds elapse with at least n changes
save 3600 1    # Trigger after 1 hour if at least 1 change occurred
save 300 10    # Trigger after 5 minutes if at least 10 changes occurred
save 60 10000  # Trigger after 1 minute if at least 10000 changes occurred

Additional configuration options:

rdbcompression yes        # Enable RDB file compression
dbfilename dump.rdb       # Set RDB filename
dir /var/lib/redis        # Set directory for RDB and AOF files

Other Automatic Triggers

• Full replication operations on slave nodes trigger master bgsave
• shutdown commands initiate RDB persistence before exit

Execution Flow

1. The parent process checks whether a save, bgsave, or bgrewriteaof operation is already running. If so, bgsave returns immediately to prevent concurrent heavy disk operations.
2. The parent process calls fork() to create a child process—this blocks the parent.
3. After fork() completes, bgsave returns immediately and the parent resumes accepting commands.
4. The child process writes the snapshot to a temporary file, then atomically replaces the existing RDB file.
5. The child process signals completion to the parent, which updates statistics.

Startup Loading

RDB files load automatically during server startup. When AOF is enabled, Redis prioritizes loading the AOF file over RDB. RDB loading only occurs when AOF is disabled. The server remains blocked during the loading process, and corrupted files cause startup failure.

AOF Persistence

AOF persistence logs every write operation (excluding queries) to an append-only file. Upon restart, Redis replays these commands to reconstruct the dataset. AOF provides superior durability compared to RDB.

Enabling AOF

appendonly yes                    # Enable AOF
appendfilename "appended.aof"     # Set AOF filename
aof-load-truncated yes            # Tolerate truncated AOF files (default)

systemctl restart redis-server

Execution Flow

AOF involves three phases: command appending, file synchronization, and rewrite.

Command Appending

Write commands append to an in-memory buffer (aof_buf) rather than directly to disk. This design avoids filesystem performance bottlenecks caused by frequent small writes. Commands use RESP (REdis Serialization Protocol) format—text-based, human-readable, and efficient to parse.

Write and Sync

Modern operating systems defer disk writes through memory buffers, improving performance but risking data loss on crashes. The fsync() system call forces immediate disk synchronization.

Redis offers three synchronization policies:

appendfsync always   # Sync after every write (slowest, highest durability)
appendfsync everysec # Sync once per second (balanced, default)
appendfsync no       # Let OS handle synchronization (fastest, risky)

• always: Maximum durability but creates a significant I/O bottleneck—typically limits throughput to a few hundred operations per second.
• no: Fastest option but data may be lost if the system crashes before OS synchronization occurs.
• everysec: Strikes an optimal balance between performance and durability, synchronizing in a background thread.

Rewrite

As Redis processes write commands over time, AOF files grow large. File rewriting compresses the AOF by:

• Removing expired data
• Eliminating redundant operations (duplicate sets, deleted keys)
• Combining multiple related commands (multiple SADD operations merged into one)

Rewriting creates a new file from current memory state rather than reading the existing AOF.

Triggering Rewrite

auto-aof-rewrite-percentage 100  # Rewrite when current size reaches 200% of last rewrite
auto-aof-rewrite-min-size 64mb  # Minimum file size to trigger rewrite

Rewrite can be triggered manually via bgrewriteaof or automatically when both thresholds are exceeded.

Rewrite Process

1. Parent process checks for running bgsave or bgrewriteaof operations.
2. Parent calls fork()—this blocks briefly.
3. After fork, the parent continues processing commands normally, appending to both aof_buf and aof_rewrite_buf.
4. Child process generates a new AOF from in-memory data.
5. Parent receives signal, updates statistics, then appends aof_rewrite_buf contents to the new file.
6. New file replaces the old one atomically.

Startup Loading

When AOF is enabled, Redis loads the AOF file first. If AOF is disabled, Redis loads the RDB file instead. If AOF is enabled but the file is missing, Redis refuses to start even if an RDB file exists. Corrupted AOF files cause startup failure, but truncated files may be loaded if aof-load-truncated is enabled.

RDB vs AOF Comparison

RDB Advantages

• Compact files suitable for backups and remote transfer
• Fast recovery compared to AOF
• Minimal performance impact during fork operations
• Ideal for full dataset backups

RDB Disadvantages

• Cannot achieve real-time persistence—data loss risk between snapshots
• Compatibility issues between versions (older versions may not read newer formats)
• Fork operations block the parenet process briefly

AOF Advantages

• Supports near-second-level persistence
• Better version compatibility
• Append-only format resists corruption

AOF Disadvantages

• Larger file sizes
• Slower recovery times
• Higher performance overhead due to frequent disk writes
• Rewrite operations require fork and additional I/O

Redis Performance Management

Memory Usage Analysis

redis-cli> info memory

Key metrics include:

• used_memory: Memory allocated by Redis for storing data
• used_memory_rss: Physical memory allocated by the operating system
• used_memory_peak: Peak memory usage recorded

Memory Fragmentation Ratio

mem_fragmentation_ratio = used_memory_rss / used_memory

Causes: Redis maintains its own memory allocator for efficiency. When keys are deleted, memory returns to Redis's internal allocator rather than the OS. This design improves utilization but causes fragmentation—allocated but unused memory.

Interpretation:

Ratio	Meaning
1.0 - 1.5	Normal—minimal fragmentation
> 1.5	Excessive—Redis using 150% of actual data size
< 1.0	Memory swapping—insufficient physical memory

Solutions for High Fragmentation:

For Redis versions before 4.0, restart the server to release fragmented memory:

redis-cli shutdown save
systemctl restart redis

For Redis 4.0 and later, enable automatic defragmentation:

activedefrag yes

Or manually trigger defragmentation:

redis-cli> memory purge

Memory Utilization

When memory usage exceeds available capacity, the OS begins swapping to disk, causing severe performance degradation.

Preventing Swap:

• Right-size Redis instances based on expected cache size
• Use Hash data structures for efficient storage
• Implement TTL on appropriate keys

Key Eviction Policies

When memory reaches the configured limit, Redis applies eviction policies:

maxmemory-policy noeviction

Available policies:

Policy	Behavior
volatile-lru	Evict least recently used keys with TTL
volatile-ttl	Evict keys with shortest TTL
volatile-random	Randomly evict keys with TTL
allkeys-lru	Evict least recently used keys
allkeys-random	Randomly evict any key
noeviction	Reject writes until memory is available

Additional Configuration Parameters

maxclients 10000           # Maximum simultaneous connections
maxmemory 2gb             # Maximum memory allocation
maxmemory-samples 5       # Sample size for LRU/TTL algorithms
tcp-backlog 511           # Connection queue depth
timeout 300               # Client idle timeout (seconds)
lazyfree-lazy-expire yes  # Async key expiration
no-appendfsync-on-rewrite yes  # Avoid AOF rewrite I/O contention

Redis Optimization Strategies

1. Enable automatic memory defragmentation with activedefrag yes or use memory purge periodically
2. Prefer Hash data structures for storage—single keys can contain multiple fields, minimizing overhead
3. Always set appropriate TTL values for keys
4. Keep key names and values concise to prevent bigkey issues. Use redis-cli --bigkeys to identify problematic keys
5. Configure memory limits and eviction policies:
   • maxmemory: Cap Redis memory usage
   • maxmemory-policy: Choose appropriate eviction strategy
   • maxmemory-samples: Balance accuracy vs. performance (3-7 typically optimal)
   • maxclients: Match expected connection load
   • tcp-backlog: Size connection backlog appropriately
   • timeout: Configure idle client timeouts
   • lazyfree-lazy-expire yes: Defer key expiration to background threads
   • no-appendfsync-on-rewrite yes: Prevent AOF rewrite from competing with normal fsync operations
6. Enable AOF persistence and implement replication for data redundancy. Use Sentinel or Cluster for automated failover
7. Secure Redis instances by setting authentication via config set requirepass or configuration file

Cache-Related Issues

When Redis operates normally, most request are served from cache, allowing the database to handle minimal load. Cache-related problems occur when Redis hit rates drop significantly, causing excessive database requests that overwhelm MySQL.

Cache Avalanche

Problem: Large numbers of cache keys expire simultaneously.

Solutions:

1. Introduce randomized TTL values to prevent synchronized expiration
2. Implement cache tags with automatic refresh mechanisms
3. Use distributed locks at the database layer

Cache Penetration

Problem: Requests target keys that exist in neither Redis nor the database.

Solutions:

1. Cache null results for non-existent keys
2. Implement Bloom filters to reject impossible requests
3. Deploy monitoring scripts to identify and block abusive patterns

Cache Breakdown

Problem: A single frequently-accessed key expires, causing simultaneous database load.

Solutions:

1. Pre-warm frequently accessed data during off-peak hours
2. Monitor access patterns and dynamically adjust expiration times
3. Use distributed locks at the database layer

Use redis-cli --hotkeys to identify frequently accessed keys requiring special handling.

Maintaining Data Consistency

Read Operations:

1. Check Redis first
2. On cache miss, query MySQL
3. Write retrieved data back to Redis

Update Operations:

1. Update MySQL first
2. Invalidate or update corresponding Redis cache

Delete Operations:

1. Remove from Redis cache
2. Delete from MySQL database

Additional Consistency Strategies:

• Use MySQL triggers to propagate changes to Redis
• Implement scheduled tasks for cache warm-up and synchronization
• Consider event-driven architectures for real-time consistency when requirements are stringent