Persistence refers to the process of creating backups of data stored in memory. This is a crucial strategy for ensuring high availability in single-node deployments. When Redis experiences a failure, these backups enable data recovery. Redis supports two primary persistence mechanisms: RDB and AOF.

1. RDB (Redis DataBase)

RDB creates point-in-time snapshots of the entire Redis dataset at configured intervals. These snapshots are saved to disk and can be used for recovery when needed.

2. AOF (Append Only File)

AOF logs every write operation received by the Redis server (similar to MySQL's binary log). When Redis starts, it replays these commands to reconstruct the dataset.

RDB Mechanism

1.1 Triggering Conditions

SAVE command: This blocks the Redis server until the entire save operation completes. During this blocking period, the server cannot process any client requests. This command is considered deprecated for production use.

BGSAVE command: This command forks a child process to handle the persistence operation. The blocking only occurs during the fork phase, which is typically brief. While the child process is writing the RDB file, the parent process continues handling client requests normally.

Automatic triggering: Configure in redis.conf:

save m n

This configuration instructs Redis to execute the BGSAVE command if there are n modification operations within m seconds.

1.2 Execution Flow

The RDB saving process follows these steps:

1. Execute the BGSAVE command. The parent process checks if any child process is currently running (such as an AOF rewrite process). If one exists, it returns immediately.
2. The parent process forks a child process. During the fork operation, the parent process is blocked and cannot respond to client requests.
3. Once the fork completes, BGSAVE returns the message "background saving started" and the parent process resumes responding to clients.
4. The child process creates an RDB file based on the parent's memory snapshot at fork time. After completion, it atomically replaces the existing RDB file.
5. The child process sends a signal to the parent process indicating the RDB operation has finished.

1.3 Advantages and Disadvantages

Advantages:

1. RDB files are compressed binary representations of Redis data at a specific point in time, making them ideal for backups and full data replication scenarios.
2. Redis can restore data from RDB files significantly faster than from AOF files.

Disadvantages:

1. RDB cannot achieve real-time persistence.
2. There may be compatibility issues between different Redis versions regarding RDB file formats.

To address the real-time persistence limitation, Redis introduced the AOF mechanism.

AOF Mechanism

2.1 Enabling AOF

AOF is disabled by default. Enable it using:

appendonly yes

2.2 AOF Workflow

The AOF mechanism operates through four fundamental stages:

1. Command apending: Every write command is appended to the aof_buf (in-memory buffer).
2. File synchronization: The AOF buffer is synchronized to disk based on the configured policy.
3. File rewriting: As the AOF file grows, it undergoes periodic rewriting to reduce its size.
4. Restart loading: When Redis starts, it loads data from the AOF file to recovery.

Command Appending

Each Redis write command is recorded in AOF using Redis Serialization Protocol (RESP) format. This format offers both compatibility and readability while avoiding re-processing overhead.

Writing commands directly to disk would degrade Redis performance. Instead, commands are first stored in aof_buf and later flushed to disk based on the synchronization policy, preventing direct I/O from impacting server performance.

File Synchronization

Redis provides multiple synchronization strategies controlled by the appendfsync parameter:

The everysec strategy represents a balanced compromise between always and no, offering both reasonable performance and data safety. It is the default configuration and generally recommended for most deployments.

File Rewriting

File rewriting converts the current dataset in Redis memory into write commands and synchronizes them to a new AOF file. This process reduces AOF file size and improves loading speed during recovery.

AOF rewriting can reduce file size for several reasons:

1. Expired data no longer needs to be included. For example, if SET k1 aaa has expired but exists in the AOF file, rewriting removes it.
2. Invalid commands are eliminated. If SET k1 111 and SET k1 222 both exist in the AOF, the first becomes redundant. Rewriting consolidates them into a single command.
3. Multiple commands can be merged.

Triggering Conditions:

1. Manual trigger: Execute the BGREWRITEAOF command.

2. Automatic trigger: Based on the following parameters:

   • auto-aof-rewrite-min-size: Minimum file size required for AOF rewrite (default: 64MB).
   • auto-aof-rewrite-percentage: Ratio of current AOF size (aof_current_size) to the size after the last rewrite (aof_base_size).

   Only when both conditions are met will automatic rewriting be triggered.

AOF Rewrite Process:

1. A rewrite request is initiated.
2. The parent process forks a child process, with overhead equivalent to the BGSAVE fork operation.
3. AOF buffer handling:
   • After the fork completes, the parent process continues responding to client requests. All modification commands are written to aof_buf and later flushed to disk based on the appendfsync policy.
   • Due to copy-on-write technology during fork, the child process can only share memory data existing at fork time. Since the parent continues processing commands, Redis uses aof_rewrite_buf to store these new modifications, preventing data loss during the new AOF generation.
4. The child process generates a new AOF file based on the memory snapshot captured at fork time.
5. Final steps:
   • The child process sends a signal to the parent indicating the AOF file is ready.
   • The parent process writes data from aof_rewrite_buf to the new AOF file.
   • The old AOF file is replaced with the new one, completing the AOF rewrite.

Startup Loading Process

When Redis starts and loads persisted data:

1. AOF files are prioritized for loading if available.
2. If AOF is disabled, Redis loads the RDB file instead.
3. Only after successful data loading can Redis start successfully.
4. If loading fails, Redis startup fails.