RDB Persistence

Redis, being an in-memory data store, requires a persistence mechanism to ensure data durability and recovery in case of failures. The two primary persistence methods in Redis are RDB (Redis Database) and AOF (Append Only File).

RDB creates point-in-time snapshots of the dataset in a compact binary format. It is efficient for backups and fast restores. However, it may result in data loss if the server crashes between snapshots.

How RDB Works

• Manual Trigger: Using SAVE or BGSAVE commands.
• Automatic Trigger: Configured using the save directive in redis.conf.

For example, the following configuration triggers an RDB snapshot if any of the conditions are met:

save 900 1
save 300 10
save 60 10000

Internally, Redis uses a dirty counter and a lastsave timestamp to determine when to trigger a snapshot. The serverCron function checks these conditions every 100 milliseconds.

Fork and Copy-on-Write

When BGSAVE is executed, Redis forks a child process. The child uses Copy-on-Write (COW) to generate the RDB file without blocking the main process. This allows Redis to continue handling requests during persistence.

RDB Configuration Options

• stop-writes-on-bgsave-error: Whether to halt writes if RDB fails.
• rdbcompression: Enables compression of RDB files.
• rdbchecksum: Enables checksum validation for RDB files.

AOF Persistence

AOF logs every write operation received by the server. It offers better durabiilty at the cost of performance and file size.

How AOF Works

Write commands are appended to a buffer, which is periodically flushed to disk based on the appendfsync setting:

• always: Every write is synced to disk (slowest, safest).
• everysec: Default mode; syncs every second.
• no: Let the OS decide (fastest, riskiest).

AOF Rewrite

Over time, the AOF file grows due to redundant commands. Redis performs AOF rewrite to compact the file by reading the current dataset and generating minimal commands to represent it.

Example:

R PUSH list "A"
R PUSH list "B"
...
R PUSH list "F"

Can be rewritten as:

R PUSH list "A" "B" "C" "D" "E" "F"

AOF Rewrite Process

1. Fork a child process to read the current dataset.
2. Create a new compact AOF file.
3. Use a rewrite buffer to capture writes during the rewrite.
4. Swap the new file with the old one after completion.

AOF Configuration Options

• appendonly: Enables AOF mode.
• appendfilename: Name of the AOF file.
• auto-aof-rewrite-percentage and auto-aof-rewrite-min-size: Controls AOF rewrite thresholds.

Hybrid Persistence

Redis 4.0 introduced hybrid persistence, combining RDB and AOF. The AOF file starts with an RDB snapshot followed by incremental AOF commands.

Enable hybrid persistence with:

appendonly yes
aof-use-rdb-preamble yes

Choosing the Right Persistence Strategy

• Use RDB for fast backups and acceptable data loss.
• Use AOF for high durability requirements.
• Use Hybrid for a balance of performance and durability.