Redis Persistence Mechanisms

RDB Snapshotting

RDB (Redis Database) persistence captures a point-in-time snapshot of the dataset in memory and writes it to disk as a binary file. During recovery, Redis loads this file directly into memory.

Triggering Methods

Manual Triggers

• SAVE: Blocks the main thread until the snapshot is fully written — no other commands can be processed during this time.
• BGSAVE: Forks a child process to handle serialization asynchronously. Only the fork() system call blocks the parent briefly; subsequent I/O occurs off the main thread.

Automatic Triggers Configuration directives in redis.conf define time-and-change thresholds:

save 900 1     # Save if ≥1 key changed in 15 minutes
save 300 10    # Save if ≥10 keys changed in 5 minutes
save 60 10000  # Save if ≥10,000 keys changed in 1 minute

dbfilename dump.rdb
dir /var/lib/redis/
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes

Other automatic triggers include:

• Full synchronization during replication (master generates RDB for slavee).
• DEBUG RELOAD command execution.
• SHUTDOWN without AOF enabled.

Snapshot Workflow

1. Redis checks whether another background operation (BGSAVE, BGREWRITEAOF) is active — if so, BGSAVE returns immediately.
2. The main process calls fork(), temporarily blocking all client requests.
3. After forking, the parent resumes handling commands while the child serializes the dataset.
4. The child writes to a temporary file, then atomically replaces the previous RDB file upon completion.
5. The child signals success; the parent updates internal metrics.

Copy-on-Write Behavior

Redis leverages OS-level copy-on-write (COW). At fork(), the child shares physical memory pages with the parent but with read-only permissions. When either process modifies a page, the kernel duplicates that 4KB page before allowing the write. This minimizes memory duplication and avoids copying entire datasets upfront.

As a result:

• Modifications made by the main thread after forking do not appear in the current RDB file.
• If Redis crashes right after the snapshot finishes, changes applied between the start and end of the BGSAVE cycle are lost.
• Under heavy write load, worst-case memory usage may double due to widespread COW duplication.

Trade-offs

Pros

• Compact, fast-to-load binary format ideal for backups and bulk transfers.
• Faster restoration then AOF.

Cons

• Not suitable for real-time durability — data loss window equals snapshot interval.
• Format incompatible across major Redis versions.
• Frequent snapshots increase CPU and I/O pressure.

AOF Logging

AOF (Append-Only File) logs every write command sequentially, similar to database transaction logs. On restart, Redis replays these commands to reconstruct the dataset.

Enabling AOF

appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
aof-rewrite-incremental-fsync yes

Commands are logged after successful execution — ensuring only syntactically valid, executed operations enter the log.

Advantages

• Lower risk of corruption: failed commands never reach the log.
• Non-blocking command execution (though logging itself happens synchronous in the main thread).

Drawbacks

• Potential data loss if the OS buffer isn’t flushed before crash.
• Increased latency under high disk I/O pressure.

Write Policies

Policy	Timing	Durability	Performance
always	After every command	Highest	Lowest
everysec	Once per second	Up to 2s loss	Balanced
no	OS-dependent	Unpredictable loss	Highest

All policies rely on fsync() to flush kernel buffers to disk. Choosing one involves balancing safety versus throughput.

AOF Rewriting

To prevent unbounded log growth, Redis supports rewriting — generating a minimal equivalent log from the current dataset state.

Example:

• Original sequence: SET name zhangsan, SET name lisi
• Rewritten: only SET name lisi remains, since earlier values are obsolete.

Rewriting runs in a background process (BGREWRITEAOF) to avoid blocking the main thread. It uses fork() and inherits the same COW semantics as RDB.

During rewriting:

• A dedicated AOF rewrite buffer collects new writes.
• Upon completion, the child process signals the parent, which appends the rewrite buffer contents to the new file and atomically swaps it in.

This design ensures resilience: partial failures leave the original AOF intact.

Trade-offs

Pros

• Higher durability guarantees than RDB.
• Human-readable log format aids debugging and disaster recovery.

Cons

• Larger file size compared to RDB.
• Slower recovery due to sequential command replay.
• Slight performance impact on write-heavy workloads.

Hybrid Persistence (RDB+AOF)

Introduced in Redis 4.0, hybrid mode combines both mechanisms within a single AOF file:

• Prefix contains an RDB-formatted full dataset snapshot.
• Suffix holds incremental AOF commands since the snapshot.

Enabled via:

aof-use-rdb-preamble yes

Benefits

• Fast startup (RDB portion loads quickly).
• Minimal data loss (AOF tail captures recent changes).

Limitations

• Incompatible with Redis < 4.0.
• Reduced readability due to mixed binary/text encoding.

Expiration Handling

Redis manages expiring keys using two complementary strategies:

• Lazy expiration: Keys are checked only when accessed. Efficient for CPU, but risks memory bloat if expired keys remain unused.
• Active expiration: Periodically scans a subset of keys with TTLs. Default behavior checks up to 20 keys per cycle; if >25% are expired, it repeats. Scans occur ~10 times per second, capped at 25ms per iteration.

Expiration metadata resides in a separate dictionary (expires) mapping keys to UNIX timestamps.

Since Redis 4.0, lazy deletion releases memory asynchronously to avoid main-thread stalls.

Memory Eviction Policies

When maxmemory is reached, Redis applies one of eight eviction policies:

TTL-based (volatile keys only)

• volatile-ttl: Evicts keys with shortest remaining TTL.
• volatile-random: Random selection among expiring keys.
• volatile-lru: Approximate LRU among expiring keys.
• volatile-lfu: Approximate LFU among expiring keys.

All-keys (any key eligible)

• allkeys-random: Uniform random selection.
• allkeys-lru: Approximate LRU across entire dataset.
• allkeys-lfu: Approximate LFU across entire dataset.

No eviction

• noeviction: Returns errors on write attempts exceeding memory limit.

LRU approximates recency using a sampled 24-bit access counter; LFU tracks frequency via logarithmic counters to prevent aging bias.

Configuration example:

maxmemory 2gb
maxmemory-policy allkeys-lfu