Direct Caching of Page Results

The most straightforward approach to caching paginated data involves storing the complete result set for a specific page query. The cache key is typically constructed using the query parameters and pagination metadata.

public List<Item> fetchItemList(String filter, int pageNum, int pageSize) {
    String cacheKey = "items:list:" + filter + ":" + pageNum + ":" + pageSize;
    List<Item> items = redisClient.get(cacheKey);

    if (items != null) {
        return items;
    }

    items = databaseRepository.query(filter, pageNum, pageSize);
    if (items != null) {
        redisClient.setex(cacheKey, 3600, items);
    }
    return items;
}

While this method is simple and offers fast read speeds for cache hits, it suffers from a significant drawback regarding data granularity. The cached unit is the entire page result. If any item within this dataset is added, modified, or deleted, maintaining consistency requires invalidating the cache.

Common invalidation strategies include relying on time-to-live (TTL) expiration, which introduces latency in consistency, or actively finding and deleting keys. However, using commands like KEYS to scan for matching patterns in production is highly discouraged due to the blocking nature of the operation, which can degrade Redis performance severely.

Separating ID Lists from Object Caching

To achieve finer control over cache invalidation and improve hit rates, the strategy shifts from caching the full payload to caching only the list of identifiers (IDs) for the current page. The actual data objects are then cached individually, mapped by their IDs.

This process involves several distinct steps:

1. Fetch Identifiers from the Database
Query the database solely for the primary keys corresponding to the requested page.

SELECT id 
FROM items 
WHERE category = ? 
ORDER BY created_at DESC 
LIMIT ? OFFSET ?;

List<Long> itemIds = itemRepository.findIds(filter, offset, limit);

2. Retrieve Objects from Cache
Use the list of IDs to perform a batch retrieval of item objects from the cache.

Map<Long, Item> cachedItems = redisClient.mget(itemIds);

3. Identify Cache Misses
Iterate through the requested ID list to determine which items are missing from the cache.

List<Long> missingIds = itemIds.stream()
    .filter(id -> !cachedItems.containsKey(id))
    .collect(Collectors.toList());

4. Backfill Missing Data
Perform a batch query on the database for the missing IDs and update the cache.

SELECT * FROM items WHERE id IN (?, ?, ?, ?);

if (!missingIds.isEmpty()) {
    List<Item> loadedItems = itemRepository.findByIds(missingIds);
    
    // Update cache using pipeline or mset for efficiency
    Map<Long, Item> loadedMap = loadedItems.stream()
        .collect(Collectors.toMap(Item::getId, Function.identity()));
    
    redisClient.mset(loadedMap);
    
    // Merge into existing map
    cachedItems.putAll(loadedMap);
}

5. Assemble the Result Reconstruct the final list by mapping the original ordered ID list to the fully populated item map.

List<Item> finalResult = itemIds.stream()
    .map(cachedItems::get)
    .filter(Objects::nonNull)
    .collect(Collectors.toList());

This architecture effectively decouples the pagination logic from the data storage. The source of the ID list can be flexible—it might come from a SQL database, a search engine like Elasticsearch, or another Redis structure—while the detailed object data is always served from a high-speed object cache.

Caching ID Lists and Objects via Redis Sorted Sets

In scenarios requiring time-based or score-based ordering, such as social media feeds or activity logs, storing the ID list in a standard database result set is inefficient. Instead, Redis Sorted Sets (ZSets) provide an optimal solution.

ZSets store unique members (the object IDs) associated with a score (typically a timestamp or priority).

Structure:
- Member: The ID of the dynamic entity.
- Score: The creation timestamp or sorting weight.

Using the ZREVRANGE command allows for efficient retrieval of a paginated list of IDs based on the score in descending order.

ZREVRANGE feed:user:123 0 20

Once the ID list is retrieved, the system retrieves the corresponding detailed objects. For complex data structures involving nested entities (e.g., posts with comments and likes), fetching details might require multiple cache lookups. To maximize performance, batch operations are essential.

Depending on complexity, tools like MGET for simple hashes, or Redis Pipelines and Lua scripts for complex multi-key retrieval, should be employed to minimize network round-trips.

Workflow:
1. Fetch the paginated ID list using ZREVRANGE.
2. Use a Redis Pipeline to batch-fetch all associated data objects (content, user info, metrics) in a single network operation.
3. Construct the final response.