File System Layer Architecture

Before exploring page cache, understanding the file system layer architecture provides essential context for comprehending how data flows through the Linux kernel.

1 VFS Layer

The Virtual File System (VFS) layer serves as a generic abstraction for file operations in Linux. It consolidates common functionalities shared across different file systems into unified data structures such as file, inode, and dentry. The VFS layer exposes standardized API interfaces that concrete file system implementations must adhere to, enabling seamless transitions between different file system types during I/O operations.

This layer provides a unified interface for upper layers while handling the complexity of switching between various file system implementations during I/O requests. Developers creating custom kernel file systems only need to implement the predefined API interfaces.

2 File System Layer

The specific file system layer receives I/O requests from VFS and performs the critical task of mapping file concepts to physical storage on block devices. File systems manage linear address spaces (partitions or block devices) while presenting users with hierarchical file abstractions. This transformation layer determines how user data gets stored physically—whether in 4K blocks or 1MB segments, depending on the file system's design.

The address_space interface serves as the core mechanism for mapping virtual file space to actual linear device space.

3 Block Layer

The block layer introduces an abstraction above actual hardware,屏蔽不同的硬件驱动 (sheilding different hardware drivers) and presenting block devices as simple linear address spaces. Beyond hardware abstraction, this layer implements I/O scheduling algorithms that batch and aggregate requests, optimizing for sequential access patterns and reducing total I/O operations.

The block layer primarily handles I/O scheduling optimizations, with the famous elevator algorithm being a prominent example. Linux allows configurable scheduling policies including CFQ, Deadline, and NOOP.

4 SCSI Layer

The SCSI layer represents the hardware driver interface, functioning as a translation layer between the kernel and disk hardware. This layer handles the final transformation before I/O operations reach physical disk hardware.

What is Page Cache

Files stored on hard drives or solid-state drives cannot be accessed directly by the CPU. Data must first be read into memory before the CPU can process it.

Since disk access speeds significantly lag behind memory access speeds (DDR4 memory performs approximately 500 times faster than mechanical hard drives and 200 times faster than SSDs), the Linux kernel implements Page Cache to avoid repeated disk operations for file access. This mechanism caches file data in memory pages, collectively known as page cache.

Page Cache constitutes a memory management mechanism within the operating system that buffers file data from disk. It represents a portion of kernel-managed memory that stores recently read or written file content, thereby enhancing file access performance.

When an application requests file data, the kernel first checks whether the requested data exists in the page cache. If found, the data returns directly from memory, eliminating disk access. Otherwise, the kernel reads data from disk into the page cache before returning it to the application.

Similarly, when applications write data, information gets stored in the page cache first, with the kernel periodically flushing cached data back to disk.

Page Cache significantly improves file read and write efficiency since memory access substantially outperforms disk access. By reducing actual disk operations, page cache dramatically enhances file system performance, particularly during repeated access to identical files or specific file sections.

It is important to note that page cache serves purely as a caching mechanism without providing data persistence guarantees. File data remains stored on disk, with page cache merely holding a temporary copy that can be updated or replaced. File operations require explicit data flushing to ensure persistence and consistency.

Additional Notes:

I/O operations fall into two categories: cached I/O and direct I/O. Direct I/O bypasses page cache, interacting directly with disk storage.

Kernel Buffer I/O and Direct I/O represent distinct approaches for disk data transfer:

1. Buffer I/O:

   • Buffer I/O routes data through the kernel's page cache during disk operations. Read operations transfer data from disk to kernel page cache, then copy to application buffers. Write operations place data in kernel page cache first, with the OS periodically flushing to disk.
   • Advantages: Leverages system memory for caching, reducing frequent disk reads/writes and improving access speed.
   • Disadvantages: Requires additional memory overhead for cache maintenance. Data consistency concerns exist because writes initially target cache rather than disk, potentially causing data loss or inconsistency during system crashes.

2. Direct I/O:

   • Direct I/O bypasses kernel page cache, transferring data directly between user space and disk.
   • Read operations transfer data directly from disk to application buffers; write operations similarly bypass kernel cache.
   • Advantages: Eliminates multiple copies within kernel cache, reduces memory overhead, and provides more controllable data consistency.
   • Disadvantages: Bypassing kernel cache subjects performance to disk physical limitations, potentially reducing read/write performance.

The red section in diagrams represents Page Cache, which constitutes memory regions managed by the Linux kernel. File reads through mmap and buffered I/O operations load data into page cache.

Examining Page Cache Through System Information

System memory statistics available through /proc/meminfo provide real-time visibility into page cache utilization:

$ cat /proc/meminfo
...
Buffers:            1536 kB
Cached:           108284 kB
SwapCached:        41280 kB
Active:          6348296 kB
Inactive:         1023456 kB
Active(anon):    6290492 kB
Inactive(anon):   985672 kB
Active(file):      57804 kB
Inactive(file):    38784 kB
...
Shmem:             12400 kB
...
SReclaimable:      38764 kB
...

Based on this data, the following relationship holds (both sides total 123100 KB):

Buffers + Cached + SwapCached = Active(file) + Inactive(file) + Shmem + SwapCached

Both sides represent Page Cache:

Page Cache = Buffers + Cached + SwapCached

Derived relationship:

Cached + Buffers = Active(file) + Inactive(file) + Shmem

Page cache comprises three primary components:

• Cached: file page data cache
• Buffer: block device (disk) block data cache
• Swap cache: swapped page cache

Cache operates at the logical level (page-based), aligning with file system hierarchy, while buffer operates at the physical level (block-based), aligning with block device drivers. In practical terms, buffer handles lower-level disk I/O while cache manages file system I/O.

Cache encompasses not only file data pages but also tmpfs and shared memory (shmem). This inclusion is logical when considering tmpfs but might seem unexpected for shared memory.

Furthermore, not all pages organize into page cache structures.

Linux provides two distinct memory types accessible to users:

• File-backed pages: Pages in page cache corresponding to physical disk data blocks. The primary concern for these pages involves dirty page writeback.
• Anonymous pages: Pages without corresponding persistent disk data, typically storing process stacks, heaps, and dynamically allocated memory. These contain runtime-generated data requiring no disk persistence.

The active and inactive classifications represent an additional dimension for page categorization:

Active pages reference recently accessed memory containing data currently or recently used by processes. The kernel determines memory retention priority based on access frequency, generally preserving active pages in memory for performance. Only memory pressure triggers active page eviction.

Inactive pages have not been accessed for extended periods, containing previously used but currently inactive data. While not frequently accessed, these pages remain available when memory充足 (sufficient). The kernel may reclaim memory from inactive pages when requirements arise for other processes or purposes.

Active and inactive designations merely indicate access patterns and kernel management priorities, not page content validity. Both concepts optimize memory utilization within the kernel's memory management subsystem.

Page Cache Advantages and Disadvantages

Advantages

1. Accelerated Data Access

Caching data in memory eliminates subsequent disk I/O requirements, with cache hits serving data directly from memory. Since memory access significantly outperforms disk access, this represents a primary benefit of page cache.

2. Reduced I/O Operations and Improved Throughput

Page cache caching and read-ahead capabilities, combined with program locality patterns, enable loading multiple pages through single I/O operations, reducing disk access frequency and increasing system I/O throughput.

Disadvantages

Page cache introduces notable drawbacks. First, it consumes additional physical memory, which becomes problematic during memory scarcity, potentially triggering frequent swap operations and escalating disk I/O load.

Second, page cache provides limited management APIs for applications, remaining largely transparent to user space. Applications seeking to optimize page cache utilization face significant challenges. Some applications implement custom page management in user space, with MySQL InnoDB managing 16KB pages independently.

Third, under certain workloads, page cache introduces additional disk read and write operations compared to Direct I/O.