Web caching operates between origin servers and client browsers. When users request a URL, the cache server fetches content from backend web servers. Subsequent identical requests are served directly from cache, eliminating redundant origin server queries. This architecture reduces database load, decreases network latency, improves response times, and enhances user experience.

While Squid Cache remains a prominent solution, modern deployments increasingly favor Nginx for caching static assets like images, JavaScript, and CSS files.

Nginx Caching Architecture

Nginx provides two primary caching mechanisms:

1. proxy_cache - Caches content from backend origin servers during reverse proxy operations, delivering performance comparable to Squid with superior stability.
2. fastcgi_cache - Specifically caches dynamic content generated by FastCGI applications.

Beyond caching, Nginx offers advanced capabilities incluidng multi-core CPU optimization, load balancing, health checks, failover mechanisms, and URL rewriting, enabling it to function simultaneously as both a load balancer and cache server.

Proxy Cache Directives

1. proxy_cache

Syntax: proxy_cache zone_name;

Specifies which cache zone to utilize. The zone_name must match a zone created via proxy_cache_path. This directive works in conjunction with proxy_pass to fetch and store content from designated upstream servers.

2. proxy_cache_path

Syntax: proxy_cache_path path [levels=levels] keys_zone=name:size [inactive=time] [max_size=size];

Configures cache storage parameters.

Example:

proxy_cache_path /var/nginx/cache levels=1:2 keys_zone=cache_main:512m inactive=24h max_size=50g;

• path - Physical storage directory
• levels - Directory hierarchy (1:2 creates two-level hash structure: /c/29/filename)
• keys_zone - Zone identifier and memory allocation (512MB stores approximately 4 milion keys)
• inactive - Automatic purge after 24 hours of no access
• max_size - Maximum disk usage (50GB)

3. proxy_cache_methods

Syntax: proxy_cache_methods [GET HEAD POST];

Defines which HTTP methods to cache. Default: GET and HEAD only.

4. proxy_cache_min_uses

Syntax: proxy_cache_min_uses number;

Minimum request threshold before caching occurs. Default: 1.

5. proxy_cache_valid

Syntax: proxy_cache_valid code [code...] time;

Sets cache duration based on HTTP response codes.

proxy_cache_valid 200 302 15m;
proxy_cache_valid 404 2m;

6. proxy_cache_key

Syntax: proxy_cache_key string;

Defines the cache key for MD5 hashing. Typically combines host, URI, and query parameters:

proxy_cache_key "$scheme$host$request_uri$is_args$args";

Complete Proxy Cache Configuration

Installation with Cache Purge Module

# Install dependencies
yum install -y pcre pcre-devel openssl openssl-devel gcc

cd /opt/src
wget http://labs.frickle.com/files/ngx_cache_purge-2.3.tar.gz
wget http://nginx.org/download/nginx-1.18.0.tar.gz

tar -zxf ngx_cache_purge-2.3.tar.gz
tar -zxf nginx-1.18.0.tar.gz

cd nginx-1.18.0
./configure --user=www --group=www \
  --add-module=../ngx_cache_purge-2.3 \
  --prefix=/opt/nginx \
  --with-http_ssl_module \
  --with-http_stub_status_module \
  --with-http_gzip_static_module

make && make install

Directory Setup

Create cache directories on the same filesystem (required for hard links):

mkdir -p /var/nginx/cache
mkdir -p /var/nginx/temp
chown -R www:www /var/nginx/cache /var/nginx/temp
chmod -R 755 /var/nginx/cache /var/nginx/temp

Main Configuration

user www;
worker_processes auto;

events {
    worker_connections 65535;
}

http {
    include mime.types;
    default_type application/octet-stream;
    charset utf-8;

    log_format main '$http_x_forwarded_for $remote_addr [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" $request_time';

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;

    # Cache configuration
    proxy_cache_path /var/nginx/cache levels=1:2 keys_zone=cache_main:512m inactive=24h max_size=50g;
    proxy_temp_path /var/nginx/temp;

    # FastCGI settings
    fastcgi_connect_timeout 3000;
    fastcgi_send_timeout 3000;
    fastcgi_read_timeout 3000;
    fastcgi_buffer_size 256k;
    fastcgi_buffers 8 256k;

    # Client settings
    client_max_body_size 100m;
    client_body_buffer_size 256k;

    # Gzip compression
    gzip on;
    gzip_min_length 1k;
    gzip_comp_level 6;
    gzip_types text/plain text/css application/json application/javascript;

    include /opt/nginx/conf/vhosts/*.conf;
}

Virtual Host Configuration

upstream backend_pool {
    ip_hash;
    server 10.0.0.101:80 max_fails=3 fail_timeout=30s;
    server 10.0.0.102:80 max_fails=3 fail_timeout=30s;
    server 10.0.0.103:80 max_fails=3 fail_timeout=30s;
}

server {
    listen 80;
    server_name www.example.com;

    access_log /var/nginx/logs/access.log main;
    error_log /var/nginx/logs/error.log;

    location / {
        proxy_pass http://backend_pool;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_connect_timeout 30s;
        proxy_read_timeout 60s;
    }

    # Cache static assets
    location ~* \.(gif|jpg|jpeg|png|bmp|swf|js|css)$ {
        proxy_cache cache_main;
        proxy_cache_valid 200 304 12h;
        proxy_cache_valid 301 302 1m;
        proxy_cache_valid any 1m;
        proxy_cache_key $host$uri$is_args$args;
        proxy_pass http://backend_pool;
    }

    # Purge endpoint
    location ~ /purge(/.*) {
        allow 127.0.0.1;
        allow 10.0.0.0/16;
        deny all;
        proxy_cache_purge cache_main $host$1$is_args$args;
    }
}

FastCGI Cache Directives

1. fastcgi_cache

Syntax: fastcgi_cache zone_name;

Specifies the FastCGI cache zone to use.

2. fastcgi_cache_path

Syntax: fastcgi_cache_path path [levels=levels] keys_zone=name:size [inactive=time] [max_size=size];

Example:

fastcgi_cache_path /var/nginx/fcgi_cache levels=1:2 keys_zone=fcgi_main:512m inactive=24h max_size=50g;

3. fastcgi_cache_methods

Syntax: fastcgi_cache_methods [GET HEAD POST];

Default: GET and HEAD.

4. fastcgi_cache_min_uses

Syntax: fastcgi_cache_min_uses number;

Default: 1.

5. fastcgi_cache_valid

Syntax: fastcgi_cache_valid code [code...] time;

fastcgi_cache_valid 200 302 15m;
fastcgi_cache_valid 404 2m;

6. fastcgi_cache_key

Syntax: fastcgi_cache_key string;

Example:

fastcgi_cache_key "$scheme$request_method$host$request_uri";

Complete FastCGI Cache Configuration

# Create directories
mkdir -p /var/nginx/fcgi_temp
mkdir -p /var/nginx/fcgi_cache
chown -R www:www /var/nginx/fcgi_*

# nginx.conf
http {
    fastcgi_temp_path /var/nginx/fcgi_temp;
    fastcgi_cache_path /var/nginx/fcgi_cache levels=1:2 keys_zone=fcgi_main:512m inactive=24h max_size=50g;
    ...
}

# Virtual host
server {
    location ~* \.php$ {
        fastcgi_cache fcgi_main;
        fastcgi_cache_valid 200 15m;
        fastcgi_cache_valid 301 302 1h;
        fastcgi_cache_valid any 1m;
        fastcgi_cache_key "$scheme$request_method$host$request_uri";
        fastcgi_pass 127.0.0.1:9000;
        include fastcgi_params;
    }
}

Cache Purge Module

The ngx_cache_purge module enables selective cache invalidation. Access http://domain/purge/uri to remove specific cached entries. The module supports proxy, FastCGI, SCGI, and uWSGI caches.

Single-Machine Deployment Example

Scenario: Deploy cache server on a single host.

Main Configuration

user www;
worker_processes auto;

events {
    worker_connections 65535;
}

http {
    include mime.types;
    default_type application/octet-stream;
    
    log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" "$http_user_agent"';
    
    sendfile on;
    keepalive_timeout 65;
    
    proxy_temp_path /var/nginx/temp;
    proxy_cache_path /var/nginx/cache levels=1:2 keys_zone=static_cache:512m inactive=24h max_size=30g;
    
    client_max_body_size 50m;
    gzip on;
    gzip_types text/plain text/css application/javascript;
    
    include /opt/nginx/conf/vhosts/*.conf;
}

Backend Server (Port 8080)

server {
    listen 8080;
    server_name localhost;
    
    access_log /var/nginx/logs/backend.log main;
    
    location / {
        root /var/www/assets;
        index index.html;
    }
}

Cache Server (Port 80)

upstream local_backend {
    server 127.0.0.1:8080 max_fails=3 fail_timeout=30s;
}

server {
    listen 80;
    server_name cache.example.com;
    
    access_log /var/nginx/logs/cache.log main;
    error_log /var/nginx/logs/cache_error.log;
    
    location / {
        proxy_pass http://local_backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        
        proxy_cache static_cache;
        proxy_cache_valid 200 304 12h;
        proxy_cache_valid 301 302 1m;
        proxy_cache_valid any 1m;
        proxy_cache_key $host$uri$is_args$args;
    }
    
    location ~ /purge(/.*) {
        allow 127.0.0.1;
        allow 10.0.0.0/16;
        deny all;
        proxy_cache_purge static_cache $host$1$is_args$args;
    }
}

Verification and Testing

After starting Nginx, verify cache generation:

ls -la /var/nginx/cache/

Expected output shows hashed directory structure (e.g., a/bc/).

Common Issues

Cache Directory Not Populating

1. Permissions: Ensure Nginx user owns cache and temp directories
2. Configuration: Even in single-server setups, proxy_pass must point to an upstream or alternate port
3. Syntax Errors: Check for invalid whitespace in proxy_cache_path directives

Disk Space Management

When cache disks fill up:

1. Use RAID arrays (reduces usable capacity)
2. Leverage the levels directory structure by symlinking top-level cache directories to larger filesystems

Buffer and Cache Optimization

Buffer Tuning

Buffer configuration impacts performance significantly:

proxy_buffering on;
proxy_buffers 16 8k;
proxy_buffer_size 4k;
proxy_busy_buffers_size 24k;
proxy_max_temp_file_size 2048m;
proxy_temp_file_write_size 32k;

Cache Control Headers

Implement proper cache-control for mixed content:

location /public/ {
    expires 1h;
    add_header Cache-Control "public";
}

location /private/ {
    expires -1;
    add_header Cache-Control "private, no-store";
}

location /api/ {
    expires 5m;
    add_header Cache-Control "public, must-revalidate";
}

Configuration Without Purge Module

For environments where manual cache invalidation isn't required:

./configure --prefix=/opt/nginx \
  --with-http_ssl_module \
  --with-http_gzip_static_module \
  --with-pcre

# Cache configuration without purge endpoint
proxy_cache_path /var/nginx/cache levels=1:2 keys_zone=auto_cache:256m inactive=1h max_size=20g;

server {
    location / {
        proxy_pass http://application_backend;
        proxy_cache auto_cache;
        proxy_cache_valid 200 304 1h;
        proxy_cache_key $scheme$host$request_uri;
    }
}

To clear cache manually, delete cache directory contents and reload Nginx:

rm -rf /var/nginx/cache/*
nginx -s reload