When processing bulk data operations, applications may encounter system resource limitations that lead to critical failures. A common scenario involves Redis connection pooling exhausting available file descriptors.

During batch processing of over 2000 records, a application experienced complete service failure with the following stack trace:

Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketException: Too many open files
        at redis.clients.jedis.Connection.connect(Connection.java:164)
        at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:80)
        at redis.clients.jedis.BinaryJedis.connect(BinaryJedis.java:1677)
        at redis.clients.jedis.JedisFactory.makeObject(JedisFactory.java:87)
        at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:883)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:436)
        at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:365)
        at redis.clients.util.Pool.getResource(Pool.java:49)
        ... 111 common frames omitted
Caused by: java.net.SocketException: Too many open files
        at java.net.Socket.createImpl(Socket.java:460)
        at java.net.Socket.getImpl(Socket.java:520)
        at java.net.Socket.setReuseAddress(Socket.java:1449)
        at redis.clients.jedis.Connection.connect(Connection.java:148)
        ... 118 common frames omitted
redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool

This issue occurred because the application ran under a regular user account with a file descriptor limit of 1024. Once this threshold was exceeded, new socket connections could not be established.

To diagnose the problem, check current limits and active connections:

# View current file descriptor limit
ulimit -n

# Count open file descriptors for a specific process
lsof -p 2915 | wc -l

System-wide limits can be adjusted by modifying /etc/security/limits.conf:

*               soft    nofile          1024
*               hard    nofile          1024

How ever, the root cause was identified in the Redis connection management code. The original implementation failed to properly release connections back to the pool:

public class RedisConnector {
    private JedisPool connectionPool;
    
    public Jedis acquireConnection() throws JedisException {
        Jedis resource = null;
        try {
            resource = connectionPool.getResource();
        } catch (JedisException exception) {
            if (resource != null) {
                connectionPool.returnResource(resource);
            }
            throw exception;
        }
        return resource;
    }
}

The empty finally block meant connection were never returned to the pool, causing resource exhaustion. Proper resource management requires explicit connection closure after use:

public class ImprovedRedisConnector {
    private JedisPool pool;
    
    public void executeOperation() {
        Jedis connection = null;
        try {
            connection = pool.getResource();
            // Perform Redis operations
            connection.set("key", "value");
        } finally {
            if (connection != null) {
                connection.close();
            }
        }
    }
}

Using try-with-resources provides cleaner automatic resource management:

public class ModernRedisConnector {
    private JedisPool jedisPool;
    
    public void performTask() {
        try (Jedis connection = jedisPool.getResource()) {
            // Redis operations automatically cleaned up
            connection.get("example-key");
        }
        // Connection automatically closed here
    }
}