Single-System Locks
In single-system architectures, locks like ReentrantLock are effective for controlling concurrent access within a single JVM. However, when multiple instances of an application are running behind a load balancer, these locks become ineffective as each instance operates independently.
Testing with JMeter to simulate high concurrency through Nginx reveals the problem: even with 100 concurrent requests, inventory may show 12 items remaining instead of zero. This indicates that locks are only effective within their own JVM instance.
Distributed System Locks
Version 1: Basic Redis Locking
The initial approach uses Redis' set and get operations for locking. When a thread needs to access a shared resource, it attempts to set a unique key-value pair. If successful, it proceeds with the critical section; otherwise, it retries.
@RequestMapping("/sale")
public String sale(){
String retMessage = "";
Object key = "RedisLock";
Object value = UUID.randomUUID() + ":" + Thread.currentThread().getId();
Boolean flag = redisTemplate.opsForValue().setIfAbsent(key, value);
if (!flag){
try {
TimeUnit.MILLISECONDS.sleep(20);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
sale();
} else {
try {
String result = redisTemplate.opsForValue().get("inventory001").toString();
Integer inventoryNum = result == null ? 0 : Integer.parseInt(result);
if (inventoryNum != 0){
redisTemplate.opsForValue().set("inventory001", String.valueOf(--inventoryNum));
retMessage = "Successfully sold item, remaining inventory: " + inventoryNum;
System.out.println(inventoryNum);
} else {
retMessage = "Item sold out!";
}
} finally {
redisTemplate.delete(key);
}
}
return retMessage;
}However, this recursive approach fails in high-concurrency scenarios due to potential stack overflow issues.
Version 2: Spin-Waiting Approach
Replacing recursion with a spin-waiting mechanism solves the stack overflow problem:
@RequestMapping("/sale")
public String sale(){
String retMessage = "";
Object key = "RedisLock";
Object value = UUID.randomUUID() + ":" + Thread.currentThread().getId();
while(!redisTemplate.opsForValue().setIfAbsent(key, value)){
try {
TimeUnit.MILLISECONDS.sleep(20);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
try {
String result = redisTemplate.opsForValue().get("inventory001").toString();
Integer inventoryNum = result == null ? 0 : Integer.parseInt(result);
if (inventoryNum != 0){
redisTemplate.opsForValue().set("inventory001", String.valueOf(--inventoryNum));
retMessage = "Successfully sold item, remaining inventory: " + inventoryNum;
System.out.println(inventoryNum);
} else {
retMessage = "Item sold out!";
}
} finally {
redisTemplate.delete(key);
}
return retMessage;
}While this prevents stack overflow, it introduces a new problem: if an exception occurs before the finally block, the lock remains indefinitely.
Version 3: Adding Expiration
To prevent indefinite lock retention, we add expiration times to locks:
@RequestMapping("/sale")
public String sale(){
String retMessage = "";
Object key = "RedisLock";
Object value = UUID.randomUUID() + ":" + Thread.currentThread().getId();
while(!redisTemplate.opsForValue().setIfAbsent(key, value)){
try {
TimeUnit.MILLISECONDS.sleep(20);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
redisTemplate.expire(key, 30L, TimeUnit.SECONDS);
try {
String result = redisTemplate.opsForValue().get("inventory001").toString();
Integer inventoryNum = result == null ? 0 : Integer.parseInt(result);
if (inventoryNum != 0){
redisTemplate.opsForValue().set("inventory001", String.valueOf(--inventoryNum));
retMessage = "Successfully sold item, remaining inventory: " + inventoryNum;
System.out.println(inventoryNum);
} else {
retMessage = "Item sold out!";
}
} finally {
redisTemplate.delete(key);
}
return retMessage;
}However, this approach has a race condition: if an exception occurs after setting the lock but before setting the expiration, we still face the same problem.
Version 4: Atomic Operations
Making the lock acquisition and expiration atomic solves the race condition:
@RequestMapping("/sale")
public String sale(){
String retMessage = "";
Object key = "RedisLock";
Object value = UUID.randomUUID() + ":" + Thread.currentThread().getId();
while(!redisTemplate.opsForValue().setIfAbsent(key, value, 30L, TimeUnit.SECONDS)){
try {
TimeUnit.MILLISECONDS.sleep(20);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
try {
String result = redisTemplate.opsForValue().get("inventory001").toString();
Integer inventoryNum = result == null ? 0 : Integer.parseInt(result);
if (inventoryNum != 0){
redisTemplate.opsForValue().set("inventory001", String.valueOf(--inventoryNum));
retMessage = "Successfully sold item, remaining inventory: " + inventoryNum;
System.out.println(inventoryNum);
} else {
retMessage = "Item sold out!";
}
} finally {
if (redisTemplate.opsForValue().get(key).toString().equalsIgnoreCase(value.toString())){
redisTemplate.delete(key);
}
}
return retMessage;
}This ensures only the lock owner can delete their lock, but the delete operation itself isn't atomic.
Version 5: Atomic Unlock with Lua
Using Lua scripts makes the unlock operation atomic:
@RequestMapping("/sale")
public String sale(){
String retMessage = "";
Object key = "RedisLock";
Object value = UUID.randomUUID() + ":" + Thread.currentThread().getId();
while(!redisTemplate.opsForValue().setIfAbsent(key, value, 30L, TimeUnit.SECONDS)){
try {
TimeUnit.MILLISECONDS.sleep(20);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
try {
String result = redisTemplate.opsForValue().get("inventory001").toString();
Integer inventoryNum = result == null ? 0 : Integer.parseInt(result);
if (inventoryNum != 0){
redisTemplate.opsForValue().set("inventory001", String.valueOf(--inventoryNum));
retMessage = "Successfully sold item, remaining inventory: " + inventoryNum;
System.out.println(inventoryNum);
} else {
retMessage = "Item sold out!";
}
} finally {
String luaScript = "if redis.call('get',KEYS[1]) == ARGV[1] then " +
"return redis.call('del', KEYS[1]) " +
"else " +
"return 0 " +
"end";
redisTemplate.execute(new DefaultRedisScript<>(luaScript, Boolean.class),
Arrays.asList(key), value);
}
return retMessage;
}Version 6: Reentrant Locks
To support reentrancy, we use Redis hash structures to track lock ownership and reentrancy counts:
// Lock acquisition script
if redis.call('exists', KEYS[1]) == 0 or redis.call('hexists', KEYS[1], ARGV[1]) == 1 then
redis.call('hincrby', KEYS[1], ARGV[1], 1)
redis.call('expire', KEYS[1], ARGV[2])
return 1
else
return 0
end// Lock release script
if redis.call('hexists', KEYS[1], ARGV[1]) == 0 then
return nil
elseif redis.call('hincrby', KEYS[1], ARGV[1], -1) == 0 then
return redis.call('del', KEYS[1])
else
return 0
endImplementing these scripts in Java:
public class RedisDistributedLock implements Lock {
@Autowired
private StringRedisTemplate redisTemplate;
private String lockName;
private String uuidValue;
private Long expireTime;
public RedisDistributedLock(StringRedisTemplate redisTemplate, String lockName) {
this.redisTemplate = redisTemplate;
this.lockName = lockName;
this.uuidValue = UUID.randomUUID() + ":" + Thread.currentThread().getId();
this.expireTime = 50L;
}
@Override
public void lock() {
try {
tryLock();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
@Override
public boolean tryLock() {
try {
return tryLock(-1L, TimeUnit.SECONDS);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
@Override
public boolean tryLock(long time, TimeUnit unit) throws InterruptedException {
if (time == -1L) {
String script = "if redis.call('exists', KEYS[1]) == 0 or redis.call('hexists', KEYS[1], ARGV[1]) == 1 then " +
"redis.call('hincrby', KEYS[1], ARGV[1], 1) redis.call('expire', KEYS[1], ARGV[2]) " +
"return 1 " +
"else " +
"return 0 " +
"end";
while (!redisTemplate.execute(new DefaultRedisScript<>(script, Boolean.class),
Arrays.asList(lockName), uuidValue, String.valueOf(expireTime))) {
TimeUnit.MILLISECONDS.sleep(20);
}
return true;
}
return false;
}
@Override
public void unlock() {
String script = "if redis.call('hexists', KEYS[1], ARGV[1]) == 0 then " +
"return nil " +
"elseif redis.call('hincrby', KEYS[1], ARGV[1], -1) == 0 then " +
"return redis.call('del', KEYS[1]) " +
"else return 0 " +
"end";
Long flag = redisTemplate.execute(new DefaultRedisScript<>(script, Long.class),
Arrays.asList(lockName), uuidValue, String.valueOf(expireTime));
if (null == flag) {
throw new RuntimeException("This lock doesn't exist!");
}
}
// Other Lock interface methods...
}Version 7: Automatic Renewal
To handle long-running operations, we add automatic renewal functionality:
private void renewExpire() {
String script = "if redis.call('HEXISTS', KEYS[1], ARGV[1]) == 1 then " +
"return redis.call('expire', KEYS[1], ARGV[2]) " +
"else " +
"return 0 " +
"end";
new Timer().schedule(new TimerTask() {
@Override
public void run() {
if (redisTemplate.execute(new DefaultRedisScript<>(script, Boolean.class),
Arrays.asList(lockName), uuidValue, String.valueOf(expireTime))) {
renewExpire();
}
}
}, (this.expireTime * 1000) / 3);
}Redisson Implementation
For production use, consider using Redisson, a mature Redis client with built-in distributed lock support:
@Configuration
public class RedisConfig {
@Bean
public RedissonClient redisson() {
Config config = new Config();
config.useSingleServer()
.setAddress("redis://127.0.0.1:6379")
.setDatabase(0);
return Redisson.create(config);
}
}@Autowired
private RedissonClient redissonClient;
@RequestMapping("/sale")
public String sale() {
String retMessage = "";
RLock lock = redissonClient.getLock("RedisLock");
lock.lock();
try {
String result = redisTemplate.opsForValue().get("inventory001").toString();
Integer inventoryNum = result == null ? 0 : Integer.parseInt(result);
if (inventoryNum != 0) {
redisTemplate.opsForValue().set("inventory001", String.valueOf(--inventoryNum));
retMessage = "Successfully sold item, remaining inventory: " + inventoryNum;
System.out.println(inventoryNum);
} else {
retMessage = "Item sold out!";
}
} finally {
if (lock.isLocked() && lock.isHeldByCurrentThread()) {
lock.unlock();
}
}
return retMessage;
}Redisson provides robust distributed lock implementations with features like fair locking, lock watching, and automatic renewal, making it suitable for production environments.