Performance and Stress Testing

Stress testing evaluates the maximum load a system can handle under current hardware and software conditions, helping identify bottlenecks. The goal is to maintain system processing power and stability within acceptable parameters.

Effective stress testing helps uncover errors difficult to find with other testing methods, particularly memory leaks and concurrency/synchronization issues. A robust stress testing system applies these key conditions: repetition, concurrency, scale, and random variations.

Performance Metrics

• Response Time (RT): The time from when a user initiates a request to when they receive the server's response.
• HPS (Hits Per Second): Number of clicks per second.
• TPS (Transactions Per Second): Number of business transactions processed per second.
• QPS (Queries Per Second): Number of queries processed per second.

For internet businesses where each business request has exactly one connection, TPS = QPS = HPS. TPS typically measures entire business processes, QPS measures interface queries, and HPS represents server click requests.

Industry benchmarks for TPS:

• Financial industry: 1,000-50,000 TPS (excluding internet-enabled activities)
• Insurance industry: 100-100,000 TPS (excluding internet-enabled activities)
• Manufacturing industry: 10-5,000 TPS
• Internet e-commerce: 10,000-1,000,000 TPS
• Medium-sized internet websites: 1,000-50,000 TPS
• Small internet websites: 500-10,000 TPS

Other important metrics include maximum response time, minimum response time, 90% response time, throughput, and error rate.

JMeter Installation and Usage

JMeter can be downloaded from the official website. After extracting the package, run jmeter.bat to start the application.

JMeter Stress Testing Example

Thread Group Configuration

• Number of Threads: Virtual users (each occupies a process or thread).
• Ramp-Up Period (seconds): Time to start all virtual users. For 10 threads with a 2-second ramp-up, 5 threads start per second.
• Loop Count: Number of requests each thread sends. For 10 threads with 100 loops, total requests = 10 * 100 = 1,000.
• Scheduler: Set start and end times for the test.

HTTP Request Configuration

Add HTTP requests to simulate user actions. Configure request parameters, headers, and body as needed.

Listeners

Listeners capture and display test results. Common listeners include View Results Tree, Summary Report, and Aggregate Report.

Test Execution and Analysis

Run the test and analyze results focusing on:

• Error rates and their acceptability
• Throughput compared to concurrency levels
• Server resource utilization (CPU, memory)
• Maximum TPS and maximum concurrency
• Bottlenecks in databases, applications, middleware, networks, or operating systems
• Whether the application is CPU-intensive or I/O-intensive

Windows JMeter Address Already in Use Bug Fix

The error occurs because Windows provides limited ports (1024-5000) with a four-minute recycling time. To fix:

1. Open Registry Editor with regedit
2. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
3. Add a new DWORD named MaxUserPort with value 65534 (decimal)
4. Restart the machine

Performance Monitoring

JVM Memory Model

• Program Counter Register: Records the address of currently executing JVM bytecode instructions. The only JVM memory area that doesn't throw OutOfMemoryError.
• Virtual Machine Stack: Models Java method execution. Each method creates a stack frame storing local variables, operand stack, dynamic linking, and method exits. StackOverflowError occurs when thread request depth exceeds limit; OutOfMemoryError occurs when stack expansion capacity is insufficient.
• Native Method Stack: Similar to VM stack but for native methods.
• Heap: Where all object instances and arrays are allocated. Managed primarily by garbage collectors.

Heap Structure

The heap can be divided into:

• New Generation: Eden space, From Survivor space, To Survivor space
• Old Generation
• Permanent Generation/Metaspace: Java 8 and earlier used Permanent Generation; Java 8+ uses Metaspace, which directly uses physical memory.

Monitoring Tools

JDK provides jconsole and jvisualvm for monitoring local and remote applications. jvisualvm offers plugins for enhanced monitoring.

Key Metrics

• Middleware Metrics: Active threads should not exceed maximum; JDBC connections should be within limits; GC frequency should be minimal.
• Database Metrics: SQL execution time should be minimal; cache hit rate should exceed 95%; lock waits should be minimal.

Static and Dynamic Resource Separation with Nginx

Static and dynamic resource separation improves performance by offloading static resource requests to Nginx instead of Tomcat. Implementation steps:

1. Move static resources to Nginx's HTML directory
2. Update resource paths in HTML files with /static/ prefix
3. Configure Nginx to handle static resources directly
4. Restart Nginx

Benefits include reduced Tomcat resource consumption and improved throughput.

Caching Strategies

Cache Usage

Caching accelerates access to frequently accessed data while databases handle data persistence. Suitable data includes:

• Data with low immediacy and consistency requirements
• High-read, low-write data (read-heavy, write-light)

Basic cache pattern:

data = cache.load(id);
if (data == null) {
    data = db.load(id);
    cache.put(id, data);
}
return data;

All cached data should have expiration times to prevent permanent inconsistency issues.

Local Cache Limitations in Distributed Systems

Local caches in distributed systems face two main issues:

1. Cache Inconsistency: Each service has its own cache, leading to multiple database queries for the same data.
2. Data Inconsistency: When data is modified, only the service handling the request updates its cache, causing inconsistencies across services.

Distributed Cache Solution

Centralized cache middleware (typically Redis) solves these issues by providing a shared cache across all services. Benefits include:

• Eliminates local cache capacity limitations
• Provides simple usage and maintenance
• Enables high availability and performance through clustering

Integrating Redis as Cache

Add Redis dependency to your project:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>

Configure Redis in application.yml:

spring:
  redis:
    host: 192.168.2.190
    port: 6379

Use RedisTemplate for operations:

@Autowired
StringRedisTemplate stringRedisTemplate;

@Test
public void testStringRedisTemplate() {
    ValueOperations<String, String> ops = stringRedisTemplate.opsForValue();
    ops.set("hello", "world_" + UUID.randomUUID().toString());
    String hello = ops.get("hello");
    System.out.println(hello);
}

Cache Problems and Solutions

Cache Penetration

Occurs when requests are made for non-existent data, causing unnecessary database queries. Solutions:

• Cache null values
• Use Bloom filters
• Implement MVC interceptors

Cache Avalanche

Happens when many cache keys expire simultaneously, overwhelming the database. Solutions:

• Set random expiration times
• Distribute hot data across multiple cache instances
• Set hot data to never expire
• Implement circuit breakers and degradation

Cache Breakdown

Occurs when high concurrency accesses a key that has just expired. Solutions:

• Set hot data to never expire
• Implement mutex locks

Distributed Locking

Basic Principles

Distributed locks ensure only one request can access a critical resource at a time. Redis provides atomic operations for implementing locks:

SET key value [EX seconds] [PX milliseconds] [NX|XX]

Evolution of Distributed Locks

Phase 1: Basic Redis Lock

Boolean lock = stringRedisTemplate.opsForValue().setIfAbsent("lock", "1111");
if (lock) {
    // Execute business logic
    stringRedisTemplate.delete("lock");
} else {
    // Retry logic
}

Phase 2: Adding Expiration Time

Boolean lock = stringRedisTemplate.opsForValue().setIfAbsent("lock", "1111");
if (lock) {
    stringRedisTemplate.expire("lock", 300, TimeUnit.MILLISECONDS);
    // Execute business logic
    stringRedisTemplate.delete("lock");
} else {
    // Retry logic
}

Phase 3: Atomic Set with Expiration

Boolean lock = stringRedisTemplate.opsForValue().setIfAbsent("lock", "1111", 300, TimeUnit.MILLISECONDS);
if (lock) {
    // Execute business logic
    stringRedisTemplate.delete("lock");
} else {
    // Retry logic
}

Phase 4: Preventing Accidental Deletion

String token = UUID.randomUUID().toString();
Boolean lock = stringRedisTemplate.opsForValue().setIfAbsent("lock", token, 300, TimeUnit.MILLISECONDS);
if (lock) {
    // Execute business logic
    if (stringRedisTemplate.opsForValue().get("lock").equals(token)) {
        stringRedisTemplate.delete("lock");
    }
} else {
    // Retry logic
}

Phase 5: Atomic Deletion with Lua Script

String script = "if redis.call('get', KEYS[1]) == ARGV[1] then return redis.call('del', KEYS[1]) else return 0 end";
Long result = stringRedisTemplate.execute(
    new DefaultRedisScript<Long>(script, Long.class),
    Arrays.asList("lock"), token);

Redisson Implementation

Redisson provides a distributed Java service framework with enhanced locking capabilities:

Basic Usage

<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson</artifactId>
    <version>3.12.0</version>
</dependency>

Configuration

@Configuration
public class RedissonConfig {
    @Bean(destroyMethod = "shutdown")
    RedissonClient redisson() {
        Config config = new Config();
        config.useSingleServer().setAddress("redis://192.168.2.190:6379");
        return Redisson.create(config);
    }
}

Reentrant Lock

RLock lock = redissonClient.getLock("myLock");
lock.lock();
try {
    // Business logic
} finally {
    lock.unlock();
}

ReadWrite Lock

<n>RReadWriteLock rwLock = redissonClient.getReadWriteLock("myRWLock");
// Read lock (shared)
rwLock.readLock().lock();
// Write lock (exclusive)
rwLock.writeLock().lock();

Cache Consistency Solutions

Common approaches for maintaining cache and database consistency:

• Double Write Pattern: Write to database first, then update cache.
• Invalidation Pattern: Write to database first, then delete cache.
• Read-Write Locks: Use distributed read-write locks for concurrent access.
• Canal Integration: Use Canal to listen to database binlogs and update cache accordingly.

Spring Cache Integration

Introduction

Spring Cache provides a unified caching abstraction across different caching technologies. Key annotations include:

• @Cacheable: Method result is cached
• @CacheEvict: Remove data from cache
• @CachePut: Update cache with method result
• @Caching: Combine multiple caching operations
• @CacheConfig: Shared cache configuration at class level

Configuration

Enable caching and configure Redis as the cache provider:

@SpringBootApplication
@EnableCaching
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

Configure Redis cache settings:

spring:
  cache:
    type: redis
    redis:
      time-to-live: 3600000
      cache-null-values: true

Custom Cache Configuration

For custom serialization and cache behavior:

@Configuration
@EnableConfigurationProperties(CacheProperties.class)
public class MyCacheConfig {
    
    @Bean
    RedisCacheConfiguration redisCacheConfiguration(CacheProperties cacheProperties) {
        RedisCacheConfiguration config = RedisCacheConfiguration.defaultCacheConfig();
        
        config = config.serializeKeysWith(
            RedisSerializationContext.SerializationPair.fromSerializer(new StringRedisSerializer()));
        config = config.serializeValuesWith(
            RedisSerializationContext.SerializationPair.fromSerializer(new GenericJackson2JsonRedisSerializer()));
        
        CacheProperties.Redis redisProperties = cacheProperties.getRedis();
        if (redisProperties.getTimeToLive() != null) {
            config = config.entryTtl(redisProperties.getTimeToLive());
        }
        if (redisProperties.getKeyPrefix() != null) {
            config = config.prefixKeysWith(redisProperties.getKeyPrefix());
        }
        if (!redisProperties.isCacheNullValues()) {
            config = config.disableCachingNullValues();
        }
        
        return config;
    }
}

Usage Examples

Caching Method Results

@Cacheable(value = {"products"}, key = "#productId")
public Product getProductById(Long productId) {
    // Business logic to retrieve product
    return productRepository.findById(productId);
}

Evicting Cache Data

@CacheEvict(value = {"products"}, key = "#productId")
public void updateProduct(Product product) {
    // Business logic to update product
    productRepository.save(product);
}

Updating Cache

@CachePut(value = {"products"}, key = "#product.id")
public Product updateProduct(Product product) {
    // Business logic to update product
    return productRepository.save(product);
}

Spring Cache Limitations

While Spring Cache simplifies caching implementation, it has limitations:

• Read Mode: No built-in protection against cache penetration, avalanche, or breakdown
• Write Mode: No guaranteed consistency between cache and database
• For critical data requiring strong consistency, direct database access is recommended

Spring Cache is best suited for read-heavy scenarios with data that doesn't require strong consistency.

Asynchronous Processing with Thread Pools

Thread Pool Fundamentals

Thread pools improve performance by reusing threads instead of creating new ones for each task. Key parameters include:

• Core pool size: Minimum number of threads
• Maximum pool size: Maximum number of threads
• Keep-alive time: Time before excess threads are terminated
• Work queue: Holds tasks before thread creation
• Thread factory: Creates new threads
• Handler: Rejection policy when queue is full

Common Thread Pool Types

• Fixed Thread Pool: Fixed number of threads
• Cached Thread Pool: Dynamically adjusts thread count
• Scheduled Thread Pool: Supports scheduling
• Single Thread Executor: Single thread for sequential execution

CompletableFuture for Asynchronous Programming

CompletableFuture provides powerful asynchronous programming capabilities:

Creating Async Operations

// Without return value
CompletableFuture.runAsync(() -> {
    // Async task without result
}, executor);

// With return value
CompletableFuture<String> future = CompletableFuture.supplyAsync(() -> {
    // Async task with result
    return "Result";
}, executor);

Chaining Operations

CompletableFuture.supplyAsync(() -> "First")
    .thenApply(result -> result + " Second")
    .thenAccept(result -> System.out.println(result))
    .exceptionally(ex -> {
        System.err.println("Error: " + ex.getMessage());
        return null;
    });

Combining Multiple Futures

// All must complete
CompletableFuture.allOf(future1, future2, future3)
    .thenRun(() -> System.out.println("All completed"));

// Any completes
CompletableFuture.anyOf(future1, future2, future3)
    .thenAccept(result -> System.out.println("Any completed: " + result));

Thread pools and CompletableFuture enable efficient asynchronous processing in e-commerce systems, improving responsiveness and throughput.