Simulating and Analyzing Java Metaspace OutOfMemoryError

Metaspace Overview

The Metaspace replaces the legacy Permanent Generation (PermGen) in HotSpot JVMs as the storage location for class metadata. Unlike the Java heap, this area allocates memory from native address space. It holds critical data such as loaded class structures, method bytecodes, static variables, and runtime constant pools. Since it uses native memory, its size is only limited by the available system memory unless explicitly capped by JVM flags.

JVM Configuration

To reproduce a memory exhaustion scenario, the Metaspace size must be restricted. The following flags limit the initial and maximum size to 10MB and trigger a heap dump upon failure, facilitating post-mortem analysis.

-XX:MetaspaceSize=10m
-XX:MaxMetaspaceSize=10m
-XX:+HeapDumpOnOutOfMemoryError

Code Implementation

The following Java application demonstrates how continuous dynamic class generation can exhaust the Metaspace. By leveraging a library like CGLIB to create proxies and explicitly disabling the class cache, each iteration generates a new class definition, consuming metadata until the limit is reached.

import net.sf.cglib.proxy.Enhancer;
import net.sf.cglib.proxy.MethodInterceptor;
import java.lang.reflect.Method;

public class MetaspaceMemoryTest {

    public static void main(String[] args) {
        while (true) {
            Enhancer proxyGenerator = new Enhancer();
            proxyGenerator.setSuperclass(DataEntity.class);
            proxyGenerator.setUseCache(false);
            proxyGenerator.setCallback(new MethodInterceptor() {
                @Override
                public Object intercept(Object obj, Method method, Object[] params, net.sf.cglib.proxy.MethodProxy proxy) throws Throwable {
                    return proxy.invokeSuper(obj, params);
                }
            });
            proxyGenerator.create();
        }
    }

    static class DataEntity {
        // Base class for proxy generation
    }
}

Runtime Exception

When the Metaspace reaches its maximum capacity, the JVM triggers a garbage collection specifically for metadata. If space cannot be reclaimed, the application terminates with an OutOfMemoryError indicating Metaspace exhaustion. A heap dump file is also generated based on the configuration.

java.lang.OutOfMemoryError: Metaspace
	dumping heap to java_pid1234.hprof ...
	Heap dump file created [1234567 bytes in 0.023 secs]
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
	at net.sf.cglib.core.AbstractClassGenerator.generate(AbstractClassGenerator.java:345)
	at net.sf.cglib.proxy.Enhancer.generate(Enhancer.java:582)
	at MetaspaceMemoryTest.main(MetaspaceMemoryTest.java:14)

Garbage Collection Log

The garbage collection logs illustrate the JVM's attempts to manage the limited space. Initially, young generation GCs occur, followed by Full GCs triggered by the Metadata GC Threshold. The log shows the Metaspace usage remaining constant and full despite reclamation efforts before the crash.

[GC (Allocation Failure) [PSYoungGen: 8192K->512K(9216K)] 8192K->512K(19456K), 0.0041250 secs] 
[Full GC (Metadata GC Threshold) [PSYoungGen: 512K->0K(9216K)] [ParOldGen: 0K->1024K(10240K)] 512K->1024K(19456K), [Metaspace: 8192K->8192K(8192K)], 0.0123456 secs] 
[Full GC (Last ditch collection) [PSYoungGen: 0K->0K(9216K)] [ParOldGen: 1024K->1024K(10240K)] 1024K->1024K(19456K), [Metaspace: 8192K->8192K(8192K)], 0.0234567 secs] 
java.lang.OutOfMemoryError: Metaspace

Tags: java JVM Metaspace OutOfMemoryError Performance Tuning

Posted on Sun, 16 Aug 2026 16:33:32 +0000 by codygoodman