Troubleshooting HBase Snapshot Reads with LZO Compression

Issue 1: UnsatisfiedLinkError to gplcompression

When attempting to read HBase snapshot data, the following error occurs:

java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
    at com.hadoop.compression.lzo.GPLNativeCodeLoader.<clinit>(GPLNativeCodeLoader.java:31)
    at com.hadoop.compression.lzo.LzoCodec.<clinit>(LzoCodec.java:60)
    ...
ERROR LzoCodec:70 | Cannot load native-lzo without native-hadoop
</clinit></clinit>

The stack trace reveals that the JVM cannot locate the native GPL compression library. To resolve this, specify the native library path when launching the Java application:

java -Djava.library.path=/opt/hadoop/lib/native -cp lib/*:conf/*:application.jar com.example.HbaseReader

Issue 2: native-lzo library not available

After resolving the first error, a new exception appears:

org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.RuntimeException: native-lzo library not available
    at org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:102)
    ...
Caused by: java.lang.RuntimeException: native-lzo library not available
    at com.hadoop.compression.lzo.LzoCodec.getCompressorType(LzoCodec.java:135)

This error originates during compression codec verification in HRegion.checkCompressionCodecs().

Root Cause Analysis

After extensive investigation, the root cause was identified: the project dependencies include hadoop-gpl-compression-*.jar, which is an outdated library no longer maintained. This legacy dependency conflicts with proper LZO compression support.

Solution

The solution involvse removing the deprecated dependency and using the actively maintained hadoop-lzo library instead. Twitter maintains the recommended version on GitHub:

git clone https://github.com/twitter/hadoop-lzo.git

Build the project using Maven and add the resulting hadoop-lzo-*.jar to your classpath. This resolves the "native-lzo library not available" error and allows successful snapshot reads.

For compilation instructions with specific Hadoop versions, refer to the official hadoop-lzo documentation.

Tags: HBase Snapshot LZO compression Hadoop

Posted on Sun, 14 Jun 2026 16:47:10 +0000 by zMastaa