com.hadoop.compression.lzo

Class LzopDecompressor

java.lang.Object

com.hadoop.compression.lzo.LzopDecompressor

All Implemented Interfaces:

org.apache.hadoop.io.compress.Decompressor

public class LzopDecompressor
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LzoDecompressor.CompressionStrategy

Field Summary

Fields

Modifier and Type	Field and Description
static int	LZO_LIBRARY_VERSION
static int	MINIMUM_LZO_VERSION The minimum version of LZO that we can read.

Constructor Summary

Constructors

Constructor and Description
LzopDecompressor(int bufferSize) Create an LzoDecompressor with LZO1X strategy (the only lzo algorithm supported by lzop).

Method Summary

Modifier and Type	Method and Description
int	decompress(byte[] b, int off, int len)
void	end()
protected void	finalize()
boolean	finished()
int	getChecksumsCount() Get the number of checksum implementations the current lzo file uses.
int	getCompressedChecksumsCount() Get the number of compressed checksum implementations the current lzo file uses.
int	getDecompressedChecksumsCount() Get the number of decompressed checksum implementations the current lzo file uses.
int	getRemaining()
void	initHeaderFlags(EnumSet<DChecksum> dflags, EnumSet<CChecksum> cflags) Given a set of decompressed and compressed checksums,
protected boolean	isCurrentBlockUncompressed() Query the compression status of the current block as it exists in the file.
static boolean	isNativeLzoLoaded() Check if lzo decompressors are loaded and initialized.
boolean	needsDictionary()
boolean	needsInput()
void	reset()
void	resetChecksum() Reset all checksums registered for this decompressor instance.
void	setCurrentBlockUncompressed(boolean uncompressed) Note whether the current block being decompressed is actually stored as uncompressed data.
void	setDictionary(byte[] b, int off, int len)
void	setInput(byte[] b, int off, int len)
boolean	verifyCChecksum(CChecksum typ, int checksum) Given a checksum type, verity its value against that observed in compressed data.
boolean	verifyDChecksum(DChecksum typ, int checksum) Given a checksum type, verify its value against that observed in decompressed data.

Methods inherited from class java.lang.Object

clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MINIMUM_LZO_VERSION

public static int MINIMUM_LZO_VERSION

The minimum version of LZO that we can read. Set to 1.0 since there were a couple header size changes prior to that. See read_header() in lzop.c

LZO_LIBRARY_VERSION

public static final int LZO_LIBRARY_VERSION

Constructor Detail

LzopDecompressor

public LzopDecompressor(int bufferSize)

Create an LzoDecompressor with LZO1X strategy (the only lzo algorithm supported by lzop).

Parameters:

bufferSize - The buffer size

Method Detail

initHeaderFlags

public void initHeaderFlags(EnumSet<DChecksum> dflags,
                            EnumSet<CChecksum> cflags)

Given a set of decompressed and compressed checksums,

Parameters:

dflags - The decompressed flags

cflags - The compressed flags

getChecksumsCount

public int getChecksumsCount()

Get the number of checksum implementations the current lzo file uses.

Returns:

Number of checksum implementations in use.

getCompressedChecksumsCount

public int getCompressedChecksumsCount()

Get the number of compressed checksum implementations the current lzo file uses.

Returns:

Number of compressed checksum implementations in use.

getDecompressedChecksumsCount

public int getDecompressedChecksumsCount()

Get the number of decompressed checksum implementations the current lzo file uses.

Returns:

Number of decompressed checksum implementations in use.

resetChecksum

public void resetChecksum()

Reset all checksums registered for this decompressor instance.

verifyDChecksum

public boolean verifyDChecksum(DChecksum typ,
                               int checksum)

Given a checksum type, verify its value against that observed in decompressed data.

Parameters:

typ - The checksum type

checksum - The checksum of decompressed data

Returns:

Whether the checksum is correct

verifyCChecksum

public boolean verifyCChecksum(CChecksum typ,
                               int checksum)

Given a checksum type, verity its value against that observed in compressed data.

Parameters:

typ - The checksum type

checksum - The checksum of compressed data

Returns:

Whether the checksum is correct

setInput

public void setInput(byte[] b,
                     int off,
                     int len)

Specified by:

setInput in interface org.apache.hadoop.io.compress.Decompressor

decompress

public int decompress(byte[] b,
                      int off,
                      int len)
               throws IOException

Specified by:

decompress in interface org.apache.hadoop.io.compress.Decompressor

Throws:

IOException

isNativeLzoLoaded

public static boolean isNativeLzoLoaded()

Check if lzo decompressors are loaded and initialized.

Returns:

true if lzo decompressors are loaded and initialized, else false

setDictionary

public void setDictionary(byte[] b,
                          int off,
                          int len)

Specified by:

setDictionary in interface org.apache.hadoop.io.compress.Decompressor

needsInput

public boolean needsInput()

Specified by:

needsInput in interface org.apache.hadoop.io.compress.Decompressor

needsDictionary

public boolean needsDictionary()

Specified by:

needsDictionary in interface org.apache.hadoop.io.compress.Decompressor

finished

public boolean finished()

Specified by:

finished in interface org.apache.hadoop.io.compress.Decompressor

reset

public void reset()

Specified by:

reset in interface org.apache.hadoop.io.compress.Decompressor

end

public void end()

Specified by:

end in interface org.apache.hadoop.io.compress.Decompressor

getRemaining

public int getRemaining()

Specified by:

getRemaining in interface org.apache.hadoop.io.compress.Decompressor

finalize

protected void finalize()

Overrides:

finalize in class Object

setCurrentBlockUncompressed

public void setCurrentBlockUncompressed(boolean uncompressed)

Note whether the current block being decompressed is actually stored as uncompressed data. If it is, there is no need to use the lzo decompressor, and no need to update compressed checksums.

Parameters:

uncompressed - Whether the current block of data is uncompressed already.

isCurrentBlockUncompressed

protected boolean isCurrentBlockUncompressed()

Query the compression status of the current block as it exists in the file.

Returns:

true if the current block of data was stored as uncompressed.