Class SnappyOutputStream

java.lang.Object
java.io.OutputStream
org.xerial.snappy.SnappyOutputStream
All Implemented Interfaces:
Closeable, Flushable, AutoCloseable
Direct Known Subclasses:
SnappyHadoopCompatibleOutputStream
public class SnappyOutputStream extends OutputStream
This class implements a stream filter for writing compressed data using Snappy.

The input data is blocked into 32kb size (in default), and each block is compressed and then passed to the given OutputStream.

The output data format is:
  1. snappy codec header defined in SnappyCodec (8 bytes)
  2. codec version (4 bytes integer, at least 1) and compatible version (4 bytes integer)
  3. compressed block 1 : a pair of (compressed data size [4 byte integer. Big-endian], compressed data...)
  4. compressed block 2
  5. ...

Note that the compressed data created by SnappyOutputStream cannot be uncompressed by Snappy.uncompress(byte[]) since the output formats of Snappy.compress(byte[]) and SnappyOutputStream are different. Use SnappyInputStream for uncompress the data generated by SnappyOutputStream.

Author:
leo

  • Field Details

    • MAX_BLOCK_SIZE

      public static final int MAX_BLOCK_SIZE
      See Also:

    • out

      protected final OutputStream out

    • inputBuffer

      protected byte[] inputBuffer

    • outputBuffer

      protected byte[] outputBuffer

  • Constructor Details

    • SnappyOutputStream

      public SnappyOutputStream(OutputStream out)

    • SnappyOutputStream

      public SnappyOutputStream(OutputStream out, int blockSize)
      Parameters:
      out -
      blockSize - byte size of the internal buffer size
      Throws:
      IllegalArgumentException - when blockSize is larger than 512 MiB

    • SnappyOutputStream

      public SnappyOutputStream(OutputStream out, int blockSize, BufferAllocatorFactory bufferAllocatorFactory)

  • Method Details

    • write

      public void write(byte[] b, int byteOffset, int byteLength) throws IOException
      Overrides:
      write in class OutputStream
      Throws:
      IOException

    • write

      public void write(long[] d, int off, int len) throws IOException
      Compress the input long array data
      Parameters:
      d - input array
      off - offset in the array
      len - the number of elements in the array to copy
      Throws:
      IOException

    • write

      public void write(double[] f, int off, int len) throws IOException
      Compress the input double array data
      Parameters:
      f - input array
      off - offset in the array
      len - the number of elements in the array to copy
      Throws:
      IOException

    • write

      public void write(float[] f, int off, int len) throws IOException
      Compress the input float array data
      Parameters:
      f - input array
      off - offset in the array
      len - the number of elements in the array to copy
      Throws:
      IOException

    • write

      public void write(int[] f, int off, int len) throws IOException
      Compress the input int array data
      Parameters:
      f - input array
      off - offset in the array
      len - the number of elements in the array to copy
      Throws:
      IOException

    • write

      public void write(short[] f, int off, int len) throws IOException
      Compress the input short array data
      Parameters:
      f - input array
      off - offset in the array
      len - the number of elements in the array to copy
      Throws:
      IOException

    • write

      public void write(long[] d) throws IOException
      Compress the input array data
      Parameters:
      d -
      Throws:
      IOException

    • write

      public void write(double[] f) throws IOException
      Compress the input array data
      Parameters:
      f -
      Throws:
      IOException

    • write

      public void write(float[] f) throws IOException
      Compress the input array data
      Parameters:
      f -
      Throws:
      IOException

    • write

      public void write(int[] f) throws IOException
      Compress the input array data
      Parameters:
      f -
      Throws:
      IOException

    • write

      public void write(short[] f) throws IOException
      Compress the input array data
      Parameters:
      f -
      Throws:
      IOException

    • rawWrite

      public void rawWrite(Object array, int byteOffset, int byteLength) throws IOException
      Compress the raw byte array data.
      Parameters:
      array - array data of any type (e.g., byte[], float[], long[], ...)
      byteOffset -
      byteLength -
      Throws:
      IOException

    • write

      public void write(int b) throws IOException
      Writes the specified byte to this output stream. The general contract for write is that one byte is written to the output stream. The byte to be written is the eight low-order bits of the argument b. The 24 high-order bits of b are ignored.
      Specified by:
      write in class OutputStream
      Throws:
      IOException

    • flush

      public void flush() throws IOException
      Specified by:
      flush in interface Flushable
      Overrides:
      flush in class OutputStream
      Throws:
      IOException

    • dumpOutput

      protected void dumpOutput() throws IOException
      Throws:
      IOException

    • compressInput

      protected void compressInput() throws IOException
      Throws:
      IOException

    • writeHeader

      protected int writeHeader()

    • writeBlockPreemble

      protected void writeBlockPreemble()
      Optionally write a preemble before a block. Hadoop requires the actual block data size being written. This base implementation does nothing. Derive implementation can call writeCurrentDataSize().

    • writeCurrentDataSize

      protected void writeCurrentDataSize()

    • close

      public void close() throws IOException
      close the stream
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Overrides:
      close in class OutputStream
      Throws:
      IOException