Package org.h2.mvstore

Class StreamStore

  • public class StreamStore
extends java.lang.Object
    A facility to store streams in a map. Streams are split into blocks, which are stored in a map. Very small streams are inlined in the stream id.

    The key of the map is a long (incremented for each stored block). The default initial value is 0. Before storing blocks into the map, the stream store checks if there is already a block with the next key, and if necessary searches the next free entry using a binary search (0 to Long.MAX_VALUE).

    Before storing

    The format of the binary id is: An empty id represents 0 bytes of data. In-place data is encoded as 0, the size (a variable size int), then the data. A stored block is encoded as 1, the length of the block (a variable size int), then the key (a variable size long). Multiple ids can be concatenated to concatenate the data. If the id is large, it is stored itself, which is encoded as 2, the total length (a variable size long), and the key of the block that contains the id (a variable size long).

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      (package private) static class  StreamStore.Stream
      A stream backed by a map.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      private java.util.Map<java.lang.Long,​byte[]> map  
      private int maxBlockSize  
      private int minBlockSize  
      private java.util.concurrent.atomic.AtomicReference<byte[]> nextBuffer  
      private java.util.concurrent.atomic.AtomicLong nextKey  

    • Constructor Summary

      Constructors 
      Constructor Description
      StreamStore​(java.util.Map<java.lang.Long,​byte[]> map)
      Create a stream store instance.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.io.InputStream get​(byte[] id)
      Open an input stream to read data.
      private long getAndIncrementNextKey()
      Generate a new key.
      (package private) byte[] getBlock​(long key)
      Get the block.
      java.util.Map<java.lang.Long,​byte[]> getMap()  
      long getMaxBlockKey​(byte[] id)
      Get the key of the biggest block, of -1 for inline data.
      long getMaxBlockSize()  
      int getMinBlockSize()  
      long getNextKey()  
      boolean isInPlace​(byte[] id)
      Check whether the id itself contains all the data.
      long length​(byte[] id)
      Calculate the number of data bytes for the given id.
      protected void onStore​(int len)
      This method is called after a block of data is stored.
      private boolean put​(java.io.ByteArrayOutputStream id, java.io.InputStream in, int level)  
      byte[] put​(java.io.InputStream in)
      Store the stream, and return the id.
      private java.io.ByteArrayOutputStream putIndirectId​(java.io.ByteArrayOutputStream id)  
      private static byte[] read​(java.io.InputStream in, byte[] target)  
      void remove​(byte[] id)
      Remove all stored blocks for the given id.
      void setMaxBlockSize​(int maxBlockSize)
      Set the maximum block size.
      void setMinBlockSize​(int minBlockSize)
      Set the minimum block size.
      void setNextKey​(long nextKey)  
      static java.lang.String toString​(byte[] id)
      Convert the id to a human readable string.
      private long writeBlock​(byte[] data)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • map

        private final java.util.Map<java.lang.Long,​byte[]> map

      • minBlockSize

        private int minBlockSize

      • maxBlockSize

        private int maxBlockSize

      • nextKey

        private final java.util.concurrent.atomic.AtomicLong nextKey

      • nextBuffer

        private final java.util.concurrent.atomic.AtomicReference<byte[]> nextBuffer

    • Constructor Detail

      • StreamStore

        public StreamStore​(java.util.Map<java.lang.Long,​byte[]> map)
        Create a stream store instance.
        Parameters:
        map - the map to store blocks of data

    • Method Detail

      • getMap

        public java.util.Map<java.lang.Long,​byte[]> getMap()

      • setNextKey

        public void setNextKey​(long nextKey)

      • getNextKey

        public long getNextKey()

      • setMinBlockSize

        public void setMinBlockSize​(int minBlockSize)
        Set the minimum block size. The default is 256 bytes.
        Parameters:
        minBlockSize - the new value

      • getMinBlockSize

        public int getMinBlockSize()

      • setMaxBlockSize

        public void setMaxBlockSize​(int maxBlockSize)
        Set the maximum block size. The default is 256 KB.
        Parameters:
        maxBlockSize - the new value

      • getMaxBlockSize

        public long getMaxBlockSize()

      • put

        public byte[] put​(java.io.InputStream in)
           throws java.io.IOException
        Store the stream, and return the id. The stream is not closed.
        Parameters:
        in - the stream
        Returns:
        the id (potentially an empty array)
        Throws:
        java.io.IOException - If an I/O error occurs

      • put

        private boolean put​(java.io.ByteArrayOutputStream id,
                    java.io.InputStream in,
                    int level)
             throws java.io.IOException
        Throws:
        java.io.IOException

      • read

        private static byte[] read​(java.io.InputStream in,
                           byte[] target)
                    throws java.io.IOException
        Throws:
        java.io.IOException

      • putIndirectId

        private java.io.ByteArrayOutputStream putIndirectId​(java.io.ByteArrayOutputStream id)
                                             throws java.io.IOException
        Throws:
        java.io.IOException

      • writeBlock

        private long writeBlock​(byte[] data)

      • onStore

        protected void onStore​(int len)
        This method is called after a block of data is stored. Override this method to persist data if necessary.
        Parameters:
        len - the length of the stored block.

      • getAndIncrementNextKey

        private long getAndIncrementNextKey()
        Generate a new key.
        Returns:
        the new key

      • getMaxBlockKey

        public long getMaxBlockKey​(byte[] id)
        Get the key of the biggest block, of -1 for inline data. This method is used to garbage collect orphaned blocks.
        Parameters:
        id - the id
        Returns:
        the key, or -1

      • remove

        public void remove​(byte[] id)
        Remove all stored blocks for the given id.
        Parameters:
        id - the id

      • toString

        public static java.lang.String toString​(byte[] id)
        Convert the id to a human readable string.
        Parameters:
        id - the stream id
        Returns:
        the string

      • length

        public long length​(byte[] id)
        Calculate the number of data bytes for the given id. As the length is encoded in the id, this operation does not cause any reads in the map.
        Parameters:
        id - the id
        Returns:
        the length

      • isInPlace

        public boolean isInPlace​(byte[] id)
        Check whether the id itself contains all the data. This operation does not cause any reads in the map.
        Parameters:
        id - the id
        Returns:
        if the id contains the data

      • get

        public java.io.InputStream get​(byte[] id)
        Open an input stream to read data.
        Parameters:
        id - the id
        Returns:
        the stream

      • getBlock

        byte[] getBlock​(long key)
        Get the block.
        Parameters:
        key - the key
        Returns:
        the block