org.msgpack.core

Class MessageUnpacker

java.lang.Object

org.msgpack.core.MessageUnpacker

All Implemented Interfaces:

java.io.Closeable, java.lang.AutoCloseable

public class MessageUnpacker
extends java.lang.Object
implements java.io.Closeable

MessagePack deserializer that converts binary into objects. You can use factory methods of MessagePack class or MessagePack.UnpackerConfig class to create an instance. To read values as statically-typed Java objects, there are two typical use cases.

One use case is to read objects as Value using unpackValue() method. A Value object contains type of the deserialized value as well as the value itself so that you can inspect type of the deserialized values later. You can repeat unpackValue() until hasNext() method returns false so that you can deserialize sequence of MessagePack values.

The other use case is to use getNextFormat() and MessageFormat.getValueType() methods followed by unpackXxx methods corresponding to returned type. Following code snipet is a typical application code:

     MessageUnpacker unpacker = MessagePack.newDefaultUnpacker(...);
     while(unpacker.hasNext()) {
         MessageFormat format = unpacker.getNextFormat();
         ValueType type = format.getValueType();
         int length;
         ExtensionTypeHeader extension;
         switch(type) {
             case NIL:
                 unpacker.unpackNil();
                 break;
             case BOOLEAN:
                 unpacker.unpackBoolean();
                 break;
             case INTEGER:
                 switch (format) {
                 case UINT64:
                     unpacker.unpackBigInteger();
                     break;
                 case INT64:
                 case UINT32:
                     unpacker.unpackLong();
                     break;
                 default:
                     unpacker.unpackInt();
                     break;
                 }
                 break;
             case FLOAT:
                 unpacker.unpackDouble();
                 break;
             case STRING:
                 unpacker.unpackString();
                 break;
             case BINARY:
                 length = unpacker.unpackBinaryHeader();
                 unpacker.readPayload(new byte[length]);
                 break;
             case ARRAY:
                 length = unpacker.unpackArrayHeader();
                 for (int i = 0; i < length; i++) {
                     readRecursively(unpacker);
                 }
                 break;
             case MAP:
                 length = unpacker.unpackMapHeader();
                 for (int i = 0; i < length; i++) {
                     readRecursively(unpacker);  // key
                     readRecursively(unpacker);  // value
                 }
                 break;
             case EXTENSION:
                 extension = unpacker.unpackExtensionTypeHeader();
                 unpacker.readPayload(new byte[extension.getLength()]);
                 break;
             }
         }
     }

 

 Following methods correspond to the MessagePack types:

 

   
MessagePack type
Unpacker method
Java type
   
Nil
unpackNil()
null
   
Boolean
unpackBoolean()
boolean
   
Integer
unpackByte()
byte
   
Integer
unpackShort()
short
   
Integer
unpackInt()
int
   
Integer
unpackLong()
long
   
Integer
unpackBigInteger()
BigInteger
   
Float
unpackFloat()
float
   
Float
unpackDouble()
double
   
Binary
unpackBinaryHeader()
byte array
   
String
unpackRawStringHeader()
String
   
String
unpackString()
String
   
Array
unpackArrayHeader()
Array
   
Map
unpackMapHeader()
Map
   
Extension
unpackExtensionTypeHeader()
ExtensionTypeHeader
 

 

 To read a byte array, first call unpackBinaryHeader() method to get length of the byte array. Then,
 call readPayload(int) or readPayloadAsReference(int) method to read the the contents.

 

 To read an Array type, first call unpackArrayHeader() method to get number of elements. Then,
 call unpacker methods for each element.

 

 To read a Map, first call unpackMapHeader() method to get number of pairs of the map. Then,
 for each pair, call unpacker methods for key first, and then value. will call unpacker methods twice
 as many time as the returned count.

Field Summary

Fields

Modifier and Type	Field and Description
private java.nio.charset.CodingErrorAction	actionOnMalformedString
private java.nio.charset.CodingErrorAction	actionOnUnmappableString
private boolean	allowReadingBinaryAsString
private boolean	allowReadingStringAsBinary
private MessageBuffer	buffer Points to the current buffer to read
private java.nio.CharBuffer	decodeBuffer Buffer for decoding strings
private java.nio.charset.CharsetDecoder	decoder For decoding String in unpackString.
private java.lang.StringBuilder	decodeStringBuffer For decoding String in unpackString.
private static MessageBuffer	EMPTY_BUFFER
private static java.lang.String	EMPTY_STRING
private MessageBufferInput	in
private int	nextReadPosition After calling prepareNumberBuffer(), the caller should use this variable to read from the returned MessageBuffer.
private MessageBuffer	numberBuffer An extra buffer for reading a small number value across the input buffer boundary.
private int	position Cursor position in the current buffer
private int	stringDecoderBufferSize
private int	stringSizeLimit
private long	totalReadBytes Total read byte size

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	MessageUnpacker(MessageBufferInput in, MessagePack.UnpackerConfig config) Create an MessageUnpacker that reads data from the given MessageBufferInput.

Method Summary

Modifier and Type	Method and Description
void	close() Closes underlying input.
private java.lang.String	decodeStringFastPath(int length)
private boolean	ensureBuffer()
private MessageBuffer	getNextBuffer() Get the next buffer without changing the position
MessageFormat	getNextFormat() Returns format of the next value.
long	getTotalReadBytes() Returns total number of read bytes.
private void	handleCoderError(java.nio.charset.CoderResult cr)
boolean	hasNext() Returns true if this unpacker has more elements.
private void	nextBuffer()
private static MessageIntegerOverflowException	overflowI16(short i16)
private static MessageIntegerOverflowException	overflowI32(int i32)
private static MessageIntegerOverflowException	overflowI64(long i64)
private static MessageIntegerOverflowException	overflowU16(short u16)
private static MessageIntegerOverflowException	overflowU32(int u32)
private static MessageSizeException	overflowU32Size(int u32)
private static MessageIntegerOverflowException	overflowU64(long u64)
private static MessageIntegerOverflowException	overflowU8(byte u8)
private MessageBuffer	prepareNumberBuffer(int readLength) Returns a short size buffer (upto 8 bytes) to read a number value
private byte	readByte() Read a byte value at the cursor and proceed the cursor.
private double	readDouble()
private float	readFloat()
private int	readInt()
private long	readLong()
private int	readNextLength16()
private int	readNextLength32()
private int	readNextLength8()
void	readPayload(byte[] dst) Reads payload bytes of binary, extension, or raw string types.
void	readPayload(byte[] dst, int off, int len) Reads payload bytes of binary, extension, or raw string types.
void	readPayload(java.nio.ByteBuffer dst) Reads payload bytes of binary, extension, or raw string types.
byte[]	readPayload(int length) Reads payload bytes of binary, extension, or raw string types.
void	readPayload(MessageBuffer dst, int off, int len) Reads payload bytes of binary, extension, or raw string types.
MessageBuffer	readPayloadAsReference(int length) Reads payload bytes of binary, extension, or raw string types as a reference to internal buffer.
private short	readShort()
MessageBufferInput	reset(MessageBufferInput in) Replaces underlying input.
private void	resetDecoder()
private void	skipPayload(int numBytes) Skip reading the specified number of bytes.
void	skipValue() Skip the next value, then move the cursor at the end of the value
void	skipValue(int count) Skip next values, then move the cursor at the end of the value
private int	tryReadBinaryHeader(byte b)
private int	tryReadStringHeader(byte b)
boolean	tryUnpackNil() Peeks a Nil byte and reads it if next byte is a nil value.
private static MessagePackException	unexpected(java.lang.String expected, byte b) Create an exception for the case when an unexpected byte value is read
private static MessagePackException	unexpectedExtension(java.lang.String expected, int expectedType, int actualType)
int	unpackArrayHeader() Reads header of an array.
java.math.BigInteger	unpackBigInteger() Reads a BigInteger.
int	unpackBinaryHeader() Reads header of a binary.
boolean	unpackBoolean() Reads true or false.
byte	unpackByte() Reads a byte.
double	unpackDouble() Reads a double.
ExtensionTypeHeader	unpackExtensionTypeHeader()
float	unpackFloat() Reads a float.
int	unpackInt() Reads a int.
long	unpackLong() Reads a long.
int	unpackMapHeader() Reads header of a map.
void	unpackNil() Reads a Nil byte.
int	unpackRawStringHeader()
short	unpackShort() Reads a short.
java.lang.String	unpackString()
java.time.Instant	unpackTimestamp()
java.time.Instant	unpackTimestamp(ExtensionTypeHeader ext) Unpack timestamp that can be used after reading the extension type header with unpackExtensionTypeHeader.
ImmutableValue	unpackValue()
Variable	unpackValue(Variable var)
private static int	utf8MultibyteCharacterSize(byte firstByte)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

EMPTY_BUFFER

private static final MessageBuffer EMPTY_BUFFER

allowReadingStringAsBinary

private final boolean allowReadingStringAsBinary

allowReadingBinaryAsString

private final boolean allowReadingBinaryAsString

actionOnMalformedString

private final java.nio.charset.CodingErrorAction actionOnMalformedString

actionOnUnmappableString

private final java.nio.charset.CodingErrorAction actionOnUnmappableString

stringSizeLimit

private final int stringSizeLimit

stringDecoderBufferSize

private final int stringDecoderBufferSize

in

private MessageBufferInput in

buffer

private MessageBuffer buffer

Points to the current buffer to read

position

private int position

Cursor position in the current buffer

totalReadBytes

private long totalReadBytes

Total read byte size

numberBuffer

private final MessageBuffer numberBuffer

An extra buffer for reading a small number value across the input buffer boundary. At most 8-byte buffer (for readLong used by uint 64 and UTF-8 character decoding) is required.

nextReadPosition

private int nextReadPosition

After calling prepareNumberBuffer(), the caller should use this variable to read from the returned MessageBuffer.

decodeStringBuffer

private java.lang.StringBuilder decodeStringBuffer

For decoding String in unpackString.

decoder

private java.nio.charset.CharsetDecoder decoder

For decoding String in unpackString.

decodeBuffer

private java.nio.CharBuffer decodeBuffer

Buffer for decoding strings

EMPTY_STRING

private static final java.lang.String EMPTY_STRING

See Also:

Constant Field Values

Constructor Detail

MessageUnpacker

protected MessageUnpacker(MessageBufferInput in,
                          MessagePack.UnpackerConfig config)

Create an MessageUnpacker that reads data from the given MessageBufferInput. This method is available for subclasses to override. Use MessagePack.UnpackerConfig.newUnpacker method to instantiate this implementation.

Parameters:

in -

Method Detail

reset

public MessageBufferInput reset(MessageBufferInput in)
                         throws java.io.IOException

Replaces underlying input.

This method clears internal buffer, swaps the underlying input with the new given input, then returns the old input.

This method doesn't close the old input.

Parameters:

in - new input

Returns:

the old input

Throws:

java.io.IOException - never happens unless a subclass overrides this method

java.lang.NullPointerException - the given input is null

getTotalReadBytes

public long getTotalReadBytes()

Returns total number of read bytes.

This method returns total of amount of data consumed from the underlying input minus size of data remained still unused in the current internal buffer.

Calling reset(MessageBufferInput) resets this number to 0.

getNextBuffer

private MessageBuffer getNextBuffer()
                             throws java.io.IOException

Get the next buffer without changing the position

Returns:

Throws:

java.io.IOException

nextBuffer

private void nextBuffer()
                 throws java.io.IOException

Throws:

java.io.IOException

prepareNumberBuffer

private MessageBuffer prepareNumberBuffer(int readLength)
                                   throws java.io.IOException

Returns a short size buffer (upto 8 bytes) to read a number value

Parameters:

readLength -

Returns:

Throws:

java.io.IOException

MessageInsufficientBufferException - If no more buffer can be acquired from the input source for reading the specified data length

utf8MultibyteCharacterSize

private static int utf8MultibyteCharacterSize(byte firstByte)

hasNext

public boolean hasNext()
                throws java.io.IOException

Returns true if this unpacker has more elements. When this returns true, subsequent call to getNextFormat() returns an MessageFormat instance. If false, next getNextFormat() call will throw an MessageInsufficientBufferException.

Returns:

true if this unpacker has more elements to read

Throws:

java.io.IOException

ensureBuffer

private boolean ensureBuffer()
                      throws java.io.IOException

Throws:

java.io.IOException

getNextFormat

public MessageFormat getNextFormat()
                            throws java.io.IOException

Returns format of the next value.

Note that this method doesn't consume data from the internal buffer unlike the other unpack methods. Calling this method twice will return the same value.

To not throw MessageInsufficientBufferException, this method should be called only when hasNext() returns true.

Returns:

the next MessageFormat

Throws:

java.io.IOException - when underlying input throws IOException

MessageInsufficientBufferException - when the end of file reached, i.e. hasNext() == false.

readByte

private byte readByte()
               throws java.io.IOException

Read a byte value at the cursor and proceed the cursor.

Returns:

Throws:

java.io.IOException

readShort

private short readShort()
                 throws java.io.IOException

Throws:

java.io.IOException

readInt

private int readInt()
             throws java.io.IOException

Throws:

java.io.IOException

readLong

private long readLong()
               throws java.io.IOException

Throws:

java.io.IOException

readFloat

private float readFloat()
                 throws java.io.IOException

Throws:

java.io.IOException

readDouble

private double readDouble()
                   throws java.io.IOException

Throws:

java.io.IOException

skipValue

public void skipValue()
               throws java.io.IOException

Skip the next value, then move the cursor at the end of the value

Throws:

java.io.IOException

skipValue

public void skipValue(int count)
               throws java.io.IOException

Skip next values, then move the cursor at the end of the value

Parameters:

count - number of values to skip

Throws:

java.io.IOException

unexpected

private static MessagePackException unexpected(java.lang.String expected,
                                               byte b)

Create an exception for the case when an unexpected byte value is read

Parameters:

expected -

b -

Returns:

Throws:

MessageFormatException

unexpectedExtension

private static MessagePackException unexpectedExtension(java.lang.String expected,
                                                        int expectedType,
                                                        int actualType)

unpackValue

public ImmutableValue unpackValue()
                           throws java.io.IOException

Throws:

java.io.IOException

unpackValue

public Variable unpackValue(Variable var)
                     throws java.io.IOException

Throws:

java.io.IOException

unpackNil

public void unpackNil()
               throws java.io.IOException

Reads a Nil byte.

Throws:

MessageTypeException - when value is not MessagePack Nil type

java.io.IOException - when underlying input throws IOException

tryUnpackNil

public boolean tryUnpackNil()
                     throws java.io.IOException

Peeks a Nil byte and reads it if next byte is a nil value. The difference from unpackNil() is that unpackNil throws an exception if the next byte is not nil value while this tryUnpackNil method returns false without changing position.

Returns:

true if a nil value is read

Throws:

MessageInsufficientBufferException - when the end of file reached

java.io.IOException - when underlying input throws IOException

unpackBoolean

public boolean unpackBoolean()
                      throws java.io.IOException

Reads true or false.

Returns:

the read value

Throws:

MessageTypeException - when value is not MessagePack Boolean type

java.io.IOException - when underlying input throws IOException

unpackByte

public byte unpackByte()
                throws java.io.IOException

Reads a byte. This method throws MessageIntegerOverflowException if the value doesn't fit in the range of byte. This may happen when getNextFormat() returns UINT8, INT16, or larger integer formats.

Returns:

the read value

Throws:

MessageIntegerOverflowException - when value doesn't fit in the range of byte

MessageTypeException - when value is not MessagePack Integer type

java.io.IOException - when underlying input throws IOException

unpackShort

public short unpackShort()
                  throws java.io.IOException

Reads a short. This method throws MessageIntegerOverflowException if the value doesn't fit in the range of short. This may happen when getNextFormat() returns UINT16, INT32, or larger integer formats.

Returns:

the read value

Throws:

MessageIntegerOverflowException - when value doesn't fit in the range of short

MessageTypeException - when value is not MessagePack Integer type

java.io.IOException - when underlying input throws IOException

unpackInt

public int unpackInt()
              throws java.io.IOException

Reads a int. This method throws MessageIntegerOverflowException if the value doesn't fit in the range of int. This may happen when getNextFormat() returns UINT32, INT64, or larger integer formats.

Returns:

the read value

Throws:

MessageIntegerOverflowException - when value doesn't fit in the range of int

MessageTypeException - when value is not MessagePack Integer type

java.io.IOException - when underlying input throws IOException

unpackLong

public long unpackLong()
                throws java.io.IOException

Reads a long. This method throws MessageIntegerOverflowException if the value doesn't fit in the range of long. This may happen when getNextFormat() returns UINT64.

Returns:

the read value

Throws:

MessageIntegerOverflowException - when value doesn't fit in the range of long

MessageTypeException - when value is not MessagePack Integer type

java.io.IOException - when underlying input throws IOException

unpackBigInteger

public java.math.BigInteger unpackBigInteger()
                                      throws java.io.IOException

Reads a BigInteger.

Returns:

the read value

Throws:

MessageTypeException - when value is not MessagePack Integer type

java.io.IOException - when underlying input throws IOException

unpackFloat

public float unpackFloat()
                  throws java.io.IOException

Reads a float. This method rounds value to the range of float when precision of the read value is larger than the range of float. This may happen when getNextFormat() returns FLOAT64.

Returns:

the read value

Throws:

MessageTypeException - when value is not MessagePack Float type

java.io.IOException - when underlying input throws IOException

unpackDouble

public double unpackDouble()
                    throws java.io.IOException

Reads a double.

Returns:

the read value

Throws:

MessageTypeException - when value is not MessagePack Float type

java.io.IOException - when underlying input throws IOException

resetDecoder

private void resetDecoder()

unpackString

public java.lang.String unpackString()
                              throws java.io.IOException

Throws:

java.io.IOException

handleCoderError

private void handleCoderError(java.nio.charset.CoderResult cr)
                       throws java.nio.charset.CharacterCodingException

Throws:

java.nio.charset.CharacterCodingException

decodeStringFastPath

private java.lang.String decodeStringFastPath(int length)

unpackTimestamp

public java.time.Instant unpackTimestamp()
                                  throws java.io.IOException

Throws:

java.io.IOException

unpackTimestamp

public java.time.Instant unpackTimestamp(ExtensionTypeHeader ext)
                                  throws java.io.IOException

Unpack timestamp that can be used after reading the extension type header with unpackExtensionTypeHeader.

Throws:

java.io.IOException

unpackArrayHeader

public int unpackArrayHeader()
                      throws java.io.IOException

Reads header of an array.

This method returns number of elements to be read. After this method call, you call unpacker methods for each element. You don't have to call anything at the end of iteration.

Returns:

the size of the array to be read

Throws:

MessageTypeException - when value is not MessagePack Array type

MessageSizeException - when size of the array is larger than 2^31 - 1

java.io.IOException - when underlying input throws IOException

unpackMapHeader

public int unpackMapHeader()
                    throws java.io.IOException

Reads header of a map.

This method returns number of pairs to be read. After this method call, for each pair, you call unpacker methods for key first, and then value. You will call unpacker methods twice as many time as the returned count. You don't have to call anything at the end of iteration.

Returns:

the size of the map to be read

Throws:

MessageTypeException - when value is not MessagePack Map type

MessageSizeException - when size of the map is larger than 2^31 - 1

java.io.IOException - when underlying input throws IOException

unpackExtensionTypeHeader

public ExtensionTypeHeader unpackExtensionTypeHeader()
                                              throws java.io.IOException

Throws:

java.io.IOException

tryReadStringHeader

private int tryReadStringHeader(byte b)
                         throws java.io.IOException

Throws:

java.io.IOException

tryReadBinaryHeader

private int tryReadBinaryHeader(byte b)
                         throws java.io.IOException

Throws:

java.io.IOException

unpackRawStringHeader

public int unpackRawStringHeader()
                          throws java.io.IOException

Throws:

java.io.IOException

unpackBinaryHeader

public int unpackBinaryHeader()
                       throws java.io.IOException

Reads header of a binary.

This method returns number of bytes to be read. After this method call, you call a readPayload method such as readPayload(int) with the returned count.

You can divide readPayload method into multiple calls. In this case, you must repeat readPayload methods until total amount of bytes becomes equal to the returned count.

Returns:

the size of the map to be read

Throws:

MessageTypeException - when value is not MessagePack Map type

MessageSizeException - when size of the map is larger than 2^31 - 1

java.io.IOException - when underlying input throws IOException

skipPayload

private void skipPayload(int numBytes)
                  throws java.io.IOException

Skip reading the specified number of bytes. Use this method only if you know skipping data is safe. For simply skipping the next value, use skipValue().

Parameters:

numBytes -

Throws:

java.io.IOException

readPayload

public void readPayload(java.nio.ByteBuffer dst)
                 throws java.io.IOException

Reads payload bytes of binary, extension, or raw string types.

This consumes bytes, copies them to the specified buffer, and moves forward position of the byte buffer until ByteBuffer.remaining() returns 0.

Parameters:

dst - the byte buffer into which the data is read

Throws:

java.io.IOException - when underlying input throws IOException

readPayload

public void readPayload(MessageBuffer dst,
                        int off,
                        int len)
                 throws java.io.IOException

Reads payload bytes of binary, extension, or raw string types.

This consumes bytes, copies them to the specified buffer This is usually faster than readPayload(ByteBuffer) by using unsafe.copyMemory

Parameters:

dst - the Message buffer into which the data is read

off - the offset in the Message buffer

len - the number of bytes to read

Throws:

java.io.IOException - when underlying input throws IOException

readPayload

public void readPayload(byte[] dst)
                 throws java.io.IOException

Reads payload bytes of binary, extension, or raw string types. This consumes specified amount of bytes into the specified byte array.

This method is equivalent to readPayload(dst, 0, dst.length).

Parameters:

dst - the byte array into which the data is read

Throws:

java.io.IOException - when underlying input throws IOException

readPayload

public byte[] readPayload(int length)
                   throws java.io.IOException

Reads payload bytes of binary, extension, or raw string types. This method allocates a new byte array and consumes specified amount of bytes into the byte array.

This method is equivalent to readPayload(new byte[length]).

Parameters:

length - number of bytes to be read

Returns:

the new byte array

Throws:

java.io.IOException - when underlying input throws IOException

readPayload

public void readPayload(byte[] dst,
                        int off,
                        int len)
                 throws java.io.IOException

Reads payload bytes of binary, extension, or raw string types. This consumes specified amount of bytes into the specified byte array.

Parameters:

dst - the byte array into which the data is read

off - the offset in the dst array

len - the number of bytes to read

Throws:

java.io.IOException - when underlying input throws IOException

readPayloadAsReference

public MessageBuffer readPayloadAsReference(int length)
                                     throws java.io.IOException

Reads payload bytes of binary, extension, or raw string types as a reference to internal buffer. Note: This methods may return raw memory region, access to which has no strict boundary checks. To use this method safely, you need to understand the internal buffer handling of msgpack-java.

This consumes specified amount of bytes and returns its reference or copy. This method tries to return reference as much as possible because it is faster. However, it may copy data to a newly allocated buffer if reference is not applicable.

Parameters:

length - number of bytes to be read

Throws:

java.io.IOException - when underlying input throws IOException

readNextLength8

private int readNextLength8()
                     throws java.io.IOException

Throws:

java.io.IOException

readNextLength16

private int readNextLength16()
                      throws java.io.IOException

Throws:

java.io.IOException

readNextLength32

private int readNextLength32()
                      throws java.io.IOException

Throws:

java.io.IOException

close

public void close()
           throws java.io.IOException

Closes underlying input.

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface java.lang.AutoCloseable

Throws:

java.io.IOException

overflowU8

private static MessageIntegerOverflowException overflowU8(byte u8)

overflowU16

private static MessageIntegerOverflowException overflowU16(short u16)

overflowU32

private static MessageIntegerOverflowException overflowU32(int u32)

overflowU64

private static MessageIntegerOverflowException overflowU64(long u64)

overflowI16

private static MessageIntegerOverflowException overflowI16(short i16)

overflowI32

private static MessageIntegerOverflowException overflowI32(int i32)

overflowI64

private static MessageIntegerOverflowException overflowI64(long i64)

overflowU32Size

private static MessageSizeException overflowU32Size(int u32)