Package morfologik.fsa.builders

Class FSA5Serializer

java.lang.Object

morfologik.fsa.builders.FSA5Serializer

All Implemented Interfaces:

FSASerializer

public final class FSA5Serializer
extends Object
implements FSASerializer

Serializes in-memory FSA graphs to a binary format compatible with Jan Daciuk's fsa's package FSA5 format.

It is possible to serialize the automaton with numbers required for perfect hashing. See withNumbers() method.

See Also:

• FSA5
• FSA.read(java.io.InputStream)

Field Summary

Fields

Modifier and Type	Field	Description
byte	annotationByte
byte	fillerByte
private static final EnumSet<FSAFlags>	flags	Supported flags.
private static final int	MAX_ARC_SIZE	Maximum number of bytes for a serialized arc.
private static final int	MAX_NODE_DATA_SIZE	Maximum number of bytes for per-node data.
private com.carrotsearch.hppc.IntIntHashMap	numbers	A hash map of [state, right-language-count] pairs.
private com.carrotsearch.hppc.IntIntHashMap	offsets	A hash map of [state, offset] pairs.
private static final int	SIZEOF_FLAGS	Number of bytes for the arc's flags header (arc representation without the goto address).
private boolean	withNumbers	true if we should serialize with numbers.

Constructor Summary

Constructors

Constructor	Description
FSA5Serializer()

Method Summary

Modifier and Type	Method	Description
private int	emitArc(ByteBuffer bb, OutputStream os, int gtl, int flags, byte label, int targetOffset)
private boolean	emitArcs(FSA fsa, OutputStream os, int[] linearized, int gtl, int nodeDataLength)	Update arc offsets assuming the given goto length.
private int	emitNodeData(ByteBuffer bb, OutputStream os, int nodeDataLength, int number)
Set<FSAFlags>	getFlags()	Return supported flags.
private int[]	linearize(FSA fsa)	Linearization of states.
<T extends OutputStream> T	serialize(FSA fsa, T os)	Serialize root state s to an output stream in FSA5 format.
FSA5Serializer	withAnnotationSeparator(byte annotationSeparator)	Sets the annotation separator (only if FSASerializer.getFlags() returns FSAFlags.SEPARATORS).
FSA5Serializer	withFiller(byte filler)	Sets the filler separator (only if FSASerializer.getFlags() returns FSAFlags.SEPARATORS).
FSA5Serializer	withNumbers()	Serialize the automaton with the number of right-language sequences in each node.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

MAX_ARC_SIZE

private static final int MAX_ARC_SIZE

Maximum number of bytes for a serialized arc.

See Also:

• Constant Field Values

MAX_NODE_DATA_SIZE

private static final int MAX_NODE_DATA_SIZE

Maximum number of bytes for per-node data.

See Also:

• Constant Field Values

SIZEOF_FLAGS

private static final int SIZEOF_FLAGS

Number of bytes for the arc's flags header (arc representation without the goto address).

See Also:

• Constant Field Values

flags

private static final EnumSet<FSAFlags> flags

Supported flags.

fillerByte

public byte fillerByte

See Also:

• FSA5.filler

annotationByte

public byte annotationByte

See Also:

• FSA5.annotation

withNumbers

private boolean withNumbers

true if we should serialize with numbers.

See Also:

• withNumbers()

offsets

private com.carrotsearch.hppc.IntIntHashMap offsets

A hash map of [state, offset] pairs.

numbers

private com.carrotsearch.hppc.IntIntHashMap numbers

A hash map of [state, right-language-count] pairs.

Constructor Details

FSA5Serializer

public FSA5Serializer()

Method Details

withNumbers

public FSA5Serializer withNumbers()

Serialize the automaton with the number of right-language sequences in each node. This is required to implement perfect hashing. The numbering also preserves the order of input sequences.

Specified by:

withNumbers in interface FSASerializer

Returns:

Returns the same object for easier call chaining.

withFiller

public FSA5Serializer withFiller(byte filler)

Sets the filler separator (only if FSASerializer.getFlags() returns FSAFlags.SEPARATORS).

Specified by:

withFiller in interface FSASerializer

Parameters:

filler - The filler separator byte.

Returns:

Returns this for call chaining.

withAnnotationSeparator

public FSA5Serializer withAnnotationSeparator(byte annotationSeparator)

Sets the annotation separator (only if FSASerializer.getFlags() returns FSAFlags.SEPARATORS).

Specified by:

withAnnotationSeparator in interface FSASerializer

Parameters:

annotationSeparator - The filler separator byte.

Returns:

Returns this for call chaining.

serialize

public <T extends OutputStream> T serialize(FSA fsa,
 T os)
                                     throws IOException

Serialize root state s to an output stream in FSA5 format.

Specified by:

serialize in interface FSASerializer

Type Parameters:

T - A subclass of OutputStream, returned for chaining.

Parameters:

fsa - The automaton to serialize.

os - The output stream to serialize to.

Returns:

Returns os for chaining.

Throws:

IOException - Rethrown if an I/O error occurs.

See Also:

• withNumbers()

getFlags

public Set<FSAFlags> getFlags()

Return supported flags.

Specified by:

getFlags in interface FSASerializer

Returns:

Returns the set of flags supported by the serializer (and the output automaton).

linearize

private int[] linearize(FSA fsa)

Linearization of states.

emitArcs

private boolean emitArcs(FSA fsa,
 OutputStream os,
 int[] linearized,
 int gtl,
 int nodeDataLength)
                  throws IOException

Update arc offsets assuming the given goto length.

Throws:

IOException

emitArc

private int emitArc(ByteBuffer bb,
 OutputStream os,
 int gtl,
 int flags,
 byte label,
 int targetOffset)
             throws IOException

Throws:

IOException

emitNodeData

private int emitNodeData(ByteBuffer bb,
 OutputStream os,
 int nodeDataLength,
 int number)
                  throws IOException

Throws:

IOException