Package edu.berkeley.nlp.lm.values

Class CountValueContainer

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected int defaultValRank  
      protected long[] numNgramsForEachOrder  
      protected boolean storeSuffixIndexes  
      protected int[] suffixBitsForOrder  
      protected boolean useMapValueArray  
      protected VariableLengthBitCompressor valueCoder  
      protected int valueRadix  
      protected CustomWidthArray[] valueRanks  
      protected int valueWidth  

    • Constructor Summary

      Constructors 
      Constructor Description
      CountValueContainer​(LongToIntHashMap countCounter, int valueRadix, boolean storePrefixes, long[] numNgramsForEachOrder)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      boolean add​(int[] ngram, int startPos, int endPos, int ngramOrder, long offset, long prefixOffset, int word, V val_, long suffixOffset, boolean ngramIsNew)
      Adds a new value at the specified offset.
      void clearStorageAfterCompression​(int ngramOrder)  
      void clearStorageForOrder​(int ngramOrder)  
      CountValueContainer createFreshValues​(long[] numNgramsForEachOrder_)
      Creates a fresh value container for copying purposes.
      void decompress​(BitStream bits, int ngramOrder, boolean justConsume, V outputVal)
      Reads and decompresses from the bit stream bits.
      BitList getCompressed​(long offset, int ngramOrder)
      Compresses the value at the given offset into a list of bits.
      long getCount​(int ngramOrder, long index)  
      protected long getCountRank​(long val)  
      protected LongRef getDefaultVal()  
      void getFromOffset​(long index, int ngramOrder, LongRef outputVal)
      Gets the value living at a particular offset.
      protected void getFromRank​(long rank, LongRef outputVal)  
      protected long getRank​(int ngramOrder, long offset)  
      LongRef getScratchValue()
      Creates a fresh value of object (useful for passing as an output parameter)
      long getSuffixOffset​(long index, int ngramOrder)  
      protected int getSuffixOffsetFromInternalVal​(long internalVal)  
      long getUnigramSum()  
      int numValueBits​(int ngramOrder)  
      void setFromOtherValues​(ValueContainer<LongRef> o)
      Destructively sets internal storage from another object.
      void setMap​(NgramMap<V> map)
      Initializes a value container with the map that contains it
      void setSizeAtLeast​(long size, int ngramOrder)
      Sets internal storage for size for a particular n-gram order
      boolean storeSuffixoffsets()  
      void swap​(long a, long b, int ngramOrder)
      Swaps values at offsets a and b.
      void trim()
      Final clean up of storage.
      void trimAfterNgram​(int ngramOrder, long size)
      Clear storage after an n-gram order is complete
      protected boolean useValueStoringArray()  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • storeSuffixIndexes

        protected final boolean storeSuffixIndexes

      • valueRadix

        protected final int valueRadix

      • valueWidth

        protected int valueWidth

      • numNgramsForEachOrder

        protected final long[] numNgramsForEachOrder

      • suffixBitsForOrder

        protected final int[] suffixBitsForOrder

      • useMapValueArray

        protected boolean useMapValueArray

    • Constructor Detail

      • CountValueContainer

        public CountValueContainer​(LongToIntHashMap countCounter,
                           int valueRadix,
                           boolean storePrefixes,
                           long[] numNgramsForEachOrder)

    • Method Detail

      • createFreshValues

        public CountValueContainer createFreshValues​(long[] numNgramsForEachOrder_)
        Description copied from interface: ValueContainer
        Creates a fresh value container for copying purposes.
        Returns:

      • getFromOffset

        public void getFromOffset​(long index,
                          int ngramOrder,
                          LongRef outputVal)
        Description copied from interface: ValueContainer
        Gets the value living at a particular offset.

      • getFromRank

        protected void getFromRank​(long rank,
                           LongRef outputVal)

      • getCount

        public final long getCount​(int ngramOrder,
                           long index)

      • getDefaultVal

        protected LongRef getDefaultVal()

      • getUnigramSum

        public long getUnigramSum()

      • getScratchValue

        public LongRef getScratchValue()
        Description copied from interface: ValueContainer
        Creates a fresh value of object (useful for passing as an output parameter)
        Returns:

      • getCountRank

        protected long getCountRank​(long val)

      • useValueStoringArray

        protected boolean useValueStoringArray()

      • add

        public boolean add​(int[] ngram,
                   int startPos,
                   int endPos,
                   int ngramOrder,
                   long offset,
                   long prefixOffset,
                   int word,
                   V val_,
                   long suffixOffset,
                   boolean ngramIsNew)
        Description copied from interface: ValueContainer
        Adds a new value at the specified offset.
        Specified by:
        add in interface ValueContainer<V extends LongRepresentable<V>>
        ngramOrder - As always, ngramOrder is 0-based (0=unigram)
        Returns:
        Whether or not the add was successful

      • decompress

        public void decompress​(BitStream bits,
                       int ngramOrder,
                       boolean justConsume,
                       V outputVal)
        Description copied from interface: CompressibleValueContainer
        Reads and decompresses from the bit stream bits.
        Specified by:
        decompress in interface CompressibleValueContainer<V extends LongRepresentable<V>>
        justConsume - If true, nothing is returned, and the function simply consumes the appropriate number of bits from the BitStream.

      • getSuffixOffset

        public long getSuffixOffset​(long index,
                            int ngramOrder)

      • getSuffixOffsetFromInternalVal

        protected int getSuffixOffsetFromInternalVal​(long internalVal)
        Parameters:
        internalVal -
        Returns:

      • getRank

        protected long getRank​(int ngramOrder,
                       long offset)