Package org.terracotta.offheapstore.paging

Class UpfrontAllocatingPageSource

  • All Implemented Interfaces:
    PageSource
    public class UpfrontAllocatingPageSource
extends java.lang.Object
implements PageSource
    An upfront allocating direct byte buffer source.

    This buffer source implementation allocates all of its required storage up-front in fixed size chunks. Runtime allocations are then satisfied using slices from these initial chunks.

    • Field Detail

      • ALLOCATION_LOG_LOCATION

        public static final java.lang.String ALLOCATION_LOG_LOCATION

      • LOGGER

        private static final org.slf4j.Logger LOGGER

      • PROGRESS_LOGGING_STEP_SIZE

        private static final double PROGRESS_LOGGING_STEP_SIZE
        See Also:
        Constant Field Values

      • PROGRESS_LOGGING_THRESHOLD

        private static final long PROGRESS_LOGGING_THRESHOLD

      • REGION_COMPARATOR

        private static final java.util.Comparator<Page> REGION_COMPARATOR

      • risingThresholds

        private final java.util.SortedMap<java.lang.Long,​java.lang.Runnable> risingThresholds

      • fallingThresholds

        private final java.util.SortedMap<java.lang.Long,​java.lang.Runnable> fallingThresholds

      • buffers

        private final java.util.List<java.nio.ByteBuffer> buffers

      • victims

        private final java.util.List<java.util.NavigableSet<Page>> victims

      • availableSet

        private volatile int availableSet

    • Constructor Detail

      • UpfrontAllocatingPageSource

        public UpfrontAllocatingPageSource​(BufferSource source,
                                   long toAllocate,
                                   int chunkSize)
        Create an up-front allocating buffer source of toAllocate total bytes, in chunkSize byte chunks.
        Parameters:
        source - source from which initial buffers will be allocated
        toAllocate - total space to allocate in bytes
        chunkSize - chunkSize size to allocate in bytes

      • UpfrontAllocatingPageSource

        public UpfrontAllocatingPageSource​(BufferSource source,
                                   long toAllocate,
                                   int maxChunk,
                                   int minChunk)
        Create an up-front allocating buffer source of toAllocate total bytes, in maximally sized chunks, within the given bounds.
        Parameters:
        source - source from which initial buffers will be allocated
        toAllocate - total space to allocate in bytes
        maxChunk - the largest chunk size in bytes
        minChunk - the smallest chunk size in bytes

      • UpfrontAllocatingPageSource

        private UpfrontAllocatingPageSource​(BufferSource source,
                                    long toAllocate,
                                    int maxChunk,
                                    int minChunk,
                                    boolean fixed)
        Create an up-front allocating buffer source of toAllocate total bytes, in maximally sized chunks, within the given bounds.

        By default we try to allocate chunks of maxChunk size. However, unless fixed is true, in case of allocation failure, we will try to allocate half-smaller chunks. We do not allocate chunks smaller than minChunk though.

        Parameters:
        source - source from which initial buffers will be allocated
        toAllocate - total space to allocate in bytes
        maxChunk - the largest chunk size in bytes
        minChunk - the smallest chunk size in bytes
        fixed - if the chunks should all be of size maxChunk or can be smaller

    • Method Detail

      • getCapacity

        public long getCapacity()
        Return the total allocated capacity, used or not
        Returns:
        the total capacity

      • allocate

        public Page allocate​(int size,
                     boolean thief,
                     boolean victim,
                     OffHeapStorageArea owner)
        Allocates a byte buffer of at least the given size.

        This BufferSource is limited to allocating regions that are a power of two in size. Supplied sizes are therefore rounded up to the next largest power of two.

        Specified by:
        allocate in interface PageSource
        Parameters:
        size - size of page to allocate
        thief - true if the allocation can steal space from victims
        victim - true if the allocated page should be eligible for stealing
        owner - owner from which subsequent steal should occur
        Returns:
        a buffer of at least the given size

      • findVictimPages

        private java.util.List<Page> findVictimPages​(int chunk,
                                             int address,
                                             int size)

      • free

        public void free​(Page page)
        Frees the supplied buffer.

        If the given buffer was not allocated by this source or has already been freed then an AssertionError is thrown.

        Specified by:
        free in interface PageSource

      • getAllocatedSize

        public long getAllocatedSize()

      • getAllocatedSizeUnSync

        public long getAllocatedSizeUnSync()

      • isUnavailable

        private boolean isUnavailable​(int size)

      • markAllAvailable

        private void markAllAvailable()

      • markUnavailable

        private void markUnavailable​(int size)

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object

      • fireThresholds

        private void fireThresholds​(long incoming,
                            long outgoing)

      • addAllocationThreshold

        public java.lang.Runnable addAllocationThreshold​(UpfrontAllocatingPageSource.ThresholdDirection direction,
                                                 long threshold,
                                                 java.lang.Runnable action)
        Adds an allocation threshold action.

        There can be only a single action associated with each unique direction and threshold combination. If an action is already associated with the supplied combination then the action is replaced by the new action and the old action is returned.

        Actions are fired on passing through the supplied threshold and are called synchronously with the triggering allocation. This means care must be taken to avoid mutating any map that uses this page source from within the action otherwise deadlocks may result. Exceptions thrown by the action will be caught and logged by the page source and will not be propagated on the allocating thread.

        Parameters:
        direction - new actions direction
        threshold - new actions threshold level
        action - fired on breaching the threshold
        Returns:
        the replaced action or null if no action was present.

      • removeAllocationThreshold

        public java.lang.Runnable removeAllocationThreshold​(UpfrontAllocatingPageSource.ThresholdDirection direction,
                                                    long threshold)
        Removes an allocation threshold action.

        Removes the allocation threshold action for the given level and direction.

        Parameters:
        direction - registered actions direction
        threshold - registered actions threshold level
        Returns:
        the removed condition or null if no action was present.

      • allocateBackingBuffers

        private static java.util.Collection<java.nio.ByteBuffer> allocateBackingBuffers​(BufferSource source,
                                                                                long toAllocate,
                                                                                int maxChunk,
                                                                                int minChunk,
                                                                                boolean fixed)
        Allocate multiple buffers to fulfill the requested memory toAllocate. We first divide toAllocate in chunks of size maxChunk and try to allocate them in parallel on all available processors. If one chunk fails to be allocated, we try to allocate two chunks of maxChunk / 2. If this allocation fails, we continue dividing until we reach of size of minChunk. If at that moment, the allocation still fails, an IllegalArgumentException is thrown.

        When fixed is requested, we will only allocated buffers of maxChunk size. If allocation fails, an IllegalArgumentException is thrown without any division.

        If the allocation is interrupted, the method will ignore it and continue allocation. It will then return with the interrupt flag is set.

        Parameters:
        source - source used to allocate memory buffers
        toAllocate - total amount of memory to allocate
        maxChunk - maximum size of a buffer. This is the targeted size for all buffers if everything goes well
        minChunk - minimum buffer size allowed
        fixed - if all buffers should have a the same size (except the last one with toAllocate % maxChunk != 0, if true, minChunk isn't used
        Returns:
        the list of allocated buffers
        Throws:
        java.lang.IllegalArgumentException - when we fail to allocate the requested memory

      • bufferAllocation

        private static java.util.Collection<java.nio.ByteBuffer> bufferAllocation​(BufferSource source,
                                                                          int toAllocate,
                                                                          int minChunk,
                                                                          boolean fixed,
                                                                          java.io.PrintStream allocatorLog,
                                                                          long start)

      • uninterruptibleGet

        private static <T> T uninterruptibleGet​(java.util.concurrent.Future<T> future)

      • createAllocatorLog

        private static java.io.PrintStream createAllocatorLog​(long max,
                                                      int maxChunk,
                                                      int minChunk)