Package io.netty.buffer

Class AdaptivePoolingAllocator

java.lang.Object
io.netty.buffer.AdaptivePoolingAllocator
All Implemented Interfaces:
AdaptiveByteBufAllocator.AdaptiveAllocatorApi
@UnstableApi final class AdaptivePoolingAllocator extends Object implements AdaptiveByteBufAllocator.AdaptiveAllocatorApi
An auto-tuning pooling allocator, that follows an anti-generational hypothesis.

The allocator is organized into a list of Magazines, and each magazine has a chunk-buffer that they allocate buffers from.

The magazines hold the mutexes that ensure the thread-safety of the allocator, and each thread picks a magazine based on the id of the thread. This spreads the contention of multi-threaded access across the magazines. If contention is detected above a certain threshold, the number of magazines are increased in response to the contention.

The magazines maintain histograms of the sizes of the allocations they do. The histograms are used to compute the preferred chunk size. The preferred chunk size is one that is big enough to service 10 allocations of the 99-percentile size. This way, the chunk size is adapted to the allocation patterns.

Computing the preferred chunk size is a somewhat expensive operation. Therefore, the frequency with which this is done, is also adapted to the allocation pattern. If a newly computed preferred chunk is the same as the previous preferred chunk size, then the frequency is reduced. Otherwise, the frequency is increased.

This allows the allocator to quickly respond to changes in the application workload, without suffering undue overhead from maintaining its statistics.

Since magazines are "relatively thread-local", the allocator has a central queue that allow excess chunks from any magazine, to be shared with other magazines. The createSharedChunkQueue() method can be overridden to customize this queue.

  • Field Details

    • LOW_MEM_THRESHOLD

      private static final int LOW_MEM_THRESHOLD
      See Also:

    • IS_LOW_MEM

      private static final boolean IS_LOW_MEM

    • DISABLE_THREAD_LOCAL_MAGAZINES_ON_LOW_MEM

      private static final boolean DISABLE_THREAD_LOCAL_MAGAZINES_ON_LOW_MEM
      Whether the IS_LOW_MEM setting should disable thread-local magazines. This can have fairly high performance overhead.

    • MIN_CHUNK_SIZE

      static final int MIN_CHUNK_SIZE
      The 128 KiB minimum chunk size is chosen to encourage the system allocator to delegate to mmap for chunk allocations. For instance, glibc will do this. This pushes any fragmentation from chunk size deviations off physical memory, onto virtual memory, which is a much, much larger space. Chunks are also allocated in whole multiples of the minimum chunk size, which itself is a whole multiple of popular page sizes like 4 KiB, 16 KiB, and 64 KiB.
      See Also:

    • EXPANSION_ATTEMPTS

      private static final int EXPANSION_ATTEMPTS
      See Also:

    • INITIAL_MAGAZINES

      private static final int INITIAL_MAGAZINES
      See Also:

    • RETIRE_CAPACITY

      private static final int RETIRE_CAPACITY
      See Also:

    • MAX_STRIPES

      private static final int MAX_STRIPES

    • BUFS_PER_CHUNK

      private static final int BUFS_PER_CHUNK
      See Also:

    • MAX_CHUNK_SIZE

      private static final int MAX_CHUNK_SIZE
      The maximum size of a pooled chunk, in bytes. Allocations bigger than this will never be pooled.

      This number is 8 MiB, and is derived from the limitations of internal histograms.

    • MAX_POOLED_BUF_SIZE

      private static final int MAX_POOLED_BUF_SIZE

    • CHUNK_REUSE_QUEUE

      private static final int CHUNK_REUSE_QUEUE
      The capacity if the chunk reuse queues, that allow chunks to be shared across magazines in a group. The default size is twice NettyRuntime.availableProcessors(), same as the maximum number of magazines per magazine group.

    • MAGAZINE_BUFFER_QUEUE_CAPACITY

      private static final int MAGAZINE_BUFFER_QUEUE_CAPACITY
      The capacity if the magazine local buffer queue. This queue just pools the outer ByteBuf instance and not the actual memory and so helps to reduce GC pressure.

    • SIZE_CLASSES

      private static final int[] SIZE_CLASSES
      The size classes are chosen based on the following observation:

      Most allocations, particularly ones above 256 bytes, aim to be a power-of-2. However, many use cases, such as framing protocols, are themselves operating or moving power-of-2 sized payloads, to which they add a small amount of overhead, such as headers or checksums. This means we seem to get a lot of mileage out of having both power-of-2 sizes, and power-of-2-plus-a-bit.

      On the conflicting requirements of both having as few chunks as possible, and having as little wasted memory within each chunk as possible, this seems to strike a surprisingly good balance for the use cases tested so far.

    • SIZE_CLASSES_COUNT

      private static final int SIZE_CLASSES_COUNT

    • SIZE_INDEXES

      private static final byte[] SIZE_INDEXES

    • chunkAllocator

      private final AdaptivePoolingAllocator.ChunkAllocator chunkAllocator

    • chunkRegistry

      private final AdaptivePoolingAllocator.ChunkRegistry chunkRegistry

    • sizeClassedMagazineGroups

      private final AdaptivePoolingAllocator.MagazineGroup[] sizeClassedMagazineGroups

    • largeBufferMagazineGroup

      private final AdaptivePoolingAllocator.MagazineGroup largeBufferMagazineGroup

    • threadLocalGroup

      private final FastThreadLocal<AdaptivePoolingAllocator.MagazineGroup[]> threadLocalGroup

  • Constructor Details

  • Method Details