Package io.grpc.rls

Class AdaptiveThrottler

  • All Implemented Interfaces:
    Throttler
    final class AdaptiveThrottler
extends java.lang.Object
implements Throttler
    Implementation of Throttler that keeps track of recent history (the duration of which is specified to the constructor) and throttles requests at the client side based on the number of requests that the backend has accepted and the total number of requests generated. A given request will be throttled with a probability 
       throttleProbability = (requests - ratio_for_accepts * accepts) / (requests + requests_padding)
    where requests is the total number of requests, accepts is the total number of requests that the backend has accepted and ratio_for_accepts is just a constant multiplier passed to the constructor (see the description of ratio_for_accepts for more information).

    • Field Detail

      • DEFAULT_HISTORY_SECONDS

        private static final int DEFAULT_HISTORY_SECONDS
        See Also:
        Constant Field Values

      • DEFAULT_REQUEST_PADDING

        private static final int DEFAULT_REQUEST_PADDING
        See Also:
        Constant Field Values

      • DEFAULT_RATIO_FOR_ACCEPT

        private static final float DEFAULT_RATIO_FOR_ACCEPT
        See Also:
        Constant Field Values

      • historySeconds

        private final int historySeconds
        The duration of history of calls used by Adaptive Throttler.

      • requestsPadding

        private final int requestsPadding
        A magic number to tune the aggressiveness of the throttling. High numbers throttle less. The default is 8.

      • ratioForAccepts

        private final float ratioForAccepts
        The ratio by which the Adaptive Throttler will attempt to send requests above what the server is currently accepting.

      • ticker

        private final com.google.common.base.Ticker ticker

      • requestStat

        final AdaptiveThrottler.TimeBasedAccumulator requestStat
        The number of requests attempted by the client during the Adaptive Throttler instance's history of calls. This includes requests throttled at the client. The history period defaults to 30 seconds.

      • throttledStat

        final AdaptiveThrottler.TimeBasedAccumulator throttledStat
        Counter for the total number of requests that were throttled by either the client (this class) or the backend in recent history.

    • Method Detail

      • shouldThrottle

        public boolean shouldThrottle()
        Description copied from interface: Throttler
        Checks if a given request should be throttled by the client. This should be called for every request before allowing it to hit the network. If the returned value is true, the request should be aborted immediately (as if it had been throttled by the server).

        This updates internal state and should be called exactly once for each request.

        Specified by:
        shouldThrottle in interface Throttler

      • shouldThrottle

        boolean shouldThrottle​(float random)

      • getThrottleProbability

        float getThrottleProbability​(long nowNanos)
        Calculates throttleProbability. 
         throttleProbability = (requests - ratio_for_accepts * accepts) / (requests + requests_padding)

      • registerBackendResponse

        public void registerBackendResponse​(boolean throttled)
        Description copied from interface: Throttler
        Registers a response received from the backend for a request allowed by shouldThrottle. This should be called for every response received from the backend (i.e., once for each request for which ShouldThrottle returned false). This updates the internal statistics used by shouldThrottle.
        Specified by:
        registerBackendResponse in interface Throttler
        Parameters:
        throttled - specifies whether the request was throttled by the backend.

      • randomFloat

        private static float randomFloat()

      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object