Package io.grpc.rls

Class CachingRlsLbClient

java.lang.Object

io.grpc.rls.CachingRlsLbClient

@ThreadSafe
final class CachingRlsLbClient
extends Object

A CachingRlsLbClient is a core implementation of RLS loadbalancer supports dynamic request routing by fetching the decision from route lookup server. Every single request is routed by the server's decision. To reduce the performance penalty, LruCache is used.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static final class	CachingRlsLbClient.AutoCleaningEvictionListener	When any CachingRlsLbClient.CacheEntry is evicted from LruCache, it performs CachingRlsLbClient.CacheEntry.cleanup() after original LruCache.EvictionListener is finished.
private static final class	CachingRlsLbClient.BackoffCacheEntry	Implementation of CachingRlsLbClient.CacheEntry contains error.
private final class	CachingRlsLbClient.BackoffRefreshListener	LbStatusListener refreshes CachingRlsLbClient.BackoffCacheEntry when lb state is changed to ConnectivityState.READY from ConnectivityState.TRANSIENT_FAILURE.
(package private) static final class	CachingRlsLbClient.Builder	A Builder for CachingRlsLbClient.
(package private) static final class	CachingRlsLbClient.CachedRouteLookupResponse	Viewer class for cached RlsProtoData.RouteLookupResponse and associated LbPolicyConfiguration.ChildPolicyWrapper.
(package private) static class	CachingRlsLbClient.CacheEntry	Common cache entry data for CachingRlsLbClient.RlsAsyncLruCache.
(package private) final class	CachingRlsLbClient.DataCacheEntry	Implementation of CachingRlsLbClient.CacheEntry contains valid data.
private static final class	CachingRlsLbClient.HappyThrottler	A Throttler never throttles.
(package private) static final class	CachingRlsLbClient.PendingCacheEntry	A pending cache entry when the async RouteLookup RPC is still on the fly.
private static final class	CachingRlsLbClient.RlsAsyncLruCache	Implementation of LinkedHashLruCache for RLS.
private static final class	CachingRlsLbClient.RlsLbHelper
(package private) final class	CachingRlsLbClient.RlsPicker

Field Summary

Fields

Modifier and Type	Field	Description
private final BackoffPolicy.Provider	backoffProvider
static final int	BYTES_PER_CHAR
private static final LongGaugeMetricInstrument	CACHE_ENTRIES_GAUGE
private static final LongGaugeMetricInstrument	CACHE_SIZE_GAUGE
private final long	callTimeoutNanos
private final ResolvedAddressFactory	childLbResolvedAddressFactory
private static final LongCounterMetricInstrument	DEFAULT_TARGET_PICKS_COUNTER
private static final LongCounterMetricInstrument	FAILED_PICKS_COUNTER
private final MetricRecorder.Registration	gaugeRegistration
private final CachingRlsLbClient.RlsLbHelper	helper
private final LbPolicyConfiguration	lbPolicyConfig
private final CachingRlsLbClient.RlsAsyncLruCache	linkedHashLruCache
private final Object	lock
private final ChannelLogger	logger
private final long	maxAgeNanos
private final String	metricsInstanceUuid
static final long	MIN_EVICTION_TIME_DELTA_NANOS
static final int	OBJ_OVERHEAD_B	Minimum bytes for a Java Object.
private final Map<RlsProtoData.RouteLookupRequest,CachingRlsLbClient.PendingCacheEntry>	pendingCallCache
private final Future<?>	periodicCleaner
private final LbPolicyConfiguration.RefCountedChildPolicyWrapperFactory	refCountedChildPolicyWrapperFactory
private static final com.google.common.base.Converter<RlsProtoData.RouteLookupRequest,RouteLookupRequest>	REQUEST_CONVERTER
private static final com.google.common.base.Converter<RlsProtoData.RouteLookupResponse,RouteLookupResponse>	RESPONSE_CONVERTER
(package private) static final Metadata.Key<String>	RLS_DATA_KEY	A header will be added when RLS server respond with additional header data.
private final ManagedChannel	rlsChannel
private final CachingRlsLbClient.RlsPicker	rlsPicker
private final RouteLookupServiceGrpc.RouteLookupServiceStub	rlsStub
private final ScheduledExecutorService	scheduledExecutorService
private final long	staleAgeNanos
static final int	STRING_OVERHEAD_BYTES
private static final LongCounterMetricInstrument	TARGET_PICKS_COUNTER
private final Throttler	throttler
private final com.google.common.base.Ticker	ticker

Constructor Summary

Constructors

Modifier	Constructor	Description
private	CachingRlsLbClient(CachingRlsLbClient.Builder builder)

Method Summary

Modifier and Type	Method	Description
private CachingRlsLbClient.CachedRouteLookupResponse	asyncRlsCall(RlsProtoData.RouteLookupRequest request, BackoffPolicy backoffPolicy)	Populates async cache entry for new request.
(package private) void	close()	Performs any pending maintenance operations needed by the cache.
(package private) static Status	convertRlsServerStatus(Status status, String serverName)	Convert the status to UNAVAILABLE and enhance the error message.
private CachingRlsLbClient.BackoffCacheEntry	createBackOffEntry(RlsProtoData.RouteLookupRequest request, Status status, BackoffPolicy backoffPolicy)
private CachingRlsLbClient.DataCacheEntry	createDataEntry(RlsProtoData.RouteLookupRequest request, RlsProtoData.RouteLookupResponse routeLookupResponse)
private CachingRlsLbClient.PendingCacheEntry	createPendingEntry(RlsProtoData.RouteLookupRequest request, com.google.common.util.concurrent.ListenableFuture<RlsProtoData.RouteLookupResponse> pendingCall, BackoffPolicy backoffPolicy)
(package private) final CachingRlsLbClient.CachedRouteLookupResponse	get(RlsProtoData.RouteLookupRequest request)	Returns async response of the request.
(package private) void	init()
(package private) static CachingRlsLbClient.Builder	newBuilder()	Returns a Builder for CachingRlsLbClient.
private void	pendingRpcComplete(CachingRlsLbClient.PendingCacheEntry entry)
private void	periodicClean()
private void	refreshBackoffEntry(CachingRlsLbClient.BackoffCacheEntry entry)
(package private) void	requestConnection()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

REQUEST_CONVERTER

private static final com.google.common.base.Converter<RlsProtoData.RouteLookupRequest,RouteLookupRequest> REQUEST_CONVERTER

RESPONSE_CONVERTER

private static final com.google.common.base.Converter<RlsProtoData.RouteLookupResponse,RouteLookupResponse> RESPONSE_CONVERTER

MIN_EVICTION_TIME_DELTA_NANOS

public static final long MIN_EVICTION_TIME_DELTA_NANOS

BYTES_PER_CHAR

public static final int BYTES_PER_CHAR

See Also:

• Constant Field Values

STRING_OVERHEAD_BYTES

public static final int STRING_OVERHEAD_BYTES

See Also:

• Constant Field Values

OBJ_OVERHEAD_B

public static final int OBJ_OVERHEAD_B

Minimum bytes for a Java Object.

See Also:

• Constant Field Values

DEFAULT_TARGET_PICKS_COUNTER

private static final LongCounterMetricInstrument DEFAULT_TARGET_PICKS_COUNTER

TARGET_PICKS_COUNTER

private static final LongCounterMetricInstrument TARGET_PICKS_COUNTER

FAILED_PICKS_COUNTER

private static final LongCounterMetricInstrument FAILED_PICKS_COUNTER

CACHE_ENTRIES_GAUGE

private static final LongGaugeMetricInstrument CACHE_ENTRIES_GAUGE

CACHE_SIZE_GAUGE

private static final LongGaugeMetricInstrument CACHE_SIZE_GAUGE

gaugeRegistration

private final MetricRecorder.Registration gaugeRegistration

metricsInstanceUuid

private final String metricsInstanceUuid

lock

private final Object lock

linkedHashLruCache

private final CachingRlsLbClient.RlsAsyncLruCache linkedHashLruCache

periodicCleaner

private final Future<?> periodicCleaner

pendingCallCache

private final Map<RlsProtoData.RouteLookupRequest,CachingRlsLbClient.PendingCacheEntry> pendingCallCache

scheduledExecutorService

private final ScheduledExecutorService scheduledExecutorService

ticker

private final com.google.common.base.Ticker ticker

throttler

private final Throttler throttler

lbPolicyConfig

private final LbPolicyConfiguration lbPolicyConfig

backoffProvider

private final BackoffPolicy.Provider backoffProvider

maxAgeNanos

private final long maxAgeNanos

staleAgeNanos

private final long staleAgeNanos

callTimeoutNanos

private final long callTimeoutNanos

helper

private final CachingRlsLbClient.RlsLbHelper helper

rlsChannel

private final ManagedChannel rlsChannel

rlsStub

private final RouteLookupServiceGrpc.RouteLookupServiceStub rlsStub

rlsPicker

private final CachingRlsLbClient.RlsPicker rlsPicker

childLbResolvedAddressFactory

private final ResolvedAddressFactory childLbResolvedAddressFactory

refCountedChildPolicyWrapperFactory

private final LbPolicyConfiguration.RefCountedChildPolicyWrapperFactory refCountedChildPolicyWrapperFactory

logger

private final ChannelLogger logger

RLS_DATA_KEY

static final Metadata.Key<String> RLS_DATA_KEY

A header will be added when RLS server respond with additional header data.

Constructor Details

CachingRlsLbClient

private CachingRlsLbClient(CachingRlsLbClient.Builder builder)

Method Details

init

void init()

convertRlsServerStatus

static Status convertRlsServerStatus(Status status,
 String serverName)

Convert the status to UNAVAILABLE and enhance the error message.

Parameters:

status - status as provided by server

serverName - Used for error description

Returns:

Transformed status

periodicClean

private void periodicClean()

asyncRlsCall

private CachingRlsLbClient.CachedRouteLookupResponse asyncRlsCall(RlsProtoData.RouteLookupRequest request,
 @Nullable
 BackoffPolicy backoffPolicy)

Populates async cache entry for new request.

get

@CheckReturnValue
final CachingRlsLbClient.CachedRouteLookupResponse get(RlsProtoData.RouteLookupRequest request)

Returns async response of the request. The returned value can be in 3 different states; cached, pending and backed-off due to error. The result remains same even if the status is changed after the return.

close

void close()

Performs any pending maintenance operations needed by the cache.

requestConnection

void requestConnection()

createPendingEntry

private CachingRlsLbClient.PendingCacheEntry createPendingEntry(RlsProtoData.RouteLookupRequest request,
 com.google.common.util.concurrent.ListenableFuture<RlsProtoData.RouteLookupResponse> pendingCall,
 @Nullable
 BackoffPolicy backoffPolicy)

pendingRpcComplete

private void pendingRpcComplete(CachingRlsLbClient.PendingCacheEntry entry)

createDataEntry

private CachingRlsLbClient.DataCacheEntry createDataEntry(RlsProtoData.RouteLookupRequest request,
 RlsProtoData.RouteLookupResponse routeLookupResponse)

createBackOffEntry

private CachingRlsLbClient.BackoffCacheEntry createBackOffEntry(RlsProtoData.RouteLookupRequest request,
 Status status,
 @Nullable
 BackoffPolicy backoffPolicy)

refreshBackoffEntry

private void refreshBackoffEntry(CachingRlsLbClient.BackoffCacheEntry entry)

newBuilder

static CachingRlsLbClient.Builder newBuilder()

Returns a Builder for CachingRlsLbClient.