Key | Default | Type | Description
--- | --- | --- | ---
##### state.changelog.dstl.dfs.base-path | (none) | String | Base path to store changelog files.
##### state.changelog.dstl.dfs.batch.persist-delay | 10 ms | Duration | Delay before persisting changelog after receiving persist request (on checkpoint). Minimizes the number of files and requests if multiple operators (backends) or sub-tasks are using the same store. Correspondingly increases checkpoint time (async phase).
##### state.changelog.dstl.dfs.batch.persist-size-threshold | 10 mb | MemorySize | Size threshold for state changes that were requested to be persisted but are waiting for state.changelog.dstl.dfs.batch.persist-delay (from all operators). . Once reached, accumulated changes are persisted immediately. This is different from state.changelog.dstl.dfs.preemptive-persist-threshold as it happens AFTER the checkpoint and potentially for state changes of multiple operators. Must not exceed in-flight data limit (see below)
##### state.changelog.dstl.dfs.compression.enabled | false | Boolean | Whether to enable compression when serializing changelog.
##### state.changelog.dstl.dfs.discard.num-threads | 1 | Integer | Number of threads to use to discard changelog (e.g. pre-emptively uploaded unused state).
##### state.changelog.dstl.dfs.download.local-cache.idle-timeout-ms | 10 min | Duration | Maximum idle time for cache files of distributed changelog file, after which the cache files will be deleted.
##### state.changelog.dstl.dfs.preemptive-persist-threshold | 5 mb | MemorySize | Size threshold for state changes of a single operator beyond which they are persisted pre-emptively without waiting for a checkpoint. Improves checkpointing time by allowing quasi-continuous uploading of state changes (as opposed to uploading all accumulated changes on checkpoint).
##### state.changelog.dstl.dfs.upload.buffer-size | 1 mb | MemorySize | Buffer size used when uploading change sets
##### state.changelog.dstl.dfs.upload.max-attempts | 3 | Integer | Maximum number of attempts (including the initial one) to perform a particular upload. Only takes effect if state.changelog.dstl.dfs.upload.retry-policy is fixed.
##### state.changelog.dstl.dfs.upload.max-in-flight | 100 mb | MemorySize | Max amount of data allowed to be in-flight. Upon reaching this limit the task will be back-pressured. I.e., snapshotting will block; normal processing will block if state.changelog.dstl.dfs.preemptive-persist-threshold is set and reached. The limit is applied to the total size of in-flight changes if multiple operators/backends are using the same changelog storage. Must be greater than or equal to state.changelog.dstl.dfs.batch.persist-size-threshold
##### state.changelog.dstl.dfs.upload.next-attempt-delay | 500 ms | Duration | Delay before the next attempt (if the failure was not caused by a timeout).
##### state.changelog.dstl.dfs.upload.num-threads | 5 | Integer | Number of threads to use for upload.
##### state.changelog.dstl.dfs.upload.retry-policy | "fixed" | String | Retry policy for the failed uploads (in particular, timed out). Valid values: none, fixed.
##### state.changelog.dstl.dfs.upload.timeout | 1 s | Duration | Time threshold beyond which an upload is considered timed out. If a new attempt is made but this upload succeeds earlier then this upload result will be used. May improve upload times if tail latencies of upload requests are significantly high. Only takes effect if state.changelog.dstl.dfs.upload.retry-policy is fixed. Please note that timeout * max_attempts should be less than execution.checkpointing.timeout