Configuration and rollout

Documentation · API reference

An enabled request scope permits caching. The effective policy selects which layers participate and how long they can reuse values. Configure resources once, define a baseline per operation, and use a provider for runtime changes.

What belongs where

Level	Responsibility	Examples
Instance	Shared resources and instance defaults	Namespace, Redis client and compression, local capacity, metrics, shadow capacity
Operation definition	Result identity and execution contract	Key, serializer, invalidation tracking, source deadline, Policy defaults
Runtime provider	Policy for one enabled invocation	Layer TTLs and ramps, request-local, coalescing, remote-read deadline, recovery age, shadow policy

Keep operation definitions stable. Their policy defaults are snapshotted when registered or invoked; mutating the original config does not change that baseline. Use the provider to change policy. See keys and identity for key design and the read model for scopes and layer lifetimes.

Baseline and overlay precedence

The operation's static policy is its baseline. A provider result overrides only the fields it supplies, including individual entries in nested maps:

runtime field → operation field → library default

For example, a rollout can change the local ramp without repeating the TTL:

Field	Operation default	Runtime override	Effective policy
ttlSec.local	60	omitted	60 seconds
ramp.local	100	10	10% key cohort

The following executed example starts with request-local and process-local caching, overrides only coalescing, then explicitly disables both layers. The assertions check source calls, so a sparse overlay that accidentally discards inherited defaults fails the example.

// In an application, this value can come from your runtime config service.
let overlay = new DialCacheKeyConfig({ coalesce: false });
const cache = new DialCache({ cacheConfigProvider: () => overlay });
let sourceCalls = 0;
const lookup = cache.cached(async (_id: string) => ++sourceCalls, {
  keyType: "user",
  useCase: "runtimePolicy",
  cacheKey: (id) => id,
  defaultConfig: new DialCacheKeyConfig({
    requestLocal: true,
    ttlSec: { local: 60 },
    ramp: { local: 100 },
  }),
});

// The sparse overlay inherits the local TTL: separate requests reuse it.
assert.equal(await cache.enable(() => lookup("42")), 1);
assert.equal(await cache.enable(() => lookup("42")), 1);

// Explicit false and zero disable the two configured cache paths.
overlay = new DialCacheKeyConfig({
  requestLocal: false,
  ramp: { local: 0 },
});
await cache.enable(async () => {
  assert.equal(await lookup("42"), 2);
  assert.equal(await lookup("42"), 3);
});
assert.equal(sourceCalls, 3);

Complete executable example

// A live configuration service would supply a new immutable overlay.
var overlay atomic.Pointer[dialcache.PolicyOverlay]
overlay.Store(&dialcache.PolicyOverlay{Coalesce: dialcache.Ptr(false)})
cache := dialcache.MustNew(dialcache.WithPolicyProvider(
	func(context.Context, dialcache.Identity) (dialcache.RuntimePolicy, error) {
		return overlay.Load(), nil
	},
))
var sourceCalls atomic.Int64
lookup, err := dialcache.Cached(cache, dialcache.Operation[int64]{
	Identity: dialcache.Identity{KeyType: "user", UseCase: "runtimePolicy"},
	Policy:   dialcache.Policy{RequestLocal: true, LocalTTL: 60 * time.Second},
}, func(id string) (dialcache.Identity, error) {
	return dialcache.Identity{ID: id}, nil
}, func(context.Context, string) (int64, error) {
	return sourceCalls.Add(1), nil
})
if err != nil {
	t.Fatal(err)
}

// The sparse overlay inherits the local TTL across separate requests.
for range 2 {
	ctx, done := cache.Enable(context.Background())
	got, err := lookup(ctx, "42")
	done()
	if err != nil || got != 1 {
		t.Fatalf("inherited policy = %d, %v; want 1", got, err)
	}
}

// Nil leaves inherit; pointers preserve explicit false and zero.
overlay.Store(&dialcache.PolicyOverlay{
	RequestLocal: dialcache.Ptr(false),
	LocalRamp:    dialcache.Ptr(0.0),
})
ctx, done := cache.Enable(context.Background())
defer done()
for _, want := range []int64{2, 3} {
	got, err := lookup(ctx, "42")
	if err != nil || got != want {
		t.Fatalf("disabled policy = %d, %v; want %d", got, err, want)
	}
}
if sourceCalls.Load() != 3 {
	t.Fatalf("source calls = %d; want 3", sourceCalls.Load())
}

Complete executable example

// In an application, update this snapshot from your configuration service.
let overlay = Arc::new(RwLock::new(Policy::default().coalesce(false)));
let current_overlay = overlay.clone();
let cache = DialCache::builder()
    .policy_provider(move |_identity| {
        let snapshot = current_overlay.read().unwrap().clone();
        async move { Ok(Some(RuntimePolicy::from(snapshot))) }
    })
    .build()?;
let source_calls = Arc::new(AtomicUsize::new(0));
let calls = source_calls.clone();
let lookup = cache
    .use_case::<u64, usize>("user", "runtimePolicy")
    .policy(Policy::default().request_local(true).local_ttl_sec(60))
    .key(|id: &u64| KeySpec::new(id))
    .source(move |_scope, _id| {
        let value = calls.fetch_add(1, Ordering::SeqCst) + 1;
        async move { Ok(value) }
    })
    .register()?;

// The sparse overlay inherits the local TTL across separate requests.
for _ in 0..2 {
    let request = cache.enable_guard();
    assert_eq!(*lookup.get(request.scope(), 42).await?, 1);
}

// Omitted fields inherit; explicit false and zero disable these paths.
*overlay.write().unwrap() = Policy::default().request_local(false).local_ramp(0.0);
let request = cache.enable_guard();
assert_eq!(*lookup.get(request.scope(), 42).await?, 2);
assert_eq!(*lookup.get(request.scope(), 42).await?, 3);
assert_eq!(source_calls.load(Ordering::SeqCst), 3);

Complete executable example

# An application may fetch this overlay from its runtime config service.
overlay = Policy(coalesce=False)
cache = DialCache(policy_provider=lambda key: overlay)
source_calls = 0

@cache.cached(
    key_type="user",
    use_case="runtimePolicy",
    cache_key=lambda id: id,
    default_config=Policy(request_local=True, ttl_sec={"local": 60}, ramp={"local": 100}),
)
async def lookup(id):
    nonlocal source_calls
    source_calls += 1
    return source_calls

# The sparse overlay inherits the local TTL across separate requests.
async with cache.enable():
    assert await lookup("42") == 1
async with cache.enable():
    assert await lookup("42") == 1

# False and zero explicitly disable the two configured cache paths.
overlay = Policy(request_local=False, ramp={"local": 0})
async with cache.enable():
    assert await lookup("42") == 2
    assert await lookup("42") == 3
assert source_calls == 3

Complete executable example

defaultConfig accepts DialCacheKeyConfig; cacheConfigProvider returns a sparse config or null. Defensive undefined also inherits. An empty config inherits all fields. Nested local/remote maps and shadow leaves merge separately.

Operation.Policy uses time.Duration for TTLs and deadlines. WithPolicyProvider can return *PolicyOverlay, JSONPolicy or RawPolicy. Nil overlay leaves inherit; use pointers such as Ptr(false) and Ptr(0.0) to supply explicit false and zero. TTLs use whole seconds, deadlines whole milliseconds. See the Go guide.

Operations use Policy; policy_provider returns Option<RuntimePolicy>. Ok(None) inherits. Converting a Policy into a runtime overlay preserves its omitted leaves. TTL builders use whole seconds and deadline builders use milliseconds. See the Rust guide.

Operation defaults use Policy; the policy_provider returns a sparse Policy, mapping or None. A whole reply of None inherits; an explicit None leaf is invalid. TTLs use integer seconds and deadlines use integer milliseconds. Python names include ttl_sec, request_local, and remote_read_timeout_ms; mappings also accept the shared camelCase names. See the Python guide.

Within each config, the defaults or a runtime overlay, shadow.logMismatches first expands to the shadow.mismatchLogging leaves it stands for, and leaves written beside it win; only then do the leaves merge, runtime over defaults. The mismatch logging table shows the result for each combination.

The policy names in the tables below use the shared JSON configuration shape, also accepted by the Go, Rust and Python policy parsers. Native names and units differ; use the selected language's API reference when constructing typed policy. A configured TTL implies a 100% serving ramp unless overridden. Without a TTL, a local or remote layer is off. Request-local caching and shadow work are off by default. Coalescing defaults to true, but starts no flight when all serving layers are inactive.

Turning features off

Use explicit values to disable inherited policy:

Overlay	Effect on new invocations
requestLocal: false	Bypass request-local lookup and storage
ramp.local: 0	Bypass process-local serving
ramp.remote: 0	Bypass remote serving; shadow admission remains independent
shadow: { ramp: 0 }	Stop new shadow work; inherit the logging preference
staleOnErrorMaxAgeSec: 0	Disable stale recovery
coalesce: false	Use independent cache paths and source deadlines; settled cache hits still apply
Disabled-policy overlay	Disable request-local, recovery, and mismatch logging; set both serving ramps and the shadow ramp to 0

The disabled-policy overlay leaves TTLs and coalesce unset. Inherited TTLs remain inactive under the zero ramps. Replacing that overlay with a later ramp-up coalesces unless another field opts out. Disabling does not cancel admitted work, evict values, or disable explicit invalidation.

Stable key cohorts

Ramp values are thresholds from 0 to 100. 0 disables the layer, 100 enables it for every key, and an intermediate value selects keys whose DialCache-owned deterministic bucket for the full cache key and layer is below that threshold.

For a fixed cache identity and layer, increasing a ramp only adds keys and decreasing it only removes keys; it does not reshuffle existing membership. Local and remote cohorts are layer-specific.

Ramps select key cohorts, not requests or load, so a ramp of 10 does not guarantee 10% of calls, especially for a small or skewed key population. DialCache keeps the assignment stable across releases.

Applications that need an externally coordinated cohort can use the runtime provider to return a sparse per-key ramp override of 0 or 100.

shadow.ramp selects its own stable exact-key cohort, independent of both serving ramps. Shadowing additionally needs a valid remote TTL and a metrics adapter with the outcome hook. shadow.mismatchLogging controls diagnostic warnings separately, choosing the warning's content leaf by leaf, and shadow.logMismatches is shorthand for both leaves, which default to off. Shadow validation explains eligibility, comparison, clean-miss fills, capacity, and the data-handling contract.

Changing policy on a running service

New invocations resolve the current policy. A change does not evict existing values or rewrite their stored expiration times:

Change	Existing entries and work
Lower or raise the local TTL	Existing local entries keep the TTL assigned when inserted. The new TTL applies to subsequent writes. Reads do not refresh that TTL.
Lower or raise the remote TTL	A new Redis read classifies the frame's age using the current remote TTL. The key's physical expiration stays as written; a longer policy does not extend it or restore an expired key.
Change the stale-recovery maximum age	A new Redis read uses the new age policy. Existing keys keep their physical retention; shorter recovery policy restricts reuse without deleting the key.
Set a serving ramp to 0	Bypass that layer without evicting its entries. A later ramp-up can reuse values that remain valid.
Set requestLocal: false	Bypass the current request's memoized values without deleting them. Re-enabling it in that scope can reuse them.
Change TTLs, deadlines, or recovery while a flight is active	An eligible follower can still join the existing flight and inherit its leader's execution; admitted work is not reconfigured.
Return Disabled-policy overlay	Stop new cache use and shadow admission. Existing flights and detached jobs can finish and publish.

For example, reducing a local TTL from 60 seconds to 5 seconds does not make a 20-second-old local entry miss: it keeps its original 60-second lifetime. A Redis frame of the same age is no longer fresh under a new 5-second remote TTL, although a configured recovery policy may still admit it after a source failure.

When an immediate freshness boundary matters, account for every active layer. A local hit bypasses the new remote age policy and the invalidation watermark. See Freshness boundaries and What followers inherit.

Provider behavior

The provider receives the normalized key for every enabled invocation, before cache lookup or joining a flight. Each invocation gets one policy snapshot. Keep the provider cheap; cache external configuration reads inside it and give asynchronous work a finite deadline.

Provider errors run the loader uncached and record config_error; they do not activate defaults. Invalid runtime fields fail open at the affected boundary: for example, an invalid local TTL disables that layer while valid layers can continue. The native API reference specifies validation and error types.

Deadlines

The remote-read deadline resolves from the runtime overlay, then the operation, then the instance, then the library default of 50 ms. It bounds the semantic Redis read and cannot be disabled. The source deadline belongs to the operation and defaults to 60 seconds.

Neither is a total-call budget: config resolution, codecs and writes need their own settlement bounds. See application-owned budgets and the native API reference for duration types and disabling the source deadline intentionally.

Related reference

Operation definitions and validation are in the API reference.

Keys and identity covers components, namespaces, normalization, and provider input.

Direct key construction is in the API reference.

Scopes, storage lifetimes, and value ownership are in the read model.

Compression and coalescing policy are covered by their feature references.