Shadow validation

Documentation · Observability

Shadow validation checks cache coherence by comparing sampled Redis values with the source, without serving a shadow result to the caller.

It can also fill Redis misses while remote serving is ramped down, giving you a way to warm a cache before enabling it.

Shadow work is opt-in, sampled by key, and detached. Configure its own ramp and an observer that opts into shadow outcomes. The caller does not await shadow reads, comparison, confirmation, or fills.

What a check establishes

A sampled cached value is compared with a source result. Equality reports match. On disagreement, DialCache reads Redis again: if the payload changed or disappeared, the observation is superseded; if the original payload remains, it reports mismatch. A mismatch is diagnostic and does not trigger repair.

This is a sampled coherence signal, with race boundaries, not proof that every cached value is current. Missing values can be filled as a secondary capability.

Caller paths

Caller path	Additional shadow work	Caller receives
Served Redis hit	Compare that frame with a detached source read	The ordinary Redis hit
Remote serving ramped down	Read Redis and compare with, or fill from, the caller's accepted source result	The source result
Earlier local hit, disabled scope, or missing/invalid remote policy	None	The normal caller result
Normally enabled remote miss	No duplicate shadow fill	The ordinary fallback/refill result

Tracked and untracked keys are eligible. Tracked reads enforce watermarks; untracked reads and fills retain TTL-based last-writer-wins behavior.

Configure a shadow cohort

Choose a valid remote TTL, a zero remote serving ramp, and an independent shadow ramp. The following settings use a five-percent key cohort and capacity four.

This API excerpt assumes redisClient is a semantic adapter, metrics supports shadow outcomes, and db is your application data source:

import { CacheLayer, DialCache, DialCacheKeyConfig } from "dialcache";

const dialcache = new DialCache({
  namespace: "users-api",
  redis: { client: redisClient },
  metrics,
  shadowMaxInFlight: 4,
});

const getUser = dialcache.cached(
  (userId: string) => db.fetchUser(userId),
  {
    keyType: "user_id",
    useCase: "GetUser",
    cacheKey: (userId) => userId,
    trackForInvalidation: true,
    defaultConfig: new DialCacheKeyConfig({
      ttlSec: { [CacheLayer.REMOTE]: 300 },
      ramp: { [CacheLayer.REMOTE]: 0 },
      shadow: { ramp: 5 },
    }),
  },
);

Set Policy.RemoteTTL to 300 * time.Second, RemoteRamp to Ptr(0.0), and Shadow to &ShadowPolicy{Ramp: Ptr(5.0)}. Supply WithRemote, WithShadowCapacity(4) and WithShadowOutcomes alongside your observer/exporter. The outcome hook enables admission; if the exporter already receives all events, do not export the same outcome twice. See observability.

Use a policy with .remote_ttl_sec(300).remote_ramp(0.0) and a ShadowPolicy { ramp: Some(5.0), ..ShadowPolicy::default() }. Configure shadow_max_in_flight(4) on the builder and an observer whose observes_shadow_outcomes returns true. Bundled exporters opt in. See observability.

Use Policy(ttl_sec={"remote": 300}, ramp={"remote": 0}, shadow={"ramp": 5}) and instance shadow_max_in_flight=4. Supply a metrics callback or observer for terminal shadow outcomes. See observability.

Inside an enabled scope, callers use the source. Eligible keys in the independent 5% shadow cohort exercise Redis in the background. A semantic miss authorizes a fill. Use the selected language's outcome opt-in described above.

Keep earlier layers off when the rollout needs to exercise Redis: a request-local or process-local hit ends traversal before shadow admission.

Eligibility

A job needs all of the following:

• A live enabled scope and normal traversal reaching the Redis layer.
• A configured Redis client and a valid remote TTL/policy.
• A served Redis hit, or remote serving disabled specifically by ramped_down.
• A positive shadow ramp whose stable exact-key cohort selects this key.
• An observer opted into shadow outcomes and available capacity.

Missing policy, invalid policy, provider failure, or an omitted outcome hook cannot start a shadow-only path. Mismatch logging settings do not enable one either.

Serving and shadow cohorts are independent. Equal partial percentages do not select the same keys. A partial shadow ramp therefore does not guarantee every key admitted by a later serving ramp was validated or warmed. Even at 100%, capacity, lifetime, and earlier-hit gates still apply.

Serving-hit and ramped-down paths

On a served hit, DialCache retains the serialized frame that supplied the caller as C0, returns the normal decoded result, then starts detached source work. That loader runs with caching disabled for this DialCache instance, so nested readers through the instance bypass caching. The comparator later receives an independently deserialized cached value.

On a ramped-down path, the caller starts and awaits its normal source loader exactly once. Detached work reads C0 and shares that caller-accepted source result, S; it does not launch another loader. A source rejection or timeout never becomes an accepted fill value. This foreground loader retains the caller's context; it is not rerun inside the served-hit branch's disabled scope.

The C0 / S / C1 algorithm

C0 is the initial cached observation, S the source result, and C1 an optional confirmation read after a disagreement:

C0 miss ───────────────→ accepted S → eligible fill
C0 present → compare S
                 ├─ equal ────────→ match
                 └─ different → C1
                                  ├─ absent / changed → superseded
                                  └─ same payload ───→ mismatch

Clean-miss fill

A semantic C0 miss can be filled from S. For a tracked miss carrying a valid observedWatermarkMs, DialCache checks the application timestamp before serialization and again immediately before dispatch. A timestamp at or below that observation skips the fill and emits fill_fenced. Preflight suppression also avoids serialization, compression, and frame allocation.

An admitted fill uses the final timestamp exactly and sends one complete-frame SET. Without an observed fence, the adapter samples its normal dispatch-time timestamp. The physical TTL follows ordinary write policy, including M when stale recovery is active and the separate one-hour tracked cap.

A semantic miss includes absent, unsupported, logically expired, future-dated, and watermark-fenced frames. It does not include a present payload that fails load: that is deserialization_error, with no repair. Miss reason and observed fence are independent. Core classifies logical expiry after the adapter decodes the wire frame, before the application codec decoder runs. That expired miss does not carry a watermark fence; it follows normal refill behavior.

filled means the client accepted the write before the deadline. fill_error means payload preparation or writing failed. Neither proves what remains in Redis afterward. fill_fenced means this job skipped dispatch against its observed fence; another operation may still change Redis.

Comparison and confirmation

For present C0, DialCache deserializes an independent cached snapshot after the source result is available. Equal values report match. A disagreement triggers one direct Redis C1 read in the same tracked or untracked mode.

If C1 is a miss or its payload differs byte-for-byte, the result is superseded. If the original payload remains, the result is mismatch. Confirmation failure or read timeout produces confirmation_error.

Confirmation bypasses logical age solely for supersession comparison. A future-dated C1 records its offset but can be retained for comparing bytes; it cannot serve the caller. DialCache does not deserialize C1, compare it with S, or chase another version. Payloads compare by their exact stored bytes; TypeScript string payloads use their UTF-8 representation when compared with binary payloads.

Any non-null C0 is observation-only. Shadow validation never repairs a mismatch or overwrites an undecodable present value.

Comparison semantics

Comparison is a native binding boundary. Use a deterministic, bounded, side-effect-free comparator, and never mutate its inputs. Each port provides a default; these are not equivalent for every language-native value.

The default comparator is Node's util.isDeepStrictEqual. Object property insertion order does not matter; values, array order, prototypes, constructors, and collection contents remain part of strict equality.

This API excerpt assumes a source function fetchVersionedUser and compares only domain identity and version:

const getVersionedUser = dialcache.cached(fetchVersionedUser, {
  keyType: "user_id",
  useCase: "GetVersionedUser",
  cacheKey: (userId) => userId,
  shadowComparator: (cached, source) =>
    cached.id === source.id && cached.version === source.version,
  defaultConfig: new DialCacheKeyConfig({
    ttlSec: { [CacheLayer.REMOTE]: 300 },
    shadow: { ramp: 5 },
  }),
});

The comparator must synchronously return a boolean and be deterministic, side-effect-free, non-mutating, and bounded. Throws and non-boolean results produce comparison_error. Accidental thenables are consumed while retaining the shadow slot until settlement, subject to the job deadline.

The default is SemanticEqual, which implements the supported portable value domain. Set Operation.Comparator for application-specific equality; it returns (bool, error). Errors and callback panics report comparison_error. A native Go type or map representation still needs a codec and comparator that agree on its intended meaning.

The default is PartialEq for the result type. Native equality can differ from TypeScript: NaN differs from itself in Rust. Supply the operation's comparator for domain-specific equality, such as comparing entity id and version. Callback panics report comparison_error; returned values must be safe to share.

The default recursively compares JSON-like values and distinguishes booleans from numbers. Use shadow_comparator for custom native domains; it must return a synchronous boolean. Exceptions and non-boolean results report comparison_error.

Comparison uses the decoded cache value and raw source value intentionally: it can reveal lossy serialization. Ignore differences only when they are acceptable application semantics.

Data ownership and custom integrations

A sampled served hit runs the codec decoder again in detached work. It must be repeatable, non-mutating, and return independently usable values. Returned Redis payload bytes must remain stable after the adapter read settles.

DialCache retains source arguments or the inline loader closure. It cannot generically clone source-selection state. Keep arguments, captured state, and accepted source values immutable, or snapshot before the invocation, so detached work still refers to the key that was selected.

The caller's decoded hit object is not reused for comparison. No shadow copy, hash, or deep comparison is added to the served-hit request path.

Capacity, deadlines, and detachment

The instance shadow capacity defaults to 1 per instance and must be a positive safe integer. Same-key duplicates and jobs beyond capacity report dropped; there is no queue or fleet-wide cap. Confirmation and fill stay in the original slot.

Each job has one monotonic budget across C0, the source, serialization, comparison, C1, and fill. A finite source deadline is reused as that budget. When the source budget is unbounded, shadow still uses 60 seconds. Each Redis read separately has the effective remote-read deadline.

Served-hit timing starts with the detached callback. Ramped-down timing starts immediately before the caller's source invocation, including its synchronous prefix. On abandonment, DialCache releases retained C0 and stops later phases. Already-started shadow-owned work keeps capacity until it settles, even after its DialCache deadline. A shared caller-owned loader can continue without holding the shadow slot after timeout.

Scheduling and shadow deadline timers are unreferenced and do not keep an otherwise idle Node process alive. Detachment uses the event loop, not a worker; synchronous source, codec, comparator and logger work still consumes it, as do mismatch-warning rendering and shadowMismatchDiff.

Shadow work runs independently of the caller. Native callbacks still consume CPU and can outlive the cache's bounded wait. Give dependency calls finite context deadlines and bound application concurrency. A job holds its capacity until its verdict is reported, on the shadow job's goroutine: while the WithShadowOutcomes hook, observers and metrics adapters receive its shadowValidation and shadowAge events, and while its mismatch warning is built and logged, the built-in walk and Operation.MismatchDiff included. A slow callback there therefore shows up as dropped jobs beyond capacity.

The captured runtime runs detached shadow tasks. Admitted CPU work can retain capacity even after an async runtime shutdown. Mismatch-warning content runs through the bounded CPU executor too: JSON rendering, the built-in walk, preview and the mismatch_diff hook can run on worker threads and hold the job's capacity until they finish. Application codecs control their own scheduling.

The asyncio event loop owns detached shadow work. Keep it alive while work settles. Loaders, codecs and compression execute on that event loop; use native async I/O and application-controlled scheduling for blocking custom work. Mismatch-warning content, the built-in walk and shadow_mismatch_diff included, also runs there, after the job releases its slot.

Underlying I/O is not generally canceled. Give dependencies finite native budgets, including commands that can settle after a DialCache timeout.

Consistency modes and race boundaries

The initial read and later fill are not atomic. An allowed fill can overwrite a concurrent writer; it is not write-if-still-missing. For tracked keys, the next primary read applies the current watermark, including one advanced after C0. Size the future buffer for the complete source, serialization, dispatch, and clock-skew window.

An untracked fill has no watermark fence. A detached older source value can be written after a mutation and remain until expiry. Untracked reads also have no shadow-specific primary guarantee.

A confirmed mismatch means the original payload survived a second Redis read after disagreement with the source. It is useful evidence, not an atomic cross-system snapshot or a promise that the mismatch still exists.

Command amplification

Selected path	Extra work
Served Redis hit	One source read; one C1 only after disagreement
Ramped-down Redis hit	One C0, reuse caller source, one C1 only after disagreement
Ramped-down Redis miss	One C0, reuse caller source, at most one fill SET

A fenced fill adds no write; preflight fencing also avoids payload preparation. No path adds a separate fence read. Coalescing suppresses duplicate caller paths; shadow deduplication drops jobs but does not coalesce source calls when all serving layers are disabled.

Confirmed mismatch logging

Opting into mismatch warnings adds one diagnostic only after a confirmed mismatch. Logging is default-off and independent of sampling. It can contain cache keys and value previews, so enable it only for data appropriate to your application's logging policy.

Two default-off leaves under shadow.mismatchLogging choose what the warning carries, and the warning is emitted when either is on:

Leaf	Values	Adds to the warning
diff	"off", "paths", "values"	"paths": where the values differ. "values": also what each side holds there
value	true, false	Native JSON of both compared values

diff: "paths" logs where the values differ but not what they hold, so it is the level meant to stay on for user data. Member names still appear in the pointers: an object keyed by identifiers logs those identifiers, exactly as written, control characters included, so send warnings to a structured or escaping logger. cacheKey still carries the key's arguments as well. This config excerpt keeps paths-only logging on for a 5% shadow cohort:

shadow: { ramp: 5, mismatchLogging: { diff: "paths" } }

An invalid leaf in defaultConfig throws when cached() registers or getOrLoad() runs; see validation and snapshots.

Shadow: &dialcache.ShadowPolicy{
	Ramp:            dialcache.Ptr(5.0),
	MismatchLogging: &dialcache.MismatchLoggingPolicy{Diff: dialcache.Ptr(dialcache.MismatchDiffPaths)},
},

MismatchLoggingPolicy holds Value, a *bool, and Diff, a *MismatchDiffLevel: MismatchDiffOff, MismatchDiffPaths or MismatchDiffValues. ShadowPolicy.LogMismatches is the shorthand described below. Static validation rejects any other Diff, and a JSONPolicy or RawPolicy overlay uses the JSON names on this page.

ShadowPolicy {
    ramp: Some(5.0),
    mismatch_logging: Some(MismatchLogging {
        diff: Some(MismatchDiff::Paths),
        ..MismatchLogging::default()
    }),
    ..ShadowPolicy::default()
}

MismatchLogging holds value: Option<bool> and diff: Option<MismatchDiff>, whose variants are Off, Paths and Values. log_mismatches is the shorthand described below, and a runtime overlay uses the JSON names on this page.

shadow={"ramp": 5, "mismatch_logging": {"diff": "paths"}}

log_mismatches is the shorthand described below, and mappings also accept the camelCase mismatchLogging and logMismatches. Static defaults reject an invalid shorthand or leaf with ConfigError.

diff: "values" and value: true log application data and suit an investigation.

shadow.logMismatches is shorthand for the whole group: true means { value: true, diff: "values" } and false means { value: false, diff: "off" }. A leaf written in the same shadow object overrides what the shorthand expands to. Only then do the leaves merge with the other configuration layer, runtime over defaults:

Written	Effective
{ logMismatches: true, mismatchLogging: { value: false } }	{ value: false, diff: "values" }
{ logMismatches: false, mismatchLogging: { diff: "paths" } }	{ value: false, diff: "paths" }
Default mismatchLogging: { diff: "paths" }, runtime logMismatches: false	Both off
Default mismatchLogging: { value: false }, runtime logMismatches: true	{ value: true, diff: "values" }

To log less than logMismatches: true, replace the shorthand with leaves rather than writing a leaf beside it. A leaf beside logMismatches: true leaves the other expanded leaf on, and DialCache versions before mismatchLogging read only the shorthand, so a service still on such a version keeps logging both values from the same config. At runtime, an invalid leaf only turns that leaf off and an invalid shorthand expands to both leaves off; one remote config_resolution covers every invalid input of a call and is recorded only when a shadow job is admitted. A runtime mismatchLogging that is not an object fails resolution for the whole call, which then runs the loader uncached.

The warning carries these fields:

Field	Present	Content	UTF-8 cap
cacheNamespace, useCase, keyType, outcome: "mismatch"	Always	Use-case metadata
cachedValueAgeSeconds	Always	Age of the cached value at the verdict, as the shadow value-age metric records it; null when unavailable
cacheKey	Always	Logical URN, not the physical Redis key	2 KiB
mismatchPaths	diff above "off"	RFC 6901 JSON Pointers of the differing fields, at most 64	512 bytes each
mismatchPathCount	diff above "off"	Number of differing fields before the cut to 64
diffJson	diff: "values"	JSON array of { path, cached, source } entries for the same pointers; a side the field lacks is omitted	8 KiB, 1 KiB per side
cachedValueJson	value: true	Native JSON of the decoded cached snapshot	8 KiB
sourceValueJson	value: true	Native JSON of the raw source value	8 KiB
loggingOutcome	After a failure	"hook_error" or "render_error": why fields are null

The logger's warn receives the message "DialCache shadow validation mismatch" and an object with these fields, in this order.

The logger's Warn receives the message "DialCache shadow validation mismatch" and a map[string]any with these fields, so they have no order. cachedValueAgeSeconds is a float64, mismatchPaths a []string and mismatchPathCount an int; a field that failed is a nil interface value. Without WithLogger, the standard logger writes each warning on one line through the log package, with the fields as one JSON object, so mismatchPaths [""] reads differently from [] and a failed field is null. In that object, control characters, DEL, U+2028 and U+2029 are JSON escapes and a byte that is not UTF-8 becomes U+FFFD, so a member name cannot break the line or send a terminal escape, and the object still parses.

The logger receives LogEvent::ShadowMismatch with ShadowMismatchDetails: cached_value_age_seconds, cache_key, a diff group present when diff is above Off (mismatch_paths, mismatch_path_count and, at Values, diff_json), a values group present when value is on (cached_value_json, source_value_json), and logging_outcome. When the content work does not complete, as when the CPU executor rejects it, both groups are None whatever the leaves request, and logging_outcome is RenderError. Inside a present group, None marks an unavailable field. The default LogFacadeLogger writes the event's Display: name=value pairs in the order of the table, without outcome, with namespace, cachedValue and sourceValue in place of cacheNamespace, cachedValueJson and sourceValueJson, and with unavailable fields left out.

The logger's warning receives the message "DialCache shadow validation mismatch: %s" and one dict with these fields, in this order; a field that failed is None.

For example, take the GetUser use case configured above, with mismatchLogging: { diff: "paths" } added to its shadow policy. It has this user cached, as decoded from Redis:

{"id": "u_123", "email": "ana@example.com", "plan": "free",
 "roles": ["viewer"], "profile": {"name": "Ana", "locale": "en-US"}}

The source has changed the user since and now returns:

{"id": "u_123", "email": "ana@example.com", "plan": "pro",
 "roles": ["viewer", "editor"],
 "profile": {"name": "Ana", "locale": "en-GB", "timezone": "Europe/London"}}

After the mismatch is confirmed, the warning carries these fields:

{
  "cacheNamespace": "users-api",
  "useCase": "GetUser",
  "keyType": "user_id",
  "outcome": "mismatch",
  "cachedValueAgeSeconds": 42.5, // varies
  "cacheKey": "{users-api:user_id:u_123}#GetUser",
  "mismatchPaths": ["/plan", "/profile/locale", "/profile/timezone", "/roles"],
  "mismatchPathCount": 4
}

/id, /email, and /profile/name match, so they are absent. The paths follow member names in UTF-16 order, not the order the object was written in, so /roles comes after /profile. Only the source has /profile/timezone, and /roles changed length, so that array is reported once, whole, rather than element by element. With diff: "values" the same warning also carries diffJson, shown formatted here; the logged string is compact:

[
  { "path": "/plan", "cached": "free", "source": "pro" },
  { "path": "/profile/locale", "cached": "en-US", "source": "en-GB" },
  { "path": "/profile/timezone", "source": "Europe/London" },
  { "path": "/roles", "cached": ["viewer"], "source": ["viewer", "editor"] }
]

The /profile/timezone entry carries source alone because the cached value has no such member.

The paths come from the native JSON of both values: each side is rendered once with the language's native JSON writer, not the Redis serializer, and every field above derives from that rendering, so a member the rendering leaves out of cachedValueJson stays out of the diff too. The diff is not limited to the 8 KiB that cachedValueJson keeps: for a larger value it can show differing fields from past that point. The two renderings are compared structurally:

• Object members are compared in UTF-16 code-unit order of their names. A member present on one side only is reported with that side.
• Arrays of equal length are compared by index. Arrays of different length are reported once, at the array's own pointer, with both arrays; DialCache does not align their elements.
• Any other differing pair is reported at its pointer with both sides, and roots of different kinds are reported at the root pointer "".
• Numbers are compared as IEEE doubles, so 1 matches 1.0 and 0 matches -0. true and false never match a number.
• The comparison descends at most 32 levels: a differing pair of containers 32 levels down is reported whole at its own pointer.

An empty mismatchPaths means both renderings hold the same JSON, apart from member order and number precision beyond a double: the comparator saw a difference that native JSON does not show, such as a native type the rendering erased, or a custom comparator compared more than the JSON. A side longer than 1 KiB appears in diffJson as its clipped JSON text. Clipped fields end with ...[truncated] inside their cap. The caps bound what is logged, not the work: the computation parses and walks both renderings in full, so its CPU time and memory grow with the rendered size of the values and with the number of members and elements they hold.

Each smaller pair below shows one rule:

Cached	Source	mismatchPaths	Why
{"plan":"free"}	{"plan":"pro"}	["/plan"]	A changed field
{"tags":["a","b"]}	{"tags":["a","c"]}	["/tags/1"]	Same length, so elements are compared by index
{"tags":["a","b"]}	{"tags":["a","x","b"]}	["/tags"]	Different length: the whole array, once
{"name":"Ana"}	{"name":"Ana","nickname":"A"}	["/nickname"]	Only the source has it, so its diffJson entry carries source alone
{"name":"Ana","nickname":"A"}	{"name":"Ana"}	["/nickname"]	Only the cache has it, as after the source stops returning a field; its diffJson entry carries cached alone
{"count":1}	{"count":"1"}	["/count"]	A number against a string
{"address":{"city":"Paris"}}	{"address":null}	["/address"]	An object against null is reported whole
{"a/b":1,"m~n":1}	{"a/b":2,"m~n":2}	["/a~1b","/m~0n"]	RFC 6901 escapes / as ~1 and ~ as ~0
["a","b"]	{"items":["a","b"]}	[""]	Roots of different kinds, at the root pointer
Absent	{"plan":"pro"}	[""]	A cached negative-cache entry is an absent value, not a failure: the source is reported whole at the root, and cachedValueJson is null

A use case can replace the computation with a diff hook, set beside its shadow comparator. The hook receives the compared values, an absent one included as the language's own absent marker, and returns the entries, each a path with optional cached and source sides; a hook that knows only where values differ returns paths alone. It follows the comparator's rules: synchronous, deterministic, side-effect-free, non-mutating, and bounded. Its entries are the use case's own output, so they log whatever it returns.

Failures log less, never more. A value without a JSON rendering makes its value field null, along with the built-in diff fields. An absent value, the top-level marker a negative-cache entry stores, is not such a failure: only its own value field is null and the warning carries no loggingOutcome. A hook that fails makes mismatchPaths, mismatchPathCount, and diffJson null; DialCache does not fall back to the built-in computation. A hook value without a JSON rendering makes diffJson alone null. Any other failure while building the fields still logs the warning, with render_error: TypeScript writes every requested field as null, and Go, Rust and Python log the metadata alone. Failures never change the shadowValidation outcome, and the shadowMismatchLogging metric reports each warning attempt; see Confirmed mismatch warnings.

Each side is rendered with JSON.stringify, so a member that toJSON leaves out of cachedValueJson stays out of the diff too. A cycle, a bigint, or a toJSON that throws has no rendering, and a top-level undefined is the absent value. The default comparator tells a string from a Date, but both render to the same JSON: a Date that the JSON serializer stored comes back as its ISO string, so {"at":"2026-10-24T00:00:00.000Z"} against { at: new Date("2026-10-24T00:00:00.000Z") } is a mismatch whose mismatchPaths is [].

shadowMismatchDiff sets the hook; cached() and getOrLoad() throw a TypeError when it is not a function. A hook that throws, returns a promise or returns a non-array fails. DialCache logs the first 64 entries as the hook returns them, so a wrong entry logs a wrong path. This API excerpt assumes a fetchUser source:

const getUser = dialcache.cached(fetchUser, {
  keyType: "user_id",
  useCase: "GetUser",
  cacheKey: (userId) => userId,
  shadowMismatchDiff: (cached, source) =>
    cached.email === source.email ? [] : [{ path: "/email" }],
});

Rendering, the walk, and the hook run on the event loop after the job releases its capacity slot. On one Apple-silicon Mac, a 6.3 MB object of 150,000 members took about 0.12 seconds per warning at diff: "paths", against 0.06 seconds for the value fields alone.

Each side is rendered with the JSON writer JSONCodec uses, not the operation's Codec. It writes maps, slices, JSONObject values and scalars by their kind, without calling their MarshalJSON, and renders a struct, with everything inside it, through encoding/json, parsing the output back, so a struct's MarshalJSON runs once. A cycle, a *big.Int, a channel, a function, or a struct whose MarshalJSON fails, panics or writes a number outside double range, or whose JSON nests more than 10,000 levels, has no rendering, and Absent is the absent value. A map, slice or JSONObject nested that deep does render, but the built-in computation cannot parse the rendering back and fails with hook_error.

Operation.MismatchDiff, beside Comparator, sets the hook: func(cached, source T) ([]MismatchEntry, error). A MismatchEntry side set to Absent is omitted and nil is JSON null, so an entry that only names a location sets both sides to Absent; a nil slice is an empty list. A side renders like a compared value, except that the side or any value inside it that implements json.Marshaler, such as json.RawMessage or a redacting type, renders through its MarshalJSON, whose output is parsed back rather than copied. A MarshalJSON that fails, panics, or writes invalid JSON or a number outside double range leaves the side with no rendering, and a *big.Int side still has none. The hook fails by returning an error or panicking.

Rendering, the walk, and the hook run on the shadow job's goroutine after the verdict and hold the job's shadow capacity until they finish, so while a slow hook runs, jobs beyond capacity report dropped. On one Apple-silicon Mac, a 6.3 MB map of 150,000 members took about 0.6 seconds per warning at MismatchDiffPaths, most of it rendering: the value fields alone took about 0.5 seconds.

An operation built with Operation::new, or registered with register, renders each side once with serde_json, and its default preview logs that JSON. Setting preview to None logs no values, as before: like an operation built with with_codec or registered with register_custom, which has no JSON rendering, it gets no built-in diff. A custom preview replaces the JSON for the whole warning, so the value fields come from the preview and the diff fields only from a mismatch_diff hook; the built-in walk never reads what a redacting preview leaves out. A field logged without its renderer, or a preview that returns None or panics, is unavailable with render_error. A value whose Serialize fails has no rendering, so its value field and the built-in diff fields are unavailable with render_error. The built-in computation parses each rendering back with serde_json and fails with hook_error on one it cannot parse: a rendering nested 128 or more levels deep, which reaches the parser's recursion limit, or, in a build that enables serde_json's raw_value feature without arbitrary_precision, a RawValue holding a number outside double range. Rust's default codec stores no top-level undefined value, so an absent side does not arise for native values.

Operation::mismatch_diff and UseCaseBuilder::mismatch_diff set the hook, Fn(&T, &T) -> Vec<MismatchEntry>, whose entries carry a path and optional serde_json::Value sides. A panic is its only failure, and its sides always render.

Rendering, the walk, preview, and the hook run through the bounded CPU executor after confirmation and hold the job's shadow capacity until they finish; they can run on a worker thread. When the executor rejects the work, or the work panics or is dropped, the warning keeps its metadata, both field groups are None, and it carries render_error. On one Apple-silicon Mac, a 6.3 MB map of 150,000 entries took about 0.09 seconds per warning at MismatchDiff::Paths in a release build, against 0.006 seconds for the value fields alone.

By default the executor is one process-wide pool of two worker threads and two queued jobs, which every cache instance also uses for zstd decompression and for compressing large payloads. While warnings occupy it, a compressed hit whose decompression it rejects falls through to the source, and a write whose compression it rejects is skipped. At most shadow_max_in_flight warnings per instance, times the number of instances, contend for it; a custom Runtime can supply its own executor.

Each side is rendered once with json.dumps, using compact separators, keeping non-ASCII text, and escaping lone surrogates as \uXXXX. Tuples render as arrays, so an empty mismatchPaths can mean a cached list against a source tuple, which the default comparator tells apart. NaN, an infinity, a set, bytes, another non-JSON object, a nested UNDEFINED, or a cycle has no rendering, so a datetime, Decimal or UUID in a compared value makes the diff fields None with render_error unless shadow_mismatch_diff is set. A top-level UNDEFINED is the absent value, and None is JSON null.

shadow_mismatch_diff=, set beside shadow_comparator on cached() or get_or_load(), sets the hook. It returns a list of mappings, each with a str "path" and optional "cached" and "source" sides; a missing key or UNDEFINED omits a side. A hook that raises, returns an awaitable (a coroutine is closed without running), returns something other than a list, or returns one of the first 64 entries without a "path" key fails. Other entries are logged as returned, with str() of each path. A non-callable value raises ConfigError.

Rendering, the walk, and the hook run on the event loop after the job releases its capacity slot. On one Apple-silicon Mac, a 6.3 MB dict of 150,000 members took about 0.18 seconds per warning at "paths", against 0.04 seconds for the value fields alone.

Truncation is not redaction. Keys and values can contain sensitive application data. Field byte limits do not bound rendering, parsing, or the walk. Logger failures are isolated from cache behavior.

Metrics and shutdown

Shadow outcomes distinguish matches, confirmed mismatches, superseded observations, fills, fences, errors, timeouts, and drops. Only match and mismatch report observed value age at verdict time.

Detached Redis and serializer work uses layer="remote_shadow". The original served read keeps remote; a ramped-down caller keeps its ordinary disabled observation. Shadow reads time Redis settlement, while ordinary remote get latency includes fresh deserialization. Optional age metrics do not gate jobs; only the native shadow-outcome opt-in does.

Turn down both serving and shadow ramps to stop new Redis work, or use an explicit disabled-policy overlay. Already-admitted work is not cancelled. There is no public shadow drain handle; follow client lifecycle when shutting down and treat outcomes during teardown as best-effort.