ecluse:ecluse-core
Safe HaskellNone
LanguageGHC2021

Ecluse.Core.Server.Cache.Store

Description

One TTL- and STM-backed single-flight store under a resident-byte budget: the shape every store of the metadata cache (Ecluse.Core.Server.Cache) takes, factored here so the machinery is written once over any key and value.

The cache library supplies the TTL store. Two properties it does not provide on its own are layered here:

  • Resident-byte budget with recency-aware eviction. cache expires by TTL but bounds neither entry count nor memory. Each value is wrapped with an estimate of its resident footprint (from the store's injected weigher) and a last-access stamp bumped on every hit. An insert first purges expired entries, then evicts the least-recently-used entries until the incoming value fits within both the resident-byte budget and the entry-count bound. Recency keeps a re-accessed hot head resident under pressure while shedding the one-shot tail; the byte budget bounds memory more faithfully than a count alone. A value whose weight alone exceeds the byte budget is passed through uncached: the caller still serves it (the per-value ceiling is the caller's concern, an upstream body cap), but nothing resident is evicted to make room that cannot exist, and the store's budget genuinely bounds its residency. Inserts serialise on a per-store lock so two leaders' evict-then-insert sequences cannot interleave past the budget; the lock is post-fetch cold path only, and a leader publishes its marker before inserting, so no follower ever blocks on it.
  • Single-flight. cache's own fetchWithCache is lookup-then-fetch in plain IO, so two concurrent misses would both fetch. resolveSingleFlight instead installs an in-flight marker atomically, so the first miss fetches while concurrent misses wait on its result. The leader inserts the result into the store before removing its marker, so a caller arriving the instant the fetch returns still finds either the store entry or the marker (never a gap) and never re-leads a redundant fetch. A fetch's typed failure is handed to every waiter and caches nothing, and a claimed slot is always eventually filled and de-registered, even when the leader dies to an async exception (see resolveSingleFlight).

The store never knows which cache it serves: the key and value are type parameters, the weigher enters at construction, and telemetry enters per resolution as two callbacks (the hit/miss recording and the post-insert occupancy recording). The domain semantics (what a key means, which upstream a value came from, what may be shared across clients) live entirely with the caller.

Synopsis

The store

data SingleFlight e k v Source #

One TTL- and STM-backed store with the resident-byte budget, the entry-count bound, and the in-flight map that gives single-flight. Opaque: built with newSingleFlight and driven through resolveSingleFlight and the read-only views. Entries are wrapped in Weighted so the byte budget and the least-recently-used eviction have the weight and access stamp they need.

newSingleFlight :: NominalDiffTime -> Int -> Int -> (v -> Int) -> IO (SingleFlight e k v) Source #

Build a store from its tunables (the TTL, the entry-count bound, and the resident-byte budget) and a value weigher. The TTL is converted to the cache library's monotonic TimeSpec and both bounds are clamped to at least one; the access clock starts at zero and the in-flight map empty.

Resolution

resolveSingleFlight :: (Hashable k, Ord k) => IO () -> (CacheResult -> IO ()) -> (CacheOccupancy -> IO ()) -> SingleFlight e k v -> k -> IO (Either e v) -> IO (Either e v) Source #

The store's single-flight resolution: a fresh hit short-circuits; otherwise the caller leads one fetch (installing an in-flight marker) or follows an in-flight one. recordRequest records the hit/miss counter (or ignores it) and recordInsert refreshes the occupancy gauges from the post-insert CacheOccupancy after a leader insert (or ignores it), so each caller wires its own telemetry without the resolution logic knowing which store it serves. The first argument is a hook run on the leading thread at the single-flight claim → fetch-runner handoff, so a test can park a leader in that window deterministically; production callers pass pure ().

A hit bumps the entry's recency stamp before returning it, done in plain IO so recency is updated without writing the shared STM store (and so a hit never contends with a concurrent resolution). On a miss the fetch runs exactly once even under concurrent callers; a successful fetch is cached (subject to the TTL, the entry-count bound, and the resident-byte budget), a failed fetch caches nothing and its typed Left is handed to every waiter. A claimed slot is always eventually filled and de-registered even under an async exception in the claim → runner window: the claim commits under a mask and the run is handed straight to guardInFlight, which frees the slot on every exit and hands the orphaning error to any waiting follower (closing the single-flight orphan window); an async orphan re-resolves under restore (so the retried fetch and its parse stay cancellable, never masked, and the already-recorded miss is not counted again), a synchronous one re-raises (the fetch is total, so that arm is the invariant channel, not an outcome). A follower's own wait stays interruptible. The result is inserted before the slot is de-registered, so a caller arriving the instant the fetch returns becomes a follower rather than re-leading a redundant fetch.

Reads

lookupStore :: Hashable k => SingleFlight e k v -> k -> IO (Maybe v) Source #

Look up a key's stored value without fetching on a miss and without bumping recency: the store's read-only view, for inspection and tests. A Nothing is a miss or an expired entry; this never triggers a fetch and never collapses (use resolveSingleFlight on a serve path).

lookupStoreTouching :: Hashable k => SingleFlight e k v -> k -> IO (Maybe v) Source #

Look up a key's stored value like lookupStore, but bump the entry's recency on a hit: the serve path's read, so an entry read through it stays resident under the least-recently-used eviction rather than ageing out in insert order. It is the same lookup lookupStore runs, followed by the same touch a Hit takes, so a read here and a hit through resolveSingleFlight age an entry identically. Still never fetches and never collapses; a Nothing is a miss or an expired entry. The bump is a plain IORef write (never STM), so a read does not contend with a concurrent resolution.

Occupancy

data CacheOccupancy Source #

A store's occupancy after a leader's insert: the held entry count and their summed resident weight, the values the occupancy and residency gauges report.

Constructors

CacheOccupancy 

Fields