| Safe Haskell | None |
|---|---|
| Language | GHC2021 |
Ecluse.Core.Server.Cache.Store
Description
One TTL- and STM-backed single-flight store under a resident-byte budget: the shape every store of the metadata cache (Ecluse.Core.Server.Cache) takes, factored here so the machinery is written once over any key and value.
The cache library supplies the TTL store. Two properties it does not provide on
its own are layered here:
- Resident-byte budget with recency-aware eviction.
cacheexpires by TTL but bounds neither entry count nor memory. Each value is wrapped with an estimate of its resident footprint (from the store's injected weigher) and a last-access stamp bumped on every hit. An insert first purges expired entries, then evicts the least-recently-used entries until the incoming value fits within both the resident-byte budget and the entry-count bound. Recency keeps a re-accessed hot head resident under pressure while shedding the one-shot tail; the byte budget bounds memory more faithfully than a count alone. A value whose weight alone exceeds the byte budget is passed through uncached: the caller still serves it (the per-value ceiling is the caller's concern, an upstream body cap), but nothing resident is evicted to make room that cannot exist, and the store's budget genuinely bounds its residency. Inserts serialise on a per-store lock so two leaders' evict-then-insert sequences cannot interleave past the budget; the lock is post-fetch cold path only, and a leader publishes its marker before inserting, so no follower ever blocks on it. - Single-flight.
cache's ownfetchWithCacheis lookup-then-fetch in plainIO, so two concurrent misses would both fetch.resolveSingleFlightinstead installs an in-flight marker atomically, so the first miss fetches while concurrent misses wait on its result. The leader inserts the result into the store before removing its marker, so a caller arriving the instant the fetch returns still finds either the store entry or the marker (never a gap) and never re-leads a redundant fetch. A fetch's typed failure is handed to every waiter and caches nothing, and a claimed slot is always eventually filled and de-registered, even when the leader dies to an async exception (seeresolveSingleFlight).
The store never knows which cache it serves: the key and value are type parameters, the weigher enters at construction, and telemetry enters per resolution as two callbacks (the hit/miss recording and the post-insert occupancy recording). The domain semantics (what a key means, which upstream a value came from, what may be shared across clients) live entirely with the caller.
Synopsis
- data SingleFlight e k v
- newSingleFlight :: NominalDiffTime -> Int -> Int -> (v -> Int) -> IO (SingleFlight e k v)
- resolveSingleFlight :: (Hashable k, Ord k) => IO () -> (CacheResult -> IO ()) -> (CacheOccupancy -> IO ()) -> SingleFlight e k v -> k -> IO (Either e v) -> IO (Either e v)
- lookupStore :: Hashable k => SingleFlight e k v -> k -> IO (Maybe v)
- lookupStoreTouching :: Hashable k => SingleFlight e k v -> k -> IO (Maybe v)
- data CacheOccupancy = CacheOccupancy {
- occEntries :: Int
- occBytes :: Int
The store
data SingleFlight e k v Source #
One TTL- and STM-backed store with the resident-byte budget, the entry-count bound,
and the in-flight map that gives single-flight. Opaque: built with newSingleFlight and
driven through resolveSingleFlight and the read-only views. Entries are wrapped in
Weighted so the byte budget and the least-recently-used eviction have the weight and
access stamp they need.
newSingleFlight :: NominalDiffTime -> Int -> Int -> (v -> Int) -> IO (SingleFlight e k v) Source #
Build a store from its tunables (the TTL, the entry-count bound, and the
resident-byte budget) and a value weigher. The TTL is converted to the cache
library's monotonic TimeSpec and both bounds are clamped to at least one; the
access clock starts at zero and the in-flight map empty.
Resolution
resolveSingleFlight :: (Hashable k, Ord k) => IO () -> (CacheResult -> IO ()) -> (CacheOccupancy -> IO ()) -> SingleFlight e k v -> k -> IO (Either e v) -> IO (Either e v) Source #
The store's single-flight resolution: a fresh hit short-circuits; otherwise the
caller leads one fetch (installing an in-flight marker) or follows an in-flight one.
recordRequest records the hit/miss counter (or ignores it) and recordInsert
refreshes the occupancy gauges from the post-insert CacheOccupancy after a leader
insert (or ignores it), so each caller wires its own telemetry without the resolution
logic knowing which store it serves. The first argument is a hook run on the leading
thread at the single-flight claim → fetch-runner handoff, so a test can park a leader
in that window deterministically; production callers pass pure ().
A hit bumps the entry's recency stamp before returning it, done in plain IO so recency is
updated without writing the shared STM store (and so a hit never contends with a concurrent
resolution). On a miss the fetch runs exactly once even under concurrent callers; a
successful fetch is cached (subject to the TTL, the entry-count bound, and the resident-byte
budget), a failed fetch caches nothing and its typed Left is handed to every waiter.
A claimed slot is always eventually filled and de-registered even under an async
exception in the claim → runner window: the claim commits under a mask and the run is
handed straight to guardInFlight, which frees the slot on every exit
and hands the orphaning error to any waiting follower (closing the single-flight orphan
window); an async orphan re-resolves under restore (so the retried fetch and its
parse stay cancellable, never masked, and the already-recorded miss is not counted again),
a synchronous one re-raises (the fetch is total, so that arm is the invariant channel, not
an outcome). A follower's own wait stays interruptible. The result is inserted before
the slot is de-registered, so a caller arriving the instant the fetch returns becomes a
follower rather than re-leading a redundant fetch.
Reads
lookupStore :: Hashable k => SingleFlight e k v -> k -> IO (Maybe v) Source #
Look up a key's stored value without fetching on a miss and without bumping recency:
the store's read-only view, for inspection and tests. A Nothing is a miss or an expired
entry; this never triggers a fetch and never collapses (use resolveSingleFlight on a
serve path).
lookupStoreTouching :: Hashable k => SingleFlight e k v -> k -> IO (Maybe v) Source #
Look up a key's stored value like lookupStore, but bump the entry's recency on a
hit: the serve path's read, so an entry read through it stays resident under the
least-recently-used eviction rather than ageing out in insert order. It is the same
lookup lookupStore runs, followed by the same touch a Hit takes, so a read
here and a hit through resolveSingleFlight age an entry identically. Still never fetches
and never collapses; a Nothing is a miss or an expired entry. The bump is a plain IORef
write (never STM), so a read does not contend with a concurrent resolution.
Occupancy
data CacheOccupancy Source #
A store's occupancy after a leader's insert: the held entry count and their summed resident weight, the values the occupancy and residency gauges report.
Constructors
| CacheOccupancy | |
Fields
| |