ecluse:ecluse-core
Safe HaskellNone
LanguageGHC2021

Ecluse.Core.Server.Cache

Description

The short-TTL, size-bounded metadata cache shared by the serve paths.

Resolving a package re-fetches its upstream packument, parses it, and evaluates the rules. To avoid repeating the fetch and parse, the result (a coherent pair of the parsed packument metadata, PackageInfo, and the raw document it was decoded from, CacheEntry) is held here in a short-TTL, size-bounded, STM-backed cache (the cache library backs the TTL store). Both serve paths share it: a packument request and the tarball-gating fetch that follows reuse one fetch and parse, and concurrent resolutions of a popular package __collapse to one upstream call__ (single-flight).

Per-source key

A packument is fetched from two distinct upstreams, a private origin and a public origin, whose documents differ for the same package, so one entry cannot represent both. The key is (source, package): the source is the upstream's base URL, which distinguishes any cached origin without naming a credential, so distinct upstreams never cross-contaminate and the key never blurs the trust split.

Credential-free; sharing is the caller's policy

The key carries no credential dimension and the value is a canonical document, so the cache stores nothing derived from a caller's credential. Whether a given origin is handed to it, and so shared across clients, is the serve path's decision.

Under the default passthrough access strategy only the anonymous public origin is cached. The trusted private upstream is the per-client authority: it re-authorises each request with that client's own forwarded credential, so the serve path fetches it per request and never hands it here. Were a private entry cached under passthrough, the credential-free key would let one client's entry serve another client's private document within the TTL, bypassing the upstream's authorisation. The public origin is anonymous, so one shared entry serves every client without crossing a trust boundary. Other strategies make a shared private entry safe by authorising each serve before it is returned (see docs/architecture/access-model.mdCaching); that gate lives on the serve path, never in this store.

Coherent pair

An entry holds the parsed PackageInfo and the raw document (CachedDoc) it was decoded from, so a hit returns a typed view and the exact bytes that produced it. The packument serve path needs both: it decides over the typed view but rebuilds the served body from the raw document, and the two must describe the same fetch. The store holds the raw document opaquely -- it never reads it, only hands it back to the injected adapter capabilities that assemble and serialise the served body.

What is cached is the metadata, not the verdict. The rules are re-evaluated on the cached metadata each request, so time-sensitive rules (AllowIfOlderThan) and the separately-synced advisory tier stay correct; only each upstream's fetch and parse is memoised. The TTL is short, and brief staleness is benign: a brand-new publish need not appear instantly (see docs/architecture/web-layer.md → "Metadata cache").

Two properties the cache library does not provide on its own are layered onto every store by the shared machinery (Ecluse.Core.Server.Cache.Store):

  • Resident-byte budget with recency-aware eviction. cache expires by TTL but bounds neither entry count nor memory. Each entry is wrapped with an estimate of its resident footprint (a heavy packument, parsed plus raw, costs many times its wire size) and a last-access stamp bumped on every hit. An insert first purges expired entries, then evicts the least-recently-used entries until the incoming entry fits within both its store's resident-byte budget and entry count (StoreBudget). Recency keeps a re-accessed hot head resident under pressure while shedding the one-shot tail; the byte budget bounds memory more faithfully than a count alone. An entry whose estimated footprint alone exceeds its store's byte budget is served without being retained (nothing resident is evicted to make impossible room), so one pathological document can never flush a store.
  • Single-flight. cache's own fetchWithCache is lookup-then-fetch in plain IO, so two concurrent misses would both fetch. resolveMetadata instead installs an in-flight marker atomically, so the first miss fetches while concurrent misses wait on its result. The leader inserts the result into the store before removing its in-flight marker, so a caller arriving the instant the fetch returns still finds either the store entry or the marker (never a gap) and never re-leads a redundant fetch.

Two coherent stores: the full packument and one version

This handle owns two stores of the same shape (the TTL + size-bound + single-flight machinery, SingleFlight, is shared between them):

They are isolated on writes: a single-version resolution caches under its own key and never writes back to the full-packument store, so a cold tarball gate cannot materialise a whole packument into the shared full cache. The serve path's single-version read consults the warm full-packument store read-only first (a packument GET followed by its tarball gate still collapses to one upstream call), and only falls back to leading its own selective fetch into the version store when the full entry is cold. Each store enforces its own named sub-budget (StoreBudget): the three sub-budgets are carved from one cache aggregate at the composition root and sum to it, so the aggregate holds by arithmetic while each class's eviction stays isolated (a version-store flood can never evict the full store's hot head). Each reports its own residency gauge: the full-packument store under ecluse.metadata_cache.resident_bytes and the single-version store under ecluse.metadata_cache.version.resident_bytes. The hit/miss counter and the entry-count occupancy gauge stay about the full-packument store.

A third store memoises the assembled representation (resolveAssembled): the encoded merged document, keyed by its derived validator (packumentETag). The key is a fingerprint of every input the document is a function of (the origin bodies, private included by content digest; the survivor sets; the mount base), which makes the store content-addressed: an entry can never be served stale, because changed inputs produce a different key and simply miss. The resident-byte budget is the real bound here, not the TTL, which only trims dead entries early. Cross-client safety follows from the same property: a lookup key includes the digest of the private document this request's own authorised fetch returned, so a client can only hit an entry whose bytes its own inputs would deterministically re-produce. The transform is shared, never the authorisation and never another client's view (the private-origin caching prohibition is about credential-blind keying, which a content key is not). Residency gauge: ecluse.metadata_cache.assembled.resident_bytes.

Synopsis

Configuration

data CacheConfig Source #

The metadata cache's tunables, sourced from configuration: how long a parsed packument stays fresh, and each store's own StoreBudget. The three sub-budgets are carved from one cache aggregate at the composition root (Ecluse.Composition.MemoryBudget.budgetCacheConfig) and sum to it, so the cache's total resident bytes are bounded by the aggregate while each class's eviction pressure stays its own.

Constructors

CacheConfig 

Fields

Instances

Instances details
Show CacheConfig Source # 
Instance details

Defined in Ecluse.Core.Server.Cache

Eq CacheConfig Source # 
Instance details

Defined in Ecluse.Core.Server.Cache

data StoreBudget Source #

One store's bounds: the entry count and the resident-byte budget it keeps its held entries under before it evicts. Each entry is weighted by an estimate of its resident footprint, and an insert past the byte budget evicts the least-recently-used entries until the budget holds -- bounding memory more faithfully than the entry count alone.

Constructors

StoreBudget 

Fields

  • sbMaxEntries :: Int

    The maximum number of distinct entries held; an insert past this evicts.

  • sbMaxBytes :: Int

    The resident-byte budget the held entries are kept under.

Instances

Instances details
Show StoreBudget Source # 
Instance details

Defined in Ecluse.Core.Server.Cache

Eq StoreBudget Source # 
Instance details

Defined in Ecluse.Core.Server.Cache

The cache handle

data MetadataCache Source #

The metadata-cache handle: the three single-flight stores (the full-packument cache, the single-version cache, and the assembled-representation store). Opaque: built with newMetadataCache and reached only through the accessors. Lives in the composition root (one per process), so every request shares the same caches and their connection-collapsing.

newMetadataCache :: CacheConfig -> IO MetadataCache Source #

Build a metadata cache from its configuration: the full-packument store, the single-version store, and the assembled-representation store, each over the same TTL but sized from its own sub-budget (the three used to share one bound, which tripled the intended cache footprint in the worst case).

Cache entries

newtype Source Source #

Which upstream a cached packument was fetched from: the dimension that partitions the cache by source so distinct upstreams never share an entry.

The discriminator is the upstream's base URL: an upstream is addressed at a distinct URL, and the URL names a location, never a credential, so keying on it keeps the trust split intact (the cached origin is fetched with its own token, supplied through its fetch action; the source carries none). Under the default passthrough strategy only the anonymous public origin is cached, so in practice the cache holds one source per package; the dimension keeps the key honest about which upstream an entry is, never blurring the split.

Constructors

Source Text 

Instances

Instances details
Show Source Source # 
Instance details

Defined in Ecluse.Core.Server.Cache

Eq Source Source # 
Instance details

Defined in Ecluse.Core.Server.Cache

Methods

(==) :: Source -> Source -> Bool #

(/=) :: Source -> Source -> Bool #

Ord Source Source # 
Instance details

Defined in Ecluse.Core.Server.Cache

data CacheEntry Source #

A coherent cache entry: the parsed PackageInfo paired with the raw document (CachedDoc) it was decoded from. A hit returns both, so a caller gets a typed view to decide over and the exact bytes that produced it: the packument serve path rebuilds the served body from the raw document and must keep its typed decision coherent with those bytes. The store holds the raw document opaquely -- it never reads it, only weighs it (weighCachedDoc) and hands it back to the injected adapter capabilities.

Constructors

CacheEntry 

Fields

  • entryInfo :: PackageInfo

    The typed packument view the rules and merge reason over.

  • entryRaw :: CachedDoc

    The raw upstream document the served body is built from.

  • entryDigest :: ContentDigest

    Digest of the wire bytes both views were decoded from, computed once at the leader's fetch -- the public origin's contribution to the serve path's derived ETag, amortised across every hit on this entry.

Instances

Instances details
Show CacheEntry Source # 
Instance details

Defined in Ecluse.Core.Server.Cache

Eq CacheEntry Source # 
Instance details

Defined in Ecluse.Core.Server.Cache

weighCacheEntry :: CacheEntry -> Int Source #

Estimate a CacheEntry's resident footprint in bytes as a fixed multiple of its raw document's compact-encoded byte length (weighCachedDoc). The resident cost (the parsed PackageInfo plus the raw document) is a near-constant multiple of the document's size, so scaling the document's encoded length estimates the footprint without measuring the parsed structure. The encode is an O(document) pass run only on a leader's insert (the cold path after a fetch), never on a hit. The multiplier is set at the high end of the observed resident-to-encoded ratio so the estimate is an upper bound: a memory budget must not systematically under-count.

Resolution

resolveMetadata :: MetricsPort -> MetadataCache -> Source -> PackageName -> IO (Either MetadataError CacheEntry) -> IO (Either MetadataError CacheEntry) Source #

Resolve a package's metadata from one upstream Source, reusing the cache and collapsing concurrent misses.

On a fresh, unexpired hit the cached CacheEntry is returned and the fetch action is never run. On a miss the action runs exactly once even under concurrent callers: the first installs an in-flight marker and fetches, the others wait on its result. A successful fetch is cached (subject to the TTL and size bound); a failed fetch caches nothing (so a transient upstream error does not poison the cache) and its typed Left is handed to every waiter, so a coalesced follower sees exactly the fault the leader saw.

A claimed in-flight slot is always eventually filled and de-registered, even if the leader is hit by an async exception (a request timeout, a killed handler thread) between claiming the slot and completing: the claim commits under a mask and the leader's run is handed straight to guardInFlight, which frees the slot on every exit and, on an escape before the marker is filled, hands that error to every waiting follower rather than leaving them parked forever. This closes the single-flight orphan window (without it, a cancelled leader would wedge that (source, package) key until restart). A follower receiving an orphaned marker re-evaluates the resolve when the leader was cancelled (async), re-entering interruptibly and counting its miss only once, and re-raises when the leader escaped synchronously: the fetch's contract is total, so a synchronous escape is an invariant break for the outer boundary, never laundered into the typed channel. A follower's own wait on the marker stays interruptible.

The Source partitions the cache: distinct upstreams of the same package resolve under distinct keys and never cross-contaminate. The fetch action supplies the origin's own credential, so reading through one source never blurs another's trust posture. Under the default passthrough strategy only the anonymous public origin is resolved here: the trusted private origin is the per-client authority and is fetched per request, never cached, so a shared entry can never serve one client another's private document.

The result is always re-decided by the caller's rules on each request -- only the fetch+parse is memoised, never the verdict.

Each resolution records the ecluse.metadata_cache.requests hit/miss counter (a coalescing follower counts as a miss, like the leader it waits on), and a leader's insert refreshes the ecluse.metadata_cache.entries occupancy gauge and the ecluse.metadata_cache.resident_bytes residency gauge.

resolveMetadataWith :: IO () -> MetricsPort -> MetadataCache -> Source -> PackageName -> IO (Either MetadataError CacheEntry) -> IO (Either MetadataError CacheEntry) Source #

As resolveMetadata, but with a hook run on the leading thread at the single-flight claim → fetch-runner handoff: the window between the STM transaction committing the in-flight claim and the leader's exception guard taking ownership of the marker. It exists only so a test can deterministically park a leader in that window and cancel it there, exercising the orphan-window guarantee; production always passes pure () via resolveMetadata.

cachedMetadata :: MetadataCache -> Source -> PackageName -> IO (Maybe CacheEntry) Source #

Look up a package's cached full-packument entry for one Source without fetching on a miss and without bumping recency: the cache's read-only view. Two readers share it: the inspection and test probes, and the hybrid serve path's step-2 full-packument consult (the tarball gate selecting one version from a warm entry). That consult stays read-only on purpose: the full store's recency is driven by the packument GET's own resolveMetadata hit, so the gate's select need not bump it, unlike the single-version store whose only steady-state read is cachedVersion. A Nothing is a miss or an expired entry; this never triggers a fetch and never collapses (use resolveMetadata for the serve path).

Single-version resolution

resolveVersion :: MetricsPort -> MetadataCache -> Source -> PackageName -> Version -> IO (Either MetadataError (Maybe PackageDetails)) -> IO (Either MetadataError (Maybe PackageDetails)) Source #

Resolve one version's PackageDetails (or its determined absence) from the single-version cache, leading a selective fetch on a miss and collapsing concurrent misses exactly as resolveMetadata does for the full packument. The cached value is the Maybe PackageDetails the fetch yields, so a version determined absent over sound metadata is cached as Nothing (a negative entry) and re-served without a re-fetch within the TTL.

This writes to the single-version store only, never the full-packument store, so a cold tarball gate's selective parse cannot materialise a whole packument into the shared full cache. Unlike resolveMetadata, the single-version store records no hit/miss counter; a leader's insert does refresh the single-version residency gauge (ecluse.metadata_cache.version.resident_bytes), so the byte budget that bounds both stores is observable on each.

resolveVersionWith :: IO () -> MetricsPort -> MetadataCache -> Source -> PackageName -> Version -> IO (Either MetadataError (Maybe PackageDetails)) -> IO (Either MetadataError (Maybe PackageDetails)) Source #

As resolveVersion, with the single-flight claim → fetch-runner handoff hook resolveMetadataWith exposes, for the same orphan-window test (production passes pure () via resolveVersion).

cachedVersion :: MetadataCache -> Source -> PackageName -> Version -> IO (Maybe (Maybe PackageDetails)) Source #

Look up a single-version cached entry for one (source, package, version) without fetching on a miss, bumping the entry's recency on a hit: the hybrid serve path's step-1 version consult before it leads a selective fetch. This read is the version store's only steady-state access (a hit here short-circuits before resolveVersion and its recency-bumping hit), so without the bump a warm version entry would never refresh its recency and would age out of the least-recently-used eviction in insert order. The outer Maybe is the cache hit/miss (an expired or absent entry is Nothing); the inner Maybe PackageDetails is the cached result (a version determined absent is a cached Just Nothing). Never fetches and never collapses (use resolveVersion to lead a selective fetch).

Assembled-representation resolution

resolveAssembled :: MetricsPort -> MetadataCache -> Text -> IO ByteString -> IO ByteString Source #

Resolve the assembled representation for one derived validator, leading the render (assemble + encode) on a miss and collapsing concurrent identical renders, exactly as resolveMetadata does for a fetch.

The key is the rendered derived ETag -- a content address over every input the served document is a function of -- so a hit is byte-for-byte the document this request's own inputs would deterministically produce: the store can never serve stale bytes (changed inputs miss by construction) and never crosses a client boundary (a different private view is a different key; see the module header). Under the TTL-zero configuration the store degrades to pure single-flight coalescing, the same behaviour as the sibling stores.

Like the single-version store it records no hit/miss counter; a leader's insert refreshes the ecluse.metadata_cache.assembled.resident_bytes residency gauge, so the byte budget's third occupant is observable alongside the other two.

The store's error slot is Void -- the render has no domain failure -- so the resolve is folded back to a plain IO ByteString here (absurd discharges the impossible Left), keeping the serve tail's call shape unchanged.