| Safe Haskell | None |
|---|---|
| Language | GHC2021 |
Ecluse.Runtime.Cve.Sync
Description
The advisory database's sync mechanics: detect a new osv.db artifact in
object storage, download it bounded, verify it, and shadow-swap it into the
read path, one ecosystem per task, driven by the configured mounts.
The write side of Ecluse.Core.Cve.Slot: syncStep performs exactly one
detect-download-verify-swap cycle over an injected CveFetch (so unit tests
drive it without a network), and runCveSync schedules those steps: an eager
boot burst (an immediate attempt, retried with incremental backoff, that is
eventually allowed to fail so a broken bucket never wedges startup) followed by
the steady ETag poll. The proxy is rules-engine complete as early as the
artifact can be had; before then it serves deny-by-default.
The swap's file discipline: the download lands in a temp file beside the
canonical per-ecosystem path, and openCveDb verifies the
temp file (epoch stamp, table shape, ecosystem), the artifact contract's
verify-before-swap. __The connection that verified is the connection that
serves__: the accepted temp file is renamed atomically onto the canonical
name, the open connection follows the inode through the rename, and that same
CveDb is swapped in; there is no reopen and so no verify-to-serve gap. The
displaced generation drains and closes inside swapIn, releasing
the old inode's last reference; reclamation is the kernel's, never a delete
this code could mistime. A rejected artifact is deleted, its ETag remembered
(re-downloading a known-bad object every poll buys nothing), and the last-good
generation keeps serving.
Synopsis
- data CveFetch = CveFetch {
- fetchHeadEtag :: IO (Either OsvDbFetchFault (Maybe DbEtag))
- fetchDownload :: FilePath -> IO (Either OsvDbFetchFault DbEtag)
- newtype DbEtag = DbEtag Text
- data OsvDbFetchFault
- = OsvDbTooLarge Int
- | OsvDbNoEtag
- | OsvDbTransport TransportFault
- newtype OsvDbCapExceeded = OsvDbCapExceeded Int
- data S3CveSource
- newS3CveSource :: Maybe (Bool, Text, Int) -> IO S3CveSource
- s3CveFetchFor :: S3CveSource -> Text -> Text -> Int -> CveFetch
- cappedAt :: forall (m :: Type -> Type). MonadIO m => Int -> ConduitT ByteString ByteString m ()
- data SyncEnv = SyncEnv {
- syncFetch :: CveFetch
- syncEcosystem :: Ecosystem
- syncDbPath :: FilePath
- syncSlot :: CveSlot
- data SyncOutcome
- = SyncSwapped DbEtag [(Text, Text)]
- | SyncUnchanged
- | SyncAbsent
- | SyncRejected DbEtag CveDbRejected
- | SyncFetchFaulted OsvDbFetchFault
- syncStep :: SyncEnv -> Maybe DbEtag -> IO SyncOutcome
- data SyncSchedule = SyncSchedule {
- schedBootBackoff :: [Int]
- schedPollDelay :: Int
- runCveSync :: (MonadUnliftIO m, KatipContext m) => SyncEnv -> SyncSchedule -> IO () -> m ()
- bootBackoffDelays :: [Int]
- bootBurstPolicy :: forall (m :: Type -> Type). Monad m => [Int] -> RetryPolicyM m
The injected transport
The sync transport, as data: how to learn the remote artifact's current
version and how to fetch its bytes. Injected so syncStep is unit-testable
without a network; the composition root draws one from a newS3CveSource.
Constructors
| CveFetch | |
Fields
| |
data OsvDbFetchFault Source #
Why an artifact fetch did not yield usable bytes: refused by this side (the
object oversteps the configured byte cap, or the response carried no ETag to
record), or not delivered at all (a transport fault, classified into the core
vocabulary at the adapter edge). A value on the CveFetch channel, never an
exception: the sync task's step folds it into its outcome and the schedule
retries.
Constructors
| OsvDbTooLarge Int | The object exceeds the configured byte cap (carried, in bytes). |
| OsvDbNoEtag | The response carried no ETag; nothing truthful to record. |
| OsvDbTransport TransportFault | The transport could not deliver the object (carried, classified). |
Instances
| Show OsvDbFetchFault Source # | |
Defined in Ecluse.Runtime.Cve.Sync Methods showsPrec :: Int -> OsvDbFetchFault -> ShowS # show :: OsvDbFetchFault -> String # showList :: [OsvDbFetchFault] -> ShowS # | |
| Eq OsvDbFetchFault Source # | |
Defined in Ecluse.Runtime.Cve.Sync Methods (==) :: OsvDbFetchFault -> OsvDbFetchFault -> Bool # (/=) :: OsvDbFetchFault -> OsvDbFetchFault -> Bool # | |
newtype OsvDbCapExceeded Source #
The byte cap's mid-stream escape hatch: cappedAt sits inside a conduit
pipeline (no value channel of its own), so it reports an overstepped cap by
throwing this -- confined typed exception, caught at the adapter boundary
(s3Download) and folded into OsvDbTooLarge. It never crosses the CveFetch
interface.
Constructors
| OsvDbCapExceeded Int |
Instances
| Exception OsvDbCapExceeded Source # | |
Defined in Ecluse.Runtime.Cve.Sync Methods toException :: OsvDbCapExceeded -> SomeException # fromException :: SomeException -> Maybe OsvDbCapExceeded # | |
| Show OsvDbCapExceeded Source # | |
Defined in Ecluse.Runtime.Cve.Sync Methods showsPrec :: Int -> OsvDbCapExceeded -> ShowS # show :: OsvDbCapExceeded -> String # showList :: [OsvDbCapExceeded] -> ShowS # | |
| Eq OsvDbCapExceeded Source # | |
Defined in Ecluse.Runtime.Cve.Sync Methods (==) :: OsvDbCapExceeded -> OsvDbCapExceeded -> Bool # (/=) :: OsvDbCapExceeded -> OsvDbCapExceeded -> Bool # | |
data S3CveSource Source #
An S3-backed advisory-fetch source: the amazonka Env is built once at
newS3CveSource and captured, so s3CveFetchFor yields a CveFetch per (bucket,
object key, byte cap) without re-discovering credentials per mount. The runtime
adapter that seals the SDK env behind the sync's transport, matching
newSqsQueue; the composition shell never handles the env.
newS3CveSource :: Maybe (Bool, Text, Int) -> IO S3CveSource Source #
Build an S3CveSource, constructing the S3 amazonka env once (honouring the
pre-parsed endpoint override) and capturing it, so every ecosystem's CveFetch
shares one credential discovery.
s3CveFetchFor :: S3CveSource -> Text -> Text -> Int -> CveFetch Source #
A CveFetch against one bucket, object key, and byte cap, over the captured env.
cappedAt :: forall (m :: Type -> Type). MonadIO m => Int -> ConduitT ByteString ByteString m () Source #
A pass-through conduit that refuses to stream past the byte cap: the
enforcement behind the source's bounded download, where the declared content
length is only the fast-fail. A breach throws the confined OsvDbCapExceeded
(a conduit has no value channel of its own); the adapter boundary folds it into
OsvDbTooLarge.
One sync cycle
Everything one ecosystem's sync task operates on.
Constructors
| SyncEnv | |
Fields
| |
data SyncOutcome Source #
What one syncStep concluded; the caller (runCveSync) logs it and
decides scheduling.
Constructors
| SyncSwapped DbEtag [(Text, Text)] | A new artifact was verified and is now live (its ETag and provenance carried). |
| SyncUnchanged | The remote ETag matches the last seen one; nothing to do. |
| SyncAbsent | The object does not exist in the bucket (not yet published). |
| SyncRejected DbEtag CveDbRejected | The artifact was downloaded and refused by verification; the last-good generation keeps serving and the ETag is remembered. |
| SyncFetchFaulted OsvDbFetchFault | The fetch itself failed (carried); nothing was learned about the remote artifact, so the last seen ETag stands and the schedule retries. |
Instances
| Show SyncOutcome Source # | |
Defined in Ecluse.Runtime.Cve.Sync Methods showsPrec :: Int -> SyncOutcome -> ShowS # show :: SyncOutcome -> String # showList :: [SyncOutcome] -> ShowS # | |
syncStep :: SyncEnv -> Maybe DbEtag -> IO SyncOutcome Source #
One detect-download-verify-swap cycle against the last seen ETag. Total
over the fetch and over verification -- a failed fetch and a refused artifact
are both outcomes, not exceptions -- so the caller's scheduling is a plain fold
over SyncOutcome. See the module header for the file discipline.
The scheduled task
data SyncSchedule Source #
The task's timing: the boot burst's backoff delays and the steady poll
interval, both in microseconds. The composition root ships bootBackoffDelays
and the configured poll interval; tests inject tiny values.
Constructors
| SyncSchedule | |
Fields
| |
runCveSync :: (MonadUnliftIO m, KatipContext m) => SyncEnv -> SyncSchedule -> IO () -> m () Source #
One ecosystem's sync task: the boot burst, then the steady poll, forever.
The boot burst attempts a sync immediately and retries per the schedule's backoff until an artifact is live, so a healthy deployment is rules-engine complete within seconds of boot. It concedes early on a rejected artifact (retrying the same bytes cannot end differently) and gives up after the schedule with a warning. The proxy serves regardless, since an empty slot only ever abstains into deny-by-default, and the poll keeps trying.
A fetch fault arrives as a value in the step's outcome and is logged here;
residue (a filesystem fault on the temp path, a contract escape) propagates to
the supervision the composition root wraps this task in
(superviseLoop), which restarts the task -- it simply
resumes from the remote artifact. notifyFirstSync runs after each successful
swap (its consumer, the readiness signal, is an idempotent one-way flip).
bootBackoffDelays :: [Int] Source #
The shipped boot-burst backoff: an immediate first attempt, then retries after each of these, then the burst concedes and the steady poll takes over. Constants by design; the poll interval is the operator-facing knob.
bootBurstPolicy :: forall (m :: Type -> Type). Monad m => [Int] -> RetryPolicyM m Source #
The boot-burst backoff schedule compiled to a Control.Retry policy: the
n-th retry waits the n-th delay (microseconds) before it, and the policy stops
(yields Nothing) once the list is spent, so the list's length is the retry
budget. Inspect the schedule without sleeping with simulatePolicy.