| Safe Haskell | None |
|---|---|
| Language | GHC2021 |
Ecluse.Core.Cve.Sync
Description
The advisory database's sync mechanics: detect a new osv.db artifact in
object storage, download it bounded, verify it, and shadow-swap it into the
read path, one ecosystem per task, driven by the configured mounts.
The write side of Ecluse.Core.Cve.Slot: syncStep performs exactly one
detect-download-verify-swap cycle over an injected CveFetch (so unit tests
drive it without a network), and runCveSync schedules those steps: an eager
boot burst (an immediate attempt, retried with incremental backoff, that is
eventually allowed to fail so a broken bucket never wedges startup) followed by
the steady ETag poll. The proxy is rules-engine complete as early as the
artifact can be had; before then it serves deny-by-default.
The swap's file discipline: the download lands in a temp file beside the
canonical per-ecosystem path, and openCveDb verifies the
temp file (epoch stamp, table shape, ecosystem), the artifact contract's
verify-before-swap. __The connection that verified is the connection that
serves__: the accepted temp file is renamed atomically onto the canonical
name, the open connection follows the inode through the rename, and that same
CveDb is swapped in; there is no reopen and so no verify-to-serve gap. The
displaced generation drains and closes inside swapIn, releasing
the old inode's last reference; reclamation is the kernel's, never a delete
this code could mistime. A rejected artifact is deleted, its ETag remembered
(re-downloading a known-bad object every poll buys nothing), and the last-good
generation keeps serving.
Synopsis
- data CveFetch = CveFetch {
- fetchHeadEtag :: IO (Maybe DbEtag)
- fetchDownload :: FilePath -> IO DbEtag
- newtype DbEtag = DbEtag Text
- data OsvDbFetchFault
- s3CveFetch :: Env -> Text -> Text -> Int -> CveFetch
- cappedAt :: forall (m :: Type -> Type). MonadIO m => Int -> ConduitT ByteString ByteString m ()
- data SyncEnv = SyncEnv {}
- data SyncOutcome
- syncStep :: SyncEnv -> Maybe DbEtag -> IO SyncOutcome
- data SyncSchedule = SyncSchedule {
- schedBootBackoff :: [Int]
- schedPollDelay :: Int
- runCveSync :: (MonadUnliftIO m, KatipContext m) => SyncEnv -> SyncSchedule -> IO () -> m ()
- bootBackoffDelays :: [Int]
The injected transport
The sync transport, as data: how to learn the remote artifact's current
version and how to fetch its bytes. Injected so syncStep is unit-testable
without a network; the composition root supplies s3CveFetch.
Constructors
| CveFetch | |
Fields
| |
An artifact version marker: S3's ETag, opaque text compared for equality only. Two objects with equal ETags carry equal bytes, so an unchanged ETag is "nothing to do" and a rejected artifact's remembered ETag is "still the same bad artifact".
data OsvDbFetchFault Source #
A download refused by this side: the object oversteps the configured byte cap, or the response carried no ETag to record.
Constructors
| OsvDbTooLarge Int | The object exceeds the configured byte cap (carried, in bytes). |
| OsvDbNoEtag | The response carried no ETag; nothing truthful to record. |
Instances
| Exception OsvDbFetchFault Source # | |
Defined in Ecluse.Core.Cve.Sync Methods toException :: OsvDbFetchFault -> SomeException # fromException :: SomeException -> Maybe OsvDbFetchFault # displayException :: OsvDbFetchFault -> String # backtraceDesired :: OsvDbFetchFault -> Bool # | |
| Show OsvDbFetchFault Source # | |
Defined in Ecluse.Core.Cve.Sync Methods showsPrec :: Int -> OsvDbFetchFault -> ShowS # show :: OsvDbFetchFault -> String # showList :: [OsvDbFetchFault] -> ShowS # | |
| Eq OsvDbFetchFault Source # | |
Defined in Ecluse.Core.Cve.Sync Methods (==) :: OsvDbFetchFault -> OsvDbFetchFault -> Bool # (/=) :: OsvDbFetchFault -> OsvDbFetchFault -> Bool # | |
s3CveFetch :: Env -> Text -> Text -> Int -> CveFetch Source #
The real transport: S3 HEAD for the ETag, bounded streaming GET for
the bytes, against one bucket and key. A 404 on HEAD is the honest
Nothing (not yet published); every other service or transport fault throws
for the sync task's supervision to log.
cappedAt :: forall (m :: Type -> Type). MonadIO m => Int -> ConduitT ByteString ByteString m () Source #
A pass-through conduit that refuses to stream past the byte cap
(OsvDbTooLarge): the enforcement behind s3CveFetch's bounded download,
where the declared content length is only the fast-fail.
One sync cycle
Everything one ecosystem's sync task operates on.
Constructors
| SyncEnv | |
Fields
| |
data SyncOutcome Source #
What one syncStep concluded; the caller (runCveSync) logs it and
decides scheduling. Failures of the transport itself surface as exceptions.
Constructors
| SyncSwapped DbEtag [(Text, Text)] | A new artifact was verified and is now live (its ETag and provenance carried). |
| SyncUnchanged | The remote ETag matches the last seen one; nothing to do. |
| SyncAbsent | The object does not exist in the bucket (not yet published). |
| SyncRejected DbEtag CveDbRejected | The artifact was downloaded and refused by verification; the last-good generation keeps serving and the ETag is remembered. |
Instances
| Show SyncOutcome Source # | |
Defined in Ecluse.Core.Cve.Sync Methods showsPrec :: Int -> SyncOutcome -> ShowS # show :: SyncOutcome -> String # showList :: [SyncOutcome] -> ShowS # | |
syncStep :: SyncEnv -> Maybe DbEtag -> IO SyncOutcome Source #
One detect-download-verify-swap cycle against the last seen ETag. Total over verification (a refused artifact is an outcome, not an exception); transport faults propagate for the caller to log and retry. See the module header for the file discipline.
The scheduled task
data SyncSchedule Source #
The task's timing: the boot burst's backoff delays and the steady poll
interval, both in microseconds. The composition root ships bootBackoffDelays
and the configured poll interval; tests inject tiny values.
Constructors
| SyncSchedule | |
Fields
| |
runCveSync :: (MonadUnliftIO m, KatipContext m) => SyncEnv -> SyncSchedule -> IO () -> m () Source #
One ecosystem's sync task: the boot burst, then the steady poll, forever.
The boot burst attempts a sync immediately and retries per the schedule's backoff until an artifact is live, so a healthy deployment is rules-engine complete within seconds of boot. It concedes early on a rejected artifact (retrying the same bytes cannot end differently) and gives up after the schedule with a warning. The proxy serves regardless, since an empty slot only ever abstains into deny-by-default, and the poll keeps trying.
Every iteration is supervised: a transport fault is caught and logged, never
fatal to the task. notifyFirstSync runs after each successful swap (its
consumer, the readiness signal, is an idempotent one-way flip).
bootBackoffDelays :: [Int] Source #
The shipped boot-burst backoff: an immediate first attempt, then retries after each of these, then the burst concedes and the steady poll takes over. Constants by design; the poll interval is the operator-facing knob.