ecluse:ecluse-core
Safe HaskellNone
LanguageGHC2021

Ecluse.Core.Supervision

Description

One supervision combinator for every background loop: rerun a step forever, absorbing transient faults with a bounded exponential backoff and failing permanent ones up to the process supervisor.

The proxy's background loops (the mirror worker's poll-and-process, the enqueue-buffer drain, the advisory sync tasks, Pilot's export cycle) all share one robustness contract: a transient fault (a dependency outage the next iteration might clear) is logged and retried at a bounded rate, a permanent fault (a wiring error no retry can fix) fails up so the process exits loudly, and cancellation (the shutdown race tearing the loop down) passes through untouched. This module is that contract, written once, so each loop's file carries only its step and its policy rather than a private copy of the catch-log-backoff machinery.

The typed fault channels stay in the steps: a step that receives an Either fault a from a handle makes its own domain decision (its own pacing included), and what reaches this combinator's catch is residue -- an exception escaping some dependency's typed contract -- plus whichever faults a step's policy deliberately classifies Permanent.

Synopsis

The combinator

superviseLoop :: (MonadUnliftIO m, KatipContext m) => SupervisionPolicy -> m () -> m Void Source #

Run the step forever under the policy: a completed step resets the backoff and reruns at once (the step owns its own pacing -- poll waits and cycle delays live inside it); a synchronous fault classifies through the policy (Transient logs and backs off, Permanent rethrows); an asynchronous exception is never caught (tryAny), so cancellation tears the loop down like any other thread. The Void return makes "this loop never returns" a fact of the type.

data SupervisionPolicy Source #

One loop's supervision policy: the label its log lines carry, how a synchronous fault is classified, and the backoff its transient faults pace at. Loops with wiring faults that no retry can fix (an unconfigured handle reached at runtime) classify those Permanent; everything else defaults Transient.

Constructors

SupervisionPolicy 

Fields

data FaultDisposition Source #

What the supervisor does with a synchronous fault the step let escape. Asynchronous exceptions are never classified: cancellation propagates untouched, so the shutdown race can always tear a supervised loop down.

Constructors

Transient

Log at ErrorS, back off (bounded exponential), rerun the step.

Permanent

Rethrow: fail up to the process supervisor, taking the process down.

Bounded exponential backoff

data BackoffSchedule Source #

A bounded exponential backoff: doubling from the base towards the cap as consecutive failures mount, so a persistently-failing dependency is retried at most once per cap interval. A base equal to the cap is a fixed-interval retry.

Constructors

BackoffSchedule 

Fields

backoffMicros :: BackoffSchedule -> Int -> Int Source #

The delay before the next retry, given how many failures have run consecutively: base * 2^failures, saturated at the cap. The exponent is clamped so the doubling cannot overflow before the ceiling applies.