| Safe Haskell | None |
|---|---|
| Language | GHC2021 |
Ecluse.Runtime.Queue.Sqs
Description
The AWS SQS backend behind the MirrorQueue handle.
Maps the handle's receive → process → ack shape onto SQS:
enqueue→SendMessage(theMirrorJobencoded as the message body),receive→ one long-pollReceiveMessage(a batch,[]on an empty poll),ack→DeleteMessage(the message is gone, never redelivered),extendVisibility→ChangeMessageVisibility(hold a long publish),deadLetter→ChangeMessageVisibilitywith thesqsTerminalBackoffwindow and noDeleteMessage(a terminal fault rides the redrive policy to the DLQ).
The provider differences SQS embodies -- the visibility timeout, the long-poll
window, the batch limit -- are SqsConfig knobs with sane defaults, and the SQS
receipt handle is carried opaquely in a ReceiptHandle (via mkReceiptHandle),
so none of it leaks past the handle. Retry is "don't ack": a job whose
processing fails transiently is simply not acked, and SQS redelivers it once the
visibility timeout lapses; persistent failures fall to the queue's native dead-letter
(max-receive-count), so there is no nack (see Ecluse.Core.Queue). A terminal
fault (deadLetter) is returned with a backoff window and never deleted, so it too
falls to the operator's dead-letter queue rather than being discarded; this assumes
the operator's redrive policy exists (the no-DLQ case is issue #935). Every
operation reports its AWS failure as the handle's typed
QueueFault value, classified into the core transport
vocabulary at this edge (Ecluse.Runtime.Aws.Fault), so a queue outage never
rides the exception channel through a caller.
The amazonka Env is built once at newSqsQueue and captured by the
handle's closures, so the backend's state never reaches the proxy's Env/App
(see docs/architecture/technology-stack.md → "Key Decisions"). The
MirrorJob wire mapping is a plain JSON object, decoded on receive; a body that
fails to parse is dropped rather than yielded as a partial, so -- like any message
left unprocessed -- it is not acked and SQS redelivers it, ultimately to the
dead-letter queue. Each drop (a missing body or receipt, or an undecodable body) is
logged at DebugS with its reason and the SQS message id when present, so a poison
message is visible rather than cycling silently; the untrusted body is never logged.
The SQS queue is a trusted, operator-declared destination (the configured queue
URL, or an endpoint override): like the OTLP telemetry endpoint (see
Ecluse.Runtime.Telemetry.Resolve), it is reached through amazonka's own client and is
not subject to the data-plane egress controls (the host allowlist and the https-only
egress posture of Ecluse.Core.Security.Egress), which guard only untrusted package
downloads, never a destination the operator configured.
Synopsis
- data SqsConfig = SqsConfig {
- sqsQueueUrl :: Text
- sqsRegion :: Text
- sqsEndpoint :: Maybe SqsEndpoint
- sqsBatchSize :: Int
- sqsWaitSeconds :: Int
- sqsVisibilityTimeout :: Seconds
- sqsTerminalBackoff :: Seconds
- data SqsEndpoint = SqsEndpoint {}
- defaultSqsConfig :: Text -> Text -> SqsConfig
- newSqsQueue :: LogEnv -> (Text -> Either Text RegistryUrl) -> SqsConfig -> IO MirrorQueue
- data ReceivedMessage = ReceivedMessage {}
- liftReceivedMessages :: LogEnv -> (Text -> Either Text RegistryUrl) -> [ReceivedMessage] -> IO [QueueMessage]
- encodeJob :: MirrorJob -> Text
- decodeJob :: (Text -> Either Text RegistryUrl) -> Text -> Either Text MirrorJob
Configuration
What the SQS backend needs. The batch size, long-poll window, and visibility
timeout are provider knobs (see Ecluse.Core.Queue) with defaults in
defaultSqsConfig.
Constructors
| SqsConfig | |
Fields
| |
data SqsEndpoint Source #
Where an SQS-compatible endpoint lives, for pointing the backend at a
non-default host: a local emulator (ministack) in tests, or a VPC endpoint. A
non-default host: a local emulator (ministack) in tests, or a VPC endpoint.
Constructors
| SqsEndpoint | |
Fields
| |
Instances
| Show SqsEndpoint Source # | |
Defined in Ecluse.Runtime.Queue.Sqs Methods showsPrec :: Int -> SqsEndpoint -> ShowS # show :: SqsEndpoint -> String # showList :: [SqsEndpoint] -> ShowS # | |
| Eq SqsEndpoint Source # | |
Defined in Ecluse.Runtime.Queue.Sqs | |
defaultSqsConfig :: Text -> Text -> SqsConfig Source #
A SqsConfig for a queue URL and region with the provider knobs at sane
defaults: a full batch of 10, the maximum 20-second long poll, and a 30-second
visibility timeout. Override the record fields to tune them, or set sqsEndpoint
to target an emulator.
The backend
newSqsQueue :: LogEnv -> (Text -> Either Text RegistryUrl) -> SqsConfig -> IO MirrorQueue Source #
Build an SQS-backed MirrorQueue. The amazonka Env is constructed
once here -- region-scoped, and pointed at sqsEndpoint with its throwaway
credentials when one is given, otherwise discovering the ambient AWS credential
chain -- and captured by the returned handle's closures.
Received-message lifting
data ReceivedMessage Source #
The fields of a received SQS message the backend reads. Lifting them out of
the amazonka Message keeps the QueueMessage mapping (and its drop
decision) free of the AWS type, so the receive path's drop behaviour is exercised
directly in tests.
Constructors
| ReceivedMessage | |
Fields
| |
Instances
| Show ReceivedMessage Source # | |
Defined in Ecluse.Runtime.Queue.Sqs Methods showsPrec :: Int -> ReceivedMessage -> ShowS # show :: ReceivedMessage -> String # showList :: [ReceivedMessage] -> ShowS # | |
| Eq ReceivedMessage Source # | |
Defined in Ecluse.Runtime.Queue.Sqs Methods (==) :: ReceivedMessage -> ReceivedMessage -> Bool # (/=) :: ReceivedMessage -> ReceivedMessage -> Bool # | |
liftReceivedMessages :: LogEnv -> (Text -> Either Text RegistryUrl) -> [ReceivedMessage] -> IO [QueueMessage] Source #
Lift a received batch into deliverable QueueMessages, logging each dropped
message (a missing body or receipt, or an undecodable body) at DebugS so a poison
message is visible rather than cycling silently until the queue's max-receive count.
A dropped message is omitted from the result and left un-acked, so redelivery and
dead-letter behaviour are unchanged.
Job wire mapping
encodeJob :: MirrorJob -> Text Source #
Encode a MirrorJob as the JSON text of an SQS message body. The inverse of
decodeJob: the package identity is split into its ecosystem, optional scope, and
bare name so it round-trips through mkPackageName, and the version keeps its raw
string. The serve-time-admitted artifact's filename rides as a plain field: it is
the selection key the worker's ingest re-evaluation gates by, and the only thing
of the artifact the wire carries -- the digests and size the worker verifies and
publishes with are derived from current metadata, never the payload.
decodeJob :: (Text -> Either Text RegistryUrl) -> Text -> Either Text MirrorJob Source #
Decode an SQS message body back into a MirrorJob, or a human-readable error
if the body is not the JSON object encodeJob produces (a missing field, an
unknown ecosystem, an artifact URL the egress former refuses, malformed JSON).
The queue payload is a trust boundary, so the artifact URL is re-formed into
its RegistryUrl egress witness on decode through the given former -- the
composition root passes the https-only mkRegistryUrl;
the loopback test harnesses pass their flag-gated dev former. A URL the former
refuses fails the decode, so a tampered or misproduced message can never hand the
worker's fetch an unwitnessed URL (it redelivers and falls to the dead-letter
queue, like any undecodable body).