ecluse:ecluse-runtime
Safe HaskellNone
LanguageGHC2021

Ecluse.Runtime.Queue.Sqs

Description

The AWS SQS backend behind the MirrorQueue handle.

Maps the handle's receive → process → ack shape onto SQS:

  • enqueueSendMessage (the MirrorJob encoded as the message body),
  • receive → one long-poll ReceiveMessage (a batch, [] on an empty poll),
  • ackDeleteMessage (the message is gone, never redelivered),
  • extendVisibilityChangeMessageVisibility (hold a long publish),
  • deadLetterChangeMessageVisibility with the sqsTerminalBackoff window and no DeleteMessage (a terminal fault rides the redrive policy to the DLQ).

The provider differences SQS embodies -- the visibility timeout, the long-poll window, the batch limit -- are SqsConfig knobs with sane defaults, and the SQS receipt handle is carried opaquely in a ReceiptHandle (via mkReceiptHandle), so none of it leaks past the handle. Retry is "don't ack": a job whose processing fails transiently is simply not acked, and SQS redelivers it once the visibility timeout lapses; persistent failures fall to the queue's native dead-letter (max-receive-count), so there is no nack (see Ecluse.Core.Queue). A terminal fault (deadLetter) is returned with a backoff window and never deleted, so it too falls to the operator's dead-letter queue rather than being discarded; this assumes the operator's redrive policy exists (the no-DLQ case is issue #935). Every operation reports its AWS failure as the handle's typed QueueFault value, classified into the core transport vocabulary at this edge (Ecluse.Runtime.Aws.Fault), so a queue outage never rides the exception channel through a caller.

The amazonka Env is built once at newSqsQueue and captured by the handle's closures, so the backend's state never reaches the proxy's Env/App (see docs/architecture/technology-stack.md → "Key Decisions"). The MirrorJob wire mapping is a plain JSON object, decoded on receive; a body that fails to parse is dropped rather than yielded as a partial, so -- like any message left unprocessed -- it is not acked and SQS redelivers it, ultimately to the dead-letter queue. Each drop (a missing body or receipt, or an undecodable body) is logged at DebugS with its reason and the SQS message id when present, so a poison message is visible rather than cycling silently; the untrusted body is never logged.

The SQS queue is a trusted, operator-declared destination (the configured queue URL, or an endpoint override): like the OTLP telemetry endpoint (see Ecluse.Runtime.Telemetry.Resolve), it is reached through amazonka's own client and is not subject to the data-plane egress controls (the host allowlist and the https-only egress posture of Ecluse.Core.Security.Egress), which guard only untrusted package downloads, never a destination the operator configured.

Synopsis

Configuration

data SqsConfig Source #

What the SQS backend needs. The batch size, long-poll window, and visibility timeout are provider knobs (see Ecluse.Core.Queue) with defaults in defaultSqsConfig.

Constructors

SqsConfig 

Fields

  • sqsQueueUrl :: Text

    The fully-qualified SQS queue URL mirror jobs are sent to and received from.

  • sqsRegion :: Text

    The AWS region the queue lives in (e.g. "us-east-1").

  • sqsEndpoint :: Maybe SqsEndpoint

    An endpoint override for an emulator or VPC endpoint; Nothing uses amazonka's default resolution and the ambient credential chain.

  • sqsBatchSize :: Int

    Maximum messages to pull per receive (SQS caps this at 10). A larger batch amortises the round-trip when the queue is busy.

  • sqsWaitSeconds :: Int

    The long-poll window in seconds (SQS caps this at 20): how long a receive waits for a message before returning [], so an idle worker does not hot-loop on empty polls.

  • sqsVisibilityTimeout :: Seconds

    How long a received message stays hidden from other receives before SQS redelivers it -- the budget for processing-then-ack, extendable per message via extendVisibility.

  • sqsTerminalBackoff :: Seconds

    The visibility timeout deadLetter returns a terminal message with (ChangeMessageVisibility, never DeleteMessage): larger than the normal processing window so a permanently-unmirrorable artifact is not re-fetched in a hot loop, while it rides the operator's redrive policy to the dead-letter queue. A per-attempt incremental backoff would need the ApproximateReceiveCount attribute (deferred with the receive-count work in issue #935); this fixed backoff is the conservative default.

Instances

Instances details
Show SqsConfig Source # 
Instance details

Defined in Ecluse.Runtime.Queue.Sqs

Eq SqsConfig Source # 
Instance details

Defined in Ecluse.Runtime.Queue.Sqs

data SqsEndpoint Source #

Where an SQS-compatible endpoint lives, for pointing the backend at a non-default host: a local emulator (ministack) in tests, or a VPC endpoint. A non-default host: a local emulator (ministack) in tests, or a VPC endpoint.

Constructors

SqsEndpoint 

Fields

Instances

Instances details
Show SqsEndpoint Source # 
Instance details

Defined in Ecluse.Runtime.Queue.Sqs

Eq SqsEndpoint Source # 
Instance details

Defined in Ecluse.Runtime.Queue.Sqs

defaultSqsConfig :: Text -> Text -> SqsConfig Source #

A SqsConfig for a queue URL and region with the provider knobs at sane defaults: a full batch of 10, the maximum 20-second long poll, and a 30-second visibility timeout. Override the record fields to tune them, or set sqsEndpoint to target an emulator.

The backend

newSqsQueue :: LogEnv -> (Text -> Either Text RegistryUrl) -> SqsConfig -> IO MirrorQueue Source #

Build an SQS-backed MirrorQueue. The amazonka Env is constructed once here -- region-scoped, and pointed at sqsEndpoint with its throwaway credentials when one is given, otherwise discovering the ambient AWS credential chain -- and captured by the returned handle's closures.

Received-message lifting

data ReceivedMessage Source #

The fields of a received SQS message the backend reads. Lifting them out of the amazonka Message keeps the QueueMessage mapping (and its drop decision) free of the AWS type, so the receive path's drop behaviour is exercised directly in tests.

Constructors

ReceivedMessage 

Fields

  • rmBody :: Maybe Text

    The message body carrying the encoded MirrorJob (SQS always supplies one).

  • rmReceipt :: Maybe Text

    The receipt handle a later ack deletes the message by (SQS always supplies one).

  • rmMessageId :: Maybe Text

    The SQS-assigned message id, for the drop log; not part of the untrusted body.

liftReceivedMessages :: LogEnv -> (Text -> Either Text RegistryUrl) -> [ReceivedMessage] -> IO [QueueMessage] Source #

Lift a received batch into deliverable QueueMessages, logging each dropped message (a missing body or receipt, or an undecodable body) at DebugS so a poison message is visible rather than cycling silently until the queue's max-receive count. A dropped message is omitted from the result and left un-acked, so redelivery and dead-letter behaviour are unchanged.

Job wire mapping

encodeJob :: MirrorJob -> Text Source #

Encode a MirrorJob as the JSON text of an SQS message body. The inverse of decodeJob: the package identity is split into its ecosystem, optional scope, and bare name so it round-trips through mkPackageName, and the version keeps its raw string. The serve-time-admitted artifact's filename rides as a plain field: it is the selection key the worker's ingest re-evaluation gates by, and the only thing of the artifact the wire carries -- the digests and size the worker verifies and publishes with are derived from current metadata, never the payload.

decodeJob :: (Text -> Either Text RegistryUrl) -> Text -> Either Text MirrorJob Source #

Decode an SQS message body back into a MirrorJob, or a human-readable error if the body is not the JSON object encodeJob produces (a missing field, an unknown ecosystem, an artifact URL the egress former refuses, malformed JSON).

The queue payload is a trust boundary, so the artifact URL is re-formed into its RegistryUrl egress witness on decode through the given former -- the composition root passes the https-only mkRegistryUrl; the loopback test harnesses pass their flag-gated dev former. A URL the former refuses fails the decode, so a tampered or misproduced message can never hand the worker's fetch an unwitnessed URL (it redelivers and falls to the dead-letter queue, like any undecodable body).