SQS
Last updated: Sep 23, 2026
Rationale
Amazon SQS (Simple Queue Service) is the service we use for message queuing in the cloud. It lets a producer hand work to a consumer without waiting for it to finish, so traffic spikes are absorbed by the queue instead of overloading the consumer or dropping messages.
The main reasons why we chose it over other alternatives are:
- It is a fully managed serverless service. There are no brokers to provision or scale, and we pay per request.
- It complies with several certifications from ISO and CSA. Many of these certifications are focused on ensuring that the entity follows best practices regarding secure cloud-based environments and information security.
- It supports FIFO queues with message groups and deduplication, so events about the same entity are processed in order and a retried send does not create duplicates.
- It is a native event source for Lambda, including partial batch responses that let a consumer retry only the messages that failed.
- It supports dead-letter queues with redrive policies, so a message that keeps failing is set aside for inspection instead of blocking the queue or disappearing.
- It supports server-side encryption with KMS.
- It publishes queue metrics to CloudWatch, such as the age of the oldest message and the number of visible messages, which we alarm on to detect stuck consumers.
- Resources can be written as code using Terraform. Our queues are created by an in-house Terraform module that applies the same retention, visibility timeout and dead-letter conventions everywhere.
Alternatives
No alternatives are currently being considered, as SQS is the queue that Lambda consumes natively, and the scheduling and event routing needs it does not cover are already served by EventBridge.
Usage
We use Amazon SQS for:
- Queuing the audit and usage events received by Tracks before a Lambda consumer writes them to Aurora (see Platform audit logs). A FIFO queue keyed by message group keeps the events of each entity in order.
- Queuing GitLab webhook events for our GitLab bot and our merge request reviewer, so the webhook receiver answers immediately and a worker processes each merge request in order.
- Holding the dead-letter queue of the platform's DynamoDB Streams triggers, so a stream record whose Lambda invocation fails is kept for later inspection.
We do not use Amazon SQS for:
- Scheduling recurring workloads or routing events between AWS services, which we do with EventBridge.
- Running long or CPU-bound jobs, which we submit to Batch.