training
Decoupling on AWS

Messaging & events on AWS โ€” an interactive field guide

Learn how Amazon SQS, Amazon SNS and Amazon EventBridge (plus Kinesis, MQ, Step Functions and friends) let systems talk without depending on each other. Watch every step, replay it, then test yourself.

Facts checked against AWS documentation as of November 1, 2026, including the 2025โ€“2026 changes: 1 MiB payloads, SQS fair queues, SNS 1 MiB topics, and the relaunched EventBridge Custom Event Bus.

โ–ถ๏ธ

Step through simulations

Every module has a live diagram. Press Next (or โ†’) to move a message one step, Back (โ†) to rewind, or click any step to jump there.
๐Ÿ‘‹๐Ÿ”ง

Pick your depth

Use the switch at the top right. Plain English keeps things simple; Technical expands API calls, limits, payloads and edge cases everywhere.
๐Ÿƒ

Practise

Each module ends with flashcards and a quiz with explanations. The final exam pulls random questions from every module.

Why decouple at all?

Imagine phoning a shop to order something: if nobody answers, you can't order. Now imagine leaving a note in their mailbox instead: they'll read it when they can, and you can get on with your day. Decoupling means building software the mailbox way. Hit Play to see the difference.

The three core services, in one picture each

๐Ÿ“ฌ

Amazon SQS โ€” the to-do rail

Queue. Work tickets wait on a rail until a worker is free to take one. Each ticket is handled by one worker, then thrown away.

Pull-based ยท stores messages up to 14 days ยท one consumer per message

๐Ÿ“ข

Amazon SNS โ€” the loudspeaker

Topic. Make one announcement; everyone who signed up hears it instantly โ€” apps, queues, email, text messages.

Push-based ยท no storage ยท one message โ†’ many receivers

๐ŸšŒ

Amazon EventBridge โ€” the sorting office

Event bus. Anyone drops "this happened" notes on the bus; sorting rules read each note and send copies only to whoever asked for that kind.

Content-based routing ยท AWS & SaaS sources ยท (new) retention & ordering

Key vocabulary

Producer / publisher
The system that sends a message or event.
Consumer / subscriber / target
The system that receives it and does something.
Message
A packet of data, often a request for work: "ship order 42".
Event
A record of a fact: "order 42 was placed". Senders don't know who listens.
Queue ยท Topic ยท Bus ยท Stream
Holding line for work ยท broadcaster ยท router ยท recorded log.
Asynchronous
Hand off and move on, instead of waiting for an answer.
At-least-once delivery
Never lost, but occasionally delivered twice.
Idempotent
Doing it twice has the same result as doing it once โ€” the cure for duplicates.
Dead-letter queue (DLQ)
Where repeatedly failing messages are parked for inspection.
Fan-out
One message delivered to many receivers.

Warm-up flashcards

Warm-up quiz

๐Ÿ“ฌ Amazon SQS Simple Queue Service

A fully managed message queue. Producers drop messages in; consumers pull them out when ready. Nothing is lost if a consumer is busy, slow, or down.

๐Ÿณ The analogy

The ticket rail in a restaurant kitchen. Waiters clip tickets on; any free cook takes the next one, flips it face-down while cooking, and bins it when the dish is served. If a cook walks off mid-dish, the ticket flips back up for someone else.

โœ… Use it whenโ€ฆ

  • Work should happen in the background (emails, image processing, orders)
  • Traffic is spiky and you need a buffer
  • One worker should handle each item
  • You need ordering per key (FIFO)

โš ๏ธ Not ideal whenโ€ฆ

  • Many different systems need the same message (use SNS/EventBridge in front)
  • Multiple apps must replay history (use a stream)
  • You need a reply right now (use a synchronous API)

Interactive simulator

Deep dive

Key numbers (current defaults & limits)
Max message size1 MiB (raised from 256 KiB, Aug 2025) ยท up to 10 message attributes
RetentionDefault 4 days ยท 60 s to 14 days
Visibility timeoutDefault 30 s ยท 0 s to 12 h
Long pollingWaitTimeSeconds up to 20 s
Delay queue / message timer0 to 15 minutes (no per-message timers on FIFO)
Batch sizeUp to 10 messages per Send/Receive/Delete batch
In-flight messages~120,000 (Standard) ยท 120,000 (FIFO)
FIFO throughput300 TPS per API action; up to 3,000 msg/s with batching; much higher in high-throughput mode
FIFO de-duplication interval5 minutes
Message life-cycle: stored โ†’ in flight โ†’ deleted

A message is stored after SendMessage (no limit on stored messages). After ReceiveMessage it is in flight: invisible to other consumers for the visibility timeout. The consumer must call DeleteMessage with the receipt handle; otherwise the message becomes visible again and its ApproximateReceiveCount increments on the next receive. Extend a long job with ChangeMessageVisibility.

When ApproximateReceiveCount exceeds the redrive policy's maxReceiveCount, SQS moves the message to the dead-letter queue. Redrive it back with StartMessageMoveTask.

Standard vs. FIFO vs. fair queues
StandardFIFOStandard + fair queues
OrderingBest effortStrict within MessageGroupIdBest effort; fair across groups
DeliveryAt least onceExactly-once processing (5-min dedup)At least once
ThroughputNearly unlimited300 TPS/action (3,000 msg/s batched); higher with HT modeNearly unlimited
Use forMost background workPayments, inventory, state changes per entityMulti-tenant SaaS: stops one tenant from delaying others
Consuming with AWS Lambda
  • The Lambda event source mapping polls the queue, invokes your function with a batch, and deletes messages on success.
  • Return batchItemFailures (enable ReportBatchItemFailures) so only failed messages are retried.
  • Set the queue's visibility timeout to at least 6ร— the function timeout.
  • The event source mapping supports 1 MiB payloads. Provisioned mode supports up to 10,000 event pollers per mapping (Aug 2026).
Design tips & pitfalls
  • Idempotency: standard queues can deliver duplicates โ€” use a dedup key (order ID) with a conditional write.
  • Poison messages: always configure a DLQ and alarm on it. Make DLQ retention longer than the source queue's.
  • Large payloads: above 1 MiB, store in S3 and send the pointer (claim-check).
  • Scaling consumers: autoscale on ApproximateNumberOfMessagesVisible and ApproximateAgeOfOldestMessage.
  • Cost: batch requests and use long polling; each 64 KB chunk of a request is billed as one request.

Real-world examples

๐Ÿ–ผ๏ธ Photo-upload thumbnails

A user uploads a photo. The app replies "Uploaded!" right away and puts a "make thumbnails" message in a queue. A fleet of workers grows on busy evenings and shrinks overnight.

Code (Python consumer)
import boto3, json
sqs = boto3.client("sqs")
URL = "https://sqs.us-east-1.amazonaws.com/111122223333/thumbs"

while True:
    resp = sqs.receive_message(QueueUrl=URL, MaxNumberOfMessages=10,
                               WaitTimeSeconds=20)           # long polling
    for m in resp.get("Messages", []):
        job = json.loads(m["Body"])
        make_thumbnails(job["bucket"], job["key"])           # idempotent
        sqs.delete_message(QueueUrl=URL,
                           ReceiptHandle=m["ReceiptHandle"])

๐Ÿฆ Ordered account ledger

Deposits and withdrawals for each account must apply in order and exactly once. A FIFO queue uses the account number as the MessageGroupId, so thousands of accounts process in parallel while each stays in sequence.

Code (send to FIFO)
sqs.send_message(
    QueueUrl=FIFO_URL,
    MessageBody=json.dumps({"acct": "A-42", "op": "withdraw", "amt": 50}),
    MessageGroupId="A-42",                 # order per account
    MessageDeduplicationId="txn-9f1c")     # safe to retry

๐Ÿ›๏ธ Flash-sale buffer

A sale drives 50ร— normal traffic. Orders go into a queue so the inventory database is written at a safe, steady rate. Customers see "Order received" instantly.

๐Ÿข Multi-tenant SaaS jobs

One big customer submits 1 million report jobs. With fair queues (tenant ID as MessageGroupId on a Standard queue), small customers' jobs keep flowing instead of waiting behind the giant batch.

Security

In plain English: decide who may put messages in and take them out, keep the contents locked (encrypted), and make sure the queue can only be reached over private, encrypted connections.

Checklist

  • ๐Ÿ”‘ Least privilege: producers get only sqs:SendMessage; consumers get Receive/Delete/ChangeMessageVisibility.
  • ๐Ÿ“œ Queue policy: never allow "Principal":"*" without tight conditions; use aws:PrincipalOrgID for cross-account.
  • ๐Ÿ”’ Encryption at rest: SSE-SQS is on by default for new queues; choose SSE-KMS for your own key, rotation control and CloudTrail visibility of key use.
  • ๐Ÿท๏ธ SSE encrypts the body only โ€” don't put sensitive data in message attributes.
  • ๐Ÿ” In transit: deny non-TLS requests (aws:SecureTransport). Requests to SSE queues must use HTTPS and SigV4.
  • ๐ŸŒ Private network: VPC interface endpoints + aws:SourceVpce conditions.
  • ๐Ÿ•ต๏ธ Confused deputy: when allowing SNS/EventBridge/S3 to send, add aws:SourceArn.
  • ๐Ÿ“ˆ Audit with CloudTrail; alarm on DLQ depth and unusual age of oldest message.

Example queue policy

Lets only one SNS topic deliver, and refuses any non-TLS call.

{
  "Version": "2012-10-17",
  "Statement": [
    { "Sid": "AllowOrderTopicOnly",
      "Effect": "Allow",
      "Principal": { "Service": "sns.amazonaws.com" },
      "Action": "sqs:SendMessage",
      "Resource": "arn:aws:sqs:us-east-1:111122223333:inventory-queue",
      "Condition": { "ArnEquals": {
        "aws:SourceArn": "arn:aws:sns:us-east-1:111122223333:order-events" } } },
    { "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "sqs:*",
      "Resource": "arn:aws:sqs:us-east-1:111122223333:inventory-queue",
      "Condition": { "Bool": { "aws:SecureTransport": "false" } } }
  ]
}

Flashcards

Quiz

๐Ÿ“ข Amazon SNS Simple Notification Service

Fully managed publish/subscribe. Publish a message once to a topic, and SNS pushes a copy to every subscriber โ€” services, queues, functions, web hooks, email, SMS and phones.

๐Ÿ“ป The analogy

A group text or a PA system. You say it once; everyone who joined the group hears it at the same moment. If someone's phone is off, SNS keeps trying for a while โ€” but it doesn't keep a voicemail box. That's what SQS is for.

โœ… Use it whenโ€ฆ

  • One event must reach many systems at once
  • You need to notify people (email, SMS, mobile push)
  • You want very low-latency push delivery
  • Building SNS โ†’ SQS fan-out

โš ๏ธ Not ideal whenโ€ฆ

  • Consumers need to read later at their own pace (add SQS)
  • You need complex routing on many sources or SaaS events (EventBridge)
  • You need replay of history (stream or new EventBridge bus)

Interactive simulator

Deep dive

Key numbers
Subscriptions per topic10 million (default, adjustable)
Topics per account100,000 (default, adjustable)
Message size256 KiB default ยท up to 1 MiB with MaximumMessageSize (Sept 2026; then โ‰ค100 subscriptions; SQS/Lambda/Firehose only) ยท billed per 64 KB chunk
Filter policies200 per topic (default) ยท 5 keys per policy ยท 150 value combinations
Retries โ€” SQS, Lambda100,015 attempts over 23 days
Retries โ€” email, SMS, mobile push50 attempts over 6 hours
Retries โ€” HTTP/SCustom delivery policy; total โ‰ค 3,600 s
Standard vs. FIFO topics

Standard topics: best-effort ordering, at-least-once delivery, all protocols, very high throughput.
FIFO topics (.fifo): strict ordering per MessageGroupId, de-duplication, delivered to SQS queues; support message filtering including on the message body. Pair an SNS FIFO topic with SQS FIFO queues for ordered, exactly-once fan-out.

Filter policies in practice
{
  "eventType": ["order_placed", "order_updated"],
  "amount":    [{ "numeric": [">=", 100, "<", 5000] }],
  "region":    [{ "anything-but": ["test"] }],
  "sku":       [{ "prefix": "BOOK-" }]
}

Keys are ANDed; values within a key are ORed. Set FilterPolicyScope to MessageBody to filter on the JSON payload. Filter policy additions and changes can take several minutes to fully take effect, so test before relying on them.

Delivery, retries and DLQs

Delivery policies have four phases: immediate retries โ†’ pre-back-off โ†’ back-off (arithmetic, exponential, geometric or linear) โ†’ post-back-off. Only HTTP/S policies are customizable. Server-side errors are retried; client-side errors (deleted endpoint, permission problems) go straight to the DLQ if one is attached. The DLQ is an SQS queue on the subscription, same account and Region.

Enable delivery status logging to CloudWatch Logs for visibility into successes and failures per protocol.

Real-world examples

๐Ÿงพ Order placed โ†’ everyone knows

One "order placed" message fans out to queues for billing, shipping and analytics. Adding a loyalty-points service later is just one new subscription โ€” no change to the order service.

๐Ÿšจ Operations alerts

CloudWatch alarms publish to an SNS topic that emails the on-call team, sends an SMS, and invokes a Lambda function to open a ticket automatically.

๐Ÿ“ฑ Mobile push

A delivery app notifies millions of phones "Your order is out for delivery" using SNS mobile push to APNs and FCM.

๐Ÿ’ฐ Price updates in order

An SNS FIFO topic sends price changes for each product (group = SKU) to wholesale and retail SQS FIFO queues, filtered by business line, so no store ever applies an older price after a newer one.

Code (Python publish)
sns.publish(
    TopicArn="arn:aws:sns:us-east-1:111122223333:prices.fifo",
    Message=json.dumps({"sku": "PART-88", "price": 41.90, "business": "retail"}),
    MessageGroupId="PART-88",
    MessageDeduplicationId="PART-88-v1734")

Security

In plain English: control who can announce and who can listen, lock the message contents, and make sure listeners can prove a message really came from SNS.

Checklist

  • ๐Ÿ“œ Topic access policy: restrict sns:Publish and sns:Subscribe; use aws:SourceArn/aws:SourceAccount when AWS services publish.
  • ๐Ÿงท Limit who can subscribe what: conditions like sns:Protocol and sns:Endpoint stop someone adding an outside email or URL.
  • ๐Ÿ”’ SSE with KMS: when AWS services (CloudWatch, S3, EventBridge) publish to an encrypted topic, use a customer managed key that allows that service principal.
  • ๐Ÿ” Use HTTPS endpoints, deny non-TLS publishing (aws:SecureTransport), and use VPC interface endpoints for private publishing.
  • โœ๏ธ HTTP subscribers must verify signatures (SignatureVersion 2 = SHA256) and check SigningCertURL and TopicArn.
  • ๐Ÿšช Set AuthenticateOnUnsubscribe when confirming, so strangers can't unsubscribe you.
  • ๐Ÿงฝ Message data protection policies can audit, mask or block sensitive data (e.g., PII) flowing through a topic.

Example topic policy

Only CloudWatch alarms from this account may publish; only SQS/Lambda subscriptions within the organization.

{
  "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow",
      "Principal": { "Service": "cloudwatch.amazonaws.com" },
      "Action": "sns:Publish",
      "Resource": "arn:aws:sns:us-east-1:111122223333:ops-alerts",
      "Condition": { "StringEquals": { "aws:SourceAccount": "111122223333" } } },
    { "Effect": "Allow",
      "Principal": { "AWS": "*" },
      "Action": "sns:Subscribe",
      "Resource": "arn:aws:sns:us-east-1:111122223333:ops-alerts",
      "Condition": {
        "StringEquals": { "aws:PrincipalOrgID": "o-exampleorg" },
        "StringEqualsIfExists": { "sns:Protocol": ["sqs", "lambda"] } } }
  ]
}

Flashcards

Quiz

๐ŸšŒ Amazon EventBridge Relaunched Sept 2026

A serverless event router. Applications, 200+ AWS services and SaaS partners publish events; rules or subscribers match on any part of the content and deliver to targets.

๐Ÿ“ฎ The analogy

A big sorting office. Anyone can drop a postcard saying "this happened". Sorters read every postcard and copy it into the trays of whoever asked for that kind of postcard ("all orders over $500", "anything from S3 in bucket X"). The 2026 bus also photocopies every postcard into a binder so teams can catch up later.

โœ… Use it whenโ€ฆ

  • Reacting to AWS service or SaaS events
  • Routing on event content, not just a topic name
  • Many teams/accounts share one event backbone
  • You want to add consumers without changing producers

โš ๏ธ Not ideal whenโ€ฆ

  • You need a work queue for one consumer (SQS)
  • Very high fan-out to people via SMS/email (SNS)
  • Massive telemetry streams that need shard-level control (Kinesis/MSK)

Interactive simulators

Two scenarios: the familiar rules & targets model (default bus and "Custom event bus โ€“ classic"), and the new Custom Event Bus. Switch with the buttons at the top of the simulator.

What's new in 2026

September 2026: EventBridge relaunched custom event buses for enterprise scale. The new Custom Event Bus can be shared across accounts with AWS RAM, retains events (24 hours built in, extendable up to a year), supports strictly ordered delivery and automatic de-duplication, and uses a new Subscriber resource that can deliver to 250+ AWS services. Existing custom buses are now called "Custom event bus โ€“ classic" and keep working. At launch it's in 14 Regions, priced by data transferred rather than per event.
January 2026: maximum event payload increased from 256 KB to 1 MB (most commercial Regions).
CapabilityCustom Event Bus (new)Custom event bus โ€“ classic / default bus
Routing resourceSubscriber: filter + transformer + one target; created by the consumerRule: pattern + up to 5 targets; created by the bus owner
Cross-accountShare via AWS RAM (publish / subscribe / attach sources); RevokeResourceBus resource policy, or forward to a bus in the other account
Retention & replayBuilt in, 1โ€“365 days; subscriber starting position; pause/resumeNot retained; archive + replay
OrderingFIFO within an EventGroupId, per subscriberNone
De-duplicationAt publish time, 5-minute windowNone
Publish APIsPutEvents (JSON) + PutRawEvents (Avro, Protobuf, raw)PutEvents (JSON)
TransformsJSONataInput transformer
Default retry5 attempts / 300 s185 attempts / 24 h
Failure outputDLQ gets a JSON failure recordDLQ gets the event + error attributes

Deep dive

Event patterns โ€” the matching language
{
  "source": ["com.shop.checkout"],
  "detail-type": ["OrderPlaced", "OrderUpdated"],
  "detail": {
    "total":    [{ "numeric": [">=", 500] }],
    "country":  [{ "anything-but": ["US"] }],
    "coupon":   [{ "exists": true }],
    "email":    [{ "suffix": "@example.com" }],
    "$or": [ { "tier": ["gold"] }, { "items": [{ "numeric": [">", 10] }] } ]
  }
}

Fields are ANDed; values in an array are ORed. Use aws events test-event-pattern to try patterns. On the new bus, the same syntax is used as a subscriber filter with scope DATA (wildcards differ; PutRawEvents payloads aren't nested under detail).

Targets, delivery roles and retries
  • Targets include Lambda, SQS, SNS, Step Functions, Kinesis, Firehose, API Gateway, API destinations (any HTTPS API), other event buses, and more. The new bus adds universal targets (arn:aws:events:::aws-sdk:service:apiAction) to call almost any AWS API.
  • EventBridge needs permission to call each target: a resource-based policy (Lambda, SNS, SQS) or an IAM delivery role.
  • Retries use exponential back-off with jitter. Classic: MaximumEventAgeInSeconds 60โ€“86,400 (default 86,400) and MaximumRetryAttempts 0โ€“185 (default 185). New bus subscribers: same ranges, defaults 300 s / 5.
  • Always attach a DLQ: without one, events that exhaust retries are dropped (visible only in metrics).
The wider EventBridge family
  • Pipes โ€” point-to-point: one source (SQS, Kinesis, DynamoDB Streams, MSK, MQโ€ฆ) โ†’ filter โ†’ enrichment (Lambda, Step Functions, API Gateway, API destination) โ†’ one target. You pay only for events that pass the filter.
  • Scheduler โ€” one-time or recurring schedules invoking AWS APIs; the recommended alternative to SQS delays longer than 15 minutes.
  • Schema registry โ€” discover event schemas and generate code bindings.
  • Global endpoints โ€” Regional failover for event ingestion (classic).

Real-world examples

๐Ÿ›ก๏ธ Security automation

An AWS service event such as "security group changed" lands on the default bus. A rule sends it to a Lambda function that checks the change and, if it opens port 22 to the world, reverts it and pages security.

๐Ÿงพ SaaS integration

A support-desk SaaS partner sends "ticket escalated" events. A rule starts a Step Functions workflow that pulls customer data and alerts the account manager โ€” no polling code.

๐Ÿข Company-wide order backbone (new bus)

A platform team runs one shared orders bus. Fulfillment, billing and analytics accounts attach their own subscribers. Billing uses FIFO per order ID; analytics starts 30 days back to backfill a new dashboard.

Code (Python, new eventbridgev2 client)
import boto3, json
eb = boto3.client("eventbridgev2")   # new Custom Event Bus API

eb.put_events(
    EventBusArn="arn:aws:events:us-east-1:111122223333:event-busv2/orders/EXAMPLE1234567890abcdef",
    Entries=[{
        "Source": "com.shop.orders",
        "DetailType": "OrderShipped",
        "Detail": json.dumps({"orderId": "o-1001", "carrier": "UPS"}),
        "SystemMetadata": {
            "EventGroupId": "o-1001",              # FIFO order per order ID
            "DeduplicationId": "o-1001-shipped"}   # safe to retry
    }])
# up to 100 entries per call; PutRawEvents for Avro / Protobuf / raw bytes

โฐ Trial-ending reminders

When a user signs up, EventBridge Scheduler creates a one-time schedule for day 27 that invokes a Lambda function to send a "your trial ends soon" email.

Security

In plain English: control who can post to the bus, who can listen, and what each listener is allowed to see. Events often contain customer data โ€” trim it before it leaves.

Checklist

  • ๐Ÿ“œ Classic: bus resource policy grants events:PutEvents to specific accounts or your organization (aws:PrincipalOrgID).
  • ๐Ÿค New bus: share through AWS RAM with the narrowest managed permission (publish-only, subscribe-only, attach-sources); consumers own their subscribers; owner can RevokeResource.
  • ๐ŸŽญ Each target/subscriber uses a delivery role scoped to exactly that target action.
  • ๐Ÿ”’ Encryption at rest is on by default (AWS owned key). Choose a customer managed KMS key for control and CloudTrail visibility; new-bus KMS calls don't grow with event volume.
  • โœ‚๏ธ Use input transformers / JSONata to strip PII before third-party targets; API destination credentials are stored in AWS Secrets Manager via connections.
  • ๐Ÿ” Watch for event loops (a target publishing back to the same bus); the new bus reports LOOP_DETECTED.
  • ๐Ÿ•ต๏ธ When an SQS DLQ or target trusts events.amazonaws.com, scope with aws:SourceArn (rule or subscriber ARN).

Example classic bus policy

Any account in the organization may publish, but only with its own source names.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "OrgCanPublish",
    "Effect": "Allow",
    "Principal": "*",
    "Action": "events:PutEvents",
    "Resource": "arn:aws:events:us-east-1:111122223333:event-bus/orders",
    "Condition": {
      "StringEquals": { "aws:PrincipalOrgID": "o-exampleorg" },
      "StringLike":   { "events:source": ["com.shop.*"] } }
  }]
}

Flashcards

Quiz

๐Ÿงฉ More decoupling services & patterns

SQS, SNS and EventBridge cover most needs. These services handle streams, legacy protocols, workflows, scheduling and delivery.

๐ŸŒŠ Amazon Kinesis Data Streams

Plain English: a security-camera recording of data. Everything is kept in order for a while; many apps can watch and rewind independently.

Technical
Shards are ordered, append-only logs; partition key picks the shard. Retention 24 h default, up to 365 days. Provisioned or on-demand capacity. Enhanced fan-out gives each consumer dedicated read throughput. Use for clickstreams, IoT telemetry, CDC, real-time analytics.

โ˜• Amazon MSK

Plain English: the same idea as Kinesis, but it's the popular open-source Apache Kafka, run for you.

Technical
Managed Apache Kafka (provisioned or serverless) with Kafka APIs, Connect and the wider ecosystem. Choose it when you already use Kafka or need Kafka-specific tooling.

๐Ÿ“จ Amazon MQ

Plain English: a classic message broker for older applications, so they can move to AWS without being rewritten.

Technical
Managed Apache ActiveMQ and RabbitMQ. Industry-standard protocols: JMS, AMQP, MQTT, STOMP, OpenWire. Best for lift-and-shift; for new builds SQS/SNS are simpler and scale further.

๐Ÿ”€ AWS Step Functions

Plain English: a recipe card that a conductor follows step by step โ€” including "wait for the manager to approve" and "undo if payment fails".

Technical
State machines (ASL) with built-in retries, catch, parallel/map, wait-for-callback tokens, and 200+ service integrations. Standard workflows (long-running, exactly-once, auditable history) vs. Express (high-volume, short-duration). Use for orchestration and sagas.

๐Ÿšฐ EventBridge Pipes

Plain English: a pipe with a sieve and a seasoning station: take items from one place, keep only the ones you want, add missing info, deliver to one place.

Technical
One source (SQS, Kinesis, DynamoDB Streams, MSK, MQ, self-managed Kafka) โ†’ filter โ†’ enrichment (Lambda, Step Functions, API Gateway, API destination) โ†’ one target. Replaces custom "glue" Lambda functions. Charged only for events that pass the filter.

โฐ EventBridge Scheduler

Plain English: a giant alarm clock that can trigger any AWS action at an exact time or on repeat.

Technical
One-time (at), rate and cron schedules with time zones, flexible windows, retries and DLQs; invokes SQS, Lambda, Step Functions and many AWS APIs via universal targets.

๐Ÿšš Amazon Data Firehose

Plain English: a delivery truck that collects streaming data and drops it into storage or analytics tools in batches.

Technical
Fully managed delivery to S3, Redshift, OpenSearch, Splunk, Apache Iceberg tables, HTTP endpoints and partners, with buffering, compression, format conversion and Lambda transforms.

ฮป Lambda async destinations

Plain English: when a function finishes (or fails) in the background, automatically tell someone.

Technical
For asynchronous invocations, on-success / on-failure destinations route the result to SQS, SNS, EventBridge or another Lambda function. Async payloads now up to 1 MB.

Simulator: how a stream differs from a queue

Common decoupling patterns

Fan-out (SNS โ†’ SQS)

One topic, many queues. Each service gets its own durable copy and pace.

Event router (EventBridge)

Producers emit facts; consumers subscribe by content. Teams evolve independently.

Load leveling (SQS)

A queue absorbs bursts so a fragile database or third-party API is called at a safe rate.

Claim-check (S3 + message)

Store large data in S3; send only the pointer. Encrypt the bucket and restrict access.

Saga (Step Functions)

Long business transactions across services, with compensating "undo" steps on failure.

Transactional outbox

Save the change and an outbox row in one transaction, then publish โ€” e.g., with EventBridge Pipes reading DynamoDB Streams.

Flashcards

Quiz

๐Ÿ” Security for messaging & events

Messages often carry orders, payments and personal data. Protect them in layers: who can access, how they travel, how they're stored, and how you'd notice a problem.

Defense in depth โ€” five layers

๐Ÿชช
1. Identity & access โ€” Who can send, read, subscribe or change settings?
IAM identity policies + resource policies (queue, topic, bus) ยท least privilege ยท separate producer and consumer roles ยท aws:PrincipalOrgID ยท AWS RAM for the new EventBridge bus.
๐ŸŒ
2. Network โ€” Can it be reached only the way we expect?
VPC interface endpoints (PrivateLink) for SQS, SNS and EventBridge ยท aws:SourceVpce conditions ยท endpoint policies.
๐Ÿ”’
3. Data protection โ€” Is the content unreadable to outsiders?
TLS in transit (deny aws:SecureTransport=false) ยท encryption at rest: SSE-SQS / SSE-KMS, SNS SSE-KMS, EventBridge AWS-owned or customer managed keys ยท minimize PII with transformers ยท SNS message data protection.
๐Ÿ›ก๏ธ
4. Service-to-service trust โ€” Can AWS services be tricked into acting for someone else?
Confused-deputy protection with aws:SourceArn / aws:SourceAccount ยท KMS key policies granting only the needed service principal ยท SNS signature verification on HTTP endpoints.
๐Ÿ”ญ
5. Detect & respond โ€” Would we notice?
CloudTrail for API activity ยท CloudWatch alarms on DLQ depth, message age, failed invocations ยท SNS delivery-status logs ยท AWS Config rules and Security Hub for encryption and policy drift.

Interactive checklist

Tick what's true for your workload to see your coverage.

Common mistakes

"SNS can't deliver to my encrypted queue."
The queue uses the AWS managed key (whose policy you can't edit) or the CMK policy doesn't allow sns.amazonaws.com. Use a customer managed key and grant the service principal kms:GenerateDataKey and kms:Decrypt.
"Anyone can send to our queue."
A queue policy with Principal: * and no conditions. Add aws:SourceArn, aws:PrincipalOrgID or aws:SourceVpce.
"Customer emails ended up in a partner's logs."
The whole event was forwarded. Use an input transformer / JSONata to send only what the partner needs.
"Failed events vanished."
No DLQ was attached, so retries ran out and events were dropped. Treat DLQs as mandatory and alarm on them.

Example: KMS key policy statement for SNS โ†’ encrypted SQS

{
  "Sid": "AllowSNSToUseKeyForOrderQueue",
  "Effect": "Allow",
  "Principal": { "Service": "sns.amazonaws.com" },
  "Action": ["kms:GenerateDataKey", "kms:Decrypt"],
  "Resource": "*",
  "Condition": { "ArnEquals": {
    "aws:SourceArn": "arn:aws:sns:us-east-1:111122223333:order-events" } }
}

Flashcards

Quiz

๐Ÿงญ Compare & choose

Side-by-side differences, and a quick decision helper.

Decision helper

Side-by-side

SQSSNSEventBridge (classic)EventBridge Custom Event Bus (2026)Kinesis Data Streams
Plain EnglishTo-do railLoudspeakerSorting officeSorting office that records everythingRecorded log
ModelQueue, consumers pullPub/sub, pushEvent bus, rules push to targetsEvent bus, subscribers push to targetsStream, consumers read
Keeps data?Until deleted (โ‰ค14 days)NoNo (archive optional)Yes, 1โ€“365 daysYes, 24 h โ€“ 365 days
Receivers per messageOneManyMany (โ‰ค5 targets/rule, many rules)Many (one target per subscriber)Many, independent
OrderingFIFO queuesFIFO topicsNoneFIFO per EventGroupIdPer shard
DuplicatesStandard: possible ยท FIFO: 5-min dedupStandard: possible ยท FIFO: dedupPossiblePublish-time dedup (5 min)Possible (producer retries)
FilteringNone (consumer decides)Filter policy per subscriptionRich event patternsRich patterns per subscriberNone (consumer decides)
Max payload1 MiB256 KiB (1 MiB opt-in)1 MBsee service quotassee service quotas
Failure handlingVisibility timeout + DLQDelivery policy + subscription DLQRetry 24 h/185 + DLQRetry 300 s/5 + DLQ recordConsumer retries from checkpoint
Sweet spotBackground jobs, bufferingBroadcast, notify peopleAWS/SaaS events, content routingOrg-wide event backbone across accountsHigh-volume telemetry & analytics
They work best together. A typical design: EventBridge routes business events โ†’ SNS fans out notifications โ†’ SQS buffers work for each service โ†’ Step Functions orchestrates long processes โ†’ Kinesis carries telemetry to analytics.

๐ŸŽ“ Final exam

15 random questions drawn from every module. Retake for a fresh set.

Review: all flashcards