AWS - MQ - SAM
SNS - Amazon simple notification service
architecture
Overall
Amazon SNS Amazon Simple Notification Service
- a highly available, durable, secure, fully managed pub/sub messaging service
enables you to decouple micro-services, distributed systems, and serverless applications.
provides topics for high-throughput, push-based, many-to-many messaging.
a web service that enables to set up, operate, and send notifications to subscribing services from the cloud.
Using SNS topics, your publisher systems can fan out messages to a large number of subscriber endpoints for parallel processing, including SQS queues, AWS Lambda functions, and HTTP/S web-hooks.
can be used to fan out notifications to end users using mobile push, SMS, and email.
EC2, S3, and Lambda can all work with SNS.
The default maximum for SNS topics per account is 100,000.
The default limit to the number of SNS subscriptions per topic is 12,500,000.
- When using SNS,
- owner create a topic and control access to
- defining policies that determine which publishers and subscribers can communicate with the topic.
- Publisher
- human/alarm/event that gives SNS the message that needs to be sent
- send messages to topics they created / have publishing permission rights to.
- Instead of including a specific destination address in each message, a publisher sends a message to the topic.
- Topic
- an “access point” for allowing recipients to dynamically subscribe for identical copies of the same notification.
- How to label and group different endpoints that send messages to.
- can also use topics to group multiple recipients.
- One topic can support deliveries to multiple endpoint types.
SNS matches the topic to a list of subscribers who have subscribed to that topic, and delivers the message to each of those subscribers.
Each topic has a unique name that identifies the SNS endpoint for publishers to post messages and subscribers to register for notifications.
- Subscribers receive all messages published to their subscribed topics, and all topic subscribers receive the same messages.
- an “access point” for allowing recipients to dynamically subscribe for identical copies of the same notification.
- Subscriptions: the endpoints that a topic send messages to.
- owner create a topic and control access to
SNS also can push messages to mobile devices or distributed services via API or an easy-to-use management console.
You can seamlessly scale from a handful of messages per day to millions of messages, or more.
- With SNS, can publish a message once and deliver it one or more times. can choose to direct unique messages to individual Apple, Google, or Amazon devices, or broadcast deliveries to many mobile devices, with a single publish request.
Amazon SNS supports a number of subscriber types for communication :
- Email, plain or JSON
- Web ports 80 or 443
- Short message service
- Amazon SQS
- Mobile push messaging
- AWS Lambda function
For Amazon SNS, you are guaranteed a one-to-many communication
- Single published message
: all notification messages contain a single published message.
- each message is delivered at least once.
- no guarantee that a message will be received or processed, which is why SNS should be used with a managed service, like SQS.
- message order is not guaranteed or relevant
- Amazon SNS attempts to deliver messages from the publisher in the order in which they were published into the topic.
- However, network issues can potentially result in out-of-order messages at the subscriber end.
- messages cannot be recalled.
- once a message is delivered successfully, there is no recall feature.
- SNS allows for HTTP/HTTPS retry
- the maximum size per message is 256 KB.
- HTTP/HTTPS retry means that an Amazon SNS delivery policy can be used to control the retry pattern (linear, geometric, or exponential backoff…), maximum and minimum retry delays, and other parameters.
- For non-SMS messages, such as XML, JSON, and unformatted text
- Amazon SNS allows up to 64 KB per message, with a maximum of 256 KB, for four requests of 64 KB each.
- Each 64 KB chunk of published data is billed as one request.
- For example, a single API call with a 256 KB payload is billed as four requests.
massage persistency
One key difference is SNS does not offer message persistency
This is why using Amazon SQS with SNS is important.
If you are using a fan-out case, SQS provides a place where the message will be caught or processed, because SQS is always available.
Due to the invisibility timeout and locks, SQS has message persistence, and messages are guaranteed to be processed.
- As far as delivery differences
- SNS allows applications to send time-critical messages to multiple subscribers through a push mechanism.
- SQS exchanges messages through a polling model—sending and receiving components are decoupled.
- When comparing producer and consumer differences
- SNS
- uses a publish and subscribe mechanism,
- distribution model is one to many
- SQS
- is a send and receive model.
- distribution model is one to one .
- provides flexibility for distributed components of applications to send and receive messages, without requiring each component to be concurrently available.
- SNS
- SNS works closely with SQS.
- Both services provide different benefits for developers.
- SNS allows applications to send time-critical messages to multiple subscribers through a “push” mechanism, eliminates the need to periodically check or “poll” for updates.
- SQS is a message queue service used by distributed applications
- to exchange messages through a polling model
- can be used to decouple sending and receiving components without requiring each component to be concurrently available.
- By using SNS and SQS together, messages can be delivered to applications that require immediate notification of an event and also persisted in an SQS queue for other applications to process at a later time.
use case
use case for fan-out.
a mobile phone uploading a cute kitten picture.
an SNS topic for the kitten picture.
- Once the upload is complete, the kitten picture is sent to three SQS queues:
- One to generate a thumbnail
- One to size the image for mobile
- One to size the image for the webIn this scenario
the SNS and SQS work together to produce the fan-out.
- It is the same message but it has three sets of instructions for one image.
end-to-end image processing scenario
phone uploads the kitten picture to an S3 bucket for ingestion.
- uploading an image to S3 triggers an event notification
- The S3 bucket sends an event notification/message to the SNS topic automatically, which sends it to the SQS queues to process the work based on the size of the image.
Next, SQS connects to the Auto Scaling groups that each have a job to do .
Once the message is processed, the Auto Scaling groups send it to the S3 bucket, which serving content to Amazon CloudFront.
- From CloudFront, the appropriately sized picture can be downloaded.
- Microservices example.
- This service-oriented architecture is a design pattern
- application components provide services to other components via a communications protocol, typically over a network.
- A service is a self-contained unit of functionality, such as retrieving an order status. Using that definition, a service is a discretely invoke-able operation.
- SOA is widely adopted by large systems to integrate business processes, such as banking.
- The architecture shown on the left illustrates a monolithic application.
- Users connect to three separate instances, and all three do file ingestion, processing, and publishing.
- Anything affecting any instances could impact the application.
- In contrast, the microservices architecture
- uses file ingestion, SQS, and SNS, which are all components that can scale to meet demand.
- Additionally, the entire architecture is serverless, except for the instances in the automatic scaling groups that perform certain functions, like file ingestion and processing.
- In this example, a file is uploaded for ingestion, a put message is created, and then the message goes to a queue for processing. The automatic scaling file processing group processes the file and sends a notification when it’s complete. At the same time, the file is pushed into a bucket.
Comments powered by Disqus.