We will be discussing the following topics in this blog:
Watch this AWS Tutorial for beginners to learn more about AWS
What is AWS SQS?
Amazon SQS was the first service introduced in AWS. It gives users access to message queues. This queue is used to keep messages while it waits to be processed. Amazon SQS enables services on the web to quickly queue messages that are sent by one component to be processed by another component.
AWS SQS helps in sending, storing, and receiving messages without the fear of losing them in the process. These messages can store up to 256 KB of data as text in formats such as JSON, XML, etc. All these messages can be retrieved by the applications with the use of AWS SQS API. The SQS queue also resolves any issues that arise when the component sending messages is working a lot faster than the component that has to process the messages.
Types of Queues in SQS
There are two types of queues in Amazon SQS:
| Standard Queue | First-in-first-out (FIFO) Queue |
| Default queue type in Amazon SQS | Complements the standard queue |
| Unlimited transactions per second; messages can be delivered in a different order than how they were sent | 300 messages per second; messages are received in the same order they were sent |
| A message is delivered at least once; more than one copy can be delivered | A message is delivered only once and is available until processed by the consumer; no duplicates allowed into the queue |
Amazon SQS Functionality
Amazon SQS comes with the following functionalities:
- Users can creates unlimited queues with unlimited messages in any particular region.
- Message payload in SQS can contain upto 256KB of text. Every 64KB of message is processed as one request.
- Users can send, receive or delete messages in a batch of upto 10 messages (or 256KB). A batch costs the same as a single message so using batches could be a cost effective solution for users.
- Users can reduce irrelevant polling, minimizing costs in turn. When the queue is empty, long poll requests take up to 20 seconds for the next message. The cost for long poll requests is the same as regular requests.
- Messages in queues can be retained for up to 14 days.
- Received messages are locked while they are being processed. This way other systems won’t be able to process the same message at the same time. In case the message processing fails, the lock expires and the messages becomes available again.
- Users can share queues securely with specific AWS accounts or anonymously.
Amazon SQS Visibility Timeout
Amazon SQS Visibility Timeout is basically the total time a message remains invisible in the SQS queue after it is picked up by a user. The message stands to be deleted from the queue if the job is processed before the visibility timeout expires. If the job is not done until then, then the message becomes visible again with another reader processing it. This sometimes results in the same message being delivered twice.
The default visibility timeout is at 30 seconds. This can be increased if the job takes longer. The maximum visibility timeout is 12 hours.
Benefits of Amazon SQS
There are four main benefits of Amazon SQS:
- Eliminate Administrative Overhead: There is no need to install, assemble, or acquire messaging packs with AWS SQS. There is no need for building or maintaining infrastructure either. Amazon SQS queues are created so that they can scale accordingly, and hence, they eliminate a huge chunk of administrative work.
- Reliably Deliver Messages: AWS SQS can be used to deliver any amount of data without losing any of it. It helps in increasing the fault tolerance of the process by decoupling application parts. Several copies of each message are put out on hold in various accessible zones. This makes sure that the messages are available whenever required.
- Keep Sensitive Information Secure: Serverside Secret Writing or SSE is used to exchange sensitive data between apps via Amazon SQS. It also comes with integration with AWS Key Management Service (KMS), so we can manage keys that secure other AWS resources.
- Scale Elastically and Cost-effectively: Since Amazon SQS is a part of the AWS Cloud, it can use its scalability feature when required. It scales elastically with apps.
Limitations
Even though there are a lot of benefits of Amazon SQS, it has some limitations as well:
- Inflight Messages: Inflight messages are messages that are never deleted by a customer after receiving them. Each queue has a limit of 120,000 inflight messages and this limit goes down to 20,000, if it is a FIFO queue. If there are too many inflight messages in a queue, then the OverLimit error pops up.
- Message Size: The maximum message size is 256 KB, which is quite less, and hence, one has to be careful so as to not surpass it. Whenever one enters a message that is approaching the upper size limit, the system detects it; so, one always has to leave at least 10 percent overhead on the message size.
- FIFO Queue Throughput: As already discussed, in FIFO queues, the maximum transaction limit is 300 messages per second. One needs to use multiple queues to beat this limit or increase the limit from AWS Support.
Get 100% Hike!
Master Most in Demand Skills Now!
Companies Using Amazon SQS
There are a lot of companies that use Amazon SQS. The following are just a few of them:
- NASA
NASA’s image and video library contains over 140,000 videos, still images, and audio recordings. Its architecture includes Amazon SQS to help decouple the incoming jobs from all the pipeline processes. It also includes AWS SNS which triggers the processing pipeline as new content is updated.
- Capital One
In their quest to modernize their retail message queuing, Capital One decided to move to Amazon SQS. They are using SQS to migrate important bank applications to the cloud. This will help ensure high availability along with cost efficiency.
- BMW
BMW was able to build its car-as-a-sensor or CARASSO service in six months using various AWS services – Amazon SNS was one of them.
- redBus
AWS SQS has been helping redBus enable messaging between their external and internal applications. It also comes in handy for monitoring and alerting purposes.
- EMS Driving Fuel IQ
Fuelsite by EMS, an AWS IoT-enabled solution, is helping petroleum retailers in Australia by improving safety and performance. Fuelsite helps retailers by gathering data from the sensors located in service stations. The AWS IoT Device Management is used to control the edge devices and Amazon SQS facilitates the scheduling of messages to and from them.
- Change Healthcare
Change Healthcare utilizes Amazon SQS to handle confidential transactions from its clients on a daily basis.
- Oyster
Oyster.com uses Amazon SQS to process and store images for its website. SQS helps communicate the photos that need to be processed and the status of the jobs.
Conclusion
Amazon SQS helps companies send an unlimited number of messages around the world. These messages can be up to 256 KB in size. Sending and receiving messages can be done at the same time. These messages can be secured by using the keys generated by AWS KMS.