Oyster with AWS SQS

Kushank Patel
4 min readMar 1, 2021

How SQS provide support to your application

AWS Provides different services to their customers to deploy their apps in a more reliable way and they provide different managed services like lambda and SQS. In these managed services they provide their managed servers so we don’t have to manage our own. SQS is a simple queue service that is fully managed by aws. it provides a service to manage the message queue in your production environment. It enables you to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message-oriented middleware and empowers developers to focus on differentiating work. By using SQS service developers have the capability to send, store, receive and queue the message between different components of the software. AWS manages all ongoing operations and underlying infrastructure needed to provide a highly available and scalable message queuing service. With SQS, there is no upfront cost, no need to acquire, install, and configure messaging software, and no time-consuming build-out and maintenance of supporting infrastructure.

Benefits of SQS

By using the SQS service we can achieve reliable data transfer between different components of our software. It provides the guarantee of delivering the message in a reliable way with encryption. So You can use Amazon SQS to exchange sensitive data between applications using server-side encryption (SSE) to encrypt each message body. SQS queues are dynamically created and scale automatically so you can build and grow applications quickly and efficiently.

What difficulties face by the oyster company before using aws services

Oyster.com is a new-york-based company that provides reviews of hotels in nearly 200 destinations worldwide. It provides high-quality photographs. Oyster takes hundreds of photos for each property, and every review includes dozens of untouched images that allow different visitors to compare hotels based on the hotel location and their property. Before using aws services they store these images in their databases but maintaining their own databases cost them more. And Oyster reprocesses its entire collection of photographic images a few times each year to update the copyright year and, if necessary, to change the watermarks. Using their previous solution, reprocessing the entire collection of photographs required about 800 hours to complete. In addition, Oyster often recreated existing images in new formats and sizes for mobile and tablet devices. Resizing existing images and adding new ones was slowing down the rate at which the company was able to process the collection. Oyster considered purchasing additional hardware but found the cost of new hardware and routine maintenance was too high, especially when the machines would sit idle most of the time. By using this method they have to manage more things and spend more upfront on resources.

Oyster with AWS

“We were already using Amazon S3 to store the images, so using Amazon EC2 to process the images was a natural choice,” Seidman says. Chris McBride, a software engineer at Oyster

Now they use S3 to store the data and EC2 to compute Within this new environment, the company connects to Amazon S3 and Amazon simple queue service (Amazon SQS) using boto, a Python interface to AWS. The images themselves are processed with the ImageMagick software available in the AMI package. Oyster’s old system needed approximately 400 hours to process one million photos. By using AWS, the company can process the same number of photos in about 20 hours a 95 percent improvement. It also Reduced in-house hardware expenses. Saves $10,000 in capital expenditure and reduced operating expenses by an additional $10,000.

--

--

Kushank Patel

I am pursuing my master's degree at University of Windsor and interested in devops field.