New York-based Oyster.com shares unvarnished reviews of hotels in nearly 200 destinations worldwide. The company’s own investigators visit each location to assess cleanliness, amenities, service and overall quality.
What sets Oyster apart from similar sites is its extensive collection of photographs. Oyster takes hundreds of photos for each property, and every review includes dozens of untouched images (submitted by guests as well as investigators) that allow potential visitors to compare a hotel’s marketing material with reality
The Challenge
Since its 2009 launch, Oyster has published more than one million high-quality digital images. When this massive volume of images became too cumbersome to handle in-house, the company decided to offload the content to a central repository on Amazon Simple Storage Service (Amazon S3).Oyster reprocesses its entire collection of photographic images a few times each year to update the copyright year and, if necessary, to change the watermarks.
Using their previous solution, reprocessing the entire collection of photographs required about 800 hours to complete. Resizing existing images and adding new ones was slowing down the rate at which the company was able to process the collection.
Why Amazon Web Service?
While the company is still running one local server, the bulk of the processing work now takes place on the AWS Cloud. Oyster is using a customized Amazon Linux AMI within Amazon EC2. Within this new environment, the company connects to Amazon S3 and Amazon Simple Queue Service (Amazon SQS) using boto, a Python interface to AWS. The images themselves are processed with the ImageMagick software available in the AMI package.
What is SQS?
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
Oyster uses Amazon EC2 instances and Amazon SQS in an integrated workflow to generate the sizes they need for each photo. The team processes a few thousand photos each night, using Amazon EC2 Spot Instances. When Oyster processes the entire collection, it can use up to 100 Amazon EC2 instances. The team uses Amazon SQS to communicate the photos that need to be processed and the status of the jobs.
The Benefits
Oyster’s old system needed approximately 400 hours to process one million photos. By using AWS, the company can process the same number of photos in about 20 hours — a 95 percent improvement.
It used to take close to a week to produce photos specifically for the iPad. With AWS, we can create the photos in just a few hours. The documentation is straightforward and the dashboards are incredibly helpful.
It took less time to rewrite the code and do a full processing job with AWS than it took to do a single run with the old method.”
Eytan Seidman,
Cofounder and Vice President of Product, Oyster