In 2007, VivaReal was a small Internet start-up that had just developed the first version of their real estate marketplace system. When it came time to launch, the team (based out of Bogotá, Colombia) decided to use Amazon Web Services as its hosting environment. Today, the VivaReal Network includes 6 unique marketplaces across Latin America and the U.S. with a central property database of approximately one million properties.
“Flexibility was one of the key factors why we chose to use Amazon Elastic Compute Cloud (Amazon EC2) for hosting,” recalls Thomas Floracks, COO of the VivaReal Network. “As a start-up we were unsure about our hardware requirements and we had no time or resources to do extensive load testing or scalability simulations. So we just had to jump in and hope that everything went well. Amazon gave us the possibility to be totally flexible with the amount of servers we used and hardware configurations.”
“The utility computing model was also very attractive. There was no risk of buying, renting the wrong servers or renting too many servers. In other classical hosting or collocation services, where we had to rent or buy the hardware, upgrades and changes to the hardware would have been very expensive.”
Why Amazon Web Services
Floracks explains their architecture, “The VivaReal Network infrastructure consists of 3 layers: web servers, applications servers and backend (database) servers. In total we use 16 EC2 server instances to run the VivaReal platform. Each marketplace (domain) has its own instance running on Apache TomCat 6.x. For the TomCat web servers we use a medium instance with 1.7GB of memory and two processor cores running at 2.5GHz. Additional AWS services allow us to dynamically scale the number of servers of each marketplace up and down depending on the load: Amazon CloudWatch, Auto Scaling and Elastic Load Balancing. Amazon CloudWatch allows us to monitor the use of resources of each Web Server, e.g. free memory, CPU time or bandwidth used. When a web server has a high load and one of the monitored resources passes a critical limit the CloudWatch service reports to the elastic load balancing service which will then use the auto scaling service to replicate the web server and divide the load between two TomCats. As AWS only charges for the resources we actually use this service helps cut cost as we do not have to buy the hardware for the highest load scenario. When needed, the amount of servers scales up (during the day) and when not needed (at night) the additional servers will shut down and we don’t pay for them anymore. The maximum and minimum amount of servers can be defined when using elastic load balancing. Elastic Load Balancing additionally provides a basic failover mechanism. If one server stops responding on a port monitored by CloudWatch the elastic load balancer will shut down the instance and replace it with a new, functional instance.”
Floracks continues, “Another AWS service that has proven to be very helpful in the VivaReal platform is Amazon Simple Query Service (Amazon SQS). SQS provides a FiFo queue for plain text objects. The lifetime of an object in the queue is 4 days the maximum allowed size of an object 8kb. It has proven to be very useful whenever our application requires an asynchronous behavior. A good example in our system is the processing of XML feeds. We receive massive amounts of properties in XML format, sometimes over 100,000 property entries in one XML file. Our system starts processing the feeds and writes objects to the SQS queue. These objects are read from the queue and written to our database in a controlled way by another process. If we would write all information directly to the database we would risk blocking or overloading the database server. The asynchronous processing with the help of the SQS allows us to buffer the information in the queue and intelligently process it depending on the load on our database servers.”
VivaReal also uses Amazon CloudFront to deliver property photos and take the load off their web servers.
In addition to hosting their marketplaces on AWS, VivaReal also finds it useful to simulate and test their systems at any time. “Typical stress testing can be very expensive to rent or buy the servers needed to deploy a 1:1 replica of a production system,” says Floracks. “Cloud hosting offers a tremendous cost advantage over classic hosting. Many companies simply cannot afford to purchase twice the amount of server to replicate a test environment and often times have to use scaled down versions of their production system for testing – with all the risks involved. Amazon Web Services, cloud hosting and utility computing make this problem an issue of the past.”