AdRoll is a global leader in digital advertising retargeting products. Companies of all sizes use AdRoll’s Real-Time Bidding (RTB) platform to create personalized ad campaigns using their own website data. RTB is a live auction marketplace for ad inventory sold on a per-impression basis for the web, social media networks, and mobile apps. The platform gives advertisers the ability to maximize ad placement quality, while minimizing price.
Established in 2007, AdRoll grew rapidly as it partnered with new exchanges to increase reach and maximize efficiency for advertisers. The company estimates that more than 20,000 advertisers worldwide use its retargeting technology to collect, analyze, and act on anonymous website visitor data. Between 2011 and 2013, the volume of data that AdRoll collected worldwide grew from 50 GB to 20 TB of data per day.
By October 2014, data collection skyrocketed to 150 TB each day and AdRoll decided to move its data- and computation-intensive processes closer to customers worldwide. This would allow the company to meet its service level agreement (SLA) of 100 millisecond bid response time. “We needed infrastructure with a presence in all of our target geographies, so we started looking for a solution provider able to handle importing and processing 30 TB of compressed data and 60 billion requests from advertisers per day,” explains Valentino Volonghi, chief technology officer at AdRoll. “We had a 100 millisecond latency requirement and wanted to get close to 100 percent uptime.”
Why Amazon Web Services
AdRoll chose Amazon Web Services (AWS) because of its global presence, the maturity of the platform, and its ability to scale services to support business growth. Additionally, the company can take advantage of AWS application programming interfaces (APIs) to manage the platform’s products and services across multiple regions without replicating staff and other resources.
AdRoll built its RTB system on more than 500 c3.4xlarge Amazon Elastic Compute Cloud (Amazon EC2) instances. The company uses Amazon DynamoDB database service with the RTB platform to process more than 500,000 requests per second. Amazon Simple Storage Service (Amazon S3) stores more than 150 TB of bid data.
“AWS provides us with the capability to handle traffic that comes from Facebook, Google, Yahoo, and other heavily traveled sites so that we can serve up more than 50 billion impressions a day,” Volonghi says. “It’s cost-effective, too—we spend more on snacks than we do on Amazon DynamoDB.”
How AdRoll Saves Money by Building and Growing on AWS
AdRoll’s leadership estimates that the company has been able to reduce IT costs well below industry benchmarks for companies processing similar sets of data on premises. “By leveraging the AWS global infrastructure and services, we’ve been able to reduce fixed costs by 75 percent and annual operational costs by 83 percent,” says Volonghi.
AdRoll uses a combination of On-Demand, Reserved, and Spot Instances to operate the RTB platform. While most of AdRoll’s tasks run on On-Demand and Reserved Instances, the company uses Spot Instances to run big data jobs on Hadoop clusters. Analytics tasks, for example, require thousands of machines in an unpredictable quantity. By taking advantage of the Spot market for intensive, non-time-critical computation processes, AdRoll is able to reduce costs by nearly 80 percent compared to On-Demand Instances.
“We run the RTB platform on more than 2,500 machines, approximately eight hours a day globally, at a cost of less than $0.05 per day per machine,” says Volonghi. “If we wanted to replicate our AWS environment on premises, we would have to operate four data centers with on-call staff in each location, provision nearly 1,000 machines in each location, add an additional 10-20 percent capacity for cold storage, and develop code for auto-scaling and a common API.”
AdRoll estimates that it would need at least 20 full-time engineers to effectively manage the physical environment:
8 full-time employees on call across four different data centers (two in each location for redundancy)
1 product manager for auto-scaling and API management
5 engineers to develop and maintain auto-scaling across the infrastructure
5 engineers to maintain Cassandra installation instead of DynamoDB
1 engineering manager
The overall annual staffing costs, with an average engineering salary of $150,000, would be $3 million.
Using Amazon S3 for storage helps the company reduce storage and management costs. AdRoll’s Amazon S3 buckets generate as much traffic as some of the most visited properties on the web. “We process the same amount of data that a storage company does, but we don’t need the same level of engineering resources dedicated to managing that storage,” says Volonghi. “We can handle this large, complex task with ease and minimal burden. For example, now that we’re running on AWS, we can manage the RTB workload successfully using just two engineers instead of eight.” AdRoll deployed a 180 TB storage environment using Amazon S3 for well under 10 percent of the cost of an on-premises equivalent.
In addition to significantly lower costs, AdRoll gained the flexibility to scale capacity quickly to meet unexpected increases in demand on its platform. “Using AWS enables AdRoll to onboard new clients of any size, including massive ones, without having to source additional engineering support,” says Volonghi.
The AWS platform also gives AdRoll the ability to quickly recover from catastrophic failures. “Thanks to our software stack on AWS, we can recreate our entire platform anywhere in the world—in fact, that’s how we execute our software releases,” Volonghi says. “We have the flexibility to migrate to different regions or availability zones in less than one hour in case of emergency.”
Using AWS has also helped free engineering resources for more valuable projects than maintaining infrastructure. “Because we’re running on AWS, we’re able to focus 95 percent of our staff on new product development,” says Volonghi. “Using AWS allows us to focus on innovating our platform and solving customer problems.”
“Leveraging the AWS platform, we’ve been able to seamlessly scale our infrastructure, better serve our customers across the globe, and reduce our fixed costs by 75% and operational costs by 83%”
Valentino Volonghi, CTO