VivaKi is part of Publicis Groupe, a French communications company headquartered in Paris, France. Publicis Groupe is one of the largest communications companies in the world, providing a full range of advertising services through a global network of agencies.
The company is active in 104 countries on five continents and has more than 96,000 employees. Publicis Groupe formed VivaKi in 2008 to accelerate the digital transformation and expertise of Publicis Groupe and its agencies. VivaKi is based in Chicago, Illinois and specializes in developing services, tools and next-generation digital platforms.
Publicis Groupe has nearly 50 agencies in its worldwide network who create digital advertising campaigns to meet the goals that its customers define, including audience reach, impressions and conversation rates. VivaKi works with ad servers, publishers and data management platforms (DMPs) to pull data and provide daily campaign effectiveness summary to agencies. A campaign can last from a few weeks to a year. Agencies use the summary report to make spending and channel adjustments that achieves the most efficient return on investment for their clients.
Every ad server provides a set of summary reports that provide a high level of overview of a campaign’s spend versus conversion. However, these reports didn’t supply the level of detail that VivaKi needed. “Our customers have specific requests around audience groups and other targets that ad servers cannot provide,” says Vice President, Infrastructure and Operations, Zhong Hong. “Furthermore, gathering data to identify how effective campaigns are by geographic location is critical and this data isn’t available in the reports.”
Before Amazon Web Services (AWS), VivaKi used a proprietary solution built on a cluster of 200 in-house servers to process data. It would take almost six hours to produce a summary report with data from just one ad server. Additionally, the cluster wasn’t designed to manage spikes in traffic. “During holidays, like Christmas or Thanksgiving, our daily statistics are about ten times higher than normal,” Hong explains. “The system didn’t meet our peak power needs. As a result, we had lengthy processing times during peak periods that impacted our ability to deliver data on time.”
Why Amazon Web Services
After considering several solutions, VivaKi chose AWS because of the maturity of the AWS platform, operational scale, security and pricing. “From a business perspective, processing scale means a lot because the more you can process, the more you can service,” says Hong. “Furthermore, we had a lot of conversations with AWS and it was clear that AWS builds its services with customer security in mind. And of course, the pricing, which is a result of operational efficiency and proper management.”
Publicis has a team of mathematicians who need to work with large volumes of data to create advanced analytical models and VivaKi developed a suite of products that Publicis agency teams can use for analytics. The key product is a data management platform known as SkySkraper that consolidates data and enables deep analysis for business analysis insights.
In-depth data analysis
Using AWS, Hong and his team created a highly elastic, scalable architecture to run SkySkraper. AWS is the hosting environment for processing, data storage, and access to the data. Amazon Elastic Compute Cloud (Amazon EC2) provides the computing platform to process the data and Amazon Simple Storage Service (Amazon S3) is the mechanism that VivaKi uses to store the data in an economical and reliable way.
Amazon Elastic MapReduce (Amazon EMR) uploads log level data to Amazon S3 for use on Amazon Redshift. Amazon Redshift acts as the data warehouse. VivaKi pushes data into a 2XL cluster to process and organize the data into consumable data formats such as data cubes. A data cube provides an easy-to-use mechanism for querying data with quick and uniform response times that VivaKi’s analysts can use to run research projects and perform in-depth analysis.
Hong comments, “We don’t have to pre-allocate resources and can easily scale up to meet demand and then scale down for efficiency. We can use Amazon Redshift to load six months’ worth of raw data or data from several channels onto just one Redshift cluster. To expand, you just pop in another node and work on the data distribution. The performance and resources are right there, so the analytics team can easily work with several terabytes of data.” VivaKi manages thousands of campaigns simultaneously and adds about 3 TB of data to Amazon Redshift monthly.
VivaKi uses Amazon ElastiCache for in-memory caching and Amazon DynamoDB to store reference data and as a fallback for caching data. Configuration and reference data is stored in Amazon DynamoDB.
“The summary-level data that we receive from an ad server like Google doesn’t have the critical geo data that we’re looking for,” explains Hong. “Now we can use Amazon Redshift to extract this information from log-level data.”
VivaKi keeps a floating window of six months of log level data in Amazon Redshift and can process the data to produce a summary-level report that includes geographical data. “The value of this data is twofold,” says Hong. “First, it provides data missing from the ad server summary and it also allows us to verify the accuracy of the ad server summary-level data. We can compare the data and have a higher level of confidence in our ad server partners.”
By using AWS, VivaKi improved its ability to process and deliver data to customers. Amazon Redshift pulls data into SkySkraper and runs daily processing to aggregate and link data and produce connected insights from the data. “We needed to load six months’ worth of data, about 10 TB of data, for a campaign,” says Hong. That type of load would have taken about 20 days with our previous solution. By using Amazon Redshift, it only took six hours to load the data.”
VivaKi estimates significant operational savings due to the efficiency of the AWS Cloud. “We support about 50 Publicis agencies worldwide,” says Hong. “If we were using a traditional data center, it would take a team of at least 40 people to support the infrastructure and services that we offer. With AWS, we have a team of 15 people supporting a global organization. That’s a 60 to 75 percent reduction in cost.”
“Efficiency is great but capability is even more crucial,” Hong continues. “We need to deliver what the business demands. With our previous infrastructure, we were limited in terms of what we could deliver. By using AWS, our team has delivered the products and services that the business needs, improved our solutions to keep up with growth, and managed the bottom line to stay profitable. AWS is a fantastic platform — I've been using AWS for more than three years and I'm loving it.”
VivaKi plans to use Amazon Redshift to establish a centralized data store for agency teams to hold data on all media types, including display, search, mobile, search, and video. Hong says, “The data volume will be huge. The flexibility and performance of Amazon Redshift will make this possible.”