About Financial Times
The Financial Times (FT) is one of the world’s leading business news organizations, with a combined paid print and digital circulation of almost 652,000. For more than 125 years, FT has given business leaders the information and analysis they need. FT operates in the same fast-paced world its readers do, so the company knows that maintaining its high level of customer satisfaction depends on making accurate, meaningful decisions as quickly as possible about everything from which stories to publish to the subscription options to offer. The FT family includes the FT newspaper and FT.com, Financial Publishing, FT Chinese, Medley Global Advisors (MGA) and FT Labs. FT education products now serve 37 of the world’s top 50 business schools.
FT has long embraced the opportunities presented by customer data, and has developed proprietary methods for analysing customer engagement, using business intelligence (BI) throughout its operations. Editors and journalists use BI to decide which stories to cover. Marketing uses it to understand campaign performance. Sales uses it to set appropriate subscription prices.
For nearly a decade, data growth consistently exceeded forecasts, and the data warehouse required re-platforming approximately every three years. A third party hosted, managed, and helped develop that data warehouse, which was expensive. The only way to scale up was to add headcount, and changing management was difficult and slow. To run intense analytics on the limited-capacity data warehouse, work had to be scheduled in advance. Those constraints meant analysts weren’t exploring potentially important business questions.
When FT again needed to re-platform, it went in search of a cheaper, more scalable solution, knowing that its data use would only grow with time. “Our data had already grown far beyond what we predicted it would—and it may never level off,” says John Kundert, head of BI delivery. “We needed a scalable solution that would grow with us.”
The company also wanted access to near-real-time data to make better decisions, more quickly. Issuing an request for proposal (RFP), evaluating the responses, and choosing a vendor can take several months or longer. “The environment we are working in is so dynamic and fluid that by the time we got to the end of the RFP process, we felt the conclusions would probably be invalid,” Kundert says.
The FT BI team decided to experiment with Amazon Redshift to see whether it would fit the company’s needs. The company uploaded two years’ worth of FT.com behavioural data and added a reporting tool, for an initial load of approximately 2 billion records that covered every click on FT.com. Next, analysts compared the results to results from FT’s conventional data warehouse, which was built using a Microsoft server technology stack and they matched.
Why Amazon Web Services
Amazon Redshift performed so quickly that some analysts thought it was malfunctioning—they were used to running queries overnight. They found that the results were indeed correct, just much faster. “Some of the queries we’re running are 98 percent faster, and most things are running 90 percent faster,” says FT CTO John O’Donovan. “And the ability to try Redshift out before having to invest a significant amount of capital was a huge bonus.” FT decided to use Amazon Redshift as its pure-play data warehousing layer.
“Amazon Redshift is the single source of truth for our user data,” O’Donovan says. “It stores data on customer usage, customer service, and advertising, and then presents those data back to the business in multiple views.”
For FT, increased speed means better business decisions. With its previous data warehouse, FT used weekly reports on subscription acquisitions and losses, which meant it was making decisions based on data that were already at least a few days out of date. Now, FT analysts can access and query clickstream data in seconds or minutes instead of hours with the previous data warehouse.
During the implementation process, the company found that data loading was fast and didn’t require much customisation. Migrating to the cloud also removed the burden of backup planning and other database administration tasks. Analysts were able to investigate business questions without a lot of training—reducing the cost and time needed to train them on a new tool.
By using Amazon Redshift, FT is supporting the same business functions with costs that are 80 percent lower than before. Headcount has not increased, and queries run much faster.
In addition, the added speed means the business can conduct more comprehensive and sophisticated analyses as well as develop new solutions to drive revenue and readership. For example, FT developed its own web app that enables journalists to see, in real time, how their articles are performing and who’s reading them. This tool delivers data directly to the employees responsible for creating FT’s content.
Thanks to its data collection and analysis, FT can show that readers spend time on articles rather than quickly clicking on a page and then away from it. FT’s standard for viewability is five seconds, compared with the industry standard of half a second. “Our standard is 10 times higher, so you know that if an ad shows as viewable on FT, it was viewable,” O’Donovan says. These insights help FT demonstrate to advertisers the value of an ad on FT.com and drive revenue. Neither of these innovations were possible with the previous data warehouse because of its capacity restrictions and slower processing speeds.
Now that FT has eliminated capacity restrictions by migrating to Amazon Redshift, it doesn’t have to choose which business questions to explore, but instead, it can explore all of them. In addition, analysts can add more variables to queries to make them more statistically relevant. The business can spot and analyse opportunities, try solutions, and assess the results in close to real time. “Being able to explore near-real-time data improves our decision making massively. We can make decisions based on what’s happening now rather than what happened three or four days ago,” O’Donovan says. “Redshift is the engine that drives our decision-making: that’s how important it is to us.”