CyberAgent is an Internet media-services company that operates one of Japan's leading streaming platforms, called FRESH! Today FRESH! offers roughly 1,400 channels and approximately 37,000 programs, ranging from broadcasters' streaming services to original series. CyberAgent uses an Amazon Aurora data store for its write-intensive microservices—such as timelines and chat—and a MySQL database on Amazon Relational Database Service (Amazon RDS) for the remaining microservices APIs. All the APIs are fronted by Amazon CloudFront and Amazon API Gateway. For processes that might affect API response time, CyberAgent implemented a set of workers who receive jobs through an Amazon Simple Queue Service (Amazon SQS) queue.
CyberAgent decided to implement all of its FRESH! microservices as Docker containers.
For local development, CyberAgent engineers have been using Docker and Docker Compose.
Very few engineers had experience deploying Docker containers in a production environment at the scale required. In addition, there was no internal track record when it came to operating a full-scale container-management solution.
CyberAgent decided to use Amazon EC2 Container Service (Amazon ECS), a highly scalable container-management service that provides cluster management and container orchestration.
The company started evaluating Amazon ECS as soon as it became available in the Asia Pacific (Tokyo) Region in Spring 2015.
It took the company only a few weeks to go from evaluation to its first development environment on Amazon ECS.
The company developed a custom tool, ecs-formation, to simplify the development workflow and enable engineers to deploy Docker Compose applications to Amazon ECS.
In addition to this custom tool, CyberAgent makes use of CircleCI—an AWS Partner Network Advanced Technology Partner—to easily build and deploy services to Amazon ECS.
Amazon ECS eliminates the need to install and operate a cluster and container-management software; this reduces management needs for running containers at scale, and speeds time to market.
It only took two months for the CyberAgent team to get ready for full-scale operations, including task automatic scaling.
Using Docker and Amazon ECS, engineers no longer need to worry about environment consistency; issues generated from environmental differences almost entirely stopped.
The essential parameter in the Amazon ECS Task Definition is an additional benefit and contributing factor to the company’s overall system stability. If the essential parameter of a container is marked as true, and that container fails, all other containers that are part of the task are stopped as well, preventing side effects from a single container's failure from spreading to other containers.
Since it started using Amazon ECS, CyberAgent has achieved more than 99.9 percent service availability.
“Amazon EC2 Container Service is essential to build our microservices with high availability, stability and benefit from various AWS services.”
Akinori Yamada, Engineering Group