This case study has been made with Carlos Perez, Head of Business Development at Malin Shopper S.A.
Malinshopper is a web platform whose mission is to provide local intelligence to consumers and retailers.
The Challenge and Opportunity:
All these components were hosted on a single server that was regularly stuck because of the Hadoop worker processes consuming all the memory and CPUs, making Malinshopper very slow and causing some downtimes.
Co-founders Carlos and Alex knew they had to do something about it, especially with their Version 2 coming.
The Cloud was definitively an option to consider. They came to us to understand the benefits, technical feasibility, requirements and cost considerations.
It was clear to them that moving to the Cloud would be the opportunity to decouple the application, clean up the space and automate the deployment of the different components of their application which would provide them with more Agility and a plan for future growth.
One of the key challenges was the content fetched by the main Hadoop worker generating a large amount of data that was periodically updated by two other Hadoop workers on a hourly basis.
Till version 2 of the application, the application limitation would require the content to be available on all the instances of the web application and Hadoop workers.
The solution brought up by the Continuous team consisted in using AWS CloudFormation to create a reproducible, configurable and right-sized infrastructure to suit every possible environment.
This new infrastructure is composed of:
- Automatic DNS Entries per environment
- A Virtual Private Cloud with:
- A load balanced (ELB)/Autoscaled Web servers (AS)
- Three auto-scaling groups for the Hadoop workers
- S3 Storage for the static content with S3fs for instance sharing
- Static content CDN with CloudFront
- A Mutli Availability Zone Database (RDS)
- Automated Deployment with continuousphp and CodeDeploy in Production, Staging, and any development/feature branches
Even though this migration doesn’t benefit from all the advantages the Cloud offers, it enables Malinshopper to:
- Fix the current problems
- Prepare the ground for Version 2 which would be developed using loosely coupled architecture on top of this Cloud infrastructure
- Implement testing, staging and production pipelines for rapid application validation, testing and deployment
- Prepare the ground to move from single Hadoop workers to Hadoop clusters
- Extend the programmed Infrastructure to other AWS services based on need and growth
Thanks to the Continuous team, we now have a stable, evolutive infrastructure on which we can build our application. It gives us the agility to quickly update and scale our application cluster based on our business growth.