In the fast-paced cloud environment, optimising the infrastructure for containerised applications is a top priority. Kubernetes has long been the go-to solution for managing containers, but as technology & products evolve, alternative options arise. 

In this blog post, we will explore the journey of the cloud team at In The Pocket as they moved away from Kubernetes and embraced AWS ECS (Elastic Container Service) for their container orchestration needs. Let's dive into their experience and discover the benefits and challenges they encountered along the way.

Background: A Cloud Project of Scale

The team had been diligently working on a massive cloud project for over four years. Their platform allowed users to interact with their physical devices at home through the cloud. With 50 microservices, 5 billion lambda invocations per month, and the handling of 1 million device commands per minute - all coming from or going to the 600,0000 connected devices - the project required a robust and scalable infrastructure. 

Initially, they started with Kubernetes but gradually incorporated more managed serverless services from AWS, resulting in a hybrid environment of Kubernetes and ECS.

Co-existence: Migrating Service by Service

Migrating an entire workload from Kubernetes to ECS is not an overnight process. As the cloud team followed a trunk-based development approach, they had to feature flag all their ECS-related work and set up services on both ECS and EKS (Elastic Kubernetes Service). This allowed them to gradually shift traffic from Kubernetes to ECS without disrupting the existing functionality. 

Despite the additional overhead and increased running costs, this careful migration strategy ensured a smooth transition. To maintain operational stability during this period, the team set up new projects in their monitoring tools, such as Sentry and New Relic, to ensure comprehensive observability.

Embracing Serverless with AWS ECS

As the backbone of their ECS infrastructure, our cloud people doubled down on Fargate and fully embraced the serverless approach. They had grown tired of manually terminating EC2 instances due to AWS maintenance, and Fargate provided the ideal solution. Setting up services on ECS revealed a host of exciting features, such as Container Secrets, eliminating the need for direct calls to Secrets Manager from the application code. 

Additionally, the team found Cloudwatch with Container Insights to be a terrific tool for monitoring their ECS services. With zero code changes, their services were up and running seamlessly alongside their Kubernetes counterparts.

Scaling with Confidence

One of the initial concerns the team had when transitioning to ECS was how well it would scale compared to Kubernetes. With Kubernetes, they were accustomed to rapid scaling with around 500 pods running the service. After extensive investigation, the team configured their ECS tasks to be more powerful, allocating more CPU and memory compared to the EKS pods. However, they maintained the minimum and maximum resource thresholds to prevent unnecessary changes.

To automate the scaling process, the team opted for Target Tracking Scaling, leveraging AWS's ability to determine the appropriate scaling based on CPU and memory thresholds. This approach differed from Step Scaling, which required explicit resource addition or removal instructions. By letting AWS manage the scaling process, the team reduced operational complexity.

To validate their configurations, they executed multiple load tests in a safe environment mirroring the production setup. The tests demonstrated the seamless scalability of their ECS service, allowing them to fine-tune memory provisioning after the initial setup.

Gradual Deployment with Canary Releases

Big Bang deploys are not something we’re a fan of at In The Pocket. So to ensure a gradual transition and closely monitor the behaviour of their services, the team leveraged API Gateway in conjunction with CloudFront. By using canary variables and creating stages, they were able to gradually shift traffic from one load balancer to another, splitting traffic across their Kubernetes and ECS services. 

This iterative process involved constant monitoring, rechecking, and incrementally increasing the load. The team meticulously analysed various metrics from both Kubernetes and ECS, ensuring a seamless deployment without causing any downtime or impacting user experience.

Aftermath: Enhanced Efficiency and Cost Savings

After successfully migrating their services to ECS, the cloud team experienced several tangible benefits. They eagerly terminated their Kubernetes cluster's EC2 instances and bid farewell to maintaining helm charts. 

In the weeks that followed, they closely monitored the behaviour of their ECS infrastructure and its impact on other services, such as their MongoDB database. Surprisingly, they observed a drop in database connections due to the optimized ECS pods requiring fewer instances than the Kubernetes setup. This reduction in connections translated into potential cost savings for their database operations.

Moreover, with all their computing resources now running on managed AWS services, the team fully leveraged AWS Compute Optimizer. Within a week, they received recommendations to optimise resource provisioning for their ECS service, resulting in immediate adjustments. 

To their delight, the first invoice since the migration arrived, and it was cheaper than their previous Kubernetes-based setup. With everything centralised and visible in the AWS console, including metrics, configurations, logging, and tracing, the team (and client!) enjoyed the simplicity and manageability of their consolidated cloud infrastructure.

Conclusion: Embracing Serverless and Managed Solutions

The decision to move away from Kubernetes and embrace AWS ECS proved to be a game-changer for the team and client. By consolidating their container orchestration on ECS and adopting a serverless approach, they simplified their cloud operations and improved efficiency. 

The meticulous migration process, supported by gradual canary releases and comprehensive load testing, ensured a seamless transition without any disruption to their users. With enhanced scalability, cost optimisation, and centralised management, the team found their cloud infrastructure to be more manageable and easier to maintain.

While Kubernetes remains a viable option for many use cases, In The Pocket's journey showcases the benefits of leveraging AWS ECS and the serverless paradigm. As they aptly put it: the more serverless and managed the environment, the greater their love for it.

‍