Ecommerce Infrastructure Automation and Modernization on AWS

LPM stands as one of Asia’s largest and most trusted precious metal retailers. The company carries a wide range of products including but not limited to popular newly released silver coins, gold bullion, certified coins and the newest and most innovative collectible numismatic issues. The company owns and operates the online marketplace for metal retailers.

Infrastructure Automation and Containerization with IaC

Our initial step involved a thorough examination of the existing infrastructure, pinpointing key improvement areas based on the previously outlined challenges.

The Adobe Commerce (Magento 2) platform operated on the Amazon EC2, serving as the application’s foundational engine. So to tackle the identified challenges, the primary step of the devised plan is containerization of applications services and deploying them on Amazon Elastic Container Service (ECS), following AWS best practices. Upon this implementation, the platform could easily adapt to changing workloads, ensuring optimal resource utilization and accommodating evolving business needs.

By employing the Infrastructure as Code (IaC) approach, we managed to automate infrastructure using Terraform and Terragrunt. It ensured the automation of setting up, provisioning, deploying, and managing required environments. Apart from reducing manual errors, this automation expedited the overall development and testing lifecycle, thus enhancing operational efficiency.

Building upon the golden Docker image, we established a pipeline for creating Docker images with predefined configurations. Some configurations became easily changeable and updatable during the code build. This particular solution standardized deployment pipelines, ensuring consistency in deploying various services. Maintaining this consistency effectively minimized deployment-related issues.

Containerization also extended to the client application. In order to ensure the confidentiality of sensitive information, we leveraged AWS System Manager Parameter Store, securely storing all necessary build variables in an encrypted form.

CI/CD Pipelines for Faster and Safer Deployments

We configured pipelines to facilitate the deployment of essential applications, conduct infrastructure upgrades, and manage the deployment/removal of new environments. The Jenkins server was deployed to establish conceived CI/CD processes.

A dedicated pipeline was created to build a golden Docker image, serving as the foundation for future builds of services such as Varnish, Nginx, PHP-FPM, Fluent-bit, and the main Adobe Commerce (Magento 2) platform. This standardized procedure ensures consistency and efficiency in the deployment and maintenance of various components.

A Jenkin-based pipeline scenario was implemented to automate the creation and deletion of new environments for development and testing. It provides a streamlined and standardized approach to managing development and testing environments. This enhances operational efficiency as well as ensures consistency, scalability, and resource optimization across the software development lifecycle. The access control system additionally implemented on the Jenkins server, ensures that actions could only be executed in the relevant environments.

AWS Account Structure and Environment Separation

We established an AWS Organization and built a hierarchy of distinct AWS accounts dedicated to separate environments, whereby each has its own purpose and infrastructure:

• Management account responsible for billing and creating other accounts.
• Service account for monitoring, CI/CD processes, users, and access to other accounts of the organization.
• Production account to deploy the primary working product.
• Staging and Dev accounts configured as exact replicas of production to provide environments closest to production for testing.

To enhance the security of the environments, we have established all environments within private subnets, ensuring access to them is exclusively through a bastion server.

Centralized Monitoring and Issue Resolution

We deployed a monitoring system based on Grafana and OpenSearch Service, incorporating such capabilities as collecting logs from all the necessary services, sending alerts, and analyzing logs and metrics data. Amazon Cloudwatch was also set up to monitor the load and increase or decrease the resources used depending on the load.

The monitoring system ensures continuous logging, allowing for efficient troubleshooting and rapid issue identification. With real-time alert notifications, potential problems can be dynamically tackled, thereby contributing to improved system stability. This proactive approach also enhances preventing and addressing performance issues which lead to potential disruptions.

Application Performance Optimization

We leveraged Amazon Elastic File System (EFS) for storing static files and implementing Varnish Cache as a caching HTTP reverse proxy, contributing significantly to accelerated application performance speed. These measures, coupled with the configuration of Amazon CloudFront, not only facilitated efficient storage and retrieval of static content but also resulted in a substantial acceleration of the platform when it was loaded by the users. These optimizations in general established faster performance for the entire application.

Resilient Backup and Rollback Strategy

As part of our infrastructure automation, we proactively established a robust data backup system for essential application services. The Jenkins server’s pipeline, designed for swift error recovery and version rollback, played a pivotal role in maintaining system stability. It facilitates a swift rollback to the previous version in case of errors. Leveraging Amazon EBS snapshots and Amazon RDS Backup and Restore contributed to the system’s ability to recover from errors and added a layer of resilience to the platform’s data integrity, as well.

AWS Architecture Diagram for E-Commerce Infrastructure Automation

While the solutions implemented addressed the challenges outlined in the initial assessment, they also collectively led to a more performant, secure, scalable, and operationally efficient application for the client. The enhancements spanned different aspects of the application’s infrastructure, delivering a measurable positive impact not only on platform performance, reliability, and functionality but also on code integrity, operational efficiency, and the overall user experience.

Efficient and secure development process
The shift to Infrastructure as Code (IaC), combined with the creation of isolated environments minimized manual efforts, meanwhile offering the flexibility to create and test new features without disrupting production.

Cost optimization
All these infrastructure automation and development process improvements bring cost-efficiency for the client by expediting the creation of environments, allowing the freeing up of unused, costly resources.

Stable system performance
Implementation of a monitoring system has notably improved performance stability. The system’s ability to quickly detect and troubleshoot issues has been enhanced as well, positively impacting overall system reliability.

Reinforced business continuity
The boosted infrastructure automation provides the client with opportunities to expand and automate new processes seamlessly, fostering scalability and adaptability. A refined data backup system, coupled with the ability to swiftly roll back to stable states, has mitigated risks associated with data loss, addressing concerns about business continuity.