Database Migration from GCP into AWS Using AWS Database Migration Service
Explore how we overcame multi-cloud management challenges by designing and implementing database migration from GCP to AWS.
Our Customer
Blockchain Ecosystem
Greenfence is a blockchain platform eco-system developed for the consumer goods industry to enable cost-efficient, transparent and trustworthy commercial relationships at scale. Greenfence empowers stakeholders along the end-to-end value chain to create individualized and secured blockchain networks that can connect, collaborate and transact on a private or public basis.
The Challenge
Real-Time Data Sync Challenge
Greenfence’s primary challenge centered on an AWS migration of its database layer from GCP, with minimal downtime. The migration needed to maintain data integrity, synchronize data in near real-time, preserve the platform’s performance and functionality and avoid disruption to the live web platform. The secondary technical challenge was dealing with the limitations of AWS DMS service when migrating the AUTO_INCREMENT attribute.
Key challenges Greenfence faced included:
- Data synchronization
Managing data consistency and replication between the AWS and GCP environments was a significant complication. The lack of a unified database environment made it difficult to ensure real-time data availability and synchronization across the multi-cloud setup. - Network connectivity and security
Connecting the application layer in AWS to the DB layer in GCP requires establishing secure and reliable network connectivity between the two clouds. Configuring and managing virtual private networks (VPNs), firewalls, and security policies across different cloud providers can be complex, requiring careful planning and coordination. - Performance optimization
Greenfence’s teams needed to optimize the performance of their database layer to meet the growing demands of their application. However, the distributed nature of the database across different cloud providers made the implementation of efficient performance-tuning strategies demanding. - Cost management
Maintaining a multi-cloud setup can lead to increased operational costs due to the need for separate management and infrastructure resources. Greenfence sought a solution to optimize costs while ensuring the required functionality and scalability of its platform.
The Solution
GCP to AWS Database Migration
As we at Romexsoft settled on addressing the obstacles Greenfence faced and simultaneously meeting the no-downtime migration requirement, we implemented the following solution:
- Database migration
We recommended using AWS Database Migration Service (DMS) to perform a homogeneous migration from the existing MySQL database in GCP to Amazon RDS for MySQL in AWS. This approach allowed for a smooth transition of the database layer while minimizing downtime. - Target database engine
Considering the compatibility and familiarity with the existing MySQL engine, we used Amazon RDS for MySQL as the target database engine. This choice ensured a seamless migration process and minimized any potential compatibility issues. - Manual configuration
In order to overcome the limitations of DMS in migrating the AUTO_INCREMENT attribute, we assisted Greenfence in manually configuring the AUTO_INCREMENT attribute on the relevant columns in the target Amazon RDS for MySQL instance. This step was crucial in maintaining the continuity of auto-incremented primary keys. - Configuration for ongoing replication
We configured AWS DMS for ongoing replication to obtain near real-time data synchronization between the source GCP MySQL database and the target Amazon RDS for MySQL in AWS. The achieved seamless continuous replication minimized data latency and enabled the migration process without disrupting the live web platform. - Increasing high availability
In the target database in AWS RDS, we enabled data replication across different availability zones (multi A-Z). - Enhancing security
We implemented data encryption at rest in the AWS RDS database and configured network topology with private subnets and corresponding security groups to provide secured private access to the DB only from the entire VPC where the application cluster is located.
Architecture Diagram Database Migration from GCP into AWS Using AWS Database Migration Service (DMS)
The Results
Continuous Service and Cost Efficiency
Near-zero downtime
By utilizing AWS DMS for ongoing replication, we reached near-zero downtime during the migration process. Greenfence’s live web platform remained accessible and functional throughout the migration, ensuring uninterrupted services for its users.
Smooth database migration
The migration process from Google Cloud Platform to Amazon Web Services was executed with minimal disruption to Greenfence’s operations. Data integrity was maintained throughout the migration, and near real-time synchronization enabled a seamless transition to the target environment.
Improved performance and scalability
With the migration to Amazon RDS for MySQL, Greenfence gained access to AWS’s scalable infrastructure, which allows optimization of the performance and scalability of the database layer. This adoption resulted in enhanced responsiveness and improved user experience.
Enhanced data security
The private subnets, protected by security groups and with disabled public endpoints, ensured that the Amazon RDS instances were shielded from direct access from the public internet. This approach significantly reduced the risk of unauthorized access and potential security breaches, providing a secure environment for storing and accessing sensitive data.
Cost optimization
Through consolidation of the database infrastructure onto AWS, Greenfence achieved tangible cost optimization: up to 40% of cost reduction. Such significant savings were achieved with benefits, such as no charges for DataTransfer, reserved RDS instances and reduced expenses on the IT staff. In addition, the consolidation reduced operational complexity, eliminated the need for separate management of multi-cloud environments, as well as allowed Greenfence to take advantage of AWS’s cost-effective pricing models.
GCP Migration to AWS FAQ
When undertaking database migration to AWS Cloud, organizations often encounter challenges such as:
Data inconsistency: Data stored without standardised measures can’t be transferred uniformly
Downtime: Database migration might cause unprecedented downtimes
Compatibility issues: Data formats might not be compatible during migration
It's crucial to conduct a thorough assessment before initiating the migration process to mitigate these risks.
Yes, AWS DMS limitations include constraints on certain data types, limited support for some database engines, and the inability to handle complex transformations natively. It's advisable to review the AWS DMS documentation to understand these limitations better before starting the migration.
AWS provides a cost calculator that can help you estimate AWS DMS costs. Factors that influence the cost include the amount of data to be transferred, the instance type selected, and the duration of the migration. It's essential to factor in these elements for budgeting accurately.
Yes, you can migrate from GCP to Amazon using AWS DMS, although the process may require additional steps for compatibility. AWS DMS supports various source and target databases, making it a viable option for GCP migrations to AWS. However, it's crucial to test the migration path thoroughly to ensure a smooth transition.
