High-Availability HashiCorp Vault Setup on AWS
We built a multi-AZ HashiCorp Vault cluster on AWS that removes the single point of failure and manual unsealing.
OUR CUSTOMER
B2B SaaS Platform Provider
Darwin CX is a B2B SaaS company. Its integrated subscription and customer experience platform serves publishers and media organizations. It covers audience management, billing, marketing, analytics, paywalls, fulfillment, and subscriber retention. The platform runs on AWS across multiple services and accounts, including payment-related workloads.
THE CHALLENGE
Single Point of Failure in Secrets Management
The client used HashiCorp Vault as the central secrets store for a subscription platform on AWS. CI/CD pipelines, Terraform, containerized workloads, data jobs, monitoring, and application services across development, staging, and production depended on it for database credentials, payment-provider keys, API tokens, and machine identities.
As the platform grew across more services and AWS accounts, the single-node Vault deployment became a critical operational risk.
One Instance Behind Every Request
All Vault requests went through one instance. If a compute failure, Availability Zone disruption, or node-level issue occurred, secrets could become inaccessible across the platform. Deployments, application restarts, and service startup would stop. Patching, upgrading, or replacing the instance had the same effect, so even planned maintenance could interrupt dependent workloads.
The risk reached payment-related operations. Vault stored credentials for payment, wallet, and cardholder-adjacent services. It did not process card data, but these services relied on credentials retrieved from it.
Manual Vault Unsealing
After every restart, engineers had to unseal Vault manually with key shares. Recovery depended on the people who held those keys. Even after the infrastructure came back, Vault stayed unavailable until someone completed the procedure.
Growing Access Management Scope
The platform already managed dozens of application identities, environment-specific policies, and access for engineering teams. More services, identities, and AWS environments increased the risk of configuration inconsistencies. The client needed clear control over policies, identities, and environment-specific settings as the platform scaled.
THE SOLUTION
Building a Highly Available Vault Architecture on AWS
Our DevOps engineers redesigned the client’s secrets management as a three-node HashiCorp Vault cluster across three AWS Availability Zones. The cluster is provisioned with Terraform and Terragrunt.
We kept a private HTTPS endpoint compatible with the client’s existing Vault integrations. Applications and engineering workflows connect the same way as before.
Multi-AZ Vault Cluster on Amazon ECS
Our engineers deployed the three Vault nodes as containerized tasks on Amazon ECS with AWS Fargate, spread across the three Availability Zones. One node is active and two are on standby.
Amazon DynamoDB serves as the shared storage backend. It keeps Vault data independent of individual Fargate tasks and provides the lock that high availability requires.
An internal Application Load Balancer is the entry point to the cluster. It routes requests to healthy Vault tasks. If the active node fails, another node acquires the lock and becomes active, and clients keep using the same endpoint.
Automated Unsealing with AWS KMS
We configured AWS Key Management Service as the Vault seal mechanism. A Vault task that starts or is replaced unseals itself with the KMS key. Operators no longer provide key shares. Together with ECS task replacement, restarted nodes rejoin the cluster without manual intervention.
Isolated and Private Secrets Management Environment
The Vault infrastructure runs in a dedicated AWS account, separate from application workloads. The Vault API is available only through private networking, and TLS protects the traffic. Application and payment-related AWS accounts connect through private connectivity.
We limited the Vault workload through IAM to the resources it needs: the designated KMS key and the DynamoDB table.
Vault Identities and Policies Managed as Code
Our DevOps engineers also moved Vault configuration under Terraform. The configuration includes:
- a versioned key-value secrets engine
- AppRole authentication for applications and CI/CD workflows
- GitHub-based authentication for engineering teams
- explicit policies for each service and environment
Automated Recovery, Deployment, and Monitoring
Amazon ECS keeps the required number of Vault tasks running and replaces failed tasks automatically. Updates roll out incrementally. Our engineers configured a deployment circuit breaker that stops a failed deployment and rolls it back to the previous working version. Amazon CloudWatch collects logs from all Vault tasks in one place.
Multi-AZ HashiCorp Vault Cluster on AWS
The Used Toolkit
THE RESULTS
Improved Secrets Management Resilience
The client now runs secrets management on a highly available Vault cluster that recovers on its own. The platform’s pipelines, infrastructure code, and services depend on a secrets store that no longer relies on a single node or on manual intervention. The main outcomes:
- No single-node dependency. The loss of one node no longer stops deployments, application startups, or services that retrieve secrets.
- Lower maintenance risk. Teams can update or replace Vault nodes without a full secrets-service outage. Integrations keep the same private endpoint.
- No manual unsealing. Vault recovers from restarts and failed tasks without the unseal procedure. Routine recovery needs less on-call involvement.
- Consistent access governance. Identities, authentication methods, and policies live in version-controlled code, so teams can review, trace, and apply changes consistently as the platform grows. The dedicated private environment keeps access boundaries clearly defined.
WHY ROMEXSOFT
AWS DevOps Engineering Services Provider
Romexsoft provides AWS infrastructure automation and DevOps engineering services for SaaS products running on AWS. Our experience with business-critical platforms helps us address both infrastructure resilience and the operational requirements of services that handle sensitive credentials.
Our experience includes:
- Design of multi-AZ, highly available architectures on Amazon ECS, AWS Fargate, and Amazon DynamoDB
- Infrastructure as code with Terraform and Terragrunt, including identities, authentication methods, and access policies
- Automated recovery, unsealing, and deployment rollback to reduce manual operations
- Private, isolated AWS environments for secrets management and payment-related workloads.
Frequently Asked Questions
Why run Vault on ECS Fargate instead of EC2?
For this project, Amazon ECS with AWS Fargate allowed Vault to run as containerized tasks without maintaining dedicated EC2 hosts. ECS handles task scheduling and replacement, while Fargate removes the need to provision, patch, and manage the underlying servers.
How does the new Vault cluster keep secrets available when a node is replaced or fails?
The new architecture retained a private HTTPS endpoint compatible with the client’s existing Vault integrations, so the cutover did not require applications to change how they connected to the secrets service.
However, the previous single-node setup could not guarantee continuous secret access during node replacement. Any workload that needed to retrieve secrets while that instance was unavailable could be affected. The HA architecture addressed this by allowing a standby Vault node to assume the active role if the current active node became unavailable.
When does a company need a highly available Vault setup?
A highly available Vault setup becomes important when access to secrets is a critical dependency for applications, CI/CD pipelines, infrastructure automation, or other production workloads. If a Vault outage can block deployments, prevent services from starting, or disrupt business-critical operations, relying on a single Vault instance creates an avoidable availability risk.
Vault HA uses Fargate tasks with an active node and standby nodes so another node can take over if the active instance fails or loses connectivity.
How does a highly available secrets store support payment-related compliance, such as PCI DSS?
A highly available secrets store does not make an environment PCI DSS compliant by itself. It can, however, support relevant security controls by centralizing credentials and cryptographic secrets, limiting access through defined identities and policies, protecting communications, and keeping access and configuration under version control.
In this case, Vault runs in a dedicated AWS account, is reachable only over private connectivity, uses TLS, and is governed through IAM permissions limited to the resources it needs and version-controlled policies. These practices can support a broader payment-security program. Whether Vault falls in PCI DSS scope depends on whether it is part of, connected to, or can affect the cardholder data environment.
