Centralized Application Monitoring and Logging Systems Development
Discover how our DevOps engineers developed custom application monitoring and logging systems from scratch for a BioTech company.
OUR CUSTOMER
Biotech Platform
Revalia Bio partnered with us to apply biomedical innovations that mitigate real-world failures. The idea behind it is that human models of disease created with deceased donor organs unsuitable for transplantation can still offer unique insights for medical researchers. Central to the company’s expertise is a technology platform: comprising custom software, hardware, and analytics. This comprehensive solution will maximize knowledge extraction from perfused organs, accelerating biomedical advancements while minimizing related risks.
The Challenge
Observability and Security Risks
After a successful launch of the core platform on AWS, the biotech company kept developing its service. For this reason, Romexsoft experts suggested building custom application log processing and monitoring systems for the platform using the corresponding AWS services.
These systems were mainly intended to detect the solution’s issues as soon as possible, to ensure quick reaction to the incidents, and to troubleshoot them effectively.
The key challenges of the possible lack of centralized logging and monitoring systems could have a significant impact on the client’s business, leading to downtimes, increased costs of ownership, and reduced customer satisfaction, more specifically:
- Lack of log monitoring could complicate getting a complete view of the system health as well as detecting and resolving the emergent issues timely.
- Limited observability of the application stack and infrastructure makes it difficult to understand and troubleshoot complex issues.
- Potential security risks in the company’s AWS environment could result in the overall vulnerability to security breaches and non-compliance with security regulations.
The Solution
Centralized Logging and Monitoring on AWS
Log Processing and Issue Detection On the Application Level
Romexsoft has already set up the biotech company’s AWS Organisation governance using AWS Control Tower with a dedicated Logs Archive AWS Account and suggested setting up a Centralized Logging System in it and building on top of AWS OpenSearch Service. This approach allows for collecting and storing logs from different sources in one place as well as to navigate easily across various facets and data attributes.
Moreover, by utilizing the OpenSearch Trace Analytics feature we will build analytics for the distributed tracing of requests and messages between the microservices in the upcoming releases.
We suggested installing Fluentd td-agent on each ECS service running Java applications in AWS accounts to stream logs into the Logs Archive AWS account. To avoid the increasing size of OpenSearch logs cluster we suggested implementing logs rotation logic and storing all logs data older than 30 days in the S3 bucket.
Metrics Collecting and Monitoring On the Infrastructure Level
For infrastructure monitoring level, we’ve suggested utilizing the AWS CloudWatch service. It is a robust monitoring service that enables us to collect and track metrics, and set alarms, allowing us to gain valuable insights into the client’s AWS infrastructure performance. With features like automated scaling and integration with Grafana as a data source, CloudWatch empowers to proactively manage and optimize AWS resources for better efficiency and cost-effectiveness.
Comprehensive Observability Stack
To react to the incidents quickly and troubleshoot them effectively in addition to the OpenSearch Service Romexsoft suggested setting up a dashboard in Grafana to visualize all critical metrics and integrate alerts about the company’s incidents into the Slack channel using AWS Chatbot service.
Grafana will be running in containers on the ECS Fargate service. It allows not only to meet common conventions of the whole ecosystem because the majority solution’s services run on ECS Fargate, but also ensures easy technical management and cost efficiency over the project.
Strong Security Posture and Compliance
In order to meet security compliances both services are located in the private subnets of AWS VPC and require secured VPN connectivity to access the dashboards. The private subnets are isolated from the public internet, which further reduces the risk of security breaches. It also means that potential attackers have fewer opportunities to gain access to the services and sensitive data.
The VPN connectivity can be audited to track who has access to the dashboards and what actions they have taken. This can help the biotech company to identify and respond to security incidents more quickly. Moreover, by placing the services in private subnets and requiring secured VPN connectivity, the client can demonstrate compliance with a variety of security regulations, such as PCI DSS, ISO 27001, and HIPAA.
Diagram of the Application Logging and Monitoring Systems Architecture
The Results
Increased System Visibility and Troubleshooting Efficiency
The centralized application logging and infrastructure monitoring systems implemented by Romexsoft have had a significant impact on the client’s AWS environment. The client got effective instrumentation for fast detecting and resolving incidents, improved visibility of the application infrastructure, and increased ability to understand and troubleshoot complex issues. Additionally, biotech software development has reduced the risk of security breaches and improved compliance with security regulations.
The project’s key accomplishments are:
- Swift detection and resolution of application issues
- Rapid identification and remediation of performance bottlenecks
- Enabled proactive monitoring of the system
- Implemented observability of the application infrastructure
- Established ability to understand and troubleshoot complex issues
- Overall reliability and stability of the system
- Reduced the risk of security breaches
- Compliance with security regulations
- Strengthened overall security posture of the system.
Why Romexsoft
AWS Observability and Monitoring Provider
Romexsoft is an AWS Partner with DevOps Competency specializing in observability for cloud-native applications. We design and deploy centralized monitoring and logging systems for AWS-based applications used in biotech and other industries.
We deliver dedicated solutions for life science applications monitoring and logging, including:
- Application log streaming configuration from containerized services
- Observability dashboards and incident alert integration
- Distributed tracing preparation for microservices communication analysis
- Log lifecycle and retention management to control storage growth
- Secure observability environment configuration using private network access.
Frequently Asked Questions
The implementation timeline depends on the platform architecture and the number of services involved. For most cloud-native platforms, setting up centralized observability – including log pipelines, monitoring dashboards, and alerting – can take from a few weeks to a few months.
Yes. A well-designed observability architecture scales with the platform by automatically collecting logs and metrics from new services as they are deployed. This ensures consistent monitoring coverage even as the system evolves and additional microservices are introduced.
Monitoring systems provide early indicators of system anomalies such as resource spikes, performance degradation, or unusual traffic patterns. These signals allow engineering teams to address issues before they escalate into outages.
Yes. Monitoring and logging systems can usually be introduced gradually by connecting existing services to centralized log pipelines and monitoring dashboards. Such an approach allows DevOps to improve visibility across the platform without interrupting ongoing workloads or deployments.
