Monitoring your databases is critical, especially in Kubernetes environments where visibility and automation are key. That’s why, in Percona Everest 1.6.0, we introduced a highly requested feature: the ability to automatically deploy Percona Monitoring and Management (PMM) as part of the Everest Helm chart using just one flag. This simplifies the process for teams who […]
15
2025
New in Percona Everest 1.6.0: Easily Deploy PMM with a Single Helm Command
16
2025
How to Extend Percona Monitoring and Management to Add Logging Functionality
Evolution is one of the inherent traits of modern software. Many people reach out to product teams daily, asking to add more functionality to the software products they use and love. This is understandable: there will always be ways to make a product better by adding more features to the users’ delight so they can […]
14
2025
Enterprise Readiness with Percona Monitoring and Management: A Look at PMM 3.0.0
Database disasters come with a hefty price tag. According to Information Technology Intelligence Consulting’s 2024 Hourly Cost of Downtime Report, 41% of enterprises face downtime costs ranging from $1 million to over $5 million per hour of outage. How can enterprises ensure their critical database infrastructure remains healthy and performant? This question keeps IT leaders […]
08
2025
Attaching a Percona Monitoring and Management Graph Image Along with an Alerting Notification
This article will be helpful if you use the Percona Monitoring and Management (PMM) instance and alert notifications, as it is nice to capture the image of the graph when you receive the alert. We will see how to capture and attach the image of the graph when receiving the alert notification (email, telegram, Slack, […]
27
2024
Important Notice: Update Process for Percona Monitoring and Management 2.38 and Earlier
We’ve received numerous reports about issues occurring with Percona Monitoring and Management (PMM) when updating to the latest version (currently 2.42.0), or updating to any version from an old version (2.37.1 and before) using the update button on the UI. Symptoms PMM won’t show any notifications on the UI about new available versions if: You’ve […]
19
2024
How to Monitor Valkey/Redis with Percona Monitoring and Management
In this blog post, we will show you how to add external exporters to Percona Monitoring and Management (PMM) and monitor various services. Our focus will be on Valkey/Redis, but the principles apply to other technologies as well. PMM uses VictoriaMetrics, which is compatible with Prometheus. Prometheus has numerous exporters that you can use. When […]
21
2024
High Availability: Choosing the Right Option for Your Percona Monitoring and Management
Ensuring your monitoring system stays up and running is vital. High availability (HA) minimizes downtime for Percona Monitoring and Management (PMM) during hardware failures, in times of disaster recovery, or increased usage of the tool. It’s not just about extra storage, RAM, or CPU but rather having redundant systems ready to take over seamlessly, like […]
08
2024
Mastering Database Monitoring: Running PMM in High Availability Mode
Percona Monitoring and Management (PMM) has become a valuable tool for database professionals, providing comprehensive insights into database health and performance. A recent update (version 2.41.0) introduced a significant enhancement: the ability to run PMM in high availability (HA) mode. This feature, currently in technical preview, offers exciting possibilities for ensuring the reliability and robustness […]
01
2024
Simplify the Use of ENV Variables in Percona Monitoring and Management AMI
The Percona Monitoring and Management (PMM) Amazon Machine Image (AMI) currently lacks native support for ENV variables. In this guide, we’ll walk through a straightforward workaround that simplifies the process of using ENV variables in PMM AMI and reapplying them after an upgrade.Step one: Adding ENV variables to /srv/.envBegin by consolidating your ENV variables in […]
21
2023
Percona Monitoring and Management High Availability – A Proof of Concept
Percona Monitoring and Management (PMM) is a state-of-the-art piece of software that exists in part thanks to great open source projects like VictoriaMetrics, PostgreSQL, and ClickHouse. The integration of those projects, plus the years of Percona expertise in the database space, makes PMM one of the best database monitoring solutions on the market.
Being software composed of different, multiple technologies can add complexity to a well-known concept: High Availability (HA). Achieving HA for PMM, as a whole, has proved to be a challenge but not an impossible task.
The easy part
Setting up the PMM cluster is the easy part. All it needs is a reverse proxy in front of a couple or more PMM instances. The go-to proxy is HAProxy configured for active/passive topology, that is, without load distribution.
For the purpose of the PoC, a single HAProxy instance is used (running as a docker container). The configuration file looks like this:
global stats socket /var/run/api.sock user haproxy group haproxy mode 660 level admin expose-fd listeners log stdout format raw local0 info defaults mode http timeout client 10s timeout connect 5s timeout server 10s timeout http-request 10s log global frontend stats bind *:8404 stats enable stats uri / stats refresh 10s frontend pmmfrontend bind :80 default_backend pmmservers backend pmmservers option tcp-check server pmm1 172.31.12.174:80 check port 80 server pmm2 172.31.11.132:80 check port 80 backup
The Docker container is run with this command:
docker run -d --name haproxy -v $(pwd):/usr/local/etc/haproxy:ro -p 80:80 -p 443:443 -p 8404:8404 haproxytech/haproxy-alpine:2.
The -v for the volume guarantees that the local copy of the haproxy.cfg file is the one used inside the container. Whenever you make a change in the cfg file, for the haproxy container to use it, just execute:
docker kill -s HUP haproxy
And to follow the haproxy logs:
docker logs -f haproxy
We have two frontends: One for the HAProxy stats and another for the PMM itself. The backend is a single one where the “passive” PMM instance (the one that is a pure “read replica”) is marked as “backup” so that traffic is only routed there in case the primary fails the health check.
For simplicity, the PMM instances are configured to listen to the 80 port (http) on the private IPs. This is made to avoid SSL certificates since everything goes through the same VPC (everything runs on ec2 instances on AWS). The health check, then, can be a simple “tcp-check” against port 80.
As you can see, stats are available via the port 8404. With this, the easy part is done.
For this example, the PMM SERVER endpoint will be where the HAProxy frontend is listening, and that’s the one used when registering a new PMM CLIENT.
And you can access PMM always using the same endpoint.
The not-so-easy part (made easy)
The proxy is configured to be aware of two different PMM instances — pmm1 and pmm2 — (using the private IPs 172.31.12.174 and 172.31.11.132 in this case), but we haven’t mentioned anything about those PMM instances.
And here is the not-so-easy part: One has to deploy at least two PMM instances on at least two different servers AND set up replicas. How to do it? This is the actual Proof of Concept: Enter the PMM HA script: https://github.com/nethalo/pmmha
The script will take care of installing PMM (if you already have it, you can skip this step), preparing the Primary, and setting up the Secondary. Simple as that. PMM will remain to be a black box.
The requirements are:
Steps to run the script:
git clone https://github.com/nethalo/pmmha.git cd pmmha bash pmm.sh
Failover
The failover will be handled by the HAProxy automatically when it detects that the current primary is no longer available. Traffic will be routed to the backup server from the backend, which, if properly set as a replica, will already have the historical data for metrics and QAN, and also the inventory will be ready to continue the data scrapping from the exporters.
Your feedback is key
We want to hear from you! Suggestions, feature requests, and comments in general are welcome, either in the comment section or via the GitHub repository.
Percona Monitoring and Management is a best-of-breed open source database monitoring solution. It helps you reduce complexity, optimize performance, and improve the security of your business-critical database environments, no matter where they are located or deployed.