May
15
2025
--

New in Percona Everest 1.6.0: Easily Deploy PMM with a Single Helm Command

New in Percona Everest 1.6.0: Easily Deploy PMM with a Single Helm CommandMonitoring your databases is critical, especially in Kubernetes environments where visibility and automation are key. That’s why, in Percona Everest 1.6.0, we introduced a highly requested feature: the ability to automatically deploy Percona Monitoring and Management (PMM) as part of the Everest Helm chart using just one flag. This simplifies the process for teams who […]

Apr
16
2025
--

How to Extend Percona Monitoring and Management to Add Logging Functionality

Extend Percona Monitoring and Management with logging capabilitiesEvolution is one of the inherent traits of modern software. Many people reach out to product teams daily, asking to add more functionality to the software products they use and love. This is understandable: there will always be ways to make a product better by adding more features to the users’ delight so they can […]

Mar
14
2025
--

Enterprise Readiness with Percona Monitoring and Management: A Look at PMM 3.0.0

Database disasters come with a hefty price tag. According to Information Technology Intelligence Consulting’s 2024 Hourly Cost of Downtime Report, 41% of enterprises face downtime costs ranging from $1 million to over $5 million per hour of outage. How can enterprises ensure their critical database infrastructure remains healthy and performant? This question keeps IT leaders […]

Jan
08
2025
--

Attaching a Percona Monitoring and Management Graph Image Along with an Alerting Notification

Attaching a Percona Monitoring and Management Graph Image Along with an Alerting NotificationThis article will be helpful if you use the Percona Monitoring and Management (PMM) instance and alert notifications, as it is nice to capture the image of the graph when you receive the alert. We will see how to capture and attach the image of the graph when receiving the alert notification (email, telegram, Slack, […]

Aug
27
2024
--

Important Notice: Update Process for Percona Monitoring and Management 2.38 and Earlier

ENV Variables in PMM AMI.jpgWe’ve received numerous reports about issues occurring with Percona Monitoring and Management (PMM) when updating to the latest version (currently 2.42.0), or updating to any version from an old version (2.37.1 and before) using the update button on the UI.  Symptoms PMM won’t show any notifications on the UI about new available versions if: You’ve […]

Jul
19
2024
--

How to Monitor Valkey/Redis with Percona Monitoring and Management

How to Monitor Valkey/Redis with Percona Monitoring and ManagementIn this blog post, we will show you how to add external exporters to Percona Monitoring and Management (PMM) and monitor various services. Our focus will be on Valkey/Redis, but the principles apply to other technologies as well. PMM uses VictoriaMetrics, which is compatible with Prometheus. Prometheus has numerous exporters that you can use.  When […]

May
21
2024
--

High Availability: Choosing the Right Option for Your Percona Monitoring and Management

high availability percona monitoring and managementEnsuring your monitoring system stays up and running is vital. High availability (HA) minimizes downtime for Percona Monitoring and Management (PMM) during hardware failures, in times of disaster recovery, or increased usage of the tool. It’s not just about extra storage, RAM, or CPU but rather having redundant systems ready to take over seamlessly, like […]

May
08
2024
--

Mastering Database Monitoring: Running PMM in High Availability Mode

Percona Monitoring and Management (PMM) has become a valuable tool for database professionals, providing comprehensive insights into database health and performance. A recent update (version 2.41.0) introduced a significant enhancement: the ability to run PMM in high availability (HA) mode. This feature, currently in technical preview, offers exciting possibilities for ensuring the reliability and robustness […]

Feb
01
2024
--

Simplify the Use of ENV Variables in Percona Monitoring and Management AMI

ENV Variables in PMM AMI.jpgThe Percona Monitoring and Management (PMM) Amazon Machine Image (AMI) currently lacks native support for ENV variables. In this guide, we’ll walk through a straightforward workaround that simplifies the process of using ENV variables in PMM AMI and reapplying them after an upgrade.Step one: Adding ENV variables to /srv/.envBegin by consolidating your ENV variables in […]

Dec
21
2023
--

Percona Monitoring and Management High Availability – A Proof of Concept

Percona Monitoring and Management (PMM) is a state-of-the-art piece of software that exists in part thanks to great open source projects like VictoriaMetrics, PostgreSQL, and ClickHouse. The integration of those projects, plus the years of Percona expertise in the database space, makes PMM one of the best database monitoring solutions on the market.

Being software composed of different, multiple technologies can add complexity to a well-known concept: High Availability (HA). Achieving HA for PMM, as a whole, has proved to be a challenge but not an impossible task.

The easy part

Setting up the PMM cluster is the easy part. All it needs is a reverse proxy in front of a couple or more PMM instances. The go-to proxy is HAProxy configured for active/passive topology, that is, without load distribution.

For the purpose of the PoC, a single HAProxy instance is used (running as a docker container). The configuration file looks like this:

global
stats socket /var/run/api.sock user haproxy group haproxy mode 660 level admin expose-fd listeners
log stdout format raw local0 info

defaults
mode http
timeout client 10s
timeout connect 5s
timeout server 10s
timeout http-request 10s
log global

frontend stats
bind *:8404
stats enable
stats uri /
stats refresh 10s

frontend pmmfrontend
bind :80
default_backend pmmservers

backend pmmservers
option tcp-check
server pmm1 172.31.12.174:80 check port 80
server pmm2 172.31.11.132:80 check port 80 backup

The Docker container is run with this command:

docker run -d --name haproxy -v $(pwd):/usr/local/etc/haproxy:ro -p 80:80 -p 443:443 -p 8404:8404 haproxytech/haproxy-alpine:2.

The -v for the volume guarantees that the local copy of the haproxy.cfg file is the one used inside the container. Whenever you make a change in the cfg file, for the haproxy container to use it, just execute:

docker kill -s HUP haproxy

And to follow the haproxy logs:

docker logs -f haproxy

We have two frontends: One for the HAProxy stats and another for the PMM itself. The backend is a single one where the “passive” PMM instance (the one that is a pure “read replica”) is marked as “backup” so that traffic is only routed there in case the primary fails the health check.

For simplicity, the PMM instances are configured to listen to the 80 port (http) on the private IPs. This is made to avoid SSL certificates since everything goes through the same VPC (everything runs on ec2 instances on AWS). The health check, then, can be a simple “tcp-check” against port 80.

Percona Monitoring and Management high availability

As you can see, stats are available via the port 8404. With this, the easy part is done.

For this example, the PMM SERVER endpoint will be where the HAProxy frontend is listening, and that’s the one used when registering a new PMM CLIENT.

PMM CLIENT

And you can access PMM always using the same endpoint.

The not-so-easy part (made easy)

The proxy is configured to be aware of two different PMM instances — pmm1 and pmm2  — (using the private IPs 172.31.12.174 and 172.31.11.132 in this case), but we haven’t mentioned anything about those PMM instances.

And here is the not-so-easy part: One has to deploy at least two PMM instances on at least two different servers AND set up replicas. How to do it? This is the actual Proof of Concept: Enter the PMM HA script: https://github.com/nethalo/pmmha

The script will take care of installing PMM (if you already have it, you can skip this step), preparing the Primary, and setting up the Secondary. Simple as that. PMM will remain to be a black box.

The requirements are:

– Docker 23.0.3 and higher
– Docker installation will fail if on Amazon Linux or RHEL9/EL9 (unless you are on a s390x architecture machine) since the Percona Easy-Install script relies on the “Get Docker” (https://get.docker.com/) script for the docker install. You will need to install docker on your own in those cases
– SSH Access to the host servers
– sudo capabilities
– Ports 443 and 9000 accessible from outside the Primary host machine

Steps to run the script:

git clone https://github.com/nethalo/pmmha.git
cd pmmha
bash pmm.sh

Failover

The failover will be handled by the HAProxy automatically when it detects that the current primary is no longer available. Traffic will be routed to the backup server from the backend, which, if properly set as a replica, will already have the historical data for metrics and QAN, and also the inventory will be ready to continue the data scrapping from the exporters.

Your feedback is key

We want to hear from you! Suggestions, feature requests, and comments in general are welcome, either in the comment section or via the GitHub repository.

Percona Monitoring and Management is a best-of-breed open source database monitoring solution. It helps you reduce complexity, optimize performance, and improve the security of your business-critical database environments, no matter where they are located or deployed.

 

Download Percona Monitoring and Management Today

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com