Personal blog of Yzmir Ramirez

Databases on Kubernetes continue their rising trend. We see the growing adoption of our Percona Kubernetes Operators and the demand to migrate workloads to the cloud-native platform. Our Operators provide built-in backup and restore capabilities, but some users are still looking for old-fashioned ways, like storage-level snapshots (i.e., AWS EBS Snapshots).

In this blog post, you will learn:

1. How to back up and restore from storage snapshots using Percona Operators
2. What the risks and limitations are of such backups

Overview

Volume Snapshots went GA in Kubernetes 1.20. Both your storage and Container Storage Interface (CSI) must support snapshots. All major cloud providers support them but might require some steps to enable it. For example, for GKE, you must create a VolumeSnapshotClass resource first.

At the high level, snapshotting on Kubernetes looks like this:

As PersistentVolume is represented by the real storage volume,

VolumeSnapshot

is the Kubernetes resource for volume snapshot in the cloud.

Getting ready for backups

First, we need to be sure that VolumeSnapshots are supported. For the major clouds, read the following docs:

• Google – Documentation on Volume Snapshots for GKE
• AWS – Using EBS Snapshots for persistent storage with your EKS cluster
• Azure – AKS CSI documentation

Once you have CSI configured and Volume Snapshot Class is in place, proceed to create a backup.

Take the backup

Identify the PersistentVolumeClaims (PVC) that you want to snapshot. For example, for my MongoDB cluster, I have six PVCs: three x replica set nodes and three x config server nodes.

$ kubectl get pvc
NAME                                STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
mongod-data-my-cluster-name-cfg-0   Bound    pvc-c9fb5afa-1fc9-41f9-88f3-4ed457f88e58   3Gi        RWO            standard-rwo   78m
mongod-data-my-cluster-name-cfg-1   Bound    pvc-b9253264-f79f-4fd0-8496-1d88105d84e5   3Gi        RWO            standard-rwo   77m
mongod-data-my-cluster-name-cfg-2   Bound    pvc-5d462005-4015-47ad-9269-c205b7a3dfcb   3Gi        RWO            standard-rwo   76m
mongod-data-my-cluster-name-rs0-0   Bound    pvc-410acf85-36ad-4bfc-a838-f311f9dfd40b   3Gi        RWO            standard-rwo   78m
mongod-data-my-cluster-name-rs0-1   Bound    pvc-a621dd8a-a671-4a35-bb3b-3f386550c101   3Gi        RWO            standard-rwo   77m
mongod-data-my-cluster-name-rs0-2   Bound    pvc-484bb835-0e2d-4a40-b5a3-1ba340ec0567   3Gi        RWO            standard-rwo   76m

Each PVC will have its own VolumeSnapshot. Example for

mongod-data-my-cluster-name-cfg-0

:

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: mongod-data-my-cluster-name-cfg-0-snap
spec:
  volumeSnapshotClassName: gke-snapshotclass
  source:
    persistentVolumeClaimName: mongod-data-my-cluster-name-cfg-0

I have listed all my VolumeSnapshots objects in one YAML manifest here.

$ kubectl apply -f https://raw.githubusercontent.com/spron-in/blog-data/master/volume-snapshots/mongo-volumesnapshots.yaml
volumesnapshot.snapshot.storage.k8s.io/mongod-data-my-cluster-name-cfg-0-snap created
volumesnapshot.snapshot.storage.k8s.io/mongod-data-my-cluster-name-cfg-1-snap created
volumesnapshot.snapshot.storage.k8s.io/mongod-data-my-cluster-name-cfg-2-snap created
volumesnapshot.snapshot.storage.k8s.io/mongod-data-my-cluster-name-rs0-0-snap created
volumesnapshot.snapshot.storage.k8s.io/mongod-data-my-cluster-name-rs0-1-snap created
volumesnapshot.snapshot.storage.k8s.io/mongod-data-my-cluster-name-rs0-2-snap created

VolumeSnapshotContent is created and bound to every

VolumeSnapshot

resource. Its status can tell you the name of the snapshot in the cloud and check if a snapshot is ready:

$ kubectl get volumesnapshotcontent snapcontent-0e67c3b5-551f-495b-b775-09d026ea3c8f -o yaml
…
status:
  creationTime: 1673260161919000000
  readyToUse: true
  restoreSize: 3221225472
  snapshotHandle: projects/percona-project/global/snapshots/snapshot-0e67c3b5-551f-495b-b775-09d026ea3c8f

• snapshot-0e67c3b5-551f-495b-b775-09d026ea3c8f is the snapshot I have in GCP for the volume.
• readyToUse: true – indicates that the snapshot is ready

Restore

The restoration process, in a nutshell, looks as follows:

1. Create persistent volumes using the snapshots. The names of the volumes must match the standard that Operator uses.
2. Provision the cluster

Like any other backup, it must have secrets in place: TLS and users.

You can use this restoration process to clone existing clusters as well, just make sure you change the cluster, PVCs, and Secret names.

Create persistent volumes from snapshots. It is the same as the creation of regular PersistentVolumeClaim, but with a

dataSource

section that points to the snapshot:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mongod-data-my-cluster-name-rs0-0
spec:
  dataSource:
    name: mongod-data-my-cluster-name-rs0-0-snap
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
  storageClassName: standard-rwo
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi

$ kubectl apply -f https://raw.githubusercontent.com/spron-in/blog-data/master/volume-snapshots/mongo-pvc-restore.yaml
persistentvolumeclaim/mongod-data-my-cluster-name-cfg-0 created
persistentvolumeclaim/mongod-data-my-cluster-name-cfg-1 created
persistentvolumeclaim/mongod-data-my-cluster-name-cfg-2 created
persistentvolumeclaim/mongod-data-my-cluster-name-rs0-0 created
persistentvolumeclaim/mongod-data-my-cluster-name-rs0-1 created
persistentvolumeclaim/mongod-data-my-cluster-name-rs0-2 created

Once done, spin up the cluster as usual. The volumes you created earlier will be used automatically. Restoration is done.

Risks and limitations

Storage support

Both storage and the storage plugin in Kubernetes must support volume snapshots. This limits the choices. Apart from public clouds, there are open source solutions like Ceph (rook.io for k8s) that can provide snapshotting capabilities.

Point-in-time recovery

Point-in-time recovery (PITR) allows you to reduce your Point Recovery Objective by restoring or rolling back the database to a specific transaction or time.

Volume snapshots in the clouds store data in increments. The first snapshot holds all the data, and the following ones only store the changes. This significantly reduces your cloud bill. But snapshots cannot provide you with the same RPO as native database mechanisms.

Data consistency and corruption

Snapshots are not data-aware. When a snapshot is taken, numerous transactions and data modifications can happen. For example, heavy write activity and simultaneous compound index creation in MongoDB might lead to snapshot corruption. The biggest problem is that you will learn about data corruption during restoration.

Locking or freezing a filesystem before the snapshot would help to avoid such issues. Solutions like Velero or Veeam make the first steps towards data awareness and can create consistent snapshots by automating file system freezes or stopping replication.

Percona Services teams use various tools to automate the snapshot creation safely. Please contact us here to ensure data safety.

Cost

Public clouds store snapshots on cheap object storage but charge you extra for convenience. For example, the AWS EBS snapshot is priced at $0.05/GB, whereas S3 is only $0.023. It is a 2x difference, which for giant data sets might significantly increase your bill.

Time to recover

It is not a risk or limitation but a common misconception I often see: recovery from snapshots takes only a few seconds. It does not. When you create an EBS volume from the snapshot, it takes a few seconds. But in reality, the volume you just created does not have any data. You can read more about the internals of EBS snapshots in this nice blog post.

Conclusion

Volume Snapshots on Kubernetes can be used for databases but come with certain limitations and risks. Data safety and consistency are the most important factors when choosing a backup solution. For Percona Operators, we strongly recommend using built-in solutions which guarantee data consistency and minimize your recovery time and point objectives.

Learn More About Percona Kubernetes Operators

The definition of Percona Advisors by ChatGPT is the following:

Percona Advisors can help organizations implement robust security measures to protect their databases and sensitive data.

It is partially true. The reason why we have Percona Advisors is to encode years of our database expertise and share it with the community, to automatically identify potential performance, configuration, and security issues, and let users resolve them before they become real incidents.

Percona Advisors is an open source framework, meaning you can create your own checks by following our documentation. Right now even if you create your own advisors, they will overwrite existing Percona checks. So it is only for debugging and testing purposes. We plan to extend the framework to enable innovation and contributions from the community. By default Percona Monitoring and Management (PMM) comes with basic advisor checks, whereas registered and paid tiers get much more. See more about it on our subscription page.

In this blog post, I will show how Percona Advisors work and demonstrate various checks in action.

Preparation

Percona Monitoring and Management (PMM)

Install PMM by following the documentation. I will deploy it on my Kubernetes cluster with the helm chart.

helm install pmm --set service.type="LoadBalancer" percona/pmm

Create your Percona Platform account

Navigate to https://portal.percona.com to create an account if you don’t have one already. By just creating an account you will get access to more advisors for free, whereas I myself on a paid tier enjoy the full power of sophisticated checks.

This is the number of checks that PMM has by default right after installation:

Once PMM is connected to the Platform, PMM downloads advisors automatically. With the paid subscription I get the following list:

Register PMM in the Platform

The process of registering PMM in the Platform is described in the documentation. In short:

1. Fetch the token from https://portal.percona.com/profile
2. Add this token in the PMM Settings -> Percona Platform section.

Action

Checks in action

I will deploy a MySQL cluster with Group Replication and monitor it with PMM. For the sake of simplicity and speed, I will use Percona Operator for MySQL (based on Percona Server for MySQL which is in tech preview).

As a result, I see the following checks firing up:

For example “Passwords don’t expire, as default_password_lifetime=0” checks alerts that the password expiration is not enabled:

Please consider imposing an expiry time for passwords by setting default_password_lifetime to a positive integer, indicating how many days can go by before the user has to renew their password; This will help prevent brute force attacks and risks of data leaks.

Embed checks in your CICD pipeline

With PMM API you can easily automate database compliance checks with your existing CICD pipelines.

Get all the services with failed checks:

$ curl -k --request POST \
     --url https://PMM_SERVER/v1/management/SecurityChecks/ListFailedServices \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --header 'Authorization: Bearer eyJrIjoiZTZxZFdyWG9GYklKNEVBZ1kwRU9OUTZIdnA4bFdlNXIiLCJuIjoidHN0IiwiaWQiOjF9'

{
  "result":  [
    {
      "service_name":  "default-test-cluster-mysql-2",
      "service_id":  "/service_id/0df4c5b6-ad15-4324-a70c-05fe3900b1d2",
      "warning_count":  6,
      "notice_count":  4
    },
    {
      "service_name":  "default-test-cluster-mysql-1",
      "service_id":  "/service_id/4f1ebbec-9383-41fd-9a32-60114e11b7aa",
      "warning_count":  6,
      "notice_count":  4
    },
    {
      "service_name":  "default-test-cluster-mysql-0",
      "service_id":  "/service_id/7e3cae47-3ee1-4488-883b-21b7450fb0d2",
      "error_count":  1,
      "warning_count":  6,
      "notice_count":  4
    }
  ]
}

Now get all the failed checks for a specific service:

curl -k --request POST \
     --url https://PMM_SERVER/v1/management/SecurityChecks/FailedChecks \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --header 'Authorization: Bearer eyJrIjoiZTZxZFdyWG9GYklKNEVBZ1kwRU9OUTZIdnA4bFdlNXIiLCJuIjoidHN0IiwiaWQiOjF9'
     --data '{"service_id":"/service_id/4f1ebbec-9383-41fd-9a32-60114e11b7aa"}'

This is the example of a summary of one check:

    {
      "summary":  "Passwords don't expire, as default_password_lifetime=0",
      "description":  "Please consider imposing an expiry time for passwords by setting default_password_lifetime to a positive integer, indicating how many days can go by before the user having to renew their password; This will help prevent brute force attacks and risks of data leaks.",
      "severity":  "SEVERITY_WARNING",
      "labels":  {
        "agent_id":  "/agent_id/e8528c89-8858-4591-800a-e5edde56df57",
        "agent_type":  "qan-mysql-perfschema-agent",
        "alert_id":  "/stt/a47e962529bc87c7ef98ba832e4d6ca20454ed65",
        "alertname":  "mysql_configuration_variables6",
        "cluster":  "test-cluster",
        "interval_group":  "standard",
        "node_id":  "/node_id/8abc903c-556d-47ca-85fe-8eec92cfbc66",
        "node_name":  "default-test-cluster-mysql-1",
        "node_type":  "container",
        "service_id":  "/service_id/4f1ebbec-9383-41fd-9a32-60114e11b7aa",
        "service_name":  "default-test-cluster-mysql-1",
        "service_type":  "mysql",
        "severity":  "warning",
        "stt_check":  "1"
      },
      "read_more_url":  "https://dev.mysql.com/doc/refman/8.0/en/server-system-variables.html#sysvar_default_password_lifetime",
      "service_name":  "default-test-cluster-mysql-1",
      "service_id":  "/service_id/4f1ebbec-9383-41fd-9a32-60114e11b7aa",
      "check_name":  "mysql_configuration_variables6",
      "alert_id":  "/stt/a47e962529bc87c7ef98ba832e4d6ca20454ed65"
    },

Conclusion

Day-2 operations for databases are complex. Keeping your database secure and performant before it is too late can be achieved without human intervention.  In this blog post, we showed how Percona Advisors can help with it.

Try PMM Now

September was and is an extremely fruitful month (especially for the black-hat hackers) for news about data leaks and breaches:

1. Uber suffers computer system breach, alerts authorities
2. GTA 6 source code and videos leaked after Rockstar Games hack
3. Revolut breach: personal and banking data exposed

In this blog post, we want to remind you how to keep your data safe when running your favorite open source databases.

Network exposure

Search engines like Shodan are an easy way to search for publicly available databases. Over 3.6 million MySQL servers found exposed on the Internet.

The best practice here is to run database servers in the isolated private network, even from the rest of your corporate network. In this case, you have a low risk of exposure even in the case of server misconfiguration.

If for some reason you run your database on the server in a public network, you still can avoid network exposure:

• Bind your server to the localhost or private IP address of the server

For example, for MySQL use bind-address option in your my.cnf:

bind-address = 192.168.0.123

• Configure your firewall to block access through a public network interface on the operating system

Users and passwords

To complement the network exposure story, ensure that your users cannot connect from just any IP address. Taking MySQL as an example, the following GRANT command allows to connect from one of the private networks only:

GRANT ALL ON db1.* TO 'perconaAdmin'@'192.168.0.0/255.255.0.0';

MySQL also has an auth_socket plugin, that controls the connection to the database through Unix sockets. Read more in this blog post: Use MySQL Without a Password (and Still be Secure).

Minimize the risk and do not use default usernames and passwords. SecList is a good example of bad choices for passwords: MySQL, PostgreSQL, and a misc list. Percona Platform provides users with Advisors (read more below) that preemptively check for misconfigured grants, weak passwords, and more.

So now we agree that a strong password is a must. Did you know that you can enforce it? This Percona post talks about Improving MySQL Password Security with Validation Plugin that performs such enforcement.

A strong password is set, great! To make your system even more resilient to security risks, it is recommended to have a password rotation policy. This policy can be manually executed, but also can be automated through various integrations, like LDAP, KMIP, HashiCorp Vault, and many more. For example, this document describes how Percona Server for MongoDB can work with LDAP.

Encryption

There are two types of encryption when you talk about databases and ideally, you’re going to use both of them:

1. Transport encryption – secure the traffic between client and server and between cluster nodes
2. Data-at-rest encryption (or Transparent Data Encryption – TDE) – encrypt the data on a disk to prevent unauthorized access

Transport

With an unencrypted connection between the client and the server, someone with access to the network could watch all your traffic and steal the credentials and sensitive data. We recommend enabling network encryption by default. Read the following blog posts highlighting the details:

• Enabling and Enforcing SSL/TLS for PostgreSQL Connections
• Percona Server for MySQL Encryption Options and Choices
• Network (Transport) Encryption for MongoDB

Data-at-rest

Someone can get access to the physical disk or a network block storage and read the data. To mitigate this risk, you can encrypt the data on the disk. It can be done on the file system, block storage level, and with the database storage engine itself.

Tools like fscrypt or in-built encryption in ZFS can help with file system encryption. Public clouds provide built-in encryption for their network storage solutions (ex AWS EBS, GCP). Private storage solutions, like Ceph, also come with the support of data-at-rest encryption on the block level.

Percona takes security seriously, which is why we recommend enabling data-at-rest encryption by default, especially for production workloads. Percona Server for MySQL and Percona Server for MongoDB provides you with a wide variety of options to perform TDE on the database level.

Preventive measures

Mistakes and misconfiguration can happen and it would be cool if there was a mechanism to alert you about issues before it is too late. Guess what – we have it! 

Percona Monitoring and Management (PMM) comes with Advisors which are the checks that identify potential security threats, vulnerabilities, data loss or data corruption, and other issues. Advisors are the software representation of the years of Percona’s expertise in database security and performance.

By connecting PMM to Percona Platform, users can get more sophisticated Advisors for free, whereas our paid customers are getting even deeper database checks, which discover various misconfiguration or non-compliance gems.

Learn more about Percona Platform with PMM on our website and check if your databases are secured and fine-tuned right away.

If you still believe you need more help, please let us know through our Community Forums or contact the Percona team directly.

Want fast access to the best technology for optimizing your databases but without the hassles? You’re in the right place. Starting now, you can purchase Percona Platform through the convenience of AWS Marketplace!

By subscribing to Percona Platform, you get access to superior open source solutions, support, services, and insights to run your databases at peak performance in the cloud or on-premises without license restrictions and with the cost effectiveness of open source. 

Regardless of which Percona products you currently use, buying Percona Platform through AWS Marketplace will speed your path to a highly optimized database environment. Learn more about the benefits below. 

Get a faster, transparent procurement process

AWS Marketplace offers a simple, transparent way to subscribe to Percona Platform, allowing you to streamline software procurement and deploy Percona Platform almost instantly, scaling up and down as you need. 

Avoid legal negotiations and explicit contracts

By subscribing to Percona Platform through AWS Marketplace, you’ll experience an easier approval cycle because you’ll already be covered under AWS standard agreements. This alleviates lengthy legal negotiations and explicit contracts, so you can start using Percona Platform fast and accelerate time to value. 

Leverage your existing cloud budget

If you have budget restrictions that have kept you from subscribing to the fully supported version of Percona Platform, your wait is over. By subscribing to Percona Platform through AWS Marketplace, you’ll also have the option to leverage your AWS cloud budget without escalating costs.

Access Premium-level Percona Support

Once signed up to Percona Platform, you’ll have access to Premium-level Percona Support to optimize your databases and meet your most complex, mission-critical needs. With Premium-level support, you and your team will have benefits like 24x7x365 help for outages, the fastest Service Level Agreements (SLAs) and response times, and advice, best practices, and guidance to optimize your database performance like never before. 

Get more granular database insights 

Percona Platform offers advanced observability capabilities, which are required to go beyond simple database availability, so you can automatically generate granular insights into database behavior. With these insights, you can uncover new performance patterns for in-depth troubleshooting and optimization.

Contact us today for a quote

To get a quote, visit the Percona Platform listing on AWS Marketplace and send us a request for purchase. We’ll get back to you to arrange everything and ensure you’re up and running on Percona Platform fast!

Get a quote

 

We at Percona are committed to delivering software that enables users to run databases anywhere. Our Operators for databases and Percona Monitoring and Management (PMM) Database as a Service (DBaaS) confirm our commitment to Kubernetes. Kubernetes is not only the most popular container orchestrator, but also becoming a de-facto standard for containerized workloads.

Even though we have an enterprise-grade solution to run and manage databases on Kubernetes, we still see that Kubernetes itself sometimes becomes a blocker for onboarding. We wrote a blog post some time ago about spinning up DBaaS in under 20 minutes. What if we can do it in two? This is why we partnered with a cloud-native service provider – Civo – to provide our users with a free temporary Kubernetes cluster. In this blog post, you will learn how to use it and try out our Private DBaaS solutions without the need of being a database or Kubernetes expert.

How do I get the cluster?

• Sign in to Percona Platform. If you don’t have an account yet, click Create one at the bottom of the sign-in form.
• Find “Free Kubernetes” in the menu on the left:

• Click “Launch a new cluster”. It will take less than 90 seconds to create one.
• Once the cluster is ready, you will be able to download kubeconfig – a file used to access the Civo Kubernetes cluster.

Save this file somewhere on your computer, we will need it later to register Kubernetes in PMM DBaaS. That is it, the cluster is up and running. 

Limitations

• The cluster will be automatically destroyed in three hours. It must not be used for any production workloads.
• The cluster comes with three nodes (4 CPUs, 8 GB RAM each) and does not have auto scaling enabled. It is enough for deploying a database cluster and an application.

Try DBaaS in Percona Monitoring and Management

Install PMM server

If you have a PMM server – skip this section. If not, we are going to deploy it using the quick install. You can also install PMM on Kubernetes with a helm chart by following our documentation and this blog post.

Run the following command to install PMM server on your docker compatible *nix based machine (see quick start guide for more details):

curl -fsSL https://www.percona.com/get/pmm | /bin/bash

When the script is done, the CLI tool will print a list of IP-based URLs you can put in a browser to access the PMM UI.  Copy/paste one into your favorite browser.  You may receive a security warning, there are instructions in the script output on how to bypass if you don’t get a “proceed anyway” option in your browser.  

DBaaS

You can find necessary information about how to utilize DBaaS in our documentation or this video. In general there are few steps:

1. At the time of writing this blog post, DBaaS is in technical preview. Do not forget to enable it in Settings -> Advanced Settings.
2. Register the Kubernetes cluster in the DBaaS using the kubeconfig generated in the Portal
3. Deploy your first database

Your database will be ready in a few minutes, you will get the endpoint to connect to and the username and password. By default the database is not exposed publicly and reachable only within the Kubernetes cluster. You can change it in the Advanced Options when creating the database.

With ‘Free Kubernetes’ we want to simplify PMM DBaaS onboarding and we also want to bring value to our community of users. It is the first version and we plan to deliver more enhancements to provide even more exciting onboarding. It would be great if you could help us to find those improvements by submitting your feedback at platform_portal@percona.com. Please spend a couple of minutes and let us know what problems or improvements you would like to see in your PMM DBaaS and Kubernetes journey. 

On the 15th of April, Percona introduced the general availability of Percona Platform, and with that step, our company started a new, interesting period.

Popular questions we have received after this date include: 

• What is Percona Platform, and how does it differ from Pecona Account? 
• What is the difference between Percona Platform and Percona Portal? 
• Why should Percona users connect to Percona Platform?

In this blog post, I will answer these questions and provide a summary of the benefits of a Percona Account. 

What is Percona Platform, and how does it differ from Pecona Account? 

First, let’s make sure we understand the following concepts:

Percona Platform – A unified experience for developers and database administrators to monitor, manage, secure, and optimize database environments on any infrastructure. It includes open source software and tools, access to Percona Expert support, automated insights, and self-service knowledge base articles.

Percona Account – A single login provides users with seamless access across all Percona properties, as well as insights into your database environment, knowledge base, support tickets, and guidance from Percona Experts. By creating a Percona Account, you get access to Percona resources like Forums, Percona Portal, ServiceNow (for current customers), and, in the future, other Percona resources (www.percona.com, Jira, etc.).

Percona Portal – One location for your Percona relationship, with a high-level overview of your subscription and key resources like the knowledge base, support tickets, and ability to access your account team from anywhere.

Next, let’s try to understand the complete flow users will have to go through when they decide to start using the Percona Platform. A Percona Account is foundational to Percona Platform. Users can create their own account here on Percona Portal. There are two options for Percona Account creation, either via an email and password or by using a social login from Google or Github.  

What is the difference between Percona Platform and Percona Portal? 

From the description above Percona Platform is a great experience in open source software, services, tools, and resources Percona is offering to all users. Percona Portal is just a part of Percona Platform and could be thought of as the core component of Percona Platform to discover the value Percona is offering to registered users.

Why should Percona users connect to Percona Platform?

Percona Platform brings together enterprise-ready distributions of MySQL, PostgreSQL, and MongoDB and a range of open source tools for database monitoring, backup, and management, making it easier to run complex database environments.

Percona Platform consists of several essential units: 

• Percona Account, 
• Percona Monitoring & Management, which can assure alerting, advisor insight for users’ environments, backups, private DBaaS, etc.
• Percona Portal, 
• Support and services.

Percona Monitoring and Management, DBaaS 

The heart of Percona Platform is Percona Monitoring and Management (PMM), which provides the observability required to understand database health while offering actionable insights to remediate database incidents or performance issues. 

PMM has already won the hearts of millions of DBAs around the world with its Query Analytics (QAN) and Advisors and now with Percona Platform release we made it even cooler! 

The new technical preview feature that I’m talking about is Private DBaaS. 

With less than 20 minutes to configure, Percona Private DBaaS enables you to provide self-service databases to your internal teams in any environment. 

Percona Private DBaaS enables Developers to create and manage database clusters through familiar UI and well-documented API, providing completely open source and ready-to-use clusters with MySQL or MongoDB that contain all necessary Kubernetes settings. Each cluster is set up with high availability, load balancing, and essential monitoring and management.

Private DBaaS feature of PMM is free from vendor lock-in and does not enforce any cloud or on-prem infrastructure usage restrictions.

Learn more about DBaaS functionality in our Documentation

Percona Portal 

Percona Portal is a simple UI to check and validate your relationship with Percona software and services. Stay tuned for updates. 

To start getting the value from Percona Portal, just create an organization in the Portal and connect Percona Monitoring & Management using a Percona account. Optionally, you can also invite other members to this group in order to share a common view of your organization, any PMM connection details, a list of current organization members, etc. 

When existing Percona customers create their accounts with corporate email, Percona Portal already knows about their organization. They are automatically added as members and can see additional details about their account, like opened support tickets, entitlements, and contact details.

Percona Account and SSO

Percona Platform is a combination of open source software and services. You can start discovering software advantages right from installing PMM and start monitoring and managing your environments. You could ask yourself what role of the Platform connection is there? Let’s briefly clarify this.

SSO is the first advantage you should see upon connecting your PMM instance to Percona Platform. All users in the organization are able to sign in to PMM using their Percona Account credentials. They also will be granted the same roles in PMM and Portal.  For example, if a user is an admin on Percona Portal, they do not need to create a new user in PMM, but will instead be automatically added as an admin in PMM. Portal Technical users will be granted a Viewer role in PMM.

Let’s assume a Percona Platform user has already created an account, has his own organization with a couple of members in it, and has also connected PMM to the Platform with a Percona Account. Then, he is also able to use SSO for the current organization. 

Percona Advisors

Advisors provide automated insights and recommendations within Percona Monitoring and Management, ensuring your database performs at its best. Constantly evolving with technology and user feedback, the Advisors check for:

• Availability at risk,
• Replication inconsistencies,
• Durability at risk,
• Passwordless users,
• Unsecure connections,
• Unstable OS configuration,
• Available performance improvements and more.

 Advisors workflow consists of the following:

1. If the user has just started PMM and set up monitoring of some databases and environments, we provide him with a set of basic Advisors.  To check the list the user should go to the Advisors section in PMM.
   
   
2. After the user connects PMM and Platform on Percona Portal, this list gets bigger and the set of advisors is updated. 
3. The final step happens with a Percona Platform subscription. If a user subscribes to Percona Platform, we provide the most advanced list of Advisors, which cover a lot of edge cases and check database environments.

See the full list of Advisors and various tiers in our documentation here. 

As an open source company, we are always eager to engage the community and invite them to innovate with us. 

Here you can find the developer guides on how to create your own Advisors.

How to register a Percona Account?

There are several options to create a Percona account. The first way is to specify your basic data – email, last name, first name, and password.

1. Go to “Don’t have an account? Create one” on the landing page for Percona Portal. 
2. Enter a minimal set of information (work email, password, first and last name), confirm that you agree with Percona’s Terms of Service and the Privacy Policy agreement, and hit Create.
3. To complete your registration, check your email and follow the steps provided there. Click the confirmation link in order to activate your account. You may see messages coming from Okta, this is because Percona Platform uses Okta as an identity provider.
4. After you have confirmed your account creation, you can log in to Percona Portal.

You can also register a Percona Account if you already have a Google or Github account created.

When you create a Percona account with Google and Github, Percona will store only your email and authenticate you using data from the given identity providers.

1. Make sure you have an account on Google/Github.
2. On the login page, there are dedicated buttons that will instruct you to confirm account information usage by Percona.
3. Confirm you are signing in to your account if 2FA for your Google is enabled.
4. Hurray! Now you have an active account on Percona Portal.

The next option to register a Percona Account is to Continue with Github. The process is similar to Google. 

1.  An important precondition has to be met prior to account usage and registration: Set your email address in Github to Public.
2. From Settings, go to the Emails section and uncheck the Keep my email address private option. These settings will be saved automatically. If you do not want to change this option in Github, so you could use other accounts like Google, or create your own with email.
3. If you already registered an account, you are ready to go with Percona Portal. Use your Google/Github credentials every time you log in to your account.

Conclusions

Start your Percona Platform experience with Percona Monitoring & Management – open source database monitoring, management, and observability solution for MySQL, PostgreSQL, and MongoDB.

Set up Percona Monitoring and Management (version >2.27.0) following PMM Documentation. 

The next step is to start exploring Percona Platform and get more value and experience from Percona database experts by creating a Percona Account on Percona Portal. The key benefits of Percona  Platform include: 

• A Percona Account as a single authentication mechanism to use across all Percona resources, 
• Access to Percona content like blogs and forums, knowledge base.
• Percona Advisors that help optimize, manage and monitor database environments.

Once you create a Percona account, you will get basic advisors, open source software, and access to community support and forums.

When connecting Percona Monitoring and Management to Percona Account you will get a more advanced set of advisors for your systems security, configuration, performance, and data design.

Percona Platform subscription is offering even more advanced advisors for Percona customers and also a fully supported software experience, private DBaaS, and an assigned Percona Expert to ensure your success with the Platform. 

Please also consider sharing your feedback and experience with Percona on our Forums.

Visit Percona Platform online help to view Platform-related documentation, and follow our updates on the Platform what’s new page.

I am a big fan of Percona Monitoring and Management (PMM) and am happy to report that setting up Percona Platform is as easy to set up and offers a lot of value. Percona Platform reached GA status recently and I think you will find it a handy addition to your infrastructure.

What is Percona Platform?

Percona Platform brings together enterprise-level distributions of MySQL, PostgreSQL, and MongoDB plus it includes a range of open source tools for data backup, availability, and management. The core is PMM which provides database management, monitoring, and automated insights, making it easier to manage database deployments. The number of sites with more than 100 separate databases has grown rapidly in the past few years.  Being able to have command and control of that many instances from a CLI has become impossible. Businesses need to move faster in increasingly complex environments which puts ever-increasing pressure on database administrators, developers, and everyone involved in database operations. The spiraling levels of demand make it harder to support, manage, and correct issues in database environments.

What Percona Platform provides is a unified view of the health of your entire database environment to quickly visually identify and remediate issues.  Developers can now self-service many of their database demands quickly and efficiently so they can easily provision and manage databases on a self-service basis across test and production instances. So you spend fewer resources and time on the management of database complexity.

The two keys to Percona Platform are Query Analytics (QAN), which provides granular insights into database behavior and helps uncover new database performance patterns for in-depth troubleshooting and performance optimization, and Percona Advisors, which are automated insights, created by Percona Experts to identify important issues for remediation such as security vulnerabilities, misconfigurations, performance problems, policy compliance, and database design issues. Automated insights within Percona Monitoring and Management ensure your database performs at its best. The Advisors check for replication inconsistencies, durability issues, password-less users, insecure connections, unstable OS configuration, and search for available performance improvements among other functions. 

Percona Platform is a point of control for your database infrastructure and augments PMM to be even more intelligent when connected to the Percona Platform. By connecting PMM with the Percona Platform, you get more advanced Advisors, centralized user account management, access to support tickets, private Database as a Service, Percona Expertise with the fastest SLAs, and more.

So How Do I Install Percona Platform?

The first step is to install PMM by following the Quick Start Guide. You need version 2.2.7 or later.

Second, You will need a Percona Account.

Third, you will need to connect that account to PMM.

I will assume that you will already have PMM installed.  Did I mention that PMM is free, open source software?

Now you can create a name for your organization.

Now login to your PMM dashboard and select the Settings / Percona Platform.  You will need to get your ‘Public Address’ which the browser can populate the value for you if need be.

The PMM Server ID is automatically generated by PMM.  You will need to provide a name for your server, and you will need a second browser window to login into Percona Platform to get the Percona Platform Access Token (this token has a thirty-minute lifetime, so be quick or regenerate another token).

Go back into PMM, paste the Access Token into the Percona Platform Access Token field, and click Connect.

On the Percona Platform page, you will see your PMM instances. Congratulations, you are using Percona Platform!

Advisor Checks

All checks are hosted on Percona Platform. PMM Server automatically downloads them from here when the Advisors and Telemetry options are enabled in PMM under Configuration > Settings > Advanced Settings. Both options are enabled by default.

Depending on the entitlements available for your Percona Account, the set of advisor checks that PMM can download from Percona Platform differ in terms of complexity and functionality.

If your PMM instance is not connected to Percona Platform, PMM can only download the basic set of Anonymous advisor checks. As soon as you connect your PMM instance to Percona Platform, has access to additional checks, available only for Registered PMM instances.

If you are a Percona customer with a Percona Customer Portal account, you also get access to Paid checks, which offer more advanced database health information. A list is provided below.

Check Name	Description	Tier
MongoDB Active vs Available Connections	Checks the ratio between Active and Available connections.	Registered, Paid
MongoDB Authentication	Warns if MongoDB authentication is disabled.	Anonymous, Registered, Paid
MongoDB Security AuthMech	Warns if MongoDB is not using the default SHA-256 hashing as SCRAM authentication method.	Paid
MongoDB IP Bindings	Warns if MongoDB network binding is not set as recommended.	Anonymous, Registered, Paid
MongoDB CVE Version	Shows an error if MongoDB or Percona Server for MongoDB version is not the latest one with CVE fixes.	Anonymous, Registered, Paid
MongoDB Journal Check	Warns if journal is disabled.	Registered, Paid
MongoDB Localhost Authentication Bypass is Enabled	Warns if MongoDB localhost bypass is enabled.	Anonymous, Registered, Paid
MongoDB Non-Default Log Level	Warns if MongoDB is not using the default log level.	Paid
MongoDB Profiling Level	Warns when the MongoDB profile level is set to collect data for all operations.	Registered, Paid
MongoDB Read Tickets	Warns if MongoDB is using more than 128 read tickets.	Paid
MongoDB Replica Set Topology	Warns if the Replica Set cluster has less than three members.	Registered, Paid
MongoDB Version	Warns if MongoDB or Percona Server for MongoDB version is not the latest one.	Anonymous, Registered, Paid
MongoDB Write Tickets	Warns if MongoDB network is using more than 128 write tickets.	Paid
Check if Binaries are 32-bits	Notifies if version_compatible_machine equals i686.	Anonymous, Registered, Paid
MySQL Automatic User Expired Password	Notifies if version_compatible_machine equals i686.	Registered, Paid
MySQL InnoDB flush method and File Format check	Checks the following settings: innodb_file_format, innodb_file_format_max, innodb_flush_method and innodb_data_file_path	Registered, Paid
MySQL Checks based on values of MySQL configuration variables	Checks the following settings: innodb_file_format,innodb_file_format_max,innodb_flush_method and innodb_data_file_path.	Paid
MySQL Binary Logs checks, Local infile and SQL Mode checks	Warns about non-optimal settings for Binary Log, Local Infile and SQL mode.	Registered, Paid
MySQL Configuration Check	Warns if parameters are not following Percona best practices, for infile, replication threads, and replica checksum.	Paid
MySQL Users With Granted Public Networks Access	Notifies about MySQL accounts allowed to be connected from public networks.	Registered, Paid
MySQL User Check	Runs a high-level check on user setup	Registered, Paid
MySQL Advanced User Check	Runs a detailed check on user setup	Paid
MySQL Security Check	Runs a detailed check on user setup	Paid
MySQL Test Database	This check returns a notice if there is a database with name ‘test’ or ‘test_%’.	Registered, Paid
MySQL Version	Warns if MySQL, Percona Server for MySQL, or MariaDB version is not the latest one.	Anonymous, Registered, Paid
PostgreSQL Archiver is Failing	Verifies if the archiver has failed.	Paid
PostgreSQL Cache Hit Ratio	Checks database hit ratio and complains when this is too low.	Paid
PostgreSQL Configuration Change Requires Restart/Reload	Warns when a configuration was changed and requires a server restart/reload	Registered, Paid
PostgreSQL fsync is Disabled	Shows an error if the fsync configuration is disabled, as this can result in unrecoverable data corruption.	Anonymous, Registered, Paid
PostgreSQL Autovacuum Logging Is Disabled	Shows an error if the fsync configuration is disabled, as this can result in unrecoverable data corruption.	Paid
PostgreSQL Checkpoints Logging Is Disabled	Notifies if the log_checkpoints configuration option is not enabled.	Paid
PostgreSQL Max_connections is too high	Notifies if the max_connections setting is set above 300.	Anonymous, Registered, Paid
PostgreSQL Stale Replication Slot	Warns for stale replication slots since these can lead to WAL file accumulation and DB server outage.	Paid
PostgreSQL Super Role	Notifies if there are users with superuser privileges.	Anonymous, Registered, Paid
PostgreSQL Version Check	Warns if the PostgreSQL minor or major versions are not the latest, and shows an error if the major version is 9.4 or older.	Anonymous, Registered, Paid

More Advisors will be added and Perona always welcomes contributions. Check out Percona Platform today!

 

Database management is hard. Not only must your database platform team work to provide consistent and reliable performance, availability, and security, but they also need to meet your developers’ ever-increasing demands to move faster. This strain has led to an environment of increasing complexity, in which you and your teams are tasked with managing hundreds or thousands of siloed databases and their supporting technologies. 

Then, of course, there’s the problem of vendor lock-in. Too often, you’re forced into non-portable platforms or environments, where it’s difficult to reclaim your data, massive and oppressive annual support obligations swell, and vendors are more than happy to hold you hostage over proprietary features. It can seem there’s little way out of the spiral. 

Fortunately, there’s a better way. 

Introducing Percona Platform 

Percona is different. Since day one, we’ve stood for Keeping Open Source Open. Whatever your environment, any cloud or on-premises, and whatever “enterprise” features you need, we offer ultimate freedom and flexibility. It’s what you’d expect from a company committed to the ideals of free and open source software and why many of you have trusted us for years.

Today, we’re proud to announce that we’re bringing together many of your favorite Percona offerings into a new product called Percona Platform. 

Percona Platform combines our software and services into a subscription-based offering that gives you all the tools and expertise you need to succeed: Whether that’s our fully open, performance-optimized database servers, our Percona-curated distributions, which include best-of-breed solutions for high availability, security, and backup, or Percona Monitoring and Management (PMM).  And we’re backing it up with our world-class services, delivered however you need them — whether that’s support for self-managed installations, or providing you with fully managed database environments.

Take a look at everything included:

This commitment to giving our customers greater control of availability and access to their data and saving them time on managing databases is why RedMonk wrote: 

“There has been an explosion of data tools available to enterprises, which in turn has led to rampant fragmentation and highly complex, disjointed workflows. Organizations that prize velocity are looking for ways to integrate various data tools into their broader development workflows and monitor them.  This is the opportunity that Percona Platform is built for. The combination of software and services in this space is intended to help customers better manage their database deployments.” –   Rachel Stephens, Sr. Analyst, RedMonk

But you really don’t have to look any further than our customers, who love the combination of Percona software and services. It’s why more than two-thirds of our Support customers run Percona Software, and 58% of our Managed Services customers choose to run Percona databases in addition to their Percona Software.

Here’s what some of them had to say: 

“Percona’s drop-in database software replacement was a big factor as migrations can be tricky! … We have more control with Percona.” – Carpages.ca

“Percona has the best engineers in the world.” –  Appsuite 

“Percona have provided us with the best solutions and identified opportunities for improvement in many areas of MySQL, Postgres, and MongoDB.” – Solera

Percona Platform is designed based on decades of experience from Percona experts, and its production-ready database software is battle-tested in mission-critical environments with the most demanding scale and performance needs.

Discover how the Percona Platform can make database management easier for you. 

 

We are thrilled to announce the Percona Platform, which brings together database distributions, support expertise, services, management, and automated insights into a single product. Building on our expertise with databases including MySQL, PostgreSQL, and MongoDB, the new Percona Platform will simplify how you can monitor, manage, and optimize your database instances across any infrastructure. In addition, the Percona Platform will enable you to run your own private Database-as-a-Service (DBaaS) instances.

Addressing Your Biggest Database Problems

We’re always listening and with the Percona Platform, we aim to address three of the biggest challenges database administrators have according to our last research report. The three are standardization, performance, and availability.  In fact, 59% of respondents lost sleep at night due to concerns around downtime and availability. Let’s be honest, there’s not much worse than losing sleep.

Developers just want things to work, and they want self-service support so they can iterate faster with fewer cross-team handoffs. Database administrators want help to manage multiple database instances in a consistent approach, so they can deliver a highly available and performant database platform for their development teams as well as their customers. With this release, we are taking the first steps towards meeting these requirements.

Announcing the Percona Platform

Percona Platform is our response, bringing together everything that developers and DBAs need to implement environments quickly, with less effort and complexity. This launch will help developers create their applications faster while building upon a strong foundation provided by their database team.

This launch will preview the Percona Platform, allowing you to unify the entire database experience from packages and backup to monitoring and management, with general availability in early 2022. Percona brings together distributions of MySQL, PostgreSQL, and MongoDB including a range of open source tools for data backup, availability, and management. It also includes Percona Monitoring and Management (PMM) for database management, monitoring, and automated insights, making it easier to manage database deployments. The Percona Platform preview will have no up-front commitments or fees to use.

Percona Platform will cover all deployment configurations, from internal data center deployments through to public, private, and hybrid cloud instances. At general availability, the platform will also cover self-managed and Percona-managed instances, allowing you the ultimate flexibility to run things the way that best suits your environment, level of confidence, and operating models.

What’s in the Percona Platform?

The Preview will include:

• Percona Portal — provides access to all Percona services and product expertise in one place

• Percona Account — simplifies user account management for all Percona products and services
• Private DBaaS — enables you to easily provision and manage databases on a self-service basis across development and testing instances
• Advisors — automated insights to ensure your database is performing at its best

The platform is available as a preview, and you can get started at percona.com/platform-preview. Try it out today!