May
26
2021
--

Compiling a Percona Monitoring and Management v2 Client in ARM: Raspberry Pi 3 Reprise

Percona Monitoring and Management Client Raspberry Pi 3

Percona Monitoring and Management Client Raspberry Pi 3In this follow-up blog to Compiling a Percona Monitoring and Management v2 Client in ARM Architecture, we will show what changes are needed to get the latest versions of PMM working on ARM architecture. In this case, we will do it using a Raspberry Pi 3, instead of the AWS EC2 ARM node. With these two blogs, we are covering a good deal of ground in respect to ways of deploying PMM in ARM. Let us know if you use any other device, though, and we’ll be happy to try if we can get ahold of one!

The latest versions will need the vmagent for supporting the newly added VictoriaMetrics functionality. In this case, the binaries are precompiled by the VM team, so we just need to download them and copy them to the appropriate directories. These steps are exactly the same for AWS, but you should use the ARM64 packages instead.

For ease of use, we have again included binaries in the following project (note that versions kept there may not be current latest at any point): https://github.com/guriandoro/pmm_arm, and a gist with the commands used.

Installing Dependencies

As always, let’s get some basic packages needed out of the way:

sudo apt-get install git
curl -LO https://golang.org/dl/go1.15.2.linux-armv6l.tar.gz
tar xzf go1.15.2.linux-armv6l.tar.gz
sudo mv go /usr/local/go
echo "export PATH=\$PATH:/usr/local/go/bin" >> .bashrc
source .bashrc

shell> go version
go version go1.15.2 linux/arm

After we install these tools, we will clone all needed projects to our Go source directory:

mkdir -p ~/go/src/github.com/percona/
cd ~/go/src/github.com/percona/
git clone https://github.com/percona/pmm-admin.git
git clone https://github.com/percona/pmm-agent.git
git clone https://github.com/percona/node_exporter.git

mkdir -p ~/go/src/github.com/percona/
cd ~/go/src/github.com/percona/
git clone https://github.com/percona/pmm-admin.git
git clone https://github.com/percona/pmm-agent.git
git clone https://github.com/percona/node_exporter.git

And get the VictioraMetrics vmagent binary ready, too:

cd /tmp/
curl -LO https://github.com/VictoriaMetrics/VictoriaMetrics/releases/download/v1.59.0/vmutils-arm-v1.59.0.tar.gz
tar xzf vmutils-arm-v1.59.0.tar.gz

Compiling

As with the previous blog, pmm-admin compilation is straightforward:

cd ~/go/src/github.com/percona/pmm-admin
make release

For node_exporter, we will need the changes for PR 1196:

cd ~/go/src/github.com/percona/node_exporter/
curl -LO https://raw.githubusercontent.com/pgier/node_exporter/1fc4d14c6613766483a8b9117ffa9cc50c1a976d/Makefile.common
make build

Lastly, for pmm-agent, we will also need an additional step. However, before we can introduce the changes, we need to compile and let it fail once:

cd ~/go/src/github.com/percona/pmm-agent
make release
...
/home/pi/go/pkg/mod/github.com/!percona-!lab/pg_query_go@v1.0.1-0.20190723081422-3fc3af54a6f7/parser/include/port/atomics.h:68:10: fatal error: port/atomics/arch-arm.h: No such file or directory
 #include "port/atomics/arch-arm.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:27: release] Error 2

We will then need to download the arch-arm.h file to that directory:

cd ~/go/pkg/mod/github.com/\!percona-\!lab/pg_query_go@v1.0.1-0.20190723081422-3fc3af54a6f7/parser/include/port/atomics/
sudo curl -LO https://raw.githubusercontent.com/postgres/postgres/master/src/include/port/atomics/arch-arm.h
sudo chown pi:pi arch-arm.h

And then we are ready to compile pmm-agent successfully:

cd ~/go/src/github.com/percona/pmm-agent
make release

Moving the Files to Their Final Destination

As the last step, let’s create the needed directories and move the files where they belong.

sudo mkdir -p /usr/local/percona/pmm2/
cd /usr/local/percona/pmm2/
sudo mkdir exporters/
sudo mkdir config/
sudo mkdir -p collectors/textfile-collector/high-resolution
sudo mkdir -p collectors/textfile-collector/medium-resolution
sudo mkdir -p collectors/textfile-collector/low-resolution

cd ~/go/src/github.com/percona/
sudo cp pmm-admin/bin/pmm-admin /usr/local/bin/
sudo cp pmm-agent/bin/pmm-agent /usr/local/bin/

sudo cp node_exporter/node_exporter /usr/local/percona/pmm2/exporters/
sudo cp /tmp/vmagent-prod /usr/local/percona/pmm2/exporters/vmagent

Starting the PMM Client

At last, we can configure and start the client!

sudo pmm-agent setup \
 --config-file=/usr/local/percona/pmm2/config/pmm-agent.yaml \
 --server-address=192.168.1.121:443 --server-insecure-tls \
 --server-username=admin --server-password=admin \
 192.168.1.247 generic raspberrypi

sudo pmm-agent --config-file=/usr/local/percona/pmm2/config/pmm-agent.yaml

Check if things are working:

shell> pmm-admin list
Service type        Service name        Address and port        Service ID

Agent type           Status           Metrics Mode        Agent ID                                              Service ID
pmm_agent            Connected                            /agent_id/2e2ed51d-f573-4510-8f4f-8b7899ef4f67
node_exporter        Running          push                /agent_id/6f34eb76-429f-47fb-a178-a1bbf4ddadcb
vmagent              Running          push                /agent_id/80a9e104-aab6-48ef-a709-475330385e32

OS and Disk Usage

Some information about the OS and kernel used:

shell> uname -a
Linux raspberrypi 5.4.83-v7+ #1379 SMP Mon Dec 14 13:08:57 GMT 2020 armv7l GNU/Linux

shell> lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description: Raspbian GNU/Linux 10 (buster)
Release: 10
Codename: buster

As for disk usage, the four compiled binaries will take up 72Mb of space.

Extra Tools

We can also download pt-summary, to get the reports in the Node Summary dashboards:

cd /usr/local/percona/pmm2/tools/
curl -LO percona.com/get/pt-summary
chmod +x pt-summary

We Come in Peace… We Mean you no Harm

Resource usage is in the expected (very low) overhead ranges, as we can see in the following before (completely idle) and after (only PMM client binaries running) top samples.

Before:

top - 05:27:42 up 28 min,  4 users,  load average: 0.00, 0.00, 0.00
Tasks: 135 total,   1 running, 134 sleeping,   0 stopped,   0 zombie
%Cpu(s):  4.9 us,  1.2 sy,  0.0 ni, 93.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :    924.8 total,    543.4 free,    104.3 used,    277.1 buff/cache
MiB Swap:    100.0 total,    100.0 free,      0.0 used.    753.9 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 1082 root      20   0   10432   2864   2480 R  11.1   0.3   0:00.05 top
    1 root      20   0   33784   8072   6392 S   0.0   0.9   0:05.36 systemd
...

After:

top - 05:46:09 up 47 min,  5 users,  load average: 0.08, 0.10, 0.09
Tasks: 144 total,   1 running, 143 sleeping,   0 stopped,   0 zombie
%Cpu(s):  2.0 us,  0.7 sy,  0.0 ni, 97.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 s
MiB Mem :    924.8 total,    495.2 free,    151.9 used,    277.8 buff/cache
MiB Swap:    100.0 total,    100.0 free,      0.0 used.    713.5 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 1136 root      20   0  805056  18368   8136 S   7.9   1.9   0:44.10 node_exporter
 1141 root      20   0  804132  33396   7244 S   1.3   3.5   0:20.27 vmagent
 1214 pi        20   0   10404   3084   2564 R   0.7   0.3   0:00.13 top
...

Conclusion

With this and the blog post linked at the beginning, you should have all the data you need to compile and run Percona Monitoring and Management (PMM) client in either a Raspberry Pi 3 or an AWS EC2 ARM node.

Feb
25
2013
--

Percona Server on the Raspberry Pi: Your own MySQL Database Server for Under $80

There are many reasons for wanting a small MySQL database server:

  • You’re a uni student who wants to learn the SQL language better and needs a mini-testbox
  • You’re a Windows user who wants to play around with Percona Server on Linux
  • You’re a corporate application developer who wants a small SQL development & test box
  • You’re a Internet startup that just needs a tiny startup database server without all the added costs

So, how about if you could setup a small Arch Linux ARMv6-based hardware device which runs Percona Server for MySQL, in a space not much bigger than your mouse, with the power consumption of only a smartphone charger, fully networked, all for under $80?

Introducing the Raspberry Pi with Percona Server:

Raspberry Pi

Raspberry Pi Fully Connected (Only the power + network connections are needed in this tutorial)

A small disclaimer first: Arch Linux is not currently on our list of supported OS’es (i.e. we do not promise to provide binaries, even though they are available from the Arch Linux AUR repository), however if you do happen to run into issues, our support service engineers are happy to provide help.

To get started, here is what you need (prices in AU$):

  • The Raspberry Pi itself (about $36)
    • Make sure to get the v2 (512MiB SDRAM) version
  • A plastic case (~$12 on eBay which includes 3 heatsinks)
  • A SD Card reader (likely build into your laptop or PC already)
  • Network setup
    • A network lead and spare network port on your router
    • A DHCP server (your router likely has this enabled already)
  • A quality 2GB (or more) SDHC Card
    • I use a SanDisk 8GB Class10 30MB/s card (~$8.50 on eBay)
  • A 1Amp+ micro USB power supply + cable
    • I use a 2Amp dual portapow supply (~$20 on eBay)

Total cost: ~76.5 AU$ (= ~79 US$).

Optionally, if your screen has a DVI port, you can get a HDMI Cable + DVI (male)-to-HDMI (female) connector (~$12). This enables you to connect the Raspberry Pi to your screen, and so follow what is happening on the console of the Raspberry Pi. Make sure to get the proper DVI connector for your screen as many connectors that are sold have a wrong pin layout (most standard computer screens use a DVI-D connector, not a DVI-I one).

Alternatively, if you really do end up needing to view the console of your Raspberry Pi (for example to configure an IP address in Arch Linux if you do not have a DHCP server), you can simply hook the Raspberry Pi to your TV set using the Composite RCA (PAL & NTSC) connector.

Once you have your hardware, here is how to get Percona Server for MySQL up and running:

  1. Insert the SD card into a slot on your laptop or desktop PC.
  2. Download Fedora ARM Installer (available for Linux and Windows).
  3. Download the Arch Linux ARM image from the Raspberry Pi website and unzip.
  4. Start Fedora ARM Installer (in Windows make sure to right-click the exe and “Run as administrator”).
    • Select the image file (about 1.8GB .img file) and select the destination drive to write the image to. Make sure you selected the right drive and click “Install”.
    • The image gets written to a 2Gb partition on the card. If you have a larger card, you can add another partition later.
  5. Software eject the SD card using your OS’es software eject system (to prevent partial writes), take out the card and insert into your Pi. Carefull, the card goes upside down :)
  6. Plug in the network lead to the Raspberry Pi, and connect the other end of the lead to your router.
  7. Plug in the Micro USB power lead to the Raspberry Pi, the other side to the power supply. Plug in the power supply.
  8. The Raspberry Pi will automatically get an IP address from the DHCP server (Arch Linux has this preset)
    • Also, Arch Linux has sshd already active (sshd=ssh daemon, a software package allowing you to connect to a [Linux] server console/command prompt).
  9. Connect via ssh to the Raspberry Pi. For windows, I use the Tunnelier SSH Client from Bitvise
    • If you do not know what IP address the Raspberry Pi got (likely the first available address in your DHCP range), you can always check your router’s DHCP table which should list all IP addresses it assigned, usually together with a host name.
    • While there, you could also assign a permanent IP address (still assigned via DHCP) by binding a MAC address to a reserved IP in your router’s DHCP server setup. This makes it easier later on.
    • You could also assign a permanent DNS name (like pi.yourdomain.com) by binding the reserved IP to a DNS name in your router’s DNS server setup.
    • The password for user ‘root’ is ‘root’, and the OS boots in about 10 seconds.
    • Connecting to the Raspberry Pi using Linux (ssh) and Windows (Tunnelier):
      $ ssh root@192.168.0.180
      root@192.168.0.180's password:
      Last login: Fri Feb 22 23:27:49 2013 from percona.<...>
      [root@alarmpi ~]#

      Tunnelier connecting with the Raspberry Pi

  10. Once connected to the Raspberry Pi, either via ssh on Linux or Tunnelier on Windows, update the root password first:
    [root@alarmpi ~]# passwd
    Enter new UNIX password:
    Retype new UNIX password:
    passwd: password updated successfully

    (If you use Windows+Tunnelier and have saved your password in the configuration file, update it in Tunnelier now also)

  11. Re-connect to see if your password change worked as expected, and install Percona Server from the Arch Linux AUR Repo at the same time:
    (pacman is the ‘Package Manager’ for Arch Linux) 

    [root@alarmpi ~]# pacman -S percona-server
    resolving dependencies...
    looking for inter-conflicts...
    
    Targets (4): libaio-0.3.109-6  libmysqlclient-5.5.30-1  mysql-clients-5.5.30-1  percona-server-5.5.29_rel29.4-1
    
    Total Download Size:    8.07 MiB
    Total Installed Size:   96.00 MiB
    
    Proceed with installation? [Y/n] y
    :: Retrieving packages from extra...
    libmysqlclient-5.5.30-1-armv6h                                           3.2 MiB   435K/s 00:08 [########################################################] 100%
    mysql-clients-5.5.30-1-armv6h                                          736.6 KiB   395K/s 00:02 [########################################################] 100%
    :: Retrieving packages from community...
    libaio-0.3.109-6-armv6h                                                  4.6 KiB  1089K/s 00:00 [########################################################] 100%
    percona-server-5.5.29_rel29.4-1-armv6h                                   4.1 MiB   323K/s 00:13 [########################################################] 100%
    (4/4) checking package integrity                                                                 [########################################################] 100%
    (4/4) loading package files                                                                      [########################################################] 100%
    (4/4) checking for file conflicts                                                                [########################################################] 100%
    (4/4) checking available disk space                                                              [########################################################] 100%
    (1/4) installing libmysqlclient                                                                  [########################################################] 100%
    (2/4) installing mysql-clients                                                                   [########################################################] 100%
    (3/4) installing libaio                                                                          [########################################################] 100%
    (4/4) installing percona-server                                                                  [########################################################] 100%
    Installing MySQL system tables...
    OK
    Filling help tables...
    OK
    [...more setup information, please read...]
  12. [Optional] Set Percona server to startup at boot time
    [root@alarmpi ~]# systemctl enable mysqld
    ln -s '/usr/lib/systemd/system/mysqld.service' '/etc/systemd/system/multi-user.target.wants/mysqld.service'
  13. Start Percona Server
    [root@alarmpi ~]# systemctl start mysqld
  14. Connect using the mysql client:
    [root@alarmpi ~]# /usr/bin/mysql
    Welcome to the MySQL monitor.  Commands end with ; or \g.
    [...]
    mysql> SHOW ENGINES;
    +--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
    | Engine             | Support | Comment                                                                    | Transactions | XA   | Savepoints |
    +--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
    | InnoDB             | DEFAULT | Percona-XtraDB, Supports transactions, row-level locking, and foreign keys | YES          | YES  | YES        |
    [...]
    mysql> SELECT "Hello World!" AS "Success!" \G
    *************************** 1. row ***************************
    Success!: Hello World!
    1 row in set (0.00 sec)

Now that was easy wasn’t it?

A few other points:

  • This setup uses Percona Server for MySQL with the latest mysql client, which is untested, but should be “ok/fine” in most situations
  • If you want to edit the configuration file, you can find it in /etc/mysql/my.cnf
  • If you setup Percona Server to startup at boot time (step 12), give your little server a reboot (type “reboot”) now and check that Percona Server works fine afterwards.
  • Make sure to secure your server properly, especially if it will be exposed on the Internet or if this will be a production server.
    • Secure the initial accounts for your installation.
    • If this will be a production server, you should use a seperate userID to run Percona Server.
    • There are more (OS and otherwise) steps necessary then what it listed here to make your server secure, but those are outside the scope of this article.
  • To check if mysqld is running, use “ps -ef | grep -v grep | grep mysqld” at the Linux command line
  • If you did get a SD card which is larger then 2GB, you can add another partition using fdisk & mkfs.ext4 (comment below if you cannot work out how, and I can post some additional information)

Enjoy!

The post Percona Server on the Raspberry Pi: Your own MySQL Database Server for Under $80 appeared first on MySQL Performance Blog.

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com