Jan
20
2017

How to Manually Build Percona Server for MySQL RPM Packages

In this blog, we’ll look at how to manually build Percona Server for MySQL RPM packages.

Several customers and other people from the open source community have asked us how they could make their own Percona Server for MySQL RPM binaries from scratch.

This request is often made by companies that want to add custom patches to our release. To do this, you need to make some modifications to the

percona-server.spec

file in the source tree, and some preparation is necessary.

This post covers how you can make your own RPMs from GIT or source tarball so that you can build RPMs from your own modified branch, or by applying patches. In this example, we’ll build Percona Server 5.7.16-10.

Making your own RPMs is not a recommended practice, and should rarely be necessary.

Prepare the Source

Using GIT Repository

We can fetch percona/percona-server from GitHub (or your own fork). As we build Percona Server 5.7.16-10, we create a new branch based on the tag of that version:

$ git clone https://github.com/percona/percona-server.git
Cloning into 'percona-server'...
remote: Counting objects: 1216597, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 1216597 (delta 1), reused 1 (delta 1), pack-reused 1216592
Receiving objects: 100% (1216597/1216597), 510.50 MiB | 5.94 MiB/s, done.
Resolving deltas: 100% (997331/997331), done.
Checking out files: 100% (28925/28925), done.
$ cd percona-server/
$ git checkout -b build-Percona-Server-5.7.16-10 Percona-Server-5.7.16-10
Switched to a new branch 'build-Percona-Server-5.7.16-10'
$ git submodule init
Submodule 'Percona-TokuBackup' (https://github.com/percona/Percona-TokuBackup.git) registered for path 'plugin/tokudb-backup-plugin/Percona-TokuBackup'
Submodule 'storage/rocksdb/rocksdb' (https://github.com/facebook/rocksdb.git) registered for path 'storage/rocksdb/rocksdb'
Submodule 'PerconaFT' (https://github.com/percona/PerconaFT.git) registered for path 'storage/tokudb/PerconaFT'
$ git submodule update
Cloning into 'plugin/tokudb-backup-plugin/Percona-TokuBackup'...
...
Submodule path 'plugin/tokudb-backup-plugin/Percona-TokuBackup': checked out '1b9bb16ad74588601d8fefe46c74cc1dac1dd1d5'
Cloning into 'storage/rocksdb/rocksdb'...
...
Submodule path 'storage/rocksdb/rocksdb': checked out '6a17b07ca856e573fabd6345d70787d4e481f57b'
Cloning into 'storage/tokudb/PerconaFT'...
...
Submodule path 'storage/tokudb/PerconaFT': checked out '0c1b53909bc62a4d9b420116ec8833c78c0c6e8e'

Downloading Source Tarball

An alternative way is to download the source tarball, which you can find at https://www.percona.com/downloads/Percona-Server-5.7/Percona-Server-5.7.16-10/source/.

Extract the source tarball, as the RPM spec file is located there:

$ wget https://www.percona.com/downloads/Percona-Server-5.7/Percona-Server-5.7.16-10/source/tarball/percona-server-5.7.16-10.tar.gz
$ tar -xzvf percona-server-5.7.16-10.tar.gz
$ cd percona-server-5.7.16-10

Making Changes with Patch Files

If you need to make any changes to the source code, you can either use your own GitHub fork or you can apply patches. If you use the former, then you can skip this section.

Why Patches?

Why would we want to use patch files? Because you won’t need to maintain your own fork. You can just build the RPM with the Percona Server source tarball and the patch file.

Create Patch Files

If you do not want to use your own fork in GitHub, you can also create some patch files and modify the RPM spec file to include them.

Create your changes to the source files

Use

diff

to create the

patch

file:

$ diff -ru FILE-orig FILE >| ~/custom.patch

Add Patch to RPM Spec File

In order for the patch to be applied during the building of the RPM, edit the

./build-ps/percona-server.spec

file and add the two lines that are highlighted:

...
Source91:       filter-requires.sh
Patch0:         mysql-5.7-sharedlib-rename.patch
Patch1:         custom.patch
BuildRequires:  cmake >= 2.8.2
...
%prep
%setup -q -T -a 0 -a 10 -c -n %{src_dir}
pushd %{src_dir}
%patch0 -p1
%patch1 -p1
%build
...

Note that you have to number the patches, in this case I gave it the name

patch1

Creating New Source Tarball

If you use your own GitHub fork, or you made manual changes to the source (and you’re not using patch files), you should use that to create your own source tarball.

First, change the Percona Server version number. In this case, we are naming it

10custom

to indicate this is not a standard build. You can adapt as you wish, just make sure the

VERSION

file looks something like this:

$ cat VERSION
MYSQL_VERSION_MAJOR=5
MYSQL_VERSION_MINOR=7
MYSQL_VERSION_PATCH=16
MYSQL_VERSION_EXTRA=-10custom

Then make the source tarball:

$ cmake . -DDOWNLOAD_BOOST=1 -DWITH_BOOST=build-ps/boost
$ make dist
...
-- Source package ~/percona-server/percona-server-5.7.16-10custom.tar.gz created
Built target dist

Now you have the tarball in your source directory, but we won’t use it yet. We need to add some TokuDB submodules to it first. The make dist also kept the uncompressed directory, which we will use to create the tarball again when the TokuDB parts are included:

$ rm percona-server-5.7.16-10custom.tar.gz
$ cp -R storage/tokudb/PerconaFT/*
    percona-server-5.7.16-10custom/storage/tokudb/PerconaFT/
$ cp -R plugin/tokudb-backup-plugin/Percona-TokuBackup/*
    percona-server-5.7.16-10custom/plugin/tokudb-backup-plugin/Percona-TokuBackup/
$ tar --owner=0 --group=0 --exclude=.git
   -czf percona-server-5.7.16-10custom.tar.gz
   percona-server-5.7.16-10custom

Preparing Build Environment

Environment Requirements

Make sure the build host has at least 10GB of free space and at least 4GB of RAM, or the build will fail at some point.

Install Dependencies

To build the RPM, we need to prepare our build environment and ensure the necessary build dependencies are installed:

$ sudo yum install epel-release
$ sudo&nbsp;yum install git gcc gcc-c++ openssl check cmake
                   bison boost-devel asio-devel libaio-devel
                   ncurses-devel readline-devel pam-devel
                   wget perl-Env time numactl-devel rpmdevtools
                   rpm-build

Prepare RPM Build Tree

Next we need to prepare our build directory structure, in this case we will install it in

~/rpmbuild

$ cd ~/
$ rpmdev-setuptree

Download Boost Source

We also need to download the boost source (http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz) and put it in the

~/rpmbuild/SOURCES/

directory:

$ cd ~/rpmbuild/SOURCES
$ wget http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz

Move Files to the RPM Build Tree

Copy the source tarball:

$ cp percona-server-5.7.16-10custom.tar.gz ~/rpmbuild/SOURCES/

Also copy the

./build-ps/rpm/*

files:

$ cp ~/percona-server/build-ps/rpm/* ~/rpmbuild/SOURCES

If you have any patch files, move them to the

~/rpmbuild/SOURCES/

directory as well:

$ cp ~/custom.patch ~/rpmbuild/SOURCES/

Then the

percona-server.spec

file goes into the

~/rpmbuild/SPECS

directory:

$ cp ~/percona-server/build-ps/percona-server.spec ~/rpmbuild/SPECS/

Setting Correct Versions in the Spec File

$ cd ~/rpmbuild/SPECS/
$ sed -i s:@@MYSQL_VERSION@@:5.7.16:g percona-server.spec
$ sed -i s:@@PERCONA_VERSION@@:10custom:g percona-server.spec
$ sed -i s:@@REVISION@@:f31222d:g percona-server.spec
$ sed -i s:@@RPM_RELEASE@@:1:g percona-server.spec
$ sed -i s:@@BOOST_PACKAGE_NAME@@:boost_1_59_0:g percona-server.spec

Note the

@@PERCONA_VERSION@@

contains our

10custom

version number.

You can add your changelog information in the

%changelog

section.

Building RPM Packages

For Percona Server, making the RPMs is a two-step process. First, we need to make the SRPMS:

$ cd ~/
$ rpmbuild -bs --define 'dist .generic' rpmbuild/SPECS/percona-server.spec
Wrote: /home/vagrant/rpmbuild/SRPMS/Percona-Server-57-5.7.16-10custom.1.generic.src.rpm

And then we can build the binary RPMs from the SRPMS:

$ rpmbuild --define 'dist .el7' --rebuild
       rpmbuild/SRPMS/Percona-Server-57-5.7.16-10custom.1.generic.src.rpm
...
Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-server-57-5.7.16-10custom.1.el7.x86_64.rpm
Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-client-57-5.7.16-10custom.1.el7.x86_64.rpm
Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-test-57-5.7.16-10custom.1.el7.x86_64.rpm
Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-devel-57-5.7.16-10custom.1.el7.x86_64.rpm
Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-shared-57-5.7.16-10custom.1.el7.x86_64.rpm
Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-shared-compat-57-5.7.16-10custom.1.el7.x86_64.rpm
Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-tokudb-57-5.7.16-10custom.1.el7.x86_64.rpm
Wrote: /home/vagrant/rpmbuild/RPMS/x86_64/Percona-Server-57-debuginfo-5.7.16-10custom.1.el7.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.xmcI8e
+ umask 022
+ cd /home/vagrant/rpmbuild/BUILD
+ cd percona-server-5.7.16-10custom
+ /usr/bin/rm -rf /home/vagrant/rpmbuild/BUILDROOT/Percona-Server-57-5.7.16-10custom.1.generic.x86_64
+ exit 0
Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.UXUTmh
+ umask 022
+ cd /home/vagrant/rpmbuild/BUILD
+ rm -rf percona-server-5.7.16-10custom
+ exit 0

Once the build is done, you can find the RPMs in the RPMS directory:

$ ls -1 ~/rpmbuild/RPMS/x86_64/
Percona-Server-57-debuginfo-5.7.16-10custom.1.el7.x86_64.rpm
Percona-Server-client-57-5.7.16-10custom.1.el7.x86_64.rpm
Percona-Server-devel-57-5.7.16-10custom.1.el7.x86_64.rpm
Percona-Server-server-57-5.7.16-10custom.1.el7.x86_64.rpm
Percona-Server-shared-57-5.7.16-10custom.1.el7.x86_64.rpm
Percona-Server-shared-compat-57-5.7.16-10custom.1.el7.x86_64.rpm
Percona-Server-test-57-5.7.16-10custom.1.el7.x86_64.rpm
Percona-Server-tokudb-57-5.7.16-10custom.1.el7.x86_64.rpm

Written by Evgeniy Patlan in: custom build,GitHub,Insight for Developers,MySQL,RPM,tarball,Zend Developer |

Personal blog of Yzmir Ramirez

The Glass is twice as large as it needs to be

How to Manually Build Percona Server for MySQL RPM Packages

Prepare the Source

Using GIT Repository

Downloading Source Tarball

Making Changes with Patch Files

Why Patches?

Create Patch Files

Add Patch to RPM Spec File

Preparing Build Environment

Environment Requirements

Install Dependencies

Prepare RPM Build Tree

Download Boost Source

Move Files to the RPM Build Tree

Setting Correct Versions in the Spec File

Building RPM Packages

Archives

Contributors

Personal blog of Yzmir Ramirez

The Glass is twice as large as it needs to be

How to Manually Build Percona Server for MySQL RPM Packages

Prepare the Source

Using GIT Repository

Downloading Source Tarball

Making Changes with Patch Files

Why Patches?

Create Patch Files

Add Patch to RPM Spec File

Preparing Build Environment

Environment Requirements

Install Dependencies

Prepare RPM Build Tree

Download Boost Source

Move Files to the RPM Build Tree

Setting Correct Versions in the Spec File

Building RPM Packages

Tag Cloud

Archives

Contributors