Aug
01
2017
--

Percona Server for MySQL 5.6.36-82.1 is Now Available

Percona Server for MySQL 5.6

Percona Server for MySQL 5.6Percona is glad to announce the release of Percona Server for MySQL 5.6.36-82.1 on August 1, 2017 (Downloads are available here and from the Percona Software Repositories).

Based on MySQL 5.6.36, including all the bug fixes in it, Percona Server for MySQL 5.6.34-79.1 is the current GA release in the Percona Server for MySQL 5.6 series. All of Percona‘s software is open-source and free, all the details of the release can be found in the 5.6.36-82.1 milestone at Launchpad.

Please note that RHEL 5, CentOS 5 and Ubuntu versions 12.04 and older are not supported in future releases of Percona Server for MySQL and no further packages are added for these distributions.

New Features

  • Percona Server for MySQL can now be built with support of OpenSSL 1.1.
  • Percona Server for MySQL is now available on Debian 9 (stretch). The support only covers the amd64 architecture.
  • TokuDB enables to kill a query that is awaiting an FT locktree lock.

Bugs Fixed

  • Row counts in TokuDB could be lost intermittently after restarts. Bug fixed #2.
  • In TokuDB, two races in the fractal tree lock manager could significantly affect transactional throughput for some applications that used a small number of concurrent transactions. These races manifested as transactions unnecessarily waiting for an available lock. Bug fixed #3.
  • TokuDB could assert when opening a dictionary with no useful information to error log. Bug fixed #23.
  • TokuDB could assert for various reasons deserializing nodes with no useful error output. Bug fixed #24.
  • Percona Server could crash when running a query over a partitioned table that uses an index to read a range of rows if this range was not covered by any existing partition. Bug fixed #1657941 (upstream #76418).
  • With two client connections to a server (debug server build), the server could crash after one of the clients set the global option userstat and flushed the client statistics (FLUSH CLIENT_STATISTICS) and then both clients were closed. Bug fixed #1661488.
  • TokuDB did not pass cmake flags on to snappy cmake. Bug fixed #41. The progress status for partitioned TokuDB table ALTERs was misleading. Bug fixed #42.
  • When a client application is connecting to the Aurora cluster end point using SSL (--ssl-verify-server-cert or --ssl-mode=VERIFY_IDENTITY option), wildcard and SAN enabled SSL certificates were ignored. See also Compatibility Matrix. Note that the --ssl-verify-server-cert option is deprecated in Percona Server 5.7. Bug fixed #1673656 (upstream #68052).
  • Killing a stored procedure execution could result in an assert failure on a debug server build. Bug fixed #1689736 (upstream #86260).
  • It was not possible to build Percona Server on Debian 9 (stretch) due to issues with OpenSSL 1.1. Bug fixed #1702903 (upstream #83814).
  • The SET STATEMENT .. FOR statement changed the global instead of the session value of a variable if the statement occurred immediately after the SET GLOBAL or SHOW GLOBAL STATUS command. Bug fixed #1385352.
  • The synchronization between the LRU manager and page cleaner threads was not done at shutdown. Bug fixed #1689552.

Other bugs fixed: #6#44#65#1160986#1676740#1689989#1689998#1690012#1699788, and #1684601 (upstream #86016).

Compatibility Matrix

Feature YaSSL OpenSSL < 1.0.2 OpenSSL >= 1.0.2
‘commonName’ validation Yes Yes Yes
SAN validation No Yes Yes
Wildcards support No No Yes
Jul
28
2017
--

Percona Server for MySQL 5.7.18-16 Is Now Available

Percona Server for MySQL 5.7Percona is glad to announce the GA release of Percona Server for MySQL 5.7.18-16 on July 28, 2017 (Downloads are available here and from the Percona Software Repositories).

Based on MySQL 5.7.18, including all the bug fixes in it, Percona Server for MySQL 5.7.18-16 is the current GA release in the Percona Server for MySQL 5.7 series. All of Percona‘s software is open-source and free, and you can find all the release details in the 5.7.18-16 milestone at Launchpad

Please note that RHEL 5, CentOS 5 and Ubuntu versions 12.04 and older are not supported in future releases of Percona Server and no further packages are added for these distributions.

New Features:

  • Percona Server for MySQL is now available on Debian 9 (stretch). The support only covers the amd64 architecture.
  • Percona Server for MySQL can now be built with the support of OpenSSL 1.1.
  • MyRocks storage engine has been merged into Percona Server.
  • TokuDB enables to kill a query that is awaiting an FT locktree lock.
  • TokuDB enables using the MySQL DEBUG_SYNC facility within Percona FT.

Bugs Fixed:

  • Row counts in TokuDB could be lost intermittently after restarts. Bug fixed #2.
  • In TokuDB, two races in the fractal tree lock manager could significantly affect transactional throughput for some applications that used a small number of concurrent transactions. These races manifested as transactions unnecessarily waiting for an available lock. Bug fixed #3.
  • Percona FT could assert when opening a dictionary with no useful information to an error log. Bug fixed #23.
  • Percona FT could assert for various reasons deserializing nodes with no useful error output. Bug fixed #24.
  • It was not possible to build Percona Server on Debian 9 (stretch) due to issues with OpenSSL 1.1. Bug fixed #1702903 (upstream #83814).
  • Packaging was using the dpkg --verify command which is not available on wheezy/precise. Bug fixed #1694907.
  • Enabling and disabling the slow query log rotation spuriously added the version suffix to the next slow query log file name. Bug fixed #1704056.
  • With two client connections to a server (debug server build), the server could crash after one of the clients set the global option userstat and flushed the client statistics (FLUSH CLIENT_STATISTICS) and then both clients were closed. Bug fixed #1661488.
  • Percona FT did not pass cmake flags on to snappy cmake. Bug fixed #41. The progress status for partitioned TokuDB table ALTERs was misleading. Bug fixed #42.
  • When a client application is connecting to the Aurora cluster end point using SSL (--ssl-verify-server-cert or --ssl-mode=VERIFY_IDENTITY option), wildcard and SAN enabled SSL certificates were ignored. Note that the --ssl-verify-server-cert option is deprecated in Percona Server 5.7. Bug fixed #1673656 (upstream #68052).
  • Killing a stored procedure execution could result in an assert failure on a debug server build. Bug fixed #1689736 (upstream #86260).
  • The SET STATEMENT .. FOR statement changed the global instead of the session value of a variable if the statement occurred immediately after the SET GLOBAL or SHOW GLOBAL STATUS command. Bug fixed #1385352.
  • When running SHOW ENGINE INNODB STATUS, the Buffer pool size, bytes entry contained 0. BUg fixed #1586262.
  • The synchronization between the LRU manager and page cleaner threads was not done at shutdown. Bug fixed #1689552.
  • Spurious lock_wait_timeout_thread wakeup in lock_wait_suspend_thread() could occur. Bug fixed #1704267 (upstream #72123).

Other bugs fixed: #1686603#6#44#65#1160986#1686934#1688319#1689989#1690012#1691682#1697700#1699788#1121072, and #1684601 (upstream #86016).

The release notes for Percona Server for MySQL 5.7.18-16 are available in the online documentation. Please report any bugs on the launchpad bug tracker.

Note

Due to new package dependency, Ubuntu/Debian users should use apt-get dist-upgrade or apt-get install percona-server-server-5.7 to upgrade.

May
30
2017
--

Percona XtraBackup 2.4.7-2 is Now Available

Percona XtraBackup 2.4.7-2

Percona XtraBackup 2.4.7-2Percona announces the GA release of Percona XtraBackup 2.4.7-2 on May 29, 2017. You can download it from our download site and apt and yum repositories.

Percona XtraBackup enables MySQL backups without blocking user queries, making it ideal for companies with large data sets and mission-critical applications that cannot tolerate long periods of downtime. Offered free as an open source solution, Percona XtraBackup drives down backup costs while providing unique features for MySQL backups.

Bug Fixed:
  • Fixed build failure on Debian 9.0 (Stretch). Bug fixed #1678947.

Release notes with all the bugfixes for Percona XtraBackup 2.4.7-2 are available in our online documentation. Please report any bugs to the launchpad bug tracker.

Oct
13
2016
--

Update the Signing Key for Percona Debian and Ubuntu Packages

signing key for Percona Debian and Ubuntu

signing key for Percona Debian and UbuntuIn this blog post, we’ll explain how to update the signing key for Percona Debian and Ubuntu packages.

Some of the users might have already noticed following warning on Ubuntu 16.04 (Xenial Xerus):

W: http://repo.percona.com/apt/dists/xenial/InRelease: Signature by key 430BDF5C56E7C94E848EE60C1C4CBDCDCD2EFD2A uses weak digest algorithm (SHA1)

when running apt-get update.

Percona .deb packages are signed with a key that uses an algorithm now considered weak. Starting with the next release, Debian and Ubuntu packages are signed with a new key that uses the much stronger SHA-512 algorithm. All future package release will also contain the new algorithm.

You’ll need to do one of the following in order to use the new key:

  • If you installed the Percona repository package as described here, this package is automatically updated to a new package version (percona-release_0.1-4). This package currently contains both the old and new keys. This helps make the transition easier (until all packages are signed with the new key).
  • Install the new Percona repository package as described in the installation guide.
  • Manually download and add the key from either keys.gnupg.net or keyserver.ubuntu.com by running:
    apt-key adv --keyserver keys.gnupg.net --recv-keys 8507EFA5 or
    apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 8507EFA5

It’s important that you add the new key before the next release. Otherwise you’ll see the following warning:

W: GPG error: http://repo.percona.com xenial InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 9334A25F8507EFA5

Leave any questions about updating the signing key for Percona Debian and Ubuntu packages in the comments below.

Mar
30
2016
--

Docker MySQL Replication 101

Docker

Precona Server DockerIn this blog post, we’ll discuss some of the basics regarding Docker MySQL replication. Docker has gained widespread popularity in recent years as a lightweight alternative to virtualization. It is ideal for building virtual development and testing environments. The solution is flexible and seamlessly integrates with popular CI tools.

 

This post walks through the setup of MySQL replication with Docker using Percona Server 5.6 images. To keep things simple we’ll configure a pair of instances and override only the most important variables for replication. You can add whatever other variables you want to override in the configuration files for each instance.

Note: the configuration described here is suitable for development or testing. We’ve also used the operating system repository packages; for the latest version use the official Docker images. The steps described can be used to setup more slaves if required, as long as each slave has a different server-id.

First, install Docker and pull the Percona images (this will take some time and is only executed once):

# Docker install for Debian / Ubuntu
apt-get install docker.io
# Docker install for Red Hat / CentOS (requires EPEL repo)
yum install epel-release # If not installed already
yum install docker-io
# Pull docker repos
docker pull percona

Now create locally persisted directories for the:

  1. Instance configuration
  2. Data files
# Create local data directories
mkdir -p /opt/Docker/masterdb/data /opt/Docker/slavedb/data
# Create local my.cnf directories
mkdir -p /opt/Docker/masterdb/cnf /opt/Docker/slavedb/cnf
### Create configuration files for master and slave
vi /opt/Docker/masterdb/cnf/config-file.cnf
# Config Settings:
[mysqld]
server-id=1
binlog_format=ROW
log-bin
vi /opt/Docker/slavedb/cnf/config-file.cnf
# Config Settings:
[mysqld]
server-id=2

Great, now we’re ready start our instances and configure replication. Launch the master node, configure the replication user and get the initial replication co-ordinates:

# Launch master instance
docker run --name masterdb -v /opt/Docker/masterdb/cnf:/etc/mysql/conf.d -v /opt/Docker/masterdb/data:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=mysecretpass -d percona:5.6
00a0231fb689d27afad2753e4350192bebc19ab4ff733c07da9c20ca4169759e
# Create replication user
docker exec -ti masterdb 'mysql' -uroot -pmysecretpass -vvv -e"GRANT REPLICATION SLAVE ON *.* TO repl@'%' IDENTIFIED BY 'slavepass'G"
mysql: [Warning] Using a password on the command line interface can be insecure.
--------------
GRANT REPLICATION SLAVE ON *.* TO repl@"%"
--------------
Query OK, 0 rows affected (0.02 sec)
Bye
### Get master status
docker exec -ti masterdb 'mysql' -uroot -pmysecretpass -e"SHOW MASTER STATUSG"
mysql: [Warning] Using a password on the command line interface can be insecure.
*************************** 1. row ***************************
             File: mysqld-bin.000004
         Position: 310
     Binlog_Do_DB:
 Binlog_Ignore_DB:
Executed_Gtid_Set:

If you look carefully at the “docker run” command for masterdb, you’ll notice we’ve defined two paths to share from local storage:

/opt/Docker/masterdb/data:/var/lib/mysql

  • This maps the local “/opt/Docker/masterdb/data” to the masterdb’s container’s “/var/lib/mysql path”
  • All files within the datadir “/var/lib/mysql” persist locally on the host running docker rather than in the container
/opt/Docker/masterdb/cnf:/etc/mysql/conf.d

  • This maps the local “/opt/Docker/masterdb/cnf” directory to the container’s “/etc/mysql/conf.d” path
  • The configuration files for the masterdb instance persist locally as well
  • Remember these files augment or override the file in “/etc/mysql/my.cnf” within the container (i.e., defaults will be used for all other variables)

We’re done setting up the master, so let’s continue with the slave instance. For this instance the “docker run” command also includes the “–link masterdb:mysql” command, which links the slave instance to the master instance for replication.

After starting the instance, set the replication co-ordinates captured in the previous step:

docker run --name slavedb -d -v /opt/Docker/slavedb/cnf:/etc/mysql/conf.d -v /opt/Docker/slavedb/data:/var/lib/mysql --link masterdb:mysql -e MYSQL_ROOT_PASSWORD=mysecretpass -d percona:5.6
eb7141121300c104ccee0b2df018e33d4f7f10bf5d98445ed4a54e1316f41891
docker exec -ti slavedb 'mysql' -uroot -pmysecretpass -e'change master to master_host="mysql",master_user="repl",master_password="slavepass",master_log_file="mysqld-bin.000004",master_log_pos=310;"' -vvv
mysql: [Warning] Using a password on the command line interface can be insecure.
--------------
change master to master_host="mysql",master_user="repl",master_password="slavepass",master_log_file="mysqld-bin.000004",master_log_pos=310
--------------
Query OK, 0 rows affected, 2 warnings (0.23 sec)
Bye

Almost ready to go! The last step is to start replication and verify that replication running:

# Start replication
docker exec -ti slavedb 'mysql' -uroot -pmysecretpass -e"START SLAVE;" -vvv
mysql: [Warning] Using a password on the command line interface can be insecure.
--------------
START SLAVE
--------------
Query OK, 0 rows affected, 1 warning (0.00 sec)
Bye
# Verify replication is running OK
docker exec -ti slavedb 'mysql' -uroot -pmysecretpass -e"SHOW SLAVE STATUSG" -vvv
mysql: [Warning] Using a password on the command line interface can be insecure.
--------------
SHOW SLAVE STATUS
--------------
*************************** 1. row ***************************
               Slave_IO_State: Waiting for master to send event
                  Master_Host: mysql
                  Master_User: repl
                  Master_Port: 3306
                Connect_Retry: 60
              Master_Log_File: mysqld-bin.000004
          Read_Master_Log_Pos: 310
               Relay_Log_File: mysqld-relay-bin.000002
                Relay_Log_Pos: 284
        Relay_Master_Log_File: mysqld-bin.000004
             Slave_IO_Running: Yes
            Slave_SQL_Running: Yes
              Replicate_Do_DB:
          Replicate_Ignore_DB:
           Replicate_Do_Table:
       Replicate_Ignore_Table:
      Replicate_Wild_Do_Table:
  Replicate_Wild_Ignore_Table:
                   Last_Errno: 0
                   Last_Error:
                 Skip_Counter: 0
          Exec_Master_Log_Pos: 310
              Relay_Log_Space: 458
              Until_Condition: None
               Until_Log_File:
                Until_Log_Pos: 0
           Master_SSL_Allowed: No
           Master_SSL_CA_File:
           Master_SSL_CA_Path:
              Master_SSL_Cert:
            Master_SSL_Cipher:
               Master_SSL_Key:
        Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
                Last_IO_Errno: 0
                Last_IO_Error:
               Last_SQL_Errno: 0
               Last_SQL_Error:
  Replicate_Ignore_Server_Ids:
             Master_Server_Id: 1
                  Master_UUID: 230d005a-f1a6-11e5-b546-0242ac110004
             Master_Info_File: /var/lib/mysql/master.info
                    SQL_Delay: 0
          SQL_Remaining_Delay: NULL
      Slave_SQL_Running_State: Slave has read all relay log; waiting for the slave I/O thread to update it
           Master_Retry_Count: 86400
                  Master_Bind:
      Last_IO_Error_Timestamp:
     Last_SQL_Error_Timestamp:
               Master_SSL_Crl:
           Master_SSL_Crlpath:
           Retrieved_Gtid_Set:
            Executed_Gtid_Set:
                Auto_Position: 0
1 row in set (0.00 sec)
Bye

Finally, we have a pair of dockerized Percona Server 5.6 master-slave servers replicating!

As mentioned before, this is suitable for a development or testing environment. Before going into production with this configuration, think carefully about the tuning of the “my.cnf” variables and the choice of disks used for the data/binlog directories. It is important to remember that newer versions of Docker recommend using “networks” rather than “linking” for communication between containers.

Jan
12
2015
--

Percona Server 5.6.22-71.0 is now available

Percona ServerPercona is glad to announce the release of Percona Server 5.6.22-71.0 on January 12, 2015. Download the latest version from the Percona web site or from the Percona Software Repositories.

Based on MySQL 5.6.22, including all the bug fixes in it, Percona Server 5.6.22-71.0 is the current GA release in the Percona Server 5.6 series. Percona Server is open-source and free – and this is the latest release of our enhanced, drop-in replacement for MySQL. Complete details of this release can be found in the 5.6.22-71.0 milestone on Launchpad.

New Features:

  • Percona Server has implemented improved slow log reporting for queries in stored procedures.
  • TokuDB storage engine package has been updated to version 7.5.4. Percona Server with an older version of TokuDB could hit an early scaling limit when the binary log was enabled. TokuDB 7.5.4 fixes this problem by using the hints supplied by the binary log group commit algorithm to avoid fsync’ing its recovery log during the commit phase of the 2 phase commit algorithm that MySQL uses for transactions when the binary log is enabled.

Bugs Fixed:

  • Debian and Ubuntu init scripts no longer have a hardcoded server startup timeout. This has been done to accommodate situations where server startup takes a very long time, for example, due to a crash recovery or buffer pool dump restore. Bugs fixed #1072538 and #1328262.
  • A read-write workload on compressed InnoDB tables might have caused an assertion error. Bug fixed #1268656.
  • Selecting from GLOBAL_TEMPORARY_TABLES table while running an online ALTER TABLE in parallel could lead to server crash. Bug fixed #1294190.
  • A wrong stack size calculation could lead to a server crash when Performance Schema tables were storing big amount of data or in case of server being under highly concurrent load. Bug fixed #1351148 (upstream #73979).
  • A query on an empty table with a BLOB column may crash the server. Bug fixed #1384568 (upstream #74644).
  • A read-write workload on compressed InnoDB tables might have caused an assertion error. Bug fixed #1395543.
  • If HandlerSocket was enabled, the server would hang during shutdown. Bug fixed #1397859.
  • The default MySQL configuration file, my.cnf, was not installed during a new installation on CentOS. Bug fixed #1405667.
  • The query optimizer did not pick a covering index for some ORDER BY queries. Bug fixed #1394967 (upstream #57430).
  • SHOW ENGINE INNODB STATUS was displaying two identical TRANSACTIONS sections. Bug fixed #1404565.
  • A race condition in Multiple user level locks per connection implementation could cause a deadlock. Bug fixed #1405076.

Other bugs fixed: #1394357, #1337251, #1399174, #1396330 (upstream #74987), and #1401776 (upstream #75189).

Known Issues:
If you’re upgrading TokuDB package on CentOS 5/6 you’ll need to restart the MySQL service after the upgrade, otherwise TokuDB storage engine won’t be initialized.

Release notes for Percona Server 5.6.22-71.0 are available in the online documentation. Please report any bugs on the launchpad bug tracker

The post Percona Server 5.6.22-71.0 is now available appeared first on MySQL Performance Blog.

Aug
01
2013
--

Percona celebrates its 7th anniversary by giving to open source ecosystem

Percona celebrates its 7th anniversaryToday we’re celebrating Percona’s 7th anniversary.  A lot has changed in these past 7 years – we have grown from a two-person outfit focused exclusively on consulting to a 100-person company with teammates in 22 different countries and 18 different states, now providing Support, Consulting, RemoteDBA, Server Development and Training services.

We also made our mark in open source software development, creating some of the most popular products for the MySQL ecosystem – Percona Toolkit, Percona Xtrabackup, Percona XtraDB Cluster, Percona Server and others. Additionally, we’re into our second year of hosting the Percona Live conference series for the MySQL community. We have grown to serve over 2,000 customers and I’m proud to say we could do it all in bootstrap mode without attracting outside investors and keeping the company owned by its employees.

So how are we celebrating our anniversary? We decided to celebrate by supporting the open source ecosystem, making donations to a number of open source initiatives that have helped us through all these years. We would not be here without you!

As such we’re supporting:

  • MariaDB Foundation for supporting MariaDB, one of the MySQL alternatives that we fully support at Percona.
  • Free Software Foundation as an organization instrumental to the success of the open source movement.
  • Linux Foundation for supporting Linux, by far the most popular platform among our customers.
  • Debian for creating a foundation for some of the most popular Linux distributions out there.
  • Jenkins for the Continuous Integration server we use for our development projects.
  • OpenSSH for software that helps us to access customer systems securely.
  • Drupal for powering our website as well as the websites of many of our customers.

We’re happy to enjoy the growth that’s allowing us to support other projects in our ecosystem. If you have the chance I encourage you do the same. There is a tremendous amount of work going into open source software, which is made free to use, but it is by far not free to create and maintain.

The post Percona celebrates its 7th anniversary by giving to open source ecosystem appeared first on MySQL Performance Blog.

Sep
30
2010
--

Drizzle, Maverick, PPA’s, and you

So, this week, Drizzle released its beta, which is really exciting. But at the same time, I decided to ask the Ubuntu MOTU pull it out of Ubuntu 10.10 (a.k.a. maverick) entirely. The reasons, may not be entirely obvious.

  • Licensing: There is some ambiguity on the licensing of certain non-critical source code in Drizzle that we weren’t certain Debian archive admins would accept. Since we like to follow Debian as closely as possible in Ubuntu, the MOTU sponsor we had was requesting that we upload into Debian first. Upon review of earlier packages, the debian archive maintainers pointed out some ambiguities in the copyright documentation, and it turns out, there are some ambiguities still in the source code. These things take time to sort out, though I’m confident we’ve figured most, if not all, of them out.
  • Beta status: Drizzle released their first beta just yesterday. This is great, and would be a good release to have in Maverick, but its going to change *a lot* before the “elliot” milestone is released in early 2011. Monty Taylor assures me that they’re going to be ready to release before feature freeze of Natty (11.04). Until then, they’re going to be fixing bugs in the betas and releasing those fixes. In the face of that, its probably better to point people at a PPA that will have the latest bug fix release in it, and tools included to help debug/fix the release as well.
  • Quality: Even though its clear that drizzle is beta to those following drizzle closely, it may not be entirely clear to everyone. Beta versions make it into Ubuntu all the time, but being a database engine, I’m hesitant to have the casual user try it out. In 6 months, Drizzle will be at a stable release stage, and all users should be feeling pretty good about running on it. That seems like the right time to put it into Debian and Ubuntu.

So, what should you do if you want to run Drizzle on Maverick?

There are two package archives maintained by the drizzle developers just for ubuntu.

The PPA for drizzle development – This should have the latest stable release, and all of the build-depends to rebuild it.
The Drizzle Trunk PPA – This should have the latest daily build of drizzle from the source code repository, which may have fixes made since the last stable release.

Those links include instructions for adding the PPA’s to your system, after that, just


apt-get install drizzle-server drizzle-client

And have fun!

Also, we’ll be discussing drizzle sometime at UDS-N in Orlando. So make sure to check the schedule out and join us (remotely or on site) if you want to chime in or hear what we’re going to do with the Narwhal and Drizzle.


Sep
07
2010
--

I learned ruby last week

Time to give myself a little pat on the back.

Last week I sat down to work for a whole working day on “whatever I wanted to”, as part of the Canonical Server Team’s pilot “Fedex Day” program. Mathias Gug and I both looked at this idea from Dan Pink’s book “Drive” and thought it made sense to try it out.

Management approved, and we set about on a day of “work on one thing, make it go, and then show it off the following week”.

I was originally going to work on improving the search capabilities of the MoinMoin wiki software that we use at Canonical. But it turns out, somebody already did that by adding Xapian support, and so we really just need to backport that to whatever version of Ubuntu canonical’s servers run on.

So, I decided to tackle another issue that has been nagging at me.

I love perl. I’ve used it for years, and I have always found that the ease with which one can get software from perl’s central repository, CPAN, was a huge differentiator from other languages.

So much so, that PHP, Python, and Ruby have in many ways gone even further than CPAN with their respective tools, PEAR, pypi, and rubygems.

Well one thing Debian developers love is perl too. So for a long time now, its been relatively simple, even braindead simple, to create a debian source package from a CPAN module.

Simply download the tarball, unpack, and ‘dh-make-perl’, that gets you 90% of the way there. All that is left is renaming a few dependencies, making sure the copyrights are done, and verifying that the module doesn’t do anything nasty.

How cool is that?

Python has a few tools like this too, like sdist, so they’re covered.

But Ruby had no such thing for rubygems. There are definitely tools to create debs from ruby source distributions, but often times projects don’t actually distribute source any other way than a .gem file or code repository. To make matters worse, there is a bit of confusion between some users of rubygems on Debian and Ubuntu about how it should work. Many people feel that gems is a poor way to distribute software, or that it lacks certain features. Others would just like to use it like any other developer tool.

Well I happen to think that many ruby users would probably grow to love .debs if they covered a large portion of rubygems software.

So, DebiGem is born! I created this project in about 8 working hours, with zero Ruby knowledge beforehand. I’m sure that shows in my style of Ruby. With some help from perusing the rubygems code base and about 80 Ruby tutorial pages on the net, I think I’ve created something that actually works.

If you want to try it, go ahead and fire it up, simply install the debs available in my PPA on Launchpad, and then download a .gem, and turn it into a source package (apologies for all the debug output, we’re talking about v0.0.2 here!).

clint@ubuntu:~/testgem$ wget http://rubygems.org/downloads/a2ws-0.1.9.gem
--2010-09-07 15:58:46-- http://rubygems.org/downloads/a2ws-0.1.9.gem
Resolving rubygems.org... 72.4.120.124
Connecting to rubygems.org|72.4.120.124|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: /gems/a2ws-0.1.9.gem [following]
--2010-09-07 15:58:46-- http://rubygems.org/gems/a2ws-0.1.9.gem
Reusing existing connection to rubygems.org:80.
HTTP request sent, awaiting response... 302 Found
Location: http://production.cf.rubygems.org/gems/a2ws-0.1.9.gem [following]
--2010-09-07 15:58:46-- http://production.cf.rubygems.org/gems/a2ws-0.1.9.gem
Resolving production.cf.rubygems.org... 216.137.37.222, 216.137.37.239, 216.137.37.11, ...
Connecting to production.cf.rubygems.org|216.137.37.222|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7168 (7.0K) []
Saving to: `a2ws-0.1.9.gem'

100%[================================================================================================================================================================>] 7,168 --.-K/s in 0.07s

2010-09-07 15:58:46 (94.9 KB/s) - `a2ws-0.1.9.gem' saved [7168/7168]

clint@ubuntu:~/testgem$ dh-make-gem a2ws-0.1.9.gem
Extracting gem file a2ws-0.1.9.gem
tar -xvf a2ws-0.1.9.gem data.tar.gz
data.tar.gz
tar: data.tar.gz: implausibly old time stamp 1969-12-31 16:00:00
tar -C liba2ws-ruby-0.1.9 -zxf data.tar.gz
tar: .document: implausibly old time stamp 1969-12-31 16:00:00
tar: .gitignore: implausibly old time stamp 1969-12-31 16:00:00
tar: LICENSE: implausibly old time stamp 1969-12-31 16:00:00
tar: README.rdoc: implausibly old time stamp 1969-12-31 16:00:00
tar: Rakefile: implausibly old time stamp 1969-12-31 16:00:00
tar: VERSION.yml: implausibly old time stamp 1969-12-31 16:00:00
tar: a2ws.gemspec: implausibly old time stamp 1969-12-31 16:00:00
tar: lib/a2ws.rb: implausibly old time stamp 1969-12-31 16:00:00
tar: lib/a2ws/base.rb: implausibly old time stamp 1969-12-31 16:00:00
tar: lib/a2ws/image.rb: implausibly old time stamp 1969-12-31 16:00:00
tar: lib/a2ws/image_search.rb: implausibly old time stamp 1969-12-31 16:00:00
tar: lib/a2ws/item.rb: implausibly old time stamp 1969-12-31 16:00:00
tar: lib/a2ws/item_search.rb: implausibly old time stamp 1969-12-31 16:00:00
tar: lib/a2ws/methodize.rb: implausibly old time stamp 1969-12-31 16:00:00
tar: spec/a2ws_spec.rb: implausibly old time stamp 1969-12-31 16:00:00
tar: spec/spec_helper.rb: implausibly old time stamp 1969-12-31 16:00:00
tar -zcf liba2ws-ruby_0.1.9.orig.tar.gz --exclude=liba2ws-ruby-0.1.9/debian liba2ws-ruby-0.1.9
changing working dir to liba2ws-ruby-0.1.9
DEBUG: a2ws-0.1.9.gem.gemspec
WARNING: no description specified
WARNING: no rubyforge_project specified
DEBUG: req = libhttparty-ruby (>= 0.4.3)
DEBUG: type=runtime
DEBUG: req = libactivesupport-ruby (>= 2.2.2)
DEBUG: type=runtime
Successfully built Debian source package from gemspec
Name: a2ws
Version: 0.1.9
PackageName: liba2ws-ruby
clint@ubuntu:~/testgem$ cat liba2ws-ruby-0.1.9/debian/control
Source: liba2ws-ruby
Section: ruby
Priority: optional
Maintainer: foo
Build-Depends: ruby, debhelper (>= 7), dh-rubygems,
Standards-Version: 3.9.1

Package: liba2ws-ruby
Section: ruby
Architecture: all
Depends: ruby, ${shlibs:Depends}, libhttparty-ruby (>= 0.4.3), libactivesupport-ruby (>= 2.2.2)
Description: Wrapper for Amazon Associates Web Service (A2WS).

clint@ubuntu:~/testgem$ cat liba2ws-ruby-0.1.9/debian/rules
#!/usr/bin/make -f
# This file was generated by 'gem debsrc'
#
%:
dh $@

override_dh_auto_build::
dh_rubygems build

override_dh_install::
dh_rubygems install

override_dh_clean::
dh_rubygems clean
dh_clean
clint@ubuntu:~/testgem$ ls -l liba2ws-ruby-0.1.9/debian
total 20
-rw-r--r-- 1 clint clint 151 2010-09-07 15:58 changelog
-rw-r--r-- 1 clint clint 2 2010-09-07 15:58 compat
-rw-r--r-- 1 clint clint 410 2010-09-07 15:58 control
-rw-r--r-- 1 clint clint 2078 2010-09-07 15:58 liba2ws-ruby.gemspec
-rw-r--r-- 1 clint clint 213 2010-09-07 15:58 rules

It also includes the dh_rubygems tool and dh-rubygems package you see mentioned above.

Given this tool, its relatively easy to create .deb files and focus on the real work of creating packages, which is reviewing licensing and code, rather than turning them into the actual .deb. Sure you still have to update the maintainer field, and will probably want to check those dependencies, but for the most part, this works and gets the gem into an installable state.

I do hope someone finds this useful, it was great fun implementing it.


Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com