Personal blog of Yzmir Ramirez

I’ll be speaking at the upcoming Open Source Developers Conference (OSDC 2013) in Auckland, New Zealand! It’s on October 21st-23rd and you should go here right now and register. I’m giving two talks at OSDC this year:

• MySQL in the cloud, As A Service (Monday 21st, 12:00pm)
  There is no one magic solution to having MySQL As A Service work well, it’s a lot of small moving parts and options that need to be set, monitored and configured. We may wish it was different, or look at other database technologies, but there is a lot of legacy code that talks to MySQL, with all it’s idiosyncrasies – and we need to be able to support this code. In this talk, we’ll cover many of the problem areas and what you can do to avoid them.
• The Agony and Ecstasy of Continuous Integration (Wednesday 23rd, 2:30pm)
  This a tale of the introduction of continuous integration testing into a well established development team. It covers both the highs and lows and discusses strategies to deal with both the positives and negatives and in turn improve your own software engineering practices.

In case you need to quickly justify to your boss why you should go to OSDC, the conference organizers have helpfully provided a page of hints on just that subject.

The post Stewart is speaking at OSDC 2013 in Auckland! appeared first on MySQL Performance Blog.

Good news everyone! I recently presented a webinar: Percona Server 5.6: Enterprise Grade MySQL. It was also recorded so you can watch along or view the slide deck. As with all my talks, I am not simply reading the slides so it really is worth to listen to the audio rather than just glance through the slide deck.

There were a number of great questions asked which I’ll answer below:

Q: How does Stewart feel about this version of 5.6 taking into consideration “Stewart’s .20 rule?” (ref 2013 Percona Live Conference).

A: For those who aren’t familiar with it, I have a rule which I call “Stewart’s dot twenty rule” which I’ve posted a few times about on my personal blog. It states: “a piece of software is never really mature until a dot twenty release.” I would say that MySQL 5.6 (and Percona Server 5.6) are both in really good states currently.

I strongly recommend the excellent series of “Fun With Bugs” posts by Valeriy Kravchuk. The latest Fun With Bugs post is: Fun with Bugs #20 – welcome MySQL 5.6.13! and certainly worth a read. I’m rather safe in saying that the first GA release of MySQL 5.6 was by far the best first GA release of any MySQL version ever and subsequent MySQL 5.6 releases have improved upon that. It is quite likely that 5.6 will work perfectly for you today.

If you are really conservative with software upgrades and want as few surprises as possible, then you can of course wait – but I’d certainly recommend kicking the tyres of 5.6 over the next few months and starting to plan a migration.

Q: Any estimate on availability of XtraDB Cluster using 5.6?

A: Since Percona XtraDB Cluster is built upon both Percona Server and Galera it’s only natural to build upon a GA release of Percona Server and a GA release of Galera.

Q: What’s the birds name?

A: (Background: at one point during the Webinar you could hear one of our pet birds start to burst into song). I’m glad you asked as it gives me an excellent opportunity to include gratuitous photos of our birds! They’re both Cockatiels. People will often think cockatoo (specifically the Sulphur-Crested Cockatoo)and not cockatiel. A cockatoo is any of the 21 species belonging to the bird family Cacatuidae and the cockatiel is the smallest of the 21 species.

We have both a boy (Spike) and a girl (Beaker). Spike is the one who sings (while Beaker, like the muppet, goes meep) and could be heard for a moment during the webinar. Beaker has also been spotted helping with Percona Server 5.6 releases.

Q: The ‘first in Percona Server’ optimizations, did Oracle implement Percona code or write their own?

A: It would be accurate to say that there are changes in MySQL 5.6 that have been inspired by our work, and previously there has been Percona code that has made its way into MySQL (see COPYING.Percona in the MySQL bzr repository). For a multitude of reasons that aren’t worth going into here, it has historically been problematic getting code into MySQL if you didn’t work for the company that owned MySQL. This has been true of MySQL AB, Sun and Oracle and is certainly nothing new or unique to Oracle. What is different now is that things seem to be changing for the better and there is likely to be more cooperation with Oracle going forward.

Q: Has HandlerSocket been cooked into your 5.6 releases yet? Have there been any other improvements on that front?

A: We don’t currently have HandlerSocket in Percona Server 5.6. There has been a very small amount of adoption of HandlerSocket and we’ve taken the approach that we’ll see if the HandlerSocket team ports to 5.6 and if there is adequate demand for HandlerSocket in 5.6. So far, you’re the first person to request it.

Q: What Oracle 5.6 features have not yet been copied or reimplemented in Percona 5.6?

A: Everything in Oracle MySQL 5.6 is in Percona Server 5.6 and has been from the very first Percona Server 5.6 release.

Q: Was innodb fake changes picked up by Oracle?

A: No, at least not yet

Q: Has Percona developed or found some solutions for migrating a production Percona server 5.5 to a production Percona server 5.6 without any downtime. Previously I solved this by making a newer version of Percona server as a replica of an older version of either Percona server or mysql db. Then I would point the application servers to the new replica to complete the deployment with a trivial downtime. It seems like this approach is not valid given the new replication design.

A: You can do the old replication trick

Q: Can Xtrabackup 5.6 be used on a system running Percona Server 5.5?

A: Percona XtraBackup 2.1 (the current stable release, which works with MySQL 5.6 and Percona Server 5.6) will also work with MySQL 5.5, Percona Server 5.5, Percona Server 5.1 and MySQL 5.1 running the innodb plugin. There is also support for various MariaDB versions.

Q: question on replication: my database has no partitioned table, multi-thread replication (feature of 5.6) is not going to help. Am I right?

A: Currently the multi-threaded replication slave will partition work up across database schema. It doesn’t matter if your tables are partitioned or not, it matters what database (schema) they’re in. If all your tables are in the same schema, then parallel slave will not currently help.

Q: Is Percona Server 5.6 a drop in replacement for 5.5 or is there an upgrade process? If so, what is involved to roll back to 5.5 if necessary?

A: The upgrade process should be fairly painless and could well be a simple drop-in replacement. It does, of course, depend on what features you may be using along with the type and size of workload. We have a In-Place upgrading from Percona Server 5.5 to Percona Server 5.6 section in our Percona Server 5.6 manual and along with the Changed in Percona Server 5.6 section this should provide a fair amount of insight into what you may expect from the Percona Server side of things. There is also the Upgrading from MySQL 5.5 to 5.6 section of the MySQL manual which is well worth a read.

There is a section in the MySQL manual on downgrading from 5.6 to 5.5 and I don’t think there should be any extra limitations imposed by Percona Server on going from 5.6 back to 5.5. That being said, downgrading is certainly not as well tested as upgrading and I would consider it more of a last resort than something to jump to quickly.

Q: When does production Percona server 5.6 release?

A: Soon. The current Percona Server 5.6 releases are fairly solid and I can certainly recommend trialling them.

Q: Are there any known mysqllib binding issues or deprecations for 5.6?

A: None that I’m aware of.

Q: Is there a white paper or other docs on migratiing from 5.1 Percona server to 5.6 Percona server?

A: Not currently. Generally, the recommended practice is to go through each major version (going through 5.5 before heading to 5.6). There is upgrade documentation for upgrading 5.1 to 5.5 and for 5.5 to 5.6 – and you can certainly run 5.5 for only a few minutes before upgrading to 5.6.

Q: Will you offer training on 5.6?

A: Yes! There is a Moving to MySQL 5.6 training course offered by Percona which covers both MySQL 5.6 and Percona Server 5.6.

Q: I didn’t notice any mention of the improved NUMA support in PS 5.5 (http://www.percona.com/doc/percona-server/5.5/performance/innodb_numa_support.html). Is this carried over to Oracle and/or Percona 5.6?

A: Yes it has made it into Percona Server 5.6. See http://www.percona.com/doc/percona-server/5.6/performance/innodb_numa_support.html for the 5.6 documentation on it. I am not aware of Oracle having implemented it though.

Q: Have you made tests of user_stats overhead compared to performance_schema in 5.6?

A: I’m not aware of any published benchmarks for 5.6 although it would be great to see some.

Q: Does this release support the live table changes?

A: For some types of changes, yes.

Q: Is “Warning: Using a password on the command line interface can be insecure.” error being filtered out in the Percona release?

A: No. It’s not a good idea to provide passwords on the command line.

Q: He also promised a migration blog post

A: As promised, I am right now going to pester people about writing various posts on migrating from 5.5 to 5.6.

The post Percona Server 5.6 Webinar follow-up and Q&A appeared first on MySQL Performance Blog.

WHEN: Wednesday, July 31, 2013  1 p.m. Pacific Standard Time. Register here.

The post Let’s talk about Percona Server 5.6: Enterprise Grade MySQL (webinar) appeared first on MySQL Performance Blog.

It was once said that “real artists ship.” In looking over the history of Percona software releases, we are currently shipping more software than ever before.

First, let’s look at Percona Server. Let’s look at all major versions: 5.1, 5.5 and 5.6 as well as the total for each year. The estimate for 2013 comes from assuming the second half of 2013 is similar to the first half.

In 2011, when Percona Server 5.5 came along, we see a sharp reduction in Percona Server 5.1 releases (remember that there’s at least one Percona Server 5.1 release for each Oracle MySQL 5.1 release, and this reduction is likely a reflection of the reduction of releases from Oracle). In 2011, 2012 and 2013 we see a pretty steady number of Percona Server 5.1 releases. It seems that 5.1 is not going anywhere yet.

For Percona Server 5.5, we see an increase in 2012, which likely mirrors an increased interest in Percona Server 5.5 in production environments with added feature and bug fix releases.

For Percona XtraBackup, we have a similar story of an increase in the number of releases we’ve been making.

You can clearly see the switch between stable versions of 1.6.x to 2.0.x and now with 2.1.x. There has been a new major version of Percona XtraBackup each year since 2010 and for 1.6 and above, there have been maintenance releases into the following year.

It’s also clear that we’ve dramatically increased the number of Percona XtraBackup releases each year. This allows earlier access to new features (in alpha and beta releases) and earlier access to bug fixes (point releases such as 2.1.4).

If we look across the entire Percona software range, there is a quite noticeable increase in the number of releases year-on-year. This graph is again only up to July 23rd, so the 2013-estimate is based on average releases per month.

It’s quite amazing to see the increase from 2011, where we shipped 24 releases, to 2012 where we shipped 56 (more than one a week) to this year where we’ve already shipped 38 and we could ship 64.

This growth trend isn’t possible without both growing the development team and continually improving development practices, process and organization. Over the past two years we’ve made many changes to how we develop and release software. We’ve automated nearly everything (and this will be a topic for a future post) and distributed the workload of things that cannot be automated without major advancements in AI.

Personally, I’m really quite amazed when faced with the hard numbers. I’m also incredibly proud of everyone involved in making every release happen.

The post Growth of Percona software releases appeared first on MySQL Performance Blog.

Run Ubuntu and want to try new features of Percona XtraBackup? Well, now you can. We have a Personal Package Archive (PPA) percona-daily/percona-xtrabackup. Every time code is pushed to our main source code repository, the PPA is updated.

We are currently building for Ubuntu lucid (10.04LTS), Precise (12.04), Quantal (12.10) and Raring (13.04).

To add the repository to your Ubuntu system, simply run:

sudo add-apt-repository ppa:percona-daily/percona-xtrabackup

You will now have daily builds of Percona XtraBackup available to install.

Currently, we are in the process of working on the 2.1.4 release. The daily builds currently include some code that is destined for 2.1.4 including: fixes for bugs: 1190716, 1192347, 1190610, 1177206, 1187071, 1197644, 1182995 along with improvements to FLUSH TABLES WITH WRITE LOCK, improvements to the test suite and updates to Debian/Ubuntu packaging.

The post An Ubuntu PPA of daily builds of Percona XtraBackup appeared first on MySQL Performance Blog.

Here at Percona we’ve spent a lot of time improving our development and testing practices. Why? Because constant innovation keeps us ahead and more productive. We want to work smarter, not harder. One of the tools we use is the Jenkins Continuous Integration server.

We use Jenkins pretty heavily to help with out development processes and workflow. The flexibility, modularity and plugin availability are great advantages to Jenkins, and it being free and open source software that we can freely modify and bug fix is immensely valuable (indeed, I now maintain the Jenkins Bazaar plugin).

We currently have:

• approximately 32 slaves performing builds
• 151 top level jobs (many of these are matrix jobs)
• 2,578 jobs (including all jobs in our matrix jobs)
• 29,124 builds (we auto-expire most builds, so we’ve done many, many more than this)

totalling 109GB of disk usage and a Jenkins master Java process that uses 4GB of resident memory (10GB virt for those keeping score at home).

Our jobs range from automated merging of approved code (and both regression and performance testing of the code before it gets merged to trunk), nightly performance QA runs, building binaries for release and parameterized builds to help developers test their code.

Our slaves are deployed using only about 800 lines of puppet manifests (along with a few templates, scripts etc) and we’ve found that using virt-manager/virsh and KVM to be the most reliable way of running slaves as virtual machines on our physical hardware.

I’ll be giving a talk at the Developer Automation and Continuous Integration miniconf at linux.conf.au 2013 in Canberra about how we got to this point, hopefully helping others improve their development process and automation.

The post Serious build and testing automation appeared first on MySQL Performance Blog.