Feb
21
2012
--

DBD::mysql 4.014 breaks pt-table-checksum 2.0

DBD::mysql 4.014 breaks pt-table-checksum 2.0.  The cause is unknown, but the effect is a lot of errors like:

DBD::mysql::st execute failed: called with 2 bind variables when 6 are needed [for Statement "..." with ParamValues: ...] at ./pt-table-checksum line 7216.

The fix is simple: upgrade (or even downgrade) DBD::mysql to any version except 4.014. To see which version of DBD::mysql a system has, execute:

perl -MDBD::mysql -e 'print $DBD::mysql::VERSION, "\n";'

This bug may affect other Percona Toolkit tools, but currently pt-table-checksum 2.0 is the only victim. This bug does not affect pt-table-checksum 1.0, and it cannot be worked around in pt-table-checksum 2.0 because the bug in in DBD::mysql.

This bug affects pt-table-checksum 2.0 but not 1.0 because the newer version uses prepared statements with parameter values, whereas the older version does not. It seems, although I have not verified this, that DBD::mysql 4.014 has some sort of caching mechanism which causes it to use the wrong prepared statement.

Feb
17
2012
--

Announcing MySQL Monitoring Plugins from Percona

We’ve released a new set of monitoring plugins for MySQL servers and related software. With these plugins, you can set up world-class graphing and monitoring for your MySQL servers, using your own on-premises Cacti and Nagios software. The Cacti plugins are derived from an existing set of templates we’ve been using for several years, but the Nagios check plugins are brand new. They are informed by the research we did into the causes and preventions of MySQL downtime.

Like all Percona software, the plugins are open-source and free, licensed under the GNU GPL. The source code and issue tracker are hosted at Launchpad. The 0.9.0 release is ready to download now, and the reference manual is also online. In the next release we will integrate this into our package management, so you can install through YUM and APT repositories.

The new monitoring plugins add first-rate support for MySQL to popular enterprise opensource monitoring systems, and that’s why we’ve added them to the list of software included in our support contracts. If you’re an existing customer of Percona’s MySQL Support services, your contract automatically covers these new plugins, too; you don’t need a new contract for that. Under the support contract, you’ll get help installing, configuring, troubleshooting, and administering your monitoring plugins, as well as being entitled to bug fixes. And as always, we’re also available to extend the plugins to support your environment or broaden the range of systems and software that they can monitor.

Please use the Launchpad bug tracker to report issues. Happy monitoring!

Feb
10
2012
--

Announcing Percona Server 5.1.61-13.2

Percona is glad to announce the release of Percona Server 5.1.61-13.2 on February 10th, 2012 (Downloads are available from Percona Server 5.1.61-13.2 downloads and from the Percona Software Repositories).

Based on MySQL 5.1.61, including all the bug fixes in it, Percona Server 5.1.61-13.2 is now the current stable release in the 5.1 series. All of Percona ‘s software is open-source and free, all the details of the release can be found in the 5.1.61-13.2 milestone at Launchpad.

The full release notes can be found in our online documentation: http://www.percona.com/doc/percona-server/5.1/release-notes/Percona-Server-5.1.61-13.2.html

Feb
10
2012
--

Announcing Percona XtraBackup 1.6.5

Percona is glad to announce the release of Percona XtraBackup 1.6.5 on 10 February, 2012 (Downloads are available here and from the Percona Software Repositories).

This release is purely composed of bug fixes and is the current stable release of Percona XtraBackup.

There are some important bug fixes around incremental backups, parallel backups and backups on databases with the system tablespace being multiple files. The full release notes and details are available here: http://www.percona.com/doc/percona-xtrabackup/release-notes/1.6/1.6.5.html

Feb
08
2012
--

Announcing Percona Server 5.5.20-24.1

Percona is glad to announce the release of Percona Server 5.5.20-24.1 on February 9th, 2012 (Downloads are available here and from the Percona Software Repositories).

Based on MySQL 5.5.20, including all the bug fixes in it, Percona Server 5.5.20-24.1 is now the current stable release in the 5.5 series. All of Percona ‘s software is open-source and free, all the details of the release can be found in the 5.5.20-24.1 milestone at Launchpad.

Feb
08
2012
--

Announcing Percona XtraBackup 2.0 Beta 1

Percona is glad to announce our first Beta release of what will become Percona XtraBackup 2.0. We’re releasing this beta to encourage testing of our upcoming 2.0 release of Percona XtraBackup.

Since this is a beta release, you should not rely on it for production workloads – use the stable 1.6 series of Percona XtraBackup instead.

In this beta we’re proud to add:

We also have a good number of general bug fixes in this release and more exciting features to come in the next beta.

The full release notes are available in our online documentation.

Binaries and source tarballs can be downloaded from our download site: http://www.percona.com/downloads/XtraBackup/BETA/1.9.0/ . As this is a BETA release, we do not recommend it for production use. As always, all our work is done on the Percona XtraBackup launchpad site and the up-to-the-minute bzr trees can be pulled from Launchpad.

Feb
06
2012
--

Announcing Percona Toolkit Release 2.0.3

We’ve released Percona Toolkit 2.0.3, with a couple of major improvements and many minor ones. You can download it, read the documentation, and get support for it.

What’s new? You can read the changelog for the details, but here are the highlights:

Brand new pt-diskstats, thanks to Brian Fraser. This tool is completely rewritten, and it’s finally the iostat replacement I always wanted. Not only does it have the functionality I want (interactive, slice and dice, smart defaults) but it has the detailed statistics on I/O, so you can see whether your reads are slow versus your writes, and whether things are waiting on the disk or waiting on the queue scheduler (cfq, hint hint). Finally, it has transparency, so you can read the documentation and understand, really, what it’s doing at the low level and what that means for your server. We really need specific, precise information on exactly how the I/O is behaving so we can make good decisions when there are problems or when doing things like capacity planning.

Brand new pt-stalk, courtesy of Daniel Nichter. This tool is also completely rewritten. Instead of a Bash script that you have to configure with environment variables and run in a screen session, this is now a first-class fault detection daemon. Everyone needs post-mortem forensic data when there is a problem, and pt-stalk aims to be a core part of your infrastructure that fills this gap. It now supports things a “real” Percona Toolkit tool ought to have, such as command-line options and a configuration file. In addition, we merged pt-collect into it, so as of Percona Toolkit 2.0.3, there isn’t a separate pt-collect tool anymore.

There’s a lot more to this release, but those are the major points. Download it and let us know how it works, please! If you find bugs, file them on Launchpad, and if you need support, you know where to get it.

Jan
25
2012
--

How to recover a single InnoDB table from a Full Backup

Sometimes we need to restore only some tables from a full backup maybe because your data loss affect a small number of your tables. In this particular scenario is faster to recover single tables than a full backup. This is easy with MyISAM but if your tables are InnoDB the process is a little bit different story.

With Oracle’s stock MySQL you cannot move your ibd files freely from one server to another or from one database to another. The reason is that the table definition is stored in the InnoDB shared tablespace (ibdata) and the transaction IDs and log sequence numbers that are stored in the tablespace files also differ between servers. Therefore our example will be very straightforward: we’ll delete some rows from a table in order to recover the table later.

Most of these limitations are solved on Percona Server . More info about this in the conclusion section of this post. This post will be focus on how to recover a single tablespace using stock MySQL server.

First, you must meet certain prerequisites to be able to restore a ibd tablespace:

  • The ibd file must be from a consistent backup with all insert buffer entries merged and have no uncommitted transactions in order to not be dependent of the shared tablespace ibdata. That is, shutting down with innodb_fast_shutdown=0. We’ll use XtraBackup to avoid the server shutdown.
  • You must not drop, truncate or alter the schema of the table after the backup has been taken.
  • The variable innodb_file_per_table must be enabled.

Then, our first step is to get a consistent backup.

First we need to copy all the data to an output directory:

The –export option is the magic trick that will help us to get a consistent backup with complete independent ibd files without shutting down the service. In the second step the use of –export option runs a recovery process on the backup with innodb_fast_shutdown=0 and therefore merging all the insert buffers.

# innobackupex --defaults-file=/etc/my.cnf --export /tmp/

Then apply the logs to get a consistent backup:

# innobackupex --defaults-file=/etc/my.cnf --apply-log --export /tmp/2012-01-22_14-13-20/

Now we’re going to delete some data from one table. In this case we’re going to delete the salary information from the user 10008:

mysql> SELECT * FROM salaries WHERE emp_no=10008;
+--------+--------+------------+------------+
| emp_no | salary | from_date | to_date |
+--------+--------+------------+------------+
| 10008 | 46671 | 1998-03-11 | 1999-03-11 |
| 10008 | 48584 | 1999-03-11 | 2000-03-10 |
| 10008 | 52668 | 2000-03-10 | 2000-07-31 |
+--------+--------+------------+------------+

mysql> DELETE FROM salaries WHERE emp_no=10008;

The next step is where we are going to save a lot of time and some headaches ;) Instead of recovering all the InnoDB data we are going to recover only the “salaries” table:

  • Discard the tablespace of the salaries table:


mysql> set FOREIGN_KEY_CHECKS=0;
mysql> ALTER TABLE salaries DISCARD TABLESPACE;

  • Copy the salaries.ibd files from the backup to the database data directory:

# cp /tmp/2012-01-22_14-13-20/employees/salaries.ibd /var/lib/mysql/data/employees/

  • Import the new tablespace:


mysql> set FOREIGN_KEY_CHECKS=0;
mysql> ALTER TABLE salaries IMPORT TABLESPACE;
mysql> set FOREIGN_KEY_CHECKS=1;
mysql> SELECT * FROM salaries WHERE emp_no=10008;
+--------+--------+------------+------------+
| emp_no | salary | from_date | to_date |
+--------+--------+------------+------------+
| 10008 | 46671 | 1998-03-11 | 1999-03-11 |
| 10008 | 48584 | 1999-03-11 | 2000-03-10 |
| 10008 | 52668 | 2000-03-10 | 2000-07-31 |
+--------+--------+------------+------------+

The salary history from the user is back again!

Conclusion:

As we learned , you can also recover single InnoDB table as with MyISAM but knowing in advance that there are some prerequisites to comply.

Percona Server relaxes a lot of limitations and is able to import tables from different Server instance, when table was altered or truncated in the meanwhile. Though this only works if table was
“exported” with Xtrabackup as this exports essential information from main tablespace which is not stored in .ibd file. innodb_import_table_from_xtrabackup=1 should be enabled for such advanced import process to work. You can read more about this feature in Percona Server Documentation

In the next blog post I’ll explain how to do recovery using Percona Data Recovery toolkit.

Jan
19
2012
--

Percona XtraDB Cluster Feature 2: Multi-Master replication

This is about the second great feature – Multi-Master replication, what you get with Percona XtraDB Cluster.

It is recommended you get familiar with general architecture of the cluster, described on the previous post.

By Multi-Master I mean the ability to write to any node in your cluster and do not worry that eventually you get out-of-sync situation, as it regularly happens with regular MySQL replication if you imprudently write to the wrong server.

This is long-waited feature, I’ve seen growing demand for it for last two years or even more.
Percona XtraDB Cluster provides it, and let’s see how it works.

With our Cluster you can write to any node, and the Cluster guarantees consistency of writes. That is the write is either committed on all nodes or not committed at all.

Let me draw some diagram there. For the simplicity I will use a two-nodes example, but the same logic is applied when you have N nodes.

As you see all queries are executed locally on the node, and only on COMMIT there is special handling.
When you issue COMMIT, your transaction has to pass certification on all nodes. If it does not pass, you
will receive “ERROR” as response on your query. After that transaction is applied on the local node.

That is response time of COMMIT consist of several parts:

  • Network roundtrip
  • Certification time
  • Local applying

Please note that applying the transaction on remote nodes does not affect the response time of COMMIT,
as it happens in the background after the response on certification.

The two important consequences of this architecture.

  • First: we can have several appliers working in parallel. This gives us true parallel replication. Slave can have many parallel threads, and you can tune it by variable wsrep_slave_threads
  • Second: There might be a small period of time when the slave is out-of-sync from master. This happens because the master may apply event faster than a slave. And if you do read from the slave, you may read data, that has not changes yet. You can see that from diagram. However you can change this behavior by using variable wsrep_causal_reads=ON. In this case the read on the slave will wait until event is applied (this however will increase the response time of the read. This gap between slave and master is the reason why this replication named “virtually synchronous replication”, not real “synchronous replication”

The described behavior of COMMIT also has the second serious implication.
If you run write transactions to two different nodes, the cluster will use an optimistic locking model.
That means a transaction will not check on possible locking conflicts during individual queries, but rather on the COMMIT stage. And you may get ERROR response on COMMIT. I am highlighting this, as this is one of incompatibilities with regular InnoDB, that you may experience. In InnoDB usually DEADLOCK and LOCK TIMEOUT errors happen in response on particular query, but not on COMMIT. Well, if you follow a good practice, you still check errors code after “COMMIT” query, but I saw many applications that do not do that.

So, if you plan to use Multi-Master capabilities of XtraDB Cluster, and run write transactions on several nodes, you may need to make sure you handle response on “COMMIT” query.

Jan
17
2012
--

Percona XtraDB Cluster Feature 1: High Availability

There and in coming posts I am going to cover main features of Percona XtraDB Cluster. The first feature is High Availability.

But before jumping to HA, let’s review general architecture of the Percona XtraDB Cluster.

1. The Cluster consists of Nodes. Recommended configuration is to have at least 3 nodes, but you can make it running with 2 nodes too.
2. Each Node is regular MySQL / Percona Server setup. The point is that you can convert your existing MySQL / Percona Server into Node and roll Cluster using it as base. Or otherwise – you can detach Node from Cluster and use it as just a regular server.
3. Each Node contains the full copy of data. That defines XtraDB Cluster behavior in many ways. And obviously there are benefits and drawbacks.

Cluster architecture

Benefits of such approach:

  • When you execute a query, it is executed locally on the node. All data is available locally, no need for remote access.
  • No central management. You can loose any node at any point of time, and the cluster will continue to function
  • Good solution for scaling a read workload. You can put read queries to any of the nodes

Drawbacks:

  • Overhead of joining new node. The new node has to copy full dataset from one of existing node. If it is 100GB, it copies 100GB.
  • This can’t be used as an effective write scaling solution. There might be some improvements in write throughput when you run write traffic to 2 nodes vs all traffic to 1 node, but you can’t expect a lot. All writes still have to go on all nodes.
  • You have several duplicates of data. For 3 nodes – 3 duplicates

This basically defines how Percona XtraDB Cluster can be used for High Availability.

Basic setup: you run 3-nodes setup.
The Percona XtraDB Cluster will continue to function when you take any of nodes down.
At any point of time you can shutdown any Node to perform maintenance or make configuration changes.
Or Node may crash or become network unavailable. The Cluster will continue to work, you can continue to run queries on working nodes.

The biggest question there, what will happen when the Node joins the cluster back, and there were changes to data while the node
was down.

Let’s focus on this with details.
There is two ways that Node may use when it joins the cluster: State Snapshot Transfer (SST) and Incremental State Transfer (IST).

  • SST is the full copy if data from one node to another. SST is used when new node joins the cluster, it has to transfer data from existing node.
    There is three methods of SST available in Percona XtraDB Cluster: mysqldump, rsync and xtrabackup ( Percona XtraBackup with support of XtraDB Cluster will be released soon, so far you need to use our source code repository).
    The downside of mysqldump and rsync is that your cluster becomes READ-ONLY for time that takes to copy data from one node to another (SST applies FLUSH TABLES WITH READ LOCK command).
    Xtrabackup SST does not require READ LOCK for full time, only for syncing .frm files (the same as with regular backup).
  • Even with that, SST may be intrusive, that’s why there is IST mechanism. If down your node for short period of time, and then start it, the node is able to fetch only changes made during period it was down.
    This is done using caching mechanism on nodes. Each node contains a cache, ring-buffer, (the size is configurable) of last N changes, and the node is able to transfer part of this cache. Obviously IST can be done only if amount of changes needed to transfer is less than N. If it exceeds N, then the joining node has to perform SST.

You can monitor current state of Node by using
SHOW STATUS LIKE 'wsrep_local_state_comment', when it is ‘Synced (6)’, the node is ready to handle traffic.

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com