Sep
09
2013
--

September: A busy month for MySQL events

September 2013 MySQL eventsNow with summer winding down and school back in session, we get into a very busy time with many great opportunities to discover more about MySQL with several excellent MySQL events.

At first I wanted to cover all autumn events here but if I did that this post would be far too long. So instead I’m focusing on a few wonderful MySQL events that I’m going to be involved with in September.

In a few short days we’re running Percona University at Washington, D.C. This free event on September 12 provides a great opportunity to learn about latest developments in the MySQL ecosystem as we at Percona see it. I personally will be speaking about Percona XtraDB Cluster, MariaDB 10.0 and Percona Server 5.6, and MySQL 5.6. We also have other great speakers talking about Continent Tungsten, Hadoop, MySQL Cluster, Percona Toolkit as well as insights on using MySQL with Go and best DBA Practices.

On September 18 I’m going to the MySQL Meetup in San Francisco where Ryan Lowe will be speaking about Running MySQL 5.6 on Amazon RDS. The space for these events is limited and they often can’t accommodate everyone, so do not delay and register if you can make it to either one.

In Europe we have Frederic Descamps speaking at the Open Source For Business conference in Prato, Italy September 19-20.  Fred will talk about Percona XtraDB Cluster as well as installing and managing MySQL like a true devops. I especially wish I could attend this talk as Fred has a lot of insights and practical experience with applying a DevOps approach to MySQL installations.

September 21-23 I’ll be at MySQL Connect in San Francisco. This is a great MySQL event that Oracle puts together as part of Oracle Open World Week, which presents great opportunities to talk with Oracle’s MySQL engineering team and learn about new products and the MySQL roadmap. There will also be many great community speakers this year as well, sharing their insights and war stories from MySQL’s front lines. I personally will deliver two presentations, “MySQL Indexing: Best Practices for MySQL 5.6” and “Optimizing MySQL Configuration in MySQL 5.6.” These are great talks for both for MySQL beginners and more advanced users which I have updated to cover MySQL 5.6

At Percona we talk and exhibit at many shows throughout the year so we have created an Events page to keep you up to date. You can also subscribe to our Events newsletter for the latest news and updates on this front.

Have a wonderful September!

The post September: A busy month for MySQL events appeared first on MySQL Performance Blog.

Aug
08
2013
--

Big Data with MySQL and Hadoop at MySQL Connect 2013

I will be talking about Big Data with MySQL and Hadoop at MySQL Connect 2013 (Sept. 21-22) in San Francisco as well as at Percona University at Washington, DC (September 12, 2013). Apache Hadoop is a very popular Big Data solution and we can nowadays easily integrate it with MySQL. I will start with a brief introduction of Apache Hadoop and its components (HFDS, Map/Reduce, Hive, HBase/HCatalog, Flume, Scoop, etc). Next I will show 2 major Big Data scenarios:

  • From file to Hadoop to MySQL. This is an example of “ELT” process: Extract data from external source; Load data into Hadoop; Transform data/Analyze data; Extract results to MySQL. It is similar to the original Data Warehouse ETL (Extract; Transfer; Load) process, however, instead of “transforming” data before loading it to the Data Warehouse, we will load it “as is” and then run the data analysis. As a result of this analysis (map/reduce process) we can generate a report and load it to MySQL (using Sqoop export). To illustrate this process I will show 2 classical examples: Clickstream analysis and Twitter feed analysis. On top of those examples I will also show how to use MySQL / Full Text Search solutions to perform a near real-time reports from HBase.

Picture 1: ELT pipeline, from File to Hadoop to MySQL

clickstream_example

  • From OLTP MySQL to Hadoop to MySQL reporting. In this scenario we extract data (potentially close to real-time) from MySQL, load it to Hadoop for storage and analysis and later generate reports to load it into another MySQL instance (reporting), which can be used to generate and display graphs.

Picture 2: From OLTP MySQL to Hadoop to MySQL reporting.

hadoop_mysql_reporting

Note: The reason why we need an additional storage for MySQL reports is that it may take a long time to generate a Hive report (as it is executed with Map/Reduce which reads all the files/no indexes). So it make sense to “offload” a common reports’ results into a separate storage (MySQL).

In both scenarios we will need a way to integrate Hadoop and MySQL. In my previous post, MySQL and Hadoop integration, I have demonstrated how to integrate Hadoop and MySQL with Sqoop and Hadoop Applier for MySQL. This case is similar, however we can use a different toolset. In the scenario 1 (i.e. Clickstream) we can use Apache Flume to grab files (or read “events”) and load them into Hadoop. With Flume we can define a “source” and a “sink”. Flume supports a range of different sources including HTTP requests, Syslog, TCP, etc. HTTP source is interesting, as we can convert all (or a number of) HTTP requests (“Source”) into an “event” which can be loaded into Hadoop (“Sink”).

During my presentation I will show the exact configurations for the sample Clickstream process, including:

  1. Flume configuration
  2. HiveQL queries to generate a report
  3. Sqoop export queries to load the report into MySQL

See you at MySQL Connect 2013!

The post Big Data with MySQL and Hadoop at MySQL Connect 2013 appeared first on MySQL Performance Blog.

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com