Personal blog of Yzmir Ramirez

Ryan Mauger has a new post today looking at how to combine the Tidy extension for PHP and Firebug with a Zend Framework application to keep your HTML neat and valid with a handy bit of feedback for debugging.

Rob James has a new post looking at configuring Apache to run Tomcat and PHP together so you can have both Java and PHP apps running through the same Apache frontend.

Chris Hartjes has a new post to his blog today looking at how to make forms in a Zend Framework application in a bit more reusable fashion.

I’ve seen my posts on Ontime Air traffic and Star Schema Benchmark got a lot of interest
(links:

• http://www.mysqlperformanceblog.com/2010/01/07/star-schema-bechmark-infobright-infinidb-and-luciddb/
• http://www.mysqlperformanceblog.com/2009/10/02/analyzing-air-traffic-performance-with-infobright-and-monetdb/
• http://www.mysqlperformanceblog.com/2009/10/26/air-traffic-queries-in-luciddb/
• http://www.mysqlperformanceblog.com/2009/11/02/air-traffic-queries-in-infinidb-early-alpha/

).
However benchmarks by itself did not cover all cases I would want, so I was thinking about better scenario. The biggest problem is to get real big enough dataset, and I thank to Bradley C. Kuszmaul, he pointed me on Wikipedia statistics on access to Wikipedia pages, and thank to Domas, who made stats accessible. Link to the archives: http://dammit.lt/wikistats/archive/ or the original Domas’s announcement .

Although the table does not have very much different information, I think it is good enough to represent cases you can face in Web application ( log processing, page visits, clickstream, etc).

I made some efforts to normalize data to have model in classic star schema and prepared queries that could be run on proposed dataset (John Sichi, lead of LucidDB helped me to draft some queries).
You can see details on our Percona Wikistat benchmark Wiki.

I have next goals with proposed benchmark:

• Compare engines in OLAP queries for planning, predicting growth, analyzing access patterns to wiki pages, draw trends.
• Compare engines in statistical queries for end users, which can be executed in real-time. I.e. How many times that or another page was accessed yesterday vs today.
• Understand specific features and characteristic of each engine.
• Compare throughput on simple queries (queries and scenario to be drafted yet)
• Check ability to load data and serve queries at the same time ( availability during data load ) (queries and scenario to be drafted yet)

So in proposed schema I have four tables:
pagestat (fact table), and pages, datesinfo, projects (dimensions tables).

Dimensions tables are supposed to be static and not changed, and we can change datasize
by varying amount of months loaded into fact table (so this is scale factor).

EER diagram

( made with MySQL Workbench )

In current dataset, which you can download from Amazon snapshot (name: “percona-wikistat”, ID:snap-a5f9bacc) we have:

• Table pages: 724.550.811 rows. data size: 40476M
• Table datesinfo: 9624 rows, one entry represents 1 hour
• Table projects: 2025 rows
• Table pagestats
  Data for 2009-06: # 3.453.013.109 rows / size 68352M
  Data for 2009-07: # 3.442.375.618 rows / size 68152M

So with two months of stats we have about 172GB of data with about 7 billion rows in fact table.

Example of query ( again, full list on Benchmark Wiki)

I am going to load data and run queries against available engines:

• MySQL MyISAM / InnoDB (to have reference results)
• InfoBright
• InfiniDB
• MonetDB
• LucidDB
• Greenplum

and I will report my results ( so stay with MySQLPerformanceBlog )

I’d like also to test also Paraccel, Vertica and KickFire systems, but I do not have access to.

I welcome your feedback on the benchmark, and what else you would like to see here.

---

Entry posted by Vadim |
16 comments

Add to: | | | |

Zend_Paginator does a great job of creating pagination for a site. In this blog post I’m demonstrating how to use Zend_Paginator to easily create pagination of Twitter tweets using Zend_Rest_Client to access the Twitter Services. As a bonus, Zend_Cache is used to store the tweets to make the process more efficient.

The next ZendCast in the user authentication with the Zend Framework’s Zend_Auth has been posted to the ZendCasts.com site today. In this new screencast, they look at how to integrate it with Doctrine to automatically validate users against the information in your databases (following up on this first part of the series).

On behalf of the Zend Framework team and the framework’s many contributors,
I’m pleased to announce the immediate availability of the stable release of
Zend Framework 1.10.0. You can download it from our downloads page:

http://framework.zend.com/download/latest

This release includes a ton of new features (more on those below), as well
as some huge changes to our documentation.

Both Don Raman and Ruslan Yakushev have posted about the latest release of the WinCache caching tool from Microsoft for Windows-based PHP installations.

Don comments:

In partnership with Microsoft Philippines, PHP User Group Philippines presents PHP Developers Summit 2010 on January 30th. We are inviting you to come and join us in this gathering of the country’s best tech-talents, professionals and web developers promoting the use of PHP and open source solutions in the enterprise and schools.

Continuing on from his previous article about making RESTful applications with the Zend Framework, Sudeer has posted the second part of the tutorial focusing on the HTTP responses back to the client.