Personal blog of Yzmir Ramirez

Data and analytics service Databricks today announced that it has acquired Redash, a company that helps data scientists and analysts visualize their data and build dashboards around it.

Redash’s customers include the likes of Atlassian, Cloudflare, Mozilla and Soundcloud and the company offers both an open-source self-hosted version of its tools, as well as paid hosted options.

The two companies did not disclose the financial details of the acquisition. According to Crunchbase, Tel Aviv-based Redash never raised any outside funding.

Databricks co-founder CEO Ali Ghodsi told me that the two companies met because one of his customers was using the product. “Since then, we’ve been impressed with the entire team and their attention to quality,” he said. “The combination of Redash and Databricks is really the missing link in the equation — an amazing backend with Lakehouse and an amazing front end built-in visualization and dashboarding feature from Redash to make the magic happen.”

Image Credits: Databricks

For Databricks, this is also a clear signal that it wants its service to become the go-to platform for all data teams and offer them all of the capabilities they would need to extract value from their data in a single platform.

“Not only are our organizations aligned in our open source heritage, but we also share in the mission to democratize and simplify data and AI so that data teams and more broadly, business intelligence users, can innovate faster,” Ghodsi noted. “We are already seeing awesome results for our customers in the combined technologies and look forward to continuing to grow together.”

In addition to the Redash acquisition, Databricks also today announced the launch of its Delta Engine, a new high-performance query engine for use with the company’s Delta Lake transaction layer.

“Databricks’ new Delta Engine for Delta Lake enables fast query execution for data analytics and data science, without moving the data out of the data lake,” the company explains. “The high-performance query engine has been built from the ground up to take advantage of modern cloud hardware for accelerated query performance. With this improvement, Databricks customers are able to move to a unified data analytics platform that can support any data use case and result in meaningful operational efficiencies and cost savings.”

 If a picture is worth a thousand words, a video is worth that times the frame rate. Matroid, a computer vision startup launching out of stealth today, enables anyone to take advantage of the information inherently embedded in video. You can build your own detector within the company’s intuitive, non-technical, web platform to detect people and most other objects. Reza Zadeh, founder… Read More

 Since 2013, The Associated Press has been making an intentional effort to put data in the hands of local reporters. In the last few years, this meant assisting with Freedom of Information Act requests and putting a team of four engineers to work building visualizations and extracting insights from massive spreadsheets. Today the AP is announcing a joint pilot program with Data.world to… Read More

 Data visualization has been done — we have publicly traded, interactive, real-time and heck even artificially intelligent companies promising data visualization. But despite all the noise, Portland-based Reflect is making a go of it in the space, opening up its public beta today. By putting developers first and letting them integrate and customize visualizations in their own… Read More

 Smartsheet, the spreadsheet-based project and work management platform, is now making it easier for companies to understand how their employees actually work (and who they mostly work with). Companies mostly use the service to manage their projects and help employees collaborate. Because of this, the service knows quite a bit about who does what in a given business and now it allows businesses… Read More

When you make a change to your MySQL configuration in production it would be great to know the impact (a “before and after” type of picture). Some changes are obvious. For many variables proper values can be determined beforehand, i.e. innodb_buffer_pool_size or innodb_log_file_size. However, there is 1 configuration variable which is much less obvious for many people working with MySQL: query_cache.

The idea of query cache is great, however, there are a lot of issues with MySQL query cache, one of the most important issues is query cache mutex which can cause a severe contention on the CPU bound workloads. In MySQL 5.6 you can remove the mutex by disabling the query cache completely (set query_cache_type=0).

There is a big issue with disabling query_cache though and it is not a technical issue. The issue is how do you convince your boss or dev team/operations team that query cache is bad. Almost all measurements available in MySQL will tell you that the query cache is utilized: there will be a good number of Qcache_hits. On the other hand you may see the contention in processlist: for example you can see one of those states associates with query cache contention:  Waiting for query cache lock or Waiting on query cache mutex (if you are running Percona Server).

Now you use Percona Cloud Tools (with Percona Server) to actually see how query cache is used globally and on per query basis.

Query cache visualization

Percona Server has this very useful additional feature: if you set “log_slow_verbosity”  option to “microtime, query_plan, innodb” it will also log the information if the query was a query cache hit or query cache miss. This can give you a valuable inside for the query cache real utilization.

Percona Cloud Tools will be really helpful here as it will visualize this information and will actually see “before and after” graph. (It will set “log_slow_verbosity” option to “microtime, query_plan, innodb” for you if you set Slow log verbosity = Full in the agent configuration options).

Here is the example. I’ve disabled the query cache and then looked at the total query and lock time.

As we can see the total query time (across all queries) decreased significantly. The Average QPS on this picture is actually a little bit misleading and should be named “Queries running at the same time” (it is calculated as query count / time). The above graphs shows clear contention on the query cache level.

Just to confirm, we can look at the number of query cache hits:

The number of Query Cache “hits” dropped to 0 when we disabled the query cache.

Please note: if your workload is readonly and you are running the same complex query over and over again, query cache may be helpful. Usually, in normal circumstances however, the query cache can cause contention and reduce the response time (as you can see above).

Percona Cloud Tools is a free beta, please give it a try and share your experience in the comments. If you want to learn more about it, join the June 25 webinar hosted by Vadim Tkachenko titled, “Monitoring All (Yes, All!) MySQL Metrics with Percona Cloud Tools.” The webinar, which starts at 10 a.m. Pacific time, is free but I recommend registering now to reserve your spot.

The post Measure the impact of MySQL configuration changes with Percona Cloud Tools appeared first on MySQL Performance Blog.