Wherever you turn, businesses are facing tremendous disruptive pressure. What’s interesting is that the theory about how firms should be dealing with this massive change is itself in flux, transforming if you will, as organizations come to grips with the idea that the most basic ways they do business are being called into question. Just over a year ago when I researched this topic,… Read More
Jan
31
2016
31
2016
--
Digital Transformation Requires Total Organizational Commitment
Jan
29
2016
29
2016
--
EXPLAIN FORMAT=JSON knows everything about UNIONs: union_result and query_specifications
Ready for another post in the EXPLAIN FORMAT=JSON is Cool series! Great! This post will discuss how to see all the information that is contained in optimized queries with
UNION
using the
union_result
and
query_specifications
commands.
When optimizing complicated queries with
UNION
, it is easy to get lost in the regular
EXPLAIN
output trying to identify which part of the output belongs to each part of the
UNION
.
Let’s consider the following example:
mysql> explain
-> select emp_no, last_name, 'low_salary' from employees
-> where emp_no in (select emp_no from salaries
-> where salary < (select avg(salary) from salaries))
-> union
-> select emp_no, last_name, 'high salary' from employees
-> where emp_no in (select emp_no from salaries
-> where salary >= (select avg(salary) from salaries))G
*************************** 1. row ***************************
id: 1
select_type: PRIMARY
table: employees
partitions: NULL
type: ALL
possible_keys: PRIMARY
key: NULL
key_len: NULL
ref: NULL
rows: 299778
filtered: 100.00
Extra: NULL
*************************** 2. row ***************************
id: 1
select_type: PRIMARY
table: salaries
partitions: NULL
type: ref
possible_keys: PRIMARY,emp_no
key: PRIMARY
key_len: 4
ref: employees.employees.emp_no
rows: 9
filtered: 33.33
Extra: Using where; FirstMatch(employees)
*************************** 3. row ***************************
id: 3
select_type: SUBQUERY
table: salaries
partitions: NULL
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 2557022
filtered: 100.00
Extra: NULL
*************************** 4. row ***************************
id: 4
select_type: UNION
table: employees
partitions: NULL
type: ALL
possible_keys: PRIMARY
key: NULL
key_len: NULL
ref: NULL
rows: 299778
filtered: 100.00
Extra: NULL
*************************** 5. row ***************************
id: 4
select_type: UNION
table: salaries
partitions: NULL
type: ref
possible_keys: PRIMARY,emp_no
key: PRIMARY
key_len: 4
ref: employees.employees.emp_no
rows: 9
filtered: 33.33
Extra: Using where; FirstMatch(employees)
*************************** 6. row ***************************
id: 6
select_type: SUBQUERY
table: salaries
partitions: NULL
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 2557022
filtered: 100.00
Extra: NULL
*************************** 7. row ***************************
id: NULL
select_type: UNION RESULT
table: <union1,4>
partitions: NULL
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: NULL
filtered: NULL
Extra: Using temporary
7 rows in set, 1 warning (0.00 sec)
Note (Code 1003): /* select#1 */ select `employees`.`employees`.`emp_no` AS `emp_no`,`employees`.`employees`.`last_name` AS `last_name`,'low_salary' AS `low_salary` from `employees`.`employees` semi join (`employees`.`salaries`) where ((`employees`.`salaries`.`emp_no` = `employees`.`employees`.`emp_no`) and (`employees`.`salaries`.`salary` < (/* select#3 */ select avg(`employees`.`salaries`.`salary`) from `employees`.`salaries`))) union /* select#4 */ select `employees`.`employees`.`emp_no` AS `emp_no`,`employees`.`employees`.`last_name` AS `last_name`,'high salary' AS `high salary` from `employees`.`employees` semi join (`employees`.`salaries`) where ((`employees`.`salaries`.`emp_no` = `employees`.`employees`.`emp_no`) and (`employees`.`salaries`.`salary` >= (/* select#6 */ select avg(`employees`.`salaries`.`salary`) from `employees`.`salaries`)))
While we can guess that subquery 3 belongs to the first query of the union, and subquery 6 belongs to the second (which has number 4 for some reason), we have to be very careful (especially in our case) when queries use the same tables in both parts of the
UNION
.
The main issue with the regular
EXPLAIN
for
UNION
is that it has to re-present the hierarchical structure as a table. The same issue occurs when you want to store objects created in programming language, such as Java, in the database.
EXPLAIN FORMAT=JSON
, on the other hand, has hierarchical structure and more clearly displays how
UNION
was optimized:
mysql> explain format=json select emp_no, last_name, 'low_salary' from employees where emp_no in (select emp_no from salaries where salary < (select avg(salary) from salaries)) union select emp_no, last_name, 'high salary' from employees where emp_no in (select emp_no from salaries where salary >= (select avg(salary) from salaries))G
*************************** 1. row ***************************
EXPLAIN: {
"query_block": {
"union_result": {
"using_temporary_table": true,
"table_name": "<union1,4>",
"access_type": "ALL",
"query_specifications": [
{
"dependent": false,
"cacheable": true,
"query_block": {
"select_id": 1,
"cost_info": {
"query_cost": "921684.48"
},
"nested_loop": [
{
"table": {
"table_name": "employees",
"access_type": "ALL",
"possible_keys": [
"PRIMARY"
],
"rows_examined_per_scan": 299778,
"rows_produced_per_join": 299778,
"filtered": "100.00",
"cost_info": {
"read_cost": "929.00",
"eval_cost": "59955.60",
"prefix_cost": "60884.60",
"data_read_per_join": "13M"
},
"used_columns": [
"emp_no",
"last_name"
]
}
},
{
"table": {
"table_name": "salaries",
"access_type": "ref",
"possible_keys": [
"PRIMARY",
"emp_no"
],
"key": "PRIMARY",
"used_key_parts": [
"emp_no"
],
"key_length": "4",
"ref": [
"employees.employees.emp_no"
],
"rows_examined_per_scan": 9,
"rows_produced_per_join": 299778,
"filtered": "33.33",
"first_match": "employees",
"cost_info": {
"read_cost": "302445.97",
"eval_cost": "59955.60",
"prefix_cost": "921684.48",
"data_read_per_join": "4M"
},
"used_columns": [
"emp_no",
"salary"
],
"attached_condition": "(`employees`.`salaries`.`salary` < (/* select#3 */ select avg(`employees`.`salaries`.`salary`) from `employees`.`salaries`))",
"attached_subqueries": [
{
"dependent": false,
"cacheable": true,
"query_block": {
"select_id": 3,
"cost_info": {
"query_cost": "516948.40"
},
"table": {
"table_name": "salaries",
"access_type": "ALL",
"rows_examined_per_scan": 2557022,
"rows_produced_per_join": 2557022,
"filtered": "100.00",
"cost_info": {
"read_cost": "5544.00",
"eval_cost": "511404.40",
"prefix_cost": "516948.40",
"data_read_per_join": "39M"
},
"used_columns": [
"salary"
]
}
}
}
]
}
}
]
}
},
{
"dependent": false,
"cacheable": true,
"query_block": {
"select_id": 4,
"cost_info": {
"query_cost": "921684.48"
},
"nested_loop": [
{
"table": {
"table_name": "employees",
"access_type": "ALL",
"possible_keys": [
"PRIMARY"
],
"rows_examined_per_scan": 299778,
"rows_produced_per_join": 299778,
"filtered": "100.00",
"cost_info": {
"read_cost": "929.00",
"eval_cost": "59955.60",
"prefix_cost": "60884.60",
"data_read_per_join": "13M"
},
"used_columns": [
"emp_no",
"last_name"
]
}
},
{
"table": {
"table_name": "salaries",
"access_type": "ref",
"possible_keys": [
"PRIMARY",
"emp_no"
],
"key": "PRIMARY",
"used_key_parts": [
"emp_no"
],
"key_length": "4",
"ref": [
"employees.employees.emp_no"
],
"rows_examined_per_scan": 9,
"rows_produced_per_join": 299778,
"filtered": "33.33",
"first_match": "employees",
"cost_info": {
"read_cost": "302445.97",
"eval_cost": "59955.60",
"prefix_cost": "921684.48",
"data_read_per_join": "4M"
},
"used_columns": [
"emp_no",
"salary"
],
"attached_condition": "(`employees`.`salaries`.`salary` >= (/* select#6 */ select avg(`employees`.`salaries`.`salary`) from `employees`.`salaries`))",
"attached_subqueries": [
{
"dependent": false,
"cacheable": true,
"query_block": {
"select_id": 6,
"cost_info": {
"query_cost": "516948.40"
},
"table": {
"table_name": "salaries",
"access_type": "ALL",
"rows_examined_per_scan": 2557022,
"rows_produced_per_join": 2557022,
"filtered": "100.00",
"cost_info": {
"read_cost": "5544.00",
"eval_cost": "511404.40",
"prefix_cost": "516948.40",
"data_read_per_join": "39M"
},
"used_columns": [
"salary"
]
}
}
}
]
}
}
]
}
}
]
}
}
}
1 row in set, 1 warning (0.00 sec)
Note (Code 1003): /* select#1 */ select `employees`.`employees`.`emp_no` AS `emp_no`,`employees`.`employees`.`last_name` AS `last_name`,'low_salary' AS `low_salary` from `employees`.`employees` semi join (`employees`.`salaries`) where ((`employees`.`salaries`.`emp_no` = `employees`.`employees`.`emp_no`) and (`employees`.`salaries`.`salary` < (/* select#3 */ select avg(`employees`.`salaries`.`salary`) from `employees`.`salaries`))) union /* select#4 */ select `employees`.`employees`.`emp_no` AS `emp_no`,`employees`.`employees`.`last_name` AS `last_name`,'high salary' AS `high salary` from `employees`.`employees` semi join (`employees`.`salaries`) where ((`employees`.`salaries`.`emp_no` = `employees`.`employees`.`emp_no`) and (`employees`.`salaries`.`salary` >= (/* select#6 */ select avg(`employees`.`salaries`.`salary`) from `employees`.`salaries`)))
First it puts member
union_result
in the
query_block
at the very top level:
EXPLAIN: {
"query_block": {
"union_result": {
The
union_result
object contains information about how the result set of the
UNION
was processed:
"using_temporary_table": true,
"table_name": "<union1,4>",
"access_type": "ALL",
And also contains the
query_specifications
array which also contains all the details about queries in the
UNION
:
"query_specifications": [
{
"dependent": false,
"cacheable": true,
"query_block": {
"select_id": 1,
<skipped>
{
"dependent": false,
"cacheable": true,
"query_block": {
"select_id": 4,
This representation is much more clear, and also contains all the details which the regular
EXPLAIN
misses for regular queries.
Conclusion:
EXPLAIN FORMAT=JSON
not only contains additional optimization information for each query in the
UNION
, but also has a hierarchical structure that is more suitable for the hierarchical nature of the
UNION
.
Jan
29
2016
29
2016
--
IBM Closes Weather Co. Purchase, Names David Kenny New Head Of Watson Platform
IBM is taking another step to expand its Watson AI business and build its presence in areas like IoT: today the company announced that its acquisition of the Weather Company — the giant weather media and data group — has now officially closed. IBM is not disclosing the value of the deal: it was originally reported to be in the region of $2 billion, but sources close to IBM tell us… Read More
Jan
29
2016
29
2016
--
Our 2016 Predictions For The IPO Market
2015 was not a good year for IPOs — it was actually the worst year for tech IPOs since the financial crisis in 2009. More and more startups are opting to remain private for longer periods of time. And while we still haven’t seen any IPOs in 2016, the slowest start since 2009, eventually venture-backed companies need to go public — if only to raise additional financing to… Read More
Jan
29
2016
29
2016
--
Percona XtraDB Cluster 5.6.28-25.14 is now available
Percona is glad to announce the new release of Percona XtraDB Cluster 5.6 on January 29, 2016. Binaries are available from the downloads area or from our software repositories.
Percona XtraDB Cluster 5.6.28-25.14 is now the current release, based on the following:
All of Percona software is open-source and free, and all the details of the release can be found in the 5.6.28-25.14 milestone at Launchpad.
For more information about relevant Codership releases, see this announcement.
Bugs Fixed:
- 1494399: Fixed issue caused by replication of events on certain system tables (for example,
mysql.slave_master_info,mysql.slave_relay_log_info). Replication in the Galera eco-system is now avoided when bin-logging is disabled for said tables.
NOTE: As part of this fix, when bin-logging is enabled, replication in the Galera eco-system will happen only ifBINLOG_FORMATis set to eitherROWorSTATEMENT. The recommended format isROW, whileSTATEMENTis required only for thept-table-checksumtool to operate correctly. IfBINLOG_FORMATis set toMIXED, replication of events in the Galera eco-system tables will not happen even with bin-logging enabled for those tables. - 1522385: Fixed GTID holes caused by skipped replication. A slave might ignore an event replicated from master, if the same event has already been executed on the slave. Such events are now propagated in the form of special GTID events to maintain consistency.
- 1532857: The installer now creates a
/var/lib/galera/directory (assigned to usernobody), which can be used bygarbdin the event it is started from a directory thatgarbdcannot write to.
Known Issues:
- 1531842: Two instances of
garbdcannot be started from the same working directory. This happens because each instance creates a state file (gvwstate.dat) in the current working directory by default. Althoughgarbdis configured to use thebase_dirvariable, it was not registered due to a bug. Untilgarbdis fixed, you should start each instance from a separate working directory.
Help us improve our software quality by reporting any bugs you encounter using our bug tracking system. As always, thanks for your continued support of Percona!
Jan
28
2016
28
2016
--
Vote Percona Server in LinuxQuestions.org Members Choice Awards
Percona is calling on you! Vote Percona for Database of the Year in LinuxQuestions.org Members Choice Awards 2015. Help our Percona Server get recognized as one of the best database options for data performance. Percona Server is a free, fully compatible, enhanced, open source drop-in replacement for MySQL® that provides superior performance, scalability and instrumentation.
LinuxQuestions.org, or LQ for short, is a community-driven, self-help web site for Linux users. Each year, LinuxQuestions.org holds an annual competition to recognize the year’s best-in-breed technologies. The winners of each category are determined by the online Linux community!
You can vote now for your favorite products of 2015 (Percona, of course!). This is your chance to be heard!
Voting ends on February 10th, 2016. You must be a registered member of LinuxQuestions.org with at least one post on their forums to vote.
Jan
28
2016
28
2016
--
Setup a MongoDB replica/sharding set in seconds
In the MySQL world, we’re used to playing in the MySQL Sandbox. It allows us to deploy a testing replication environment in seconds, without a great deal of effort or navigating multiple virtual machines. It is a tool that we couldn’t live without in Support.
In this post I am going to walk through the different ways we have to deploy a MongoDB replica/sharding set test in a similar way. It is important to mention that this is not intended for production, but to be used for troubleshooting, learning or just playing around with replication.
Replica Set regression test’s diagnostic commands
MongoDB includes a .js that allows us to deploy a replication set from the MongoDB’s shell. Just run the following:
# mongo --nodb
> var rstest = new ReplSetTest( { name: 'replicaSetTest', nodes: 3 } )
> rstest.startSet()
ReplSetTest Starting Set
ReplSetTest n is : 0
ReplSetTest n: 0 ports: [ 31000, 31001, 31002 ] 31000 number
{
"useHostName" : true,
"oplogSize" : 40,
"keyFile" : undefined,
"port" : 31000,
"noprealloc" : "",
"smallfiles" : "",
"rest" : "",
"replSet" : "replicaSetTest",
"dbpath" : "$set-$node",
"restart" : undefined,
"pathOpts" : {
"node" : 0,
"set" : "replicaSetTest"
}
}
ReplSetTest Starting....
[...]
At some point our mongod daemons will be running, each with its own data directory and port:
2133 pts/0 Sl+ 0:01 mongod --oplogSize 40 --port 31000 --noprealloc --smallfiles --rest --replSet replicaSetTest --dbpath /data/db/replicaSetTest-0 --setParameter enableTestCommands=1 2174 pts/0 Sl+ 0:01 mongod --oplogSize 40 --port 31001 --noprealloc --smallfiles --rest --replSet replicaSetTest --dbpath /data/db/replicaSetTest-1 --setParameter enableTestCommands=1 2213 pts/0 Sl+ 0:01 mongod --oplogSize 40 --port 31002 --noprealloc --smallfiles --rest --replSet replicaSetTest --dbpath /data/db/replicaSetTest-2 --setParameter enableTestCommands=1
Perfect. Now we need to initialize the replicaset:
> rstest.initiate()
{
"replSetInitiate" : {
"_id" : "replicaSetTest",
"members" : [
{
"_id" : 0,
"host" : "debian:31000"
},
{
"_id" : 1,
"host" : "debian:31001"
},
{
"_id" : 2,
"host" : "debian:31002"
}
]
}
}
m31000| 2016-01-24T10:42:36.639+0100 I REPL [ReplicationExecutor] Member debian:31001 is now in state SECONDARY
m31000| 2016-01-24T10:42:36.639+0100 I REPL [ReplicationExecutor] Member debian:31002 is now in state SECONDARY
[...]
and it is done!
> rstest.status()
{
"set" : "replicaSetTest",
"date" : ISODate("2016-01-24T09:43:41.261Z"),
"myState" : 1,
"members" : [
{
"_id" : 0,
"name" : "debian:31000",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 329,
"optime" : Timestamp(1453628552, 1),
"optimeDate" : ISODate("2016-01-24T09:42:32Z"),
"electionTime" : Timestamp(1453628554, 1),
"electionDate" : ISODate("2016-01-24T09:42:34Z"),
"configVersion" : 1,
"self" : true
},
{
"_id" : 1,
"name" : "debian:31001",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 68,
"optime" : Timestamp(1453628552, 1),
"optimeDate" : ISODate("2016-01-24T09:42:32Z"),
"lastHeartbeat" : ISODate("2016-01-24T09:43:40.671Z"),
"lastHeartbeatRecv" : ISODate("2016-01-24T09:43:40.677Z"),
"pingMs" : 0,
"configVersion" : 1
},
{
"_id" : 2,
"name" : "debian:31002",
"health" : 1,
"state" : 2,
"stateStr" : "SECONDARY",
"uptime" : 68,
"optime" : Timestamp(1453628552, 1),
"optimeDate" : ISODate("2016-01-24T09:42:32Z"),
"lastHeartbeat" : ISODate("2016-01-24T09:43:40.672Z"),
"lastHeartbeatRecv" : ISODate("2016-01-24T09:43:40.690Z"),
"pingMs" : 0,
"configVersion" : 1
}
],
"ok" : 1
}
There are many more commands you can run, just type rstest. and then press Tab twice to get the list. Follow this link if you need more info:
http://api.mongodb.org/js/current/symbols/_global_.html#ReplSetTest
What about sharding? Pretty similar:
> var shtest = new ShardingTest({ shards: 2, mongos: 1 })
This is the documentation link if you need more info:
http://api.mongodb.org/js/current/symbols/_global_.html#ShardingTest
It is important to mention that if you close the mongo shell where you run the commands, then all the spawned mongod will also shut down.
Mtools
mtools is a collection of tools and scripts that make MongoDB’s DBA lives much easier. It includes mlaunch, which can be used to start replicate sets and sharded systems for testing.
https://github.com/rueckstiess/mtools
The mlaunch tool requires pymongo, so you need to install it:
# pip install pymongo
You can also use pip to install mtools:
# pip install mtools
Then, we can just start our replica set. In this case, with two nodes and one arbiter:
# mlaunch --replicaset --nodes 2 --arbiter --name "replicaSetTest" --port 3000 launching: mongod on port 3000 launching: mongod on port 3001 launching: mongod on port 3002 replica set 'replicaSetTest' initialized. # ps -x | grep mongod 10246 ? Sl 0:03 mongod --replSet replicaSetTest --dbpath /root/data/replicaSetTest/rs1/db --logpath /root/data/replicaSetTest/rs1/mongod.log --port 3000 --logappend --fork 10257 ? Sl 0:03 mongod --replSet replicaSetTest --dbpath /root/data/replicaSetTest/rs2/db --logpath /root/data/replicaSetTest/rs2/mongod.log --port 3001 --logappend --fork 10274 ? Sl 0:03 mongod --replSet replicaSetTest --dbpath /root/data/replicaSetTest/arb/db --logpath /root/data/replicaSetTest/arb/mongod.log --port 3002 --logappend --fork
Done. You can also deploy a shared cluster, or a sharded replica set. More information in the following link:
https://github.com/rueckstiess/mtools/wiki/mlaunch
Ognom Toolkit
“It is a set of utilities, functions and tests with the goal of making the life of MongoDB/TokuMX administrators easier.”
This toolkit has been created by Fernando Ipar and Sveta Smirnova, and includes a set of scripts that allow us to deploy a testing environment for both sharding and replication configurations. The main difference is that you can specify what storage engine will be the default, something you cannot do with other to methods.
https://github.com/Percona-Lab/ognom-toolkit
We have the tools we need under “lab” directory. Most of the names are pretty self-explanatory:
~/ognom-toolkit/lab# ls README.md start_multi_dc_simulation start_sharded_test stop_all_mongo stop_sharded_test common.sh start_replica_set start_single stop_replica_set stop_single
So, let’s say we want a replication cluster with four nodes that will use PerconaFT storage engine. We have to do the following:
Set a variable with the storage engine we want to use:
# export MONGODB_ENGINE=PerconaFT
Specify where is our mongod binary:
# export MONGOD=/usr/bin/mongod
Start our 4 nodes replica set:
# ./start_replica_set
Starting 4 mongod instances
2016-01-25T12:36:04.812+0100 I STORAGE Compression: snappy
2016-01-25T12:36:04.812+0100 I STORAGE MaxWriteMBPerSec: 1024
2016-01-25T12:36:04.813+0100 I STORAGE Crash safe counters: 0
about to fork child process, waiting until server is ready for connections.
forked process: 1086
child process started successfully, parent exiting
[...]
MongoDB shell version: 3.0.8
connecting to: 127.0.0.1:27001/test
{
"set" : "rsTest",
"date" : ISODate("2016-01-25T11:36:09.039Z"),
"myState" : 1,
"members" : [
{
"_id" : 0,
"name" : "debian:27001",
"health" : 1,
"state" : 1,
"stateStr" : "PRIMARY",
"uptime" : 5,
"optime" : Timestamp(1453721767, 5),
"optimeDate" : ISODate("2016-01-25T11:36:07Z"),
"electionTime" : Timestamp(1453721767, 2),
"electionDate" : ISODate("2016-01-25T11:36:07Z"),
"configVersion" : 4,
"self" : true
},
{
"_id" : 1,
"name" : "debian:27002",
"health" : 1,
"state" : 5,
"stateStr" : "STARTUP2",
"uptime" : 1,
"optime" : Timestamp(0, 0),
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2016-01-25T11:36:07.991Z"),
"lastHeartbeatRecv" : ISODate("2016-01-25T11:36:08.093Z"),
"pingMs" : 0,
"configVersion" : 2
},
{
"_id" : 2,
"name" : "debian:27003",
"health" : 1,
"state" : 0,
"stateStr" : "STARTUP",
"uptime" : 1,
"optime" : Timestamp(0, 0),
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2016-01-25T11:36:07.991Z"),
"lastHeartbeatRecv" : ISODate("2016-01-25T11:36:08.110Z"),
"pingMs" : 2,
"configVersion" : -2
},
{
"_id" : 3,
"name" : "debian:27004",
"health" : 1,
"state" : 0,
"stateStr" : "STARTUP",
"uptime" : 1,
"optime" : Timestamp(0, 0),
"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
"lastHeartbeat" : ISODate("2016-01-25T11:36:08.010Z"),
"lastHeartbeatRecv" : ISODate("2016-01-25T11:36:08.060Z"),
"pingMs" : 18,
"configVersion" : -2
}
],
"ok" : 1
}
Now, just start using it:
rsTest:PRIMARY> db.names.insert({ "a" : "Miguel"})
rsTest:PRIMARY> db.names.stats()
{
"ns" : "mydb.names",
"count" : 1,
"size" : 36,
"avgObjSize" : 36,
"storageSize" : 16384,
"capped" : false,
"PerconaFT" : {
[...]
Conclusion
When dealing with bugs, troubleshooting or testing some application that needs a complex MongoDB infrastructure, these processes can save us lot of time. No need of set up multiple virtual machines, deal with networking and human mistakes. Just say “I want a sharded cluster, do it for me.”
Jan
28
2016
28
2016
--
More Money Into The Cloud As NewVoiceMedia Gets $30M For Its Contact Center Solutions
As more businesses move their IT from on-premise solutions and into networked, cloud-based services, startups that are building products for them to do this are raking in yet more funding as they grow. NewVoiceMedia, a UK business that offers a solution for companies to run their customer contact centers and sales services in the cloud, has picked up an additional $30 million in funding.… Read More
Jan
27
2016
27
2016
--
MongoDB revs you up: What storage engine is right for you? (Part 4)
Differentiating Between MongoDB Storage Engines: PerconaFT
In this series of posts, we discussed what a storage engine is, and how you can determine the characteristics of one versus the other:
“A database storage engine is the underlying software that a DBMS uses to create, read, update and delete data from a database. The storage engine should be thought of as a “bolt on” to the database (server daemon), which controls the database’s interaction with memory and storage subsystems.”
Generally speaking, it’s important to understand what type of work environment the database is going to interact with, and to select a storage engine that is tailored to that environment.
The first post looked at MMAPv1, the original default engine for MongoDB (through release 3.0). The second post examined WiredTiger, the new default MongoDB engine. The third post reviewed RocksDB, an engine developed for the Facebook environment.
This post will cover PerconaFT. PerconaFT was developed out of Percona’s acquisition of Tokutek, from their TokuDB product.
PerconaFT
Find it in: Percona Builds
PerconaFT is the newest version of the Fractal Tree storage engine that was designed and implemented by Tokutek, which was acquired by Percona in April of 2015. Designed at MIT, SUNY Stony Brook and Rutgers, the Fractal Tree is a data structure that aimed to remove disk bottlenecks from databases that were using the B-tree with datasets that were several times larger that cache.
PerconaFT is arguably the most “mature” storage engine for MongoDB, with support for document level concurrency and compression. The Fractal Tree was first commercially implemented in June of 2013 in TokuMX, a fork of MongoDB, with an advanced feature set.
As described previously, the Fractal Tree (which is available for MongoDB in the PerconaFT storage engine) is a write-optimized data structure utilizing many log-like “queues” called message buffers, but has an arrangement like that of a read-optimized data structure. With the combination of these properties, PerconaFT can provide high performance for applications with high insert rates, while providing very efficient lookups for update/query-based applications. This will theoretically provide very predictable and consistent performance as the database grows. Furthermore, PerconaFT typically provides, comparatively, the deepest compression rates of any of the engines we’ve discussed in this series.
An ideal fit for the PerconaFT storage engine is a system with varied workloads, where predictable vertical scaling is required in addition to the horizontal scaling provide MongoDB. Furthermore, the ability of PerconaFT to maintain performance while compressing – along with support for multiple compression algorithms (snappy, quicklz, zlib and lzma) – make it one of the best options for users looking to optimize their data footprint.
Conclusion
Most people don’t know that they have a choice when it comes to storage engines, and that the choice should be based on what the database workload will look like. Percona’s Vadim Tkachenko performed an excellent benchmark test comparing the performances of PerconaFT and WiredTiger to help specifically differentiate between these engines.
Part 1: Intro and the MMAPv1 storage engine.
Jan
27
2016
27
2016
--
Percona CEO Peter Zaitsev discusses working remotely with Fortune Magazine
As a company that believes in and supports the open source community, embracing innovation and change is par for the course at Percona. We wouldn’t be the company we are today without fostering a culture that rewards creative thinking and rapid evolution.
Part of this culture is making sure that Percona is a place where people love to work, and can transmit their passion for technology into tangible rewards – both personally and financially. One of the interesting facts about Percona’s culture is that almost 95 percent of its employees are working remotely. Engineers, support, marketing, even executive staff – most of these people interact daily via electronic medium rather than in person. Percona’s staff is worldwide across 29 countries and 19 U.S. states. How does that work? How do you make sure that the staff is happy, committed, and engaged enough to stay on? How do you attract prospective employees with this unusual model?
It turns out that not only does it work, but it works very well. It can be challenging to manage the needs of such a geographically diverse group, but the rewards (and the results) outweigh the effort.
The secret is, of course, good communication, an environment of respect and personal empowerment.
Percona’s CEO Peter Zaitsev recently provided some of his thoughts to Fortune magazine about how our business model helps to not only to foster incredible dedication and innovation, but create a work environment that encourages passion, commitment and teamwork.
Read about his ideas on Percona’s work model here.
Oh, and by the way, Percona is currently hiring! Perhaps a career here might fit in with your plans . . .