Jul
20
2017
--

Google Cloud gets a new networking algorithm that boosts internet throughput

 Google today announced that TCP BBR, a new congestion-control algorithm is now available to its Cloud Platform users. The general idea here is to improve on the existing congestion-control algorithms for internet traffic, which have been around since the 1980s and which typically only take packet loss into account (when networking buffers fill up, routers will discard any new packets). Read More

Jul
20
2017
--

Managed private cloud stacks trying to find their way in the enterprise

 For a long time, traditional IT resisted the cloud, but that has changed in recent years as companies have come to realize that they can’t survive without the agility, scalability and economics that only a public cloud approach can provide. Yet in spite of the clear advantages the public cloud brings, there are still many companies out there that resist it for a variety of… Read More

Jul
20
2017
--

Freshdesk owner Freshworks acquires Joe Hukum as it plans a move into chatbots

 After raising $55 million last year to build its business beyond its existing help desk services, today Freshworks (the parent company of Freshdesk) has made an acquisition to help it fill out that strategy. The company has acquired Joe Hukum, a startup out of India that offers a platform for businesses to build their own chatbots. I’ve asked, but the companies are not revealing any terms… Read More

Jul
19
2017
--

Multi-Threaded Slave Statistics

Multi-Threaded Slave Statistics

Multi-Threaded Slave StatisticsIn this blog post, I’ll talk about multi-threaded slave statistics printed in MySQL error log file.

MySQL version 5.6 and later allows you to execute replicated events using parallel threads. This feature is called Multi-Threaded Slave (MTS), and to enable it you need to modify the

slave_parallel_workers

 variable to a value greater than 1.

Recently, a few customers asked about the meaning of some new statistics printed in their error log files when they enable MTS. These error messages look similar to the example stated below:

[Note] Multi-threaded slave statistics for channel '': seconds elapsed = 123; events assigned = 57345; worker queues filled over overrun level = 0; waited due a Worker queue full = 0; waited due the total size = 0; waited at clock conflicts = 0 waited (count) when Workers occupied = 0 waited when Workers occupied = 0

The MySQL reference manual doesn’t show information about these statistics. I’ve filled a bug report asking Oracle to add information about these statistics in the MySQL documentation. I reported this bug as #85747.

Before they update the documentation, we can use the MySQL code to get insight as to the statistics meaning. We can also determine how often these statistics are printed in the error log file. Looking into the rpl_slave.cc file, we find that when you enable MTS – and log-warnings variable is greater than 1 (log-error-verbosity greater than 2 for MySQL 5.7) – the time to print these statistics in MySQL error log is 120 seconds. It is determined by a hard-coded constant number. The code below shows this:

/*
  Statistics go to the error log every # of seconds when --log-warnings > 1
*/
const long mts_online_stat_period= 60 * 2;

Does this mean that every 120 seconds MTS prints statistics to your MySQL error log (if enabled)? The answer is no. MTS prints statistics in the mentioned period depending on the level of activity of your slave. The following line in MySQL code verifies the level of the slave’s activity to print the statistics:

if (rli->is_parallel_exec() && rli->mts_events_assigned % 1024 == 1)

From the above code, you need MTS enabled and the modulo operation between the 

mts_events_assigned

 variable and 1024 equal to 1 in order to print the statistics. The 

mts_events_assigned

 variable stores the number of events assigned to the parallel queue. If you’re replicating a low level of events, or not replicating at all, MySQL won’t print the statistics in the error log. On the other hand, if you’re replicating a high number of events all the time, and the

mts_events_assigned

 variable increased its value until the remainder from the division between this variable and 1024 is 1, MySQL prints MTS statistics in the error log almost every 120 seconds.

You can find the explanation these statistics below (collected from information in the source code):

  1. Worker queues filled over overrun level: MTS tends to load balance events between all parallel workers, and the  
    slave_parallel_workers

     variable determines the number of workers. This statistic shows the level of saturation that workers are suffering. If a parallel worker queue is close to full, this counter is incremented and the worker replication event is delayed in order to avoid reaching worker queue limits.

  2. Waited due to a Worker queue full: This statistic is incremented when the coordinator thread must wait because of the worker queue gets overfull.
  3. Waited due to the total size: This statistic shows the number of times that the coordinator thread slept due to reaching the limit of the memory available in the worker queue to hold unapplied events. If this statistic keeps increasing when printed in your MySQL error log, you should resize the
    slave_pending_jobs_size_max

     variable to a higher value to avoid the coordinator thread waiting time.

  4. Waited at clock conflicts: In the case of possible dependencies between transactions, this statistic shows the wait time corresponding to logical timestamp conflict detection and resolution.
  5. Waited (count) when used occupied: A counter of how many times the coordinator saw Workers filled up with “enough” with assignments. The enough definition depends on the scheduler type (per-database or clock-based).
  6. Waited when workers occupied: These are statistics to compute coordinator thread waiting time for any worker available, and applies solely to the Commit-clock scheduler.
Conclusion

Multi-threaded slave is an exciting feature that allows you to replicate events faster, and keep in sync with master instances. By changing the log-warnings variable to a value greater than 1, you can get information from the slave error log file about how multi-threaded performance.

Jul
19
2017
--

LiveStories nabs $10M to organise, analyse and visualise fragmented civic data

 Big data analytics continues to be a key driver in how organisations utilise their vast quantities of information to run their businesses, and now a startup that has built a big data platform specifically focusing on data from the public sector has raised a round of funding.
LiveStories, a Seattle-based company that works with governments, education institutions and other public entities to… Read More

Jul
19
2017
--

Jeremy Bloom’s Integrate raises $8 million for marketing software

 You may know Jeremy Bloom from his days as an Olympian on the U.S. ski team or from when he was a professional football player. These days, he’s busy running an enterprise software startup for marketers. With clients like Salesforce and Dell, Phoenix-based Integrate has raised $8 million in funding from Iron Gate Capital and existing investors. Read More

Jul
19
2017
--

Clara Labs nabs $7M Series A as it positions its AI assistant to meet the needs of enterprise teams

 Clara Labs is announcing a $7 million Series A led by Basis Set Ventures. Slack Fund also joined in the round, alongside existing investors Sequoia and First Round. The startup will be looking to further differentiate within the crowded field of email-centric personal assistants by building in features and integrations to address the needs of enterprise teams. Read More

Jul
19
2017
--

Google Glass never really left

 When Google released a new enterprise version of Glass on Tuesday, a bevy of headlines suggested Glass was back. But it never really left. Even though the Explorer program aimed at consumers ended in January, 2015, Google has continued to sell Glass to businesses. In that context, yesterday’s release wasn’t a comeback so much as a continuation of an enterprise strategy for Glass. Read More

Jul
19
2017
--

The Open Container Initiative launches version 1.0 of its container specs

 It took a while, but the Open Container Initiative (OCI) today announced the launch of the 1.0 versions of both its container runtime and image specs for software container. The two-year-old open source foundation was established by Docker and other leaders in the container ecosystem to be the guardian of exactly these specifications, which are basically the industry standards for… Read More

Jul
18
2017
--

IBM expands its cloud footprint with new data centers in London, Sydney and San Jose

 IBM reported its quarterly earnings this week and, while the company’s overall results were once again disappointing, cloud revenue was up 15 percent year-over-year and accounted for $3.9 billion in revenue. It’s no surprise, then, that IBM is doubling down on its cloud strategy; to keep its momentum going, the company today announced the launch of four new data centers for its… Read More

Powered by WordPress | Theme: Aeros 2.0 by TheBuckmaker.com