Personal blog of Yzmir Ramirez

FOSDEM is for sure one of the most respected free and open source software-focused conferences in Europe, which is loved by many, myself included. It is from the point of love I am raising concerns about transparency, which seems to be lacking in regard to some processes at FOSDEM.For many years, as long as I […]

If you have not heard, this weekend is FOSDEM.  FOSDEM this year is virtual (and open to all), and it looks like it is going to be an absolutely stellar lineup of open source content.  The details:  February 6-7th, 2021, with over 600 speakers, over 700 sessions, and 113 tracks!  So much to see.   Percona is once again participating in this year’s festivities, with a total of eight speakers across 13 total sessions!  

• Marco Tusa: Review of the Percona Operator from a DBA perspective 
• Peter Zaitsev: 18 Things To Do When You Have a MySQL Bottleneck, MariaDB Observability, Hybrid Cloud Open Source DBaaS with Kubernetes, Why Public Database as a Service is Prime for Open Source Distribution, Performance Analysis and Troubleshooting Methodologies for Databases
• Ibrar Ahmed: Pg_stat_monitor – The new way to analyze query performance in PostgreSQL
• Sanja Bonic: Tools and Concepts for Successfully Open Sourcing Your Project, Watch the Asciinema Replay of Your Home-Made Honeypot
• Matt Yonkovit: The Death of Openness and Freedom? Open Source Under Attack
• Sveta Smirnova: MySQL Performance for DevOps 
• Agustín Gallego: What do we want to monitor? All the databases!
• Dmitriy Kostiuk: Community-accessible EEG monitoring of the user’s mental state in the UX/UI research
  

The schedule specifically for Percona speakers is available here:
https://www.percona.com/community/events/fosdem-2021

But I see several Percona alums are speaking at FOSDEM as well! It will be good to hear from former Percona colleagues like:

• Shlomi Noach (PlanetScale) – Open Source Database Infrastructure with Vitess
• Frédéric Descamps (Oracle) – From single MySQL instance to HA The journey to InnoDB Cluster, MySQL Router REST API, 
• Valerii Kravchuk (MariaDB) – Linux /proc filesystem for MySQL DBAs, Sampling /proc content for troubleshooting, Monitoring MariaDB Server with bpftrace on Linux
• Liz van Dijk (PlanetScale) – Hitchhiker’s Guide to MySQL Don’t panic
• Krunal Bauskar (Huawei) – Running MySQL on ARM, Migrating MariaDB Cluster to ARM
• Federico Razzoli (Vettabase)- Creating Vagrant development machines for MariaDB HowTo and Best Practices
• Alkin Tezuysal (PlanetScale)- How to shard MariaDB like a Pro?

Looking through the schedule, here are the HOSS’s picks for must-see sessions:

1. Valerii Kravchuk (MariaDB) – Linux /proc filesystem for MySQL DBAs – Sampling /proc content for troubleshooting:  Valerii has been in the MySQL/MariaDB ecosystem for as long as I can remember working for MySQL, Oracle, Percona, and now MariaDB.  He knows the ins and outs of troubleshooting the most complex problems and bugs imaginable.  For me, this is a MUST-attend session because you will get a good deep understanding of Linux performance.  His talk on BPFtrace is also excellent, and I would recommend it to anyone looking for deep performance.
2. Sveta Smirnova(Percona): MySQL Performance for DevOps:  Sveta has been solving the toughest problems the database sphere has to throw at her for years at MySQL, then Oracle, and now at Percona.  She has seen the performance issues of thousands of customers and knows the trends in the community.  Her research, knowledge, and wisdom on all things Databases is definitely something not to be missed for your database geeks!
3. Matthew Hodgson (Matrix) – Building massive virtual communities in Matrix or… Spaces: The Final Frontier! I am very intrigued by Matrix and their stated goal of “Our goal is to ensure no open source project ever uses Slack/Discord/Telegram to collaborate ever again. “.  Let’s face it; Slack is expensive and as corporate as ever, and open source alternatives are great!
4. Ibrar Ahmed (Percona) & Peter Zaitsev (Percona): Pg_stat_monitor – The new way to analyze query performance in PostgreSQL: I am an absolute sucker for performance.  I know more about pg-stat-monitor than many because I work at Percona, but setting aside my corporate hat… people who run PostgreSQL and care about performance and observability should definitely check this one out.  
5. Oleg Ivanov (Samsung AI) & Yerzhaisang Taskali:  Adaptive Query Optimization in PostgreSQL: approaches and challenges: Did you know that I first learned the internals of MySQL and InnoDB hacking debugging the optimizer and the statistics generation?  Some things pique my interest, and this talk is one of them.  Oleg and Yerzhaisang will present the work they are doing to enhance and improve the statistics collected and the overall query plan.  Very excited to see this.  
6. Stefan Hajnoczi – The Evolution of File Descriptor Monitoring in Linux From select(2) to io_uring: Deep Linux discussion on the performance and evolution of some of the most critical and some of the lowest level calls… it has the potential to be very eye-opening. Hopefully, it can give us some new and interesting insights!
7. Krunal Bauskar (Huawei) – Running MySQL on ARM: the team here at Percona ran performance benchmarks on ARM in PostgreSQL, and we were blown away by not only the price but the potential price difference in cloud compute options!  I want to see the work going on here with MySQL.  
8. Shlomi Noach (PlanetScale) – Open Source Database Infrastructure with Vitess: Vitess is a hot topic for many in the MySQL community.   It provides great scalability and performance.  This is a can’t miss if you have not heard about Vitess before!
9. Peter Zaitsev (Percona): 18 Things To Do When You Have a MySQL Bottleneck: I know, I know… you may feel I am biased and kissing up to my Boss by putting him on my must watch list, but there is an excellent reason.  Peter’s series on the 18 things to do when you have a MySQL Bottleneck was the top viewed and top shared MySQL blog in 2020 from Percona.  Honestly, those who have heard Peter speak know his knowledge and deep understanding.  
10. The HOSS!  (Percona):  The Death of Openness and Freedom? Open Source Under Attack: Ok, it may seem like a shameless plug, and maybe it is… but right now, I am very concerned about the way that corporations are dismantling open source software and turning it into something that looks like open source, but smells like a pile of poo.  So come on and watch the HOSS rant for 40 minutes about how evil corporate greed threatens our passion! 

That’s it!  Let me say FOSDEM is one of the most open and community-driven conferences each year, and this year looks no different!  I hope to see you there!

For those not familiar with it, FOSDEM is an amazing, free entry, full on celebration of open source that takes place in Brussels, Belgium every year. This year the event was held over the first weekend of February. Fringe events, such as the Pre-FOSDEM MySQL day hosted by Oracle MySQL, and the community dinner that follows, provide an opportunity to network.

In case you didn’t make it to FOSDEM this year, here are links to Percona’s presentations from the event. Organizers video and share online every talk from every dev room, a phenomenal achievement in itself. All credit to the volunteers who run this show.

Database Dev Room: Hugepages and databases presented by Fernando Laudares Camargos

 

MySQL, MariaDB and Friends Dev Room: MySQL Replication – Advanced Features presented by Peter Zaitsev

 

MySQL, MariaDB and Friends Dev Room: MySQL Performance Schema in 20 Minutes presented by Sveta Smirnova

 

Monitoring and Observability Dev Room: Using eBPF for Linux Performance Analyses by Peter Zaitsev

Percona enjoyed plenty of attention in the booth area where shared information about our open source, free-as-in-beer projects. We were in Brussels after all!

Passing artist and open source enthusiast Sandra Dannenberg took a liking to our Percona logos and painted her own versions. They’re great aren’t they? FOSDEM is that kind of event… we’re looking forward already to 2020!

Join Percona Chief Evangelist Colin Charles as he covers happenings, gives pointers and provides musings on the open source database community.

Percona Live Santa Clara 2018 update: tutorials have been announced. The committee rated over 300+ talks, and easily 70% of the schedule should go live next week as well. In practice, then, you should see about 50 talks announced next week. There’s been great competition: we only have 70 slots in total, so about 1 in 5 talks get picked — talk about a competitive ratio.

FOSDEM

FOSDEM was truly awesome last week. From a Percona standpoint, we had a lot of excellent booth traffic (being outside of the PostgreSQL room on Saturday, and not too far out from the MySQL room on Sunday). We gave away bottle openers — useful in Brussels with all the beer; we tried a new design with a magnet to attach it to your fridge — stickers, some brochures, but most of all we had plenty of great conversations. There was quite a crowd from Percona, and it was excellent to see the MySQL & Friends DevRoom almost constantly full! A few of us brave souls managed to stay there the whole day, barely with any breaks, so as to enjoy all the talks.

I find the quality of talks to be extremely high. And when it comes to a community run event, with all content picked by an independent program committee, FOSDEM really sets the bar high. There is plenty of competition to get a good talk in, and I enjoyed everything we picked (yes, I was on the committee too). We’ve had plenty of events in the ecosystem that sort of had “MySQL” or related days, but FOSDEM might be the only one that has really survived. I understand we will have a day of some sort at SCALE16x, but even that has been scaled down. So if you care about the MySQL ecosystem, you will really want to ensure that you are at FOSDEM next year.

This year, we started with the usual MySQL Day on Friday. I could not be present, as I was at the CentOS Dojo, giving a presentation. So, the highlight of Friday for me? The community dinner. Over 80 people showed up, I know there was a waiting list, and lots of people were trying to get tickets at the last minute. Many missed out too; sorry, better luck next year; and also, hopefully, we will get a larger venue going forward. I really thank the organizers for this — we affectionately refer to them as the Belconians (i.e. a handful of Perconians based in Belgium). The conversation, the food, the drink —  they were all excellent. It’s good to see representation from all parts of the community: MySQL, Percona, MariaDB, Pythian, and others. So thank you again, Liz, Dimitri, Tom, and Kenny in absentia. I think Tjerk also deserves special mention for always helping (this year with the drinks)

As for FOSDEM itself, beyond the booth, I think the most interesting stuff was the talks. There are video recordings and slides of pretty much all talks, but I will also give you the “Cliff’s Notes” of them here.

MySQL DevRoom talk quick summaries

Beyond WHERE and GROUP BY – Sergei Golubchik

• EXCEPT is in MariaDB Server 10.3
• recursive CTEs are good for hierarchical data, graphs, data generation, Turing complete (you can use it to solve Sudoku even)
• non-recursive CTEs can be an alternative syntax for subqueries in the FROM clause
• Window functions:
  • Normal: one result per row, depend on that row only
  • Aggregate: one result per group, depending on the whole group
  • Window: one result per row, depending on the whole group
• System versioned tables with AS OF
• Aggregate stored functions

MySQL 8.0 Performance: InnoDB Re-Design – Dimitri Kravtchuk

• Contention-Aware Transactions Scheduling (CATS), since 8.0.3. Not all transactions are equal, FIFO could not be optimal, unblock the most blocking transactions first
• CATS (VATS) had a few issues, and there were bugs (they thought everything worked since MariaDB Server had implemented it). They spent about 9 months before fixing everything.
• Where does CATS help? Workloads hitting row lock contentions. You can monitor via SHOW ENGINE INNODB MUTEX.
• the main problem is because of repeatable read versus read committed transaction isolation on the same workload. You really need to understand your workload when it comes to VATS.

MySQL 8.0 Roles – Giuseppe Maxia

• Created like a user, granted like privileges. You need to activate them to use them.
• Before roles, you created a user, then grant, grant, and more grant’s… Add another user? Same deal. Lots of repetitive work and a lot of chances to make mistakes.
• Faster user administration – define a role, assign it many times. Centralized grant handling – grant and revoke privileges to roles, add/edit all user profiles.
• You need to remember to set the default role.
• A user can have many roles; default role can be a list of roles.
• Roles are users without a login – roles are saved in user tables. This is useful from an account lock/unlock perspective.
• You can grant a user to a user
• SET ROLE is for session management; SET DEFAULT ROLE is a permanent assignment of a role for a user. SET ROLE DEFAULT means assign the default role for this user for this session
• The role_edges table reports which roles are assigned to which users. default_roles keeps track of the current default roles assigned to users. A default role may not exist.

Histogram support in MySQL 8.0 – Øystein Grøvlen

• You can now do ANALYZE TABLE table UPDATE HISTOGRAM on column WITH n BUCKETS;
• New storage engine API for sampling (default implementation is full table scan even when sampling)
• Histogram is stored in a JSON column in the data dictionary. Grab this from the INFORMATION_SCHEMA.
• Histograms are useful for columns that are not the first column of any index, and used in WHERE conditions of JOIN queries, queries with IN-subqueries, ORDER BY … LIMIT queries. Best fit: low cardinality columns (e.g. gender, orderStatus, dayOfWeek, enums), columns with uneven distribution (skew), stable distribution (do not change much over time)
• How many buckets? equi-height, 100 buckets should be enough.
• Histograms are stored in the data dictionary, so will persist over restarts of course.

Let’s talk database optimizers – Vicen?iu Ciorbaru

• Goal: produce a query plan that executes your query in the fastest time possible.
• Condition pushdown through PARTITION BY, there is also a comparison with PostgreSQL
• Split grouping for derived optimization only in MariaDB 10.3
• ?Read more at splitgroupingderived=on — https://github.com/MariaDB/server/commit/b14e2b044b (and the equivalent MDEV: https://jira.mariadb.org/browse/MDEV-13369)?

TLS for MySQL at Large Scale – Jaime Crespo

• Literally took 3 lines in the my.cnf to turn on TLS
• https://dbtree.wikimedia.org
• They wanted to do a data centre failover and wanted to ensure replication would be encrypted.
• They didn’t have proper orchestration in place (MySQL could have this too). Every time OpenSSL or MySQL had to be upgraded, the daemon needed restarting. If there was an incompatible change, you had to sync master/replicas too.
• The automation and orchestration that Wikipedia uses: https://fosdem.org/2018/schedule/event/cumin_automation/ (it is called Cumin: https://wikitech.wikimedia.org/wiki/Cumin)
• Server support was poor – OpenSSL – so they had to deploy wmf-mysql and wmf-mariadb of their own
• Currently using MariaDB 10.0, and looking to migrate to MariaDB 10.1
• Client library pain they’ve had
• TLSv1.2 from the beginning (2015).
• 20-50x slower for actual connecting; the impact is less than 5% for the actual query performance. Just fix client libraries, make them use persistent connections. They are now very interested in ProxySQL for this purpose.
• https://grafana.wikimedia.org/?orgId=1
• Monty asks, would a double certificate help? Jaime says sure. But he may not actually use double certificates; might not solve CA issues, and the goal is not to restart the server.
• Monty wonders why not to upgrade to 10.2? “Let’s talk outside because it’s a much larger question.”

MySQL InnoDB Cluster – Miguel Araújo

• group replication: update everywhere (multi-master), virtually synchronous replication, automatic server failover, distributed recovery, group reconfiguration, GCS (implementation of Paxos – group communication system). HA is a critical factor.
• mysqlsh: interactive and batch operations. Document store (CRUD and relational access)
• admin API in mysqlsh: create & manage clusters, hide complexity of configuration/provisioning/orchestration of the InnoDB clusters. Works with JavaScript and Python
• Usability. HA out of the box.
• It’s easy to join a new node; new node goes into recovery mode (and as long as you have all the binary logs, this is easy; otherwise start from a backup)
• SET PERSIST – run a command remotely, and the configuration is persisted in the server
• Network flapping? Group replication will just reject the node from the cluster if its flapping too often

Why we’re excited about MySQL 8 – Peter Zaitsev

• Native data dictionary – atomic, crash safe, DDLs, no more MyISAM system table requirements
• Fast INFORMATION_SCHEMA
• utf8mb4 as default character set
• Security: roles, breakdown of SUPER privileges, password history, faster cached-SHA2 authentication (default), builds using OpenSSL (like Percona Server), skip grants blocks remote connections, logs now encrypted when tablespace encryption enabled
• Persistent AUTO_INCREMENT
• auto-managed undo tablespaces – do not use system table space for undo space. Automatically reclaim space on disks.
• Self-tuning, limited to InnoDB (innodb_dedicated_server to auto-tune)
• partial in-place update for JSON – update filed in JSON object without full rewrite. Good for counters/statuses/timestamps. Update/removal of element is supported
• Invisible indexes – test impact of dropping indexes before actually dropping them. Maintained but unused by the optimizer. If not needed or used, then drop away.
• TmpTable Storage Engine – more efficient storage engine for internal temporary tables. Efficient storage for VARCHAR and VARBINARY columns. Good for GROUP BY queries. Doesn’t support BLOB/TEXT columns yet (this reverts to InnoDB temp table now)
• Backup locks – prevent operations which may result in inconsistent backups. CHECK INSTANCE FOR BACKUP (something Percona Server has had before)
• Optimizer histograms – detailed statistics on columns, not just indexes
• improved cost model for the optimizer – www.unofficialmysqlguide.com
• Performance schematic – faster (via “fake” indexes), error instrumentation, response time histograms (global & per query), digest summaries
• select * from sys.session – fast potential replacement for show processlist
• RESTART (command)
• SET PERSIST – e.g. change the buffer pool size, and this helps during a restart
• assumes default storage is SSD now
• binary log on by default, log_slave_updates enabled by default, and log expires after 30 days by default
• query cache removed. Look at ProxySQL or some other caching solution
• native partitioning only – remove partitions from MyISAM or convert to InnoDB
• resource groups – isolation and better performance (map queries to specific CPU cores; can jail your costly queries, like analytical queries)
• Feature Requests: better single thread performance, no parallel query support

MySQL Test Framework for Support and Bugs Work – Sveta Smirnova

• MTR allows you to add multiple connections
• has commands for flow control

ProxySQL – GTID Consistent Reads – René Cannaò, Nick Vyzas

• threshold is configurable in increments of 1 second. Replication lag can be monitored with ProxySQL. Want to ensure you don’t have stale reads.
• Why is GTID important? To guarantee consistently. Auto positioning for restructuring topologies.
• –session-track-gtids is an important feature which allows sending the GTID for a transaction on the OK packet for a transaction. Not available in MariaDB.
• There is a ProxySQL Binlog Reader now – GTID information about a MySQL server to all connected ProxySQL instances. Lightweight process to run on your MySQL server.
• ProxySQL can be configured to enforce GTID consistency for reads on any hostgroup/replication hostgroup.
• Live demo by René

Turbocharging MySQL with Vitess – Sugu Sougoumarane

• trend for the cloud: container instances, short-lived containers, tolerate neighbors, discoverability. No good tools yet for Kubernetes.
• non-ideal options: application sharing, NoSQL, paid solutions, NewSQL (CockroachDB, TiDB, Yugabyte)
• Vitess: leverage MySQL at massive scale, opensource, 8+ years of work, and multiple production examples
• Square uses Vitess for Square Cash application.
• Can MySQL run on Docker? Absolutely, many of the companies do huge QPS on Docker.
• YouTube does a major re-shard every 2-3 months once. No one notices nowadays when that happens.
• app server connects to vtgate, and only underneath it’s a bunch of smaller databases with vttablet + mysqld. The lockserver is what makes it run well in the cloud.
• pluggable architecture with no compromise on performance: monitoring, health check, ACLs, tracing, more.
• at most, it adds about 2ms overhead to connections
• Go coding standards are enforced, unit tests with strict coverage requirements, end-to-end tests, Travis, CodeClimate and Netlify. Readability is king.
• On February 5 2018, it will be a CNCF project. One year of due diligence. They said there was nothing to compare it with. Looked at maturity and contributors. It’s becoming a truly community-owned project! (CNCF to Host Vitess is already live as of now)
• roadmap: full cross-shard queries, migration tools, simplify configurability, documentation.
• full MySQL protocol, but a limited query set – they want to get it to a point where it accepts a full MySQL query.

Orchestrator on Raft – Shlomi Noach

• Raft: guaranteed to be in-order replication log, an increasing index. This is how nodes choose a leader based on who has the higher index. Get periodic snapshots (node runs a full backup).
• HashiCorp raft, a Golang raft implementation, used by Consul
• orchestrator manages topology for HA topologies; also want orchestrator to be highly available. Now with orchestrator/raft, remove the MySQL backend dependency, and you can have data center fencing too. Now you get: better cross-DC deploys, DC-local KV control, and also Kubernetes friendly.
• n-orchestrator nodes, each node still runs its own backend (either MySQL or SQLite). Orchestrator provides the communication for SQLite between the nodes. Only one (the Raft leader) will handle failovers
• implementation & deployment @ Github – one node per DC (deployed at 3 different DCs). 1-second raft polling interval. 2 major DCs, one in the cloud. Step-down, raft-yield, SQLite-backed log store, and still a MySQL backend (SQLite backend use case is in the works)
• They patched the HashiCorp raft library. The library doesn’t care about the identity of nodes, with Github they do want to control the identity of the leader. There is an “active” data center, and locality is important. This is what they mean by raft-yield (picking a candidate leader).
• The ability for a leader to step down is also something they had to patch.
• HashiCorp Raft only supports LMDB and another database, so the replication log is now kept in a relational SQLite backed log store. Another patch.
• once orchestrator can’t run its own self-health check, it recognizes this. The application can tell raft now that it’s stepping down. Takes 5 seconds to step down, and raft then promotes another orchestrator node to be the leader. This is their patch.
• can also grab leadership
• DC fencing handles network partitioning.
• orchestrator is Consul-aware. Upon failover, orchestrator updates Consul KV with the identity of the promoted master.
• considerations to watch out for: what happens if, upon replay of the Raft log, you hit two failovers for the same cluster? NOW() and otherwise time-based assumptions. Reapplying snapshot/log upon startup
• roadmap: use Kubernetes (cluster IP based configuration in progress, already container friendly via auto-re-provisioning of nodes via Raft)

MyRocks Roadmaps – Yoshinori Matsunobu

• Facebook has a large User Database (UDB). Social graph, massively sharded, low latency, automated operations, pure flash storage (constrained by space, not CPU/IOPS)
• They have a record cache in-front of MySQL – Tao for reads. If cache misses, then it hits the database. And all write requests go thru MySQL. UDB has to be fast to ensure a good user experience.
• they also at Facebook run 2 instances of MySQL on the same machine, because CPU wasn’t huge, but the space savings were awesome.
• design decisions: clustered index (same as InnoDB), slower for reads, faster for writes (bloom filters, column family), support for transactions including consistency between binlog and MyRocks. Faster data loading/deletes/replication, dynamic options (instead of having to restart mysqld), TTL (comparable to HBase TTL feature, specify the TTL, any data older than time, can be removed), online logical (for recovery purposes) & binary backup (for creating replicas)
• Pros: smaller space, better cache hit rate, writes are faster so you get faster replication, much smaller bytes written
• Cons: no statement based replication, GAP locks, foreign keys, full-text index, spatial index support. Need to use case sensitive collations for performance. Reads are slower, especially if the data fits in memory. Dependent on file system and OS; lack of solid direct I/O (uses buffered I/O). You need a newer than 4.6 kernel. Too many tuning options beyond buffer pool such as bloom filter, compactions, etc.
• https://twitter.com/deniszh/status/960163082642382849
• Completed InnoDB to MyRocks migration. Saved 50% space in UDB compared to compressed InnoDB.
• Roadmaps: getting in MariaDB and Percona Server for MySQL. Read Mark’s blog for matching read performance vs InnoDB. Supporting mixed engines. Better replication and bigger instance sizes.
• mixed engines: InnoDB and MyRocks on the same instance, though single transaction does not overlap engines. Plan to extend star backup to integrate `myrocks_hotbackup. Backport gtid_pos_auto_engines from MariaDB?
• Removing engine log. Could be caused by binlog and engine log, which requires 2pc and ordered commits. Use one log? Either binlog or binlog like service or RocksDB WAL? Rely on binlog now (semi-sync, binlog consumers), need to determine how much performance is gained by stopping writing to WAL.
• Parallel replication apply is important in MySQL 8
• support bigger instance sizes: shared nothing database is not a general purpose database. Today you can get 256GB+ RAM and 10TB+ flash on commodity servers. Why not run one big instance and put everything there? Bigger instances may help general purpose small-mid applications. Then you don’t have to worry about sharing. Atomic transactions, joins and secondary keys will just work. Amazon Aurora today supports a 60TB instance!
• today: you can start deploying slaves with consistency check. Many status counters for instance monitoring.

ProxySQL internals – René Cannaò

• reduce latency, scales, maximize throughput. Single instance to travel hundreds of thousands of connections and to handle thousands of backend servers.
• threading models: one thread per connection (blocking I/O), thread pooling (non-blocking I/O, scalable).
• ProxySQL thread pool implementation: known as “MySQL threads”, fixed number of worker threads (configurable), all threads listen on the same port(s), client connections are not shared between threads, all threads perform their own network I/O, and it uses poll() (does that scale? True, but there is a reason why poll over epoll)
• threads never share client connections – no need for synchronization, thread contention is reduced, each thread calls poll(). Possibly imbalanced load as a con (one thread that has way more connections that another). Is it really a problem? Most of the time, no, connections will automatically balance.
• poll() is O(N), epoll() is O(1). Poll() is faster than epoll() for fewer connections (around 1000). Performance degrees when there are a lot of connections. So by default, it uses poll() instead of epoll(), around 50,000 connections performance degrades badly – so ProxySQL has auxiliary threads.
• MySQL_Session() is implemented as a state machine. Stores metadata associated with the client session (running timers, default hostgroup, etc.)

MySQL Point-in-time recovery like a rockstar – Frederic Descamps

• to do PITR, you need a decent backup!
• manual tells you about PITR – read it well
• https://bugs.mysql.com/bug.php?id=89375
• github.com/lefred/MyUndelete

Releases

• Percona Monitoring and Management 1.7.0 (PMM) – This release features improved support for external services, which enables a PMM Server to store and display metrics for any available Prometheus exporter. For example, you could deploy the postgres_exporter and use PMM’s external services feature to store PostgreSQL metrics in PMM. Immediately, you’ll see these new metrics in the Advanced Data Exploration dashboard. Then you could leverage many of the pre-developed PostgreSQL dashboards available on Grafana.com, and with a minimal amount of edits have a working PostgreSQL dashboard in PMM!
• MariaDB Server 10.1.31 – usual updates to storage engines, and a handful of bug fixes.

Link List

• Amazon Redshift Spectrum – are you using this yet?
• Database sharding explained in plain English
• Modern SQL Window Function Questions – a quiz that’s quite poignant (MariaDB 10.2+ has window functions; upcoming MySQL 8 has it too).

Upcoming appearances

• SCALE16x – Pasadena, California, USA – March 8-11 2018

Feedback

I look forward to feedback/tips via e-mail at colin.charles@percona.com or on Twitter @bytebot.

Join Percona Chief Evangelist Colin Charles as he covers happenings, gives pointers and provides musings on the open source database community.

Percona Live Santa Clara 2018 update: tutorials have been picked, and the schedule/press release should be announced by next week. We’ve (the committee) rated over 300+ talks, and easily 70% of the schedule should go live next week as well.

There’s a lot happening for FOSDEM this week — so expect a longer report of some sort next week.

A friend, Yanwei Zhou, DBA at Qunar in China, gave an excellent presentation in Chinese on how they use Percona Toolkit. Check it out:

Are you on Twitter? Hope you’re following the @planetmysql account.

Releases

• Percona XtraDB Cluster 5.7.20-29.24 – new features, bug fixes, and a base of Percona Server 5.7.20-18.
• orchestrator 3.0.6 – faster failure detection, faster master recoveries, semi-sync support, and more. This is getting to be a real interesting release.
• MariaDB Server 10.0.34 – new XtraDB, InnoDB, TokuDB, and even PERFORMANCE_SCHEMA. Security and InnoDB fixes as well.
• Percona Server for MySQL 5.5.59-38.11 – bug fixes

Link List

• Considering the Community Effects of Introducing an Official MongoDB Go Driver – What do you do when an open-source project you rely on no longer meets your needs? When your choice affects not just you, but a larger community, what principles guide your decision?
• Postgres Indexes Under the Hood
• It’s About Time For Time Series Databases
• Database Design Decisions for Multi-Version Concurrency Control
• How to convert galera node to async slave and vice-versa with MariaDB Galera Cluster.
• DO or UNDO – there is no VACUUM – really important article from a PostgreSQL standpoint. Recommended read.
• Optimising graph lookups in MongoDB

Upcoming appearances

• CentOS Dojo Brussels – Brussels, Belgium – February 2 2018
• FOSDEM 2018 – Brussels, Belgium – February 3-4 2018
• SCALE16x – Pasadena, California, USA – March 8-11 2018

Feedback

I look forward to feedback/tips via e-mail at colin.charles@percona.com or on Twitter @bytebot.

Join Percona Chief Evangelist Colin Charles as he covers happenings, gives pointers and provides musings on the open source database community.

I think the biggest news from last week was from cPanel – if you haven’t already read the post, please do – on Being a Good Open Source Community Member: Why we hesitated on MySQL 5.7. cPanel anticipated MariaDB being the eventual replacement for MySQL, based on movements from Red Hat, Wikipedia and Google. The advantage focused on transparency around security disclosure, and the added features/improvements. Today though, “MySQL now consistently matches or outpaces MariaDB when it comes to development and releases, which in turn is increasing the demand on us for providing those upgraded versions of MySQL by our users.” And maybe a little more telling, “when MariaDB 10.2 became stable in May 2017 it included many features found in MySQL 5.7. However, MySQL reached stable nearly 18 months earlier in October 2015.” (emphasis mine).

So cPanel is going forth and supporting MySQL 5.7. They will continue supporting MariaDB Server for the foreseeable future. This really is cPanel ensuring they are responsive to users: “The people using and building database-driven applications are doing so with MySQL in mind, and are hesitant to add support for MariaDB. Responding to our community’s desires is one of the most important things to us, and this is something that we are hearing asked for from our community consistently.”

I, of course, think this is a great move. Users deserve choice. And MySQL has features that are sometimes still not included in MariaDB Server. Have you seen the Complete list of new features in MySQL 5.7? Or my high-level response to a MariaDB Corporation white paper?

I can only hope to see more people think pragmatically like cPanel. Ubuntu as a Linux distribution still does – you get MySQL 5.7 as a default (very unlike the upstream Debian which ships MariaDB Server nowadays). I used to be a proponent of MariaDB Server being everywhere, when it was community-developed, feature-enhanced, and backward-compatible. However, the moment it stopped being a branch and a true fork is the moment where trouble lies for users. I think it was still marginally fine with 10.0, and maybe even 10.1, but the ability to maintain feature parity with enhanced features has long gone. Short of a rebase? But then… what would be different to the already popular branch of MySQL called Percona Server for MySQL?

While there are wins and support from cloud vendors, like Amazon AWS RDS and Microsoft Azure, you’ll notice that they offer both MySQL and MariaDB Server. Google Cloud SQL notably only offers MySQL. IBM may be a sponsor of the MariaDB Foundation, but I don’t see their services like Compose offering anything other than MySQL (with group replication nonetheless!). Platinum member Alibaba Cloud offers MySQL and PostgreSQL. However, Tencent seems to suggest that MariaDB is coming soon? One interesting statistic to watch would be user uptake naturally.

Events

From an events standpoint, the Percona Live 2018 Call for Papers has been extended to January 12, 2018. We expect an early announcement of maybe ten talks in the week of  January 5. Please submit to the CFP. Have you got your tickets yet? Nab them during our Percona Live 2018 super saver registration when they are the best price!

FOSDEM has got Sveta and myself speaking in the MySQL and Friends DevRoom, but we also have good news in the sense that Peter Zaitsev is also going to be at FOSDEM – speaking in the main track. We’ll also have plenty of schwag at the stand.

I think it’s important to take note of the updates to Percona bug tracking: yes, its Jira all the way. Would be good for everyone to start also looking at how the sausage is made.

Dragph, a “distributed fast graph database“, just raised $3m and released 1.0. Have you used it?

On a lighter note, there seems to be a tweet going around by many, so I thought I’d share it here. Merry Christmas and Happy Holidays.

He’s making a database
He’s sorting it twice
SELECT * FROM girls_boys WHERE behaviour = “nice”
SQL Claus is coming to town!

Releases

• Percona Monitoring and Management 1.5.3
• ProxySQL 1.4.4 – some interesting features include bandwidth throttling, limit connections to backends, monitoring replication lag, and more.
• MariaDB Server 10.2 is in CentOS 6 and CentOS 7 via Software Collections.

Link List

• Good presentation on multi-user Presto usage.
• You probably must be dabbling with containers, so Google released container-diff, a tool for quickly comparing container images. Check it out on GitHub.
• James Governor (RedMonk), writes On AWS and Pivotal, opinions and overlaps.
• That time Larry Ellison allegedly tried to have a professor fired for benchmarking Oracle
• Apache Cassandra users, should you use incremental repair?

Upcoming appearances

• FOSDEM 2018 – Brussels, Belgium – February 3-4 2018
• SCALE16x – Pasadena, California, USA – March 8-11 2018

Feedback

I look forward to feedback/tips via e-mail at colin.charles@percona.com or on Twitter @bytebot.

Join Percona Chief Evangelist Colin Charles as he covers happenings, gives pointers and provides musings on the open source database community.

Hurry up – the call for papers (CFP) for FOSDEM 2018 ends December 1, 2017. I highly recommend submitting as its a really fun, free, and technically-oriented event.

Don’t forget that the CFP for Percona Live Open Source Database Conference 2018 in Santa Clara closes December 22, 2017, so please also consider submitting as soon as possible. We want to make an early announcement of the talks, so we’ll definitely do the first pass even before the CFP date closes.

Is ARM the new hotness? Marvell confirms $6 billion purchase of chip maker Cavium. This month we’ve seen Red Hat Enterprise Linux for ARM arrive. We’ve also seen a press release from MariaDB about the performance on the Qualcomm Centriq 2400 Processor.

Some new books to add to your bookshelf and read: MariaDB and MySQL Common Table Expressions and Window Functions Revealed by Daniel Bartholomew. The accompanying source code repository will also be useful. Much awaited for, by Percona Live keynote speakers, Database Reliability Engineering: Designing and Operating Resilient Database Systems by Laine Campbell and Charity Majors is now ready to read.

Releases

• Percona Toolkit 3.0.5 – improvements for MongoDB and MySQL here

Link List

• Easily restore an Amazon RDS MySQL database from your MySQL backup – using Percona XtraBackup, and uploading the resulting export to S3.
• Amazon RDS Now Supports Database Storage Size up to 16TB and Faster Scaling for MySQL, MariaDB, Oracle, and PostgreSQL Engines – 16TB for MySQL/MariaDB/PostgreSQL, storage limit is now 6TB, and provisioned IOPS (PIOPS) up to 40,000 from 30,000.
• Amazon RDS for MySQL and MariaDB Supports R4, T2 and M4 Instance Types
• The Expendables — Backends High Availability at BlaBlaCar – learn about the backend, that includes MariaDB Server, from the long-distance ridesharing community site, BlaBlaCar.
• Shenzhen MariaDB Developers Unconference Reportback – great read, wondering why the 10.4 plans aren’t public yet
• The Hacker News thread – ZFS from a MySQL perspective – commentary worth a read
• Sheeri wrote MySQL 5.7 InnoDB Temporary Tablespace – but why? The comments there are good reading, but in addition, Valerii’s Facebook thread is also worth a read.
• Several locks taken to the same record inside a transaction.
• Does PostgreSQL have transportable tablespaces or an equivalent?

Upcoming appearances

• ACMUG 2017 gathering – Beijing, China, December 9-10 2017 – it was very exciting being there in 2016, I can only imagine it’s going to be bigger and better for 2017 since it is now two days long!

Feedback

I look forward to feedback/tips via e-mail at colin.charles@percona.com or on Twitter @bytebot.

You can already feel the cold of February coming slowly… you can also smell waffles, fries and see a large amount of beards walking around with laptops… you are right, FOSDEM is coming! And as every year, the MySQL Community will also be present! For the 4th year in a row, I’ll perpetuate the organization of the MySQL & Friends Devroom.

FOSDEM 2015 edition will be held January 31 and February 1 here in Brussels. The MySQL & Friends Devroom is back on Sunday from 9 a.m. What is FOSDEM? It stands for the “Free and Open Source Software Developers’ European Meeting.” It’s a free event that offers open-source communities a place to meet, share ideas and collaborate.

As every year, the “Call for Papers” has been announced on the MySQL mailing list, and you can still read it here. CfP is open until December 7th!

This year the committee responsible for the talk’s selection is composed by:

* Dimitri Kravtchuk, representing Oracle
* Daniël van Eeden for the Community
* Roland Bouman for the Community
* Cédric Peintre for the Community
* Liz van Dijk, representing Percona
* Serge Frezefond, representing MariaDB
* René Cannaò, representing Blackbird IT

Thanks to all who have accepted playing this role and I wish them to work hard and make the best schedule as possible.

Don’t forget to submit your sessions (submit here, don’t forget to select MySQL track) in time and see you soon in Brussels to discover amazing stuff related to MySQL and have some beers with Friends!

The post MySQL & Friends Devroom FOSDEM 2015 appeared first on MySQL Performance Blog.

This weekend, on 1-2 February, FOSDEM 2014 will take place in Brussels, Belgium.
There will be MySQL talks on Saturday in the MySQL Devroom and a MySQL Community dinner afterward.

30 people have already signed up for the community dinner, we’re almost at the amount of people from last year.

Last orders for the MySQL Community dinner will be accepted on Wednesday 29 January, so if you plan to attend, make sure to buy your tickets now. After that day, we won’t be able to add more people to the list.

I also wanted to thank the sponsors. Without them, this community dinner would not have been made possible:

The post FOSDEM 2014 MySQL Devroom, Community Dinner in Brussels this Saturday! appeared first on MySQL Performance Blog.

FOSDEM is a free, yearly event that offers open source communities a place to meet, share ideas and collaborate. It takes place in Brussels, Belgium each February.

10 years ago now, the first MySQL devroom was organized there by David Axmark, marking the first time MySQL users got their own space to come together and discuss at the event. On this anniversary edition, Percona consultants will be present at the MySQL & Friends Community Booth, as well as giving talks in the devroom (full schedule):

* Frédéric Descamps at 12.35 with 15 Tips to improve your Galera Cluster
* Stéphane Combaudon at 15:05 with Handling failover with MySQL 5.6 and Global Transaction IDs
* Peter Boros at 17.35 with Practical sysbench

The MySQL & Friends devroom will end with its traditional Community Dinner, organized by Percona’s Liz, Kenny and Dimitri  and co-sponsored by Percona, Oracle and MariaDB. Registration for the dinner is MANDATORY this year, so don’t forget to reserve your spot !

Looking forward to seeing you all on Friday night during the traditional beer event opening FOSDEM 2014!

The post FOSDEM 2014 MySQL & Friends devroom appeared first on MySQL Performance Blog.