December 20, 2016

Why Marko Mäkelä, Lead Developer InnoDB, Recently Joined MariaDB Corporation

I recently joined MariaDB Corporation. You might not recognize my name, but you may have used some InnoDB features that I have worked on since I joined Innobase Oy as the first full-time employee in 2003.

My first task was to reduce the overhead of the InnoDB table storage. I introduced ROW_FORMAT=COMPACT (and named the old format ROW_FORMAT=REDUNDANT) in MySQL 5.0.3. ROW_FORMAT=COMPRESSED and ROW_FORMAT=DYNAMIC were released as part of the InnoDB Plugin for MySQL 5.1.

In the InnoDB Plugin, I also completed the ‘fast index creation’ feature. That along with the ROW_FORMAT changes and some BLOB bug fixes were among the major improvements that the InnoDB Plugin offered over the built-in InnoDB in MySQL 5.1. The InnoDB Plugin became the built-in InnoDB in MySQL 5.5.

In MySQL 5.5, I transformed the InnoDB insert buffer into a change buffer (delete-mark and purge buffering) and introduced the first regression tests based on fault injection.

In MySQL 5.6, I designed and implemented the InnoDB part of ALTER TABLE…ALGORITHM=INPLACE and LOCK=NONE, also known as the ‘online ALTER TABLE’. I also removed the famous limitation that the InnoDB redo log file size could not be changed.

In MySQL 5.7 one of my most visible contributions probably is the InnoDB redo log format tagging, to prevent old servers from failing with obscure errors when starting up with new data files.

Why did I join MariaDB? The short answer is that Monty called me and asked. The long answer is that having an academic background, I value open collaboration and the open exchange of ideas. At MariaDB it feels like being at home again, working with many of the colleagues from the 2003‒2005 era when both Innobase Oy and MySQL AB were small companies where each employee was able or forced to work on a wide range of  tasks.

The acquisitions of the companies introduced policies and processes that restrict communication, gradually transforming the ‘bazaar’ into a ‘cathedral’. While ‘open source’ seems to have largely won over ‘closed source’ when it comes to information technology and communication infrastructure, I think that The Cathedral and the Bazaar are still with us. The new ‘cathedral’ is ‘closed development’ where source code is only released in snapshots synchronized with product releases. Significant parts of the development history are often hidden by squashing merges of long-lived development branches to a gigantic commit.

MariaDB is the ‘bazaar’, encouraging feedback from end users in all stages of development.

While a for-profit business cannot provide unpaid support to users, it sometimes makes sense to work with users to obtain test cases. For instance, MDEV-11233 (a server crash in CREATE FULLTEXT INDEX) had been reported by several users, but crucial details were missing. Finally, the same assertion failure message was posted on the #maria channel in the FreeNode IRC network. After some discussion, Waqar Khan was busy executing a binary search, running a series of SQL statements to reduce the problematic table from one million rows to a single record that triggered the problem. An hour or two later we had a minimal 3-statement test case for reproduce the problem.

Another example is MDEV-6076 Persistent AUTO_INCREMENT for InnoDB tables, which I hope to be included in the upcoming MariaDB 10.2 release. Zhang Yuan at Alibaba looked at my code changes and pointed out a mistake before our internal code review had been completed.

I am looking forward to interesting times with MariaDB.