What is MariaDB 5.3

The focus for MariaDB 5.3 is to radically improve performance for subqueries, as well as for joins and single-table queries over large data sets.

MariaDB 5.3 is based on MariaDB 5.2 and thus on MariaDB 5.1 and MySQL 5.1.

Some of the code is backported from MySQL 6.0 (a MySQL version that was never released as GA by Oracle), some - re-engineered and enriched by new features, some code is written from scratch.

This is the biggest redesign of the MariaDB optimizer in 10 years and it will finally make subqueries usable in practice. We listened to the MariaDB/MySQL community, and took a conservative approach in the way new features are introduced. In the beta release of MariaDB 5.3 all optimizer-related features that are not fully tested are disabled by default. Any feature or combination of features can be enabled dynamically via the optimizer_switch system variable.

MariaDB 5.3 is in beta (feature complete) phase since July 26, 2011. You can download the binaries of MariaDB 5.3.0 here, or get latest source code from launchpad.

Query optimizer

Subquery optimizations

One of the major performance improvements in MariaDB 5.3 is that subqueries are finally usable in practice. It is no longer necessary to rewrite subqueries manually into joins or separate queries, or to forbid their use altogether in your applications. MariaDB 5.3 will take care of this, providing in many cases the best theoretically achievable performance, both when queries are run and EXPLAIN-ed.

• Semi-join subquery optimizations
  Semi-join optimizations transform subqueries into joins whenever possible. These transformations allow the database to choose among many efficient strategies, and to reuse the optimizations available for regular joins. The semi-join optimizations not only allow MariaDB to do what developers do by manual rewrites, but they also provide new and more efficient execution methods not possible to imitate outside the database server.
  • Duplicate elimination
  • Pull out
  • Loose scan
  • First match
  • Materialization-scan
  • Inside-out execution for non-semi-join materialized subqueries
  • Cost-based choice between all semi-join optimizations

• Non-semi-join optimizations
  Certain kinds of subqueries such as ones with GROUP BY, or in the SELECT clause cannot be transformed into joins. Such queries are handled by:
  • Materialization for non-correlated subqueries, with efficient NULL-aware execution
  • IN-to-EXISTS transformation (the only optimization inherited from MySQ
  • Cost-based choice between Materialization and IN->EXISTS

• Subquery Cache
  The subquery cache makes sure that subqueries are re-executed as few times as possible, improving performance of already optimized subqueries.

• Subqueries are never executed during EXPLAIN, thus resulting in almost instant EXPLAIN.

Optimizations for derived tables and views

• No early materialization of derived tables (subqueries in FROM clause) and materialized views (EXPLAIN is always instantaneous)
• Mergeable derived tables are merged like mergeable views
• Optimizer can create keys to access derived tables and views materialized in temporary tables
• Fields of mergeable views and derived tables are involved now in all optimizations employing equalities

Disk access optimization

• Index Condition Pushdown
• Multi-Range-Read optimization (MRR) (backport from MySQL 6.0)
  • Key-ordered retrieval (new development)

Join optimizations

• Block Nested Loop algorithm can be used for outer joins
• Block Hash Join (classic algorithm) is implemented and can be used for any equi-joins
• Block Index Join (Batch Key Access Join) is supported and can exploit the benefits of ordered retrievals for primary and secondary keys provided by the new implementation of MRR
• All block based algorithms for joins can use the benefits of new incremental join buffers
• All block based algorithms fully support outer joins including nested outer joins
• All block based algorithms can use the benefits of the first match optimization for semi-joins and the non-exist optimization for outer joins
• All block based algorithms for joins can exploit the benefits of index condition push-down.
• The total memory space used by the query for join buffers can be limited now, block based algorithms can allocate join buffers up to their needs not exceeding the set limits.
• Condition over outer tables extracted from ON expressions of outer joins are evaluated before inner tables are accessed (supported for both regular index join and block index join)
• Early checks for nulls for the fields from any null-rejecting conditions are performed

Index Merge improvements

• Correct optimization of index_merge vs range access: Fair choice between range and index_merge optimizations
• index_merge/sort_intersection strategy

Optimizer control

• @@optimizer_switch variable can be used to turn on/off all new optimizations.

NoSQL-style interfaces

• HandlerSocket plugin included.
• Faster HANDLER commands; HANDLER READ now also work with prepared statements.

Replication and binary logging

• Group commit for the binary log — MariaDB 5.3 implements group commit which works when using XtraDB with the binary log enabled. (In previous MariaDB releases, and all MySQL releases at the time of writing, group commit works in InnoDB/XtraDB when the binary log is disabled, but stops working when the binary log is enabled).
• Annotation of row-based replication events with the original SQL statement — When using row-based replication, the binary log does not contain SQL statements, only discrete single-row insert/update/delete events. This can make it harder to read mysqlbinlog output and understand where in an application a given event may have originated, complicating analysis and debugging.This feature adds an option to include the original SQL statement as a comment in the binary log (and shown in mysqlbinlog output) for row-based replication events.
• Checksums for binlog events. This is a backport of the same feature in MySQL 5.6. It was implemented in MWL#180.
• Enhancements for START TRANSACTION WITH CONSISTENT SNAPSHOT — In MariaDB 5.3, START TRANSACTION WITH CONSISTENT SNAPSHOT now also works with the binary log. This means it is possible to obtain the binlog position corresponding to a transactional snapshot of the database without blocking any other queries. This is used by the command "mysqldump --single-transaction --master-data" to do a fully non-blocking backup which can be used to provision a new slave. "START TRANSACTION WITH CONSISTENT SNAPSHOT" now also works consistently between transactions involving more than one storage engine (currently XTraDB and PBXT support this).
• Row-based replication for tables with no primary key — This feature can improve the performance of row-based replication on tables that do not have a primary key (or other unique key), but which do have another index that can help locate rows to update or delete. With this feature, index cardinality information from ANALYZE TABLE is considered when selecting the index to use (before this feature is implemented, the first index was selected unconditionally).
• mysqlbinlog will now omit redundant use statements around BEGIN, SAVEPOINT, COMMIT, and ROLLBACK events when reading MySQL 5.0 binlogs.

Datatypes

• Dynamic Columns support.
• Microsecond support for NOW() and timestamp, time and datetime columns.
• CAST() now supports AS DECIMAL[(M,D)] and AS INT.
• CAST() and all other datetime/time functions now supports microsecond fully.

Windows performance improvements

• Backported Windows performance patches from MySQL 5.5.
• Asynchronous IO in XtraDB is redesigned and is now faster, due to the use of IO completion ports.
• Additional durability option for XtraDB : innodb_flush_method can now be O_DSYNC, like on Unixes. The effect of using this option is that log file is opened with FILE_FLAG_WRITETHROUGH, and FlushFileBuffers() is not done. This may improve speed in write-heavy scenarios.

Miscellaneous

• New status variables: Rows_tmp_read, Handler_tmp_write, and Handler_tmp_update which count what happens with internal temporary tables. Rows_read, Handler_write and Handler_update no longer count operations on internal temporary tables.
• New variable 'in_transaction' that is 1 if you are in a transaction, 0 otherwise.
• Progress reports for ALTER TABLE and LOAD DATA INFILE. In addition Aria tables gives progress reports for REPAIR TABLE and CHECK TABLE. The progress can be seen in SHOW PROCESSLIST, INFORMATION_SCHEMA.PROCESSLIST and is sent to MariaDB clients that calls mysql_real_connect() with the new CLIENT_PROGRESS flag. mysql command line client supports the new progress indications.
• PBXT consistent commit ordering — This feature implements the new commit ordering storage engine API in PBXT. With this feature, it is possible to use "START TRANSACTION WITH CONSISTENT SNAPSHOT" and get consistency among transactions which involve both XtraDB and InnoDB. (Without this feature, there is no such consistency guarantee. For example, even after running "START TRANSACTION WITH CONSISTENT SNAPSHOT" it was still possible for the InnoDB/XtraDB part of some transaction T to be visible and the PBXT part of the same transaction T to not be visible.)
• MariaDB unique error numbers now start from 1900 to not clash with MySQL error numbers.
• A MariaDB optimized version of mytop is included in the MariaDB distribution.

For a more specific list and for things that are still planned, see the MariaDB 5.3 TODO page.

You can access the MariaDB 5.3 tree from launchpad.