MariaDB Platform X4 adds cloud-first, columnar storage

MariaDB Corporation, the commercial entity behind the MariaDB open source relational database that began life as a "fork" of MySQL, is today announcing a new version of its platform. The release, dubbed Platform X4, brings new a storage paradigm to MariaDB. This results in both cloud-friendly deployment and accommodation of analytical workloads for the platform. Shane Johnson, MariaDB Corporation's senior director of product marketing, briefed ZDNet on the new capabilities in Platform X4.

As relational OLTP (online transactional processing) databases across the industry add operational analytics capabilities, it's a logical next step for MariaDB to do likewise. Johnson explained that MariaDB will now ship with a plug-in that, when activated, causes the tables in a database to be stored in column store format as well as conventional row store format. Since analytical queries tend to aggregate values stored in a one or a small number of columns, storing all of a column's values together facilitates efficiency in such queries.

One for all

In fact, there already was a MariaDB variant that offered column store capabilities, but this forced database architects and developers to use different versions/distributions of the database for different applications and workloads. Platform X4 provides a unified platform that can handle OLTP workloads, analytical workloads or both.

MariaDB Corporation is promoting the concept of what it calls "smart transactions" to emphasize this dual capability. For example, rather than just querying a database to determine products that are low in quantity, users may wish to cross-reference or rank that information by product popularity -- essentially querying for both factual and analytical information in one go. Another example might involve an airline querying for a list of all flights in certain categories and at the same time bring back their on-time performance to date.

Implementation and economics

In terms of physical implementation, the column store versions of database tables must be kept in object storage compatible with the S3 API. That API is derived from Amazon Web Services' Simple Storage Service (S3), making AWS a natural deployment environment for Platform X4. But AWS is not the only supported environment, since several on-premises/private cloud storage solutions are S3 API-compatible as well, as is Google Cloud Storage (see details here). Platform X4 will also be available on SkySQL, MariaDB's forthcoming Database as a Service (DBaaS) platform.

MariaDB keeps its column store data in a proprietary format rather than an open one like Parquet or ORC. Regardless, leveraging object storage yields cost savings similar to those derived from object storage-based data lakes that do store data in those open formats. Row store data, meanwhile, can be kept in cloud block storage (like Amazon EBS) and X4 supports independent data archiving policies for each. This can provide additional cost savings, since object storage is cheaper on a unit basis.

Optimizations and availability

For maximum control, developers can specify whether they wish to hit the row store or column store versions of their data when issuing queries. But a new feature called "smart query routing" lets developers delegate that decision to MariaDB itself. Another column store-related optimization allows sorting operations to be pushed down to the storage layer, providing up to 50x better performance on sort-intensive queries, according to the company. In our briefing, Mr. Johnson also explained that Platform X4 provides a 2x improvement for hash joins.

Platform X4 features are available now to subscription customers as part of MariaDB Enterprise Server 10.4. According to Johnson, an alpha release of MariaDB Community Edition 10.5 will follow in a few weeks, and will also support columnar storage for analytical processing.