Introducing MariaDB Exa: 1,000x Faster Analytics on Live Data
The Challenge: Grappling with Growing Datasets
We are excited to announce our partnership with Exasol, a collaboration set to revolutionize real-time analytics for MariaDB users. This strategic alliance is a critical step toward eliminating the barrier between live operational data and analytics, thereby simplifying the data stack and allowing you to get immediate insights from your analytical data.
For too long, organizations have grappled with the challenge of running complex analytics queries on ever-growing datasets. While performance for MariaDB and its default storage engine, InnoDB, is exceptional for in-memory queries, it typically decreases as the data size grows beyond the memory. InnoDB is also optimized for handling OLTP queries and is not optimized for running analytical queries. For example, our internal testing found that some TPC-H queries perform over 1,000x faster on MariaDB with Exasol compared to MariaDB with InnoDB. Our initial analytics solution, MariaDB ColumnStore, has evolved to focus on tight coupling, co-existence on shared hardware, and real-time performance enhancements within MariaDB. However, it’s become increasingly clear that a dedicated solution for extreme analytics is essential to the modern database platform.
Our conversations with customers, many of whom were utilizing other analytics platforms, highlighted a critical flaw: external solutions invariably demand intricate, time-consuming, and often fragile ETL processes to transfer data into analytical stores. These processes are not only costly but also introduce inherent lag, effectively preventing real-time insights.
The Solution: A Groundbreaking Partnership with Exasol
In close collaboration with the Exasol team, we have engineered a groundbreaking solution that runs analytical and transactional queries concurrently – with predictable speed and reliability – on the same live data. We have also developed a system that facilitates near real-time data transfer from the OLTP side to the analytical engine. Concurrently, we provide a unified interface, powered by our intelligent proxy, MaxScale, to interact seamlessly with both the OLTP and analytics environments. By compacting the OLTP operations into batches and executing them on the analytics side, our initial tests have yielded remarkable results, demonstrating the CDC (change data capture) and analytics engine’s ability to effortlessly keep pace with a 450K TPM workload on the MariaDB Server side (validated through a TPC-C workload using HammerDB). Comprehensive benchmarking results will be published shortly, however our initial findings confirm this exceptional performance.
Architecture Overview: MaxScale as the Unified Interface
Figure 1: Architecture overview
This innovative architecture establishes MaxScale, our advanced database proxy, as a singular point of connection which abstracts the complexities of CDC, MariaDB and Exasol. All write operations are automatically directed to MariaDB and subsequently processed through the MariaDB binlog for the CDC process, which then dispatches batched and compacted operations to an Exasol server. The Exasol server processes the CDC data, making it instantly accessible to all Exasol nodes. The continuous CDC process ensures that the data in Exasol nodes remain as near real-time as possible.
When a read query is initiated, MaxScale intelligently routes it to either MariaDB or Exasol, choosing the fastest execution based on the query’s characteristics. This eliminates the need for users to determine the appropriate destination, making deployment exceptionally straightforward. No changes to existing applications are required – simply replace your current MariaDB deployment with MariaDB Exa.
Who Benefits? Real-Time Analytics on Live Data
MariaDB Exa is engineered for organizations that can no longer afford the delay between transactions and insights. It delivers profound, real-time impacts for:
1. Real-time business analytics. This solution is designed for analytics leaders and BI teams in high-velocity sectors like finance, e-commerce, and logistics, where delays in insights translate directly to lost revenue. Powered by Exasol’s high-performance MPP engine – which has a proven track record of impressive performance in competitive benchmarks – MariaDB Exa delivers near real-time analytic insights directly from live operational data, eliminating the complexity and lag of nightly ETL runs.
2. Modern AI & machine learning. MariaDB Exa is tailored to meet the needs of data scientists and engineers. AI and machine learning depend on data freshness. Training and inference on outdated datasets – even data that is weeks or even days old – can lead to missed patterns and wrong predictions. With MariaDB Exa, models can now be trained and deployed directly against live operational data. Exasol’s integrated AI/ML execution framework makes it possible to score transactions, detect anomalies, and optimize operations in real time.
If you are ready to experience MariaDB Exa in action, contact us to request a demo. Discover how live operational data and high-speed analytics converge: no pipelines, no waiting, just immediate insight.