Deploy Xpand Storage Engine Topology

Overview

This procedure describes the deployment of the Xpand Storage Engine topology with MariaDB Enterprise Server 10.5, MariaDB Xpand 5.3, and MariaDB MaxScale 2.5.

MariaDB products can be deployed to form other topologies to leverage advanced product capabilities or combine the capabilities in multiple topologies.

The goal of the Xpand Storage Engine topology is to combine the performance and scaling features of MariaDB Xpand 5.3 with modern SQL features of MariaDB Enterprise Server 10.5, like window functions, CTEs, and cross-engine JOINs

This procedure has 9 steps, which are executed in sequence.

This page provides an overview of the topology, requirements, and deployment procedure.

Please read and understand this procedure before executing.

License

A license key is required to use MariaDB Xpand.

Obtain an MariaDB Xpand license key:

  • Users who want to try MariaDB Xpand with a 45-day non-production trial can obtain a license key on the Xpand trial page.

  • Existing MariaDB Enterprise customers with an Xpand license for production use can obtain their license key from MariaDB Sales.

Support

Support for this procedure is available in the MariaDB Xpand forum.

Production customers can obtain support in the MariaDB Xpand forum or by submitting a support case.

Components

The following components are deployed during this procedure:

Component

Function

MariaDB MaxScale 2.5

An advanced database proxy, firewall, and query router.

MariaDB Enterprise Server 10.5

Modern SQL RDBMS with high availability, pluggable storage engines, hot online backups, and audit logging.

MariaDB Xpand 5.3

Distributed SQL with fault tolerance, write scaling, and horizontal scale-out for transactional workloads.

MariaDB Enterprise Server Components

Component

Function

Xpand Storage Engine

The Xpand storage engine plugin allows MariaDB Enterprise Server to read and write data to MariaDB Xpand, providing all the advantages of Xpand with the modern SQL features of MariaDB Enterprise Server, such as window functions, CTEs, and cross-engine JOINs.

MariaDB MaxScale Components

Component

Function

Listener

Listens for client connections to MaxScale, then passes them to the router service associated with the listener

MariaDB Monitor

Tracks changes in the state of Enterprise Server nodes.

Read Connection Router

Routes connections from the listener to any available node.

Read/Write Split Router

Routes read operations from the listener to any available node, and routes write operations from the listener to a specific server that MaxScale uses as the primary server.

Server Module

Connection configuration in MaxScale to Enterprise Server and Xpand nodes.

Xpand Monitor

Tracks changes in the state of Xpand nodes.

Topology

Xpand Storage Engine Topology

Xpand Storage Engine topology combines the performance, fault tolerance, and scalability of MariaDB Xpand 5.3 with the modern SQL features of MariaDB Enterprise Server 10.5.

The Xpand Storage Engine topology consists of:

  • 1 or more MaxScale nodes

  • 3 or more Enterprise Server nodes

  • 3 or more Xpand nodes

The MaxScale nodes:

  • Monitor the health and availability of each Enterprise Server node using MariaDB Monitor (mariadbmon)

  • Monitor the health and availability of each Xpand node using Xpand Monitor (xpandmon)

  • Route queries to Enterprise Server nodes using the Read/Write Split (readwritesplit) router.

  • Route administrative connections to Xpand nodes using the Read Connection (readconnroute) router.

The Enterprise Server nodes:

  • Receive queries from MaxScale

  • Provide storage for non-Xpand data

  • Pass Xpand queries to Xpand nodes

The Xpand nodes:

  • Receive administrative queries from MaxScale

  • Receive application queries from Enterprise Server

  • Store data in a distributed manner

  • Execute queries using parallel query evaluation

MariaDB products can be deployed to form other topologies to leverage advanced product capabilities or combine the capabilities in multiple topologies.

Requirements

These requirements are for the Xpand Storage Engine topology when deployed with MariaDB Enterprise Server 10.5, MariaDB Xpand 5.3, and MariaDB MaxScale 2.5.

Node Count

  • 3 or more Enterprise Server nodes are required.

  • 1 or more MaxScale nodes are required.

  • 3 or more Xpand nodes are required.

To avoid problems establishing quorum during a network partition, it is recommended to deploy an odd number of Xpand nodes. If you deploy nodes in multiple zones, it is recommended to use an odd number of zones.

Production Xpand license keys define the maximum number of Xpand nodes which can be deployed.

Node Sizing

Resource

MaxScale Node

Xpand Nodes

CPU Cores

8+

8-32

Memory

16 GiB

64 GiB

Production Xpand license keys define the maximum number of CPU cores which can be deployed.

AWS EC2 Instance Types

MariaDB Xpand 5.3 supports many different instance types in AWS EC2.

For MariaDB Xpand 5.3, the following EC2 instance types are recommended:

  • C3

  • I2

  • I3

  • R3

  • X1

  • X1E

For MariaDB Xpand 5.3, the following EC2 instance types are not recommended, so the xpdnode_install.py script throws a warning during installation:

  • C4

  • D2

  • F1

  • G2

  • G3

  • M3

  • M4

  • P2

  • R4

For MariaDB Xpand 5.3, the following EC2 instance types are rejected, so the xpdnode_install.py script exits with an error during installation:

  • T2

Operating System

In alignment to the enterprise lifecycle, the Xpand Storage Engine topology with MariaDB Enterprise Server 10.5, MariaDB Xpand 5.3, and MariaDB MaxScale 2.5 is provided for:

  • CentOS 7 (x86_64)

  • Red Hat Enterprise Linux 7 (x86_64)

Storage Considerations

Xpand requires the /data/clustrix directory to reside on the ext4 file system.

Each Xpand node should have 20+ GiB of SSD storage.

System User Accounts

By default, Xpand and MaxScale use the following OS user accounts. These accounts are created during the installation process:

User Account

Purpose

maxscale

MaxScale process owner

mysql

Enterprise Server process owner

xpand

Xpand process owner

xpandm

Used for administrative tasks, such as running the clx command

For additional information, see "MariaDB Xpand System User Accounts".

Quick Reference

MariaDB Enterprise Server Configuration Management

Method

Description

Configuration File

Configuration files (such as /etc/my.cnf) can be used to set system-variables and options. The server must be restarted to apply changes made to configuration files.

Command-line

The server can be started with command-line options that set system-variables and options.

SQL

Users can set system-variables that support dynamic changes on-the-fly using the SET statement.

MariaDB Enterprise Server packages are configured to read configuration files from different paths, depending on the operating system. Making custom changes to Enterprise Server default configuration files is not recommended because custom changes may be overwritten by other default configuration files that are loaded later.

To ensure that your custom changes will be read last, create a custom configuration file with the z- prefix in one of the include directories.

Distribution

Example Configuration File Path

  • CentOS

  • Red Hat Enterprise Linux (RHEL)

/etc/my.cnf.d/z-custom-mariadb.cnf

MariaDB Enterprise Server Service Management

The systemctl command is used to start and stop the MariaDB Enterprise Server service.

Operation

Command

Start

sudo systemctl start mariadb

Stop

sudo systemctl stop mariadb

Restart

sudo systemctl restart mariadb

Enable during startup

sudo systemctl enable mariadb

Disable during startup

sudo systemctl disable mariadb

Status

sudo systemctl status mariadb

MariaDB Enterprise Server Logs

MariaDB Enterprise Server produces log data that can be helpful in problem diagnosis.

Log filenames and locations may be overridden in the server configuration. The default location of logs is the data directory. The data directory is specified by the datadir system variable.

Log

System Variable/Option

Default Filename

MariaDB Error Log

log_error

<hostname>.err

MariaDB Enterprise Audit Log

server_audit_file_path

server_audit.log

Slow Query Log

slow_query_log_file

<hostname>-slow.log

General Query Log

general_log_file

<hostname>.log

Binary Log

log_bin

<hostname>-bin

MaxScale Configuration Management

MaxScale can be configured using several methods. These methods use the MaxScale REST API.

Method

Benefits

MaxCtrl

  • MaxCtrl is a command-line utility that can perform administrative tasks.

  • It supports many different commands.

  • See MaxCtrl Commands in 2.5 for a list of commands.

MaxGUI

  • MaxGUI is a graphical utility that can perform administrative tasks.

  • It was introduced in MaxScale 2.5.

REST API

  • The REST API can be used directly.

  • The curl utility can make REST API calls from the command-line.

  • Many programming languages also have libraries to interact with REST APIs.

This deployment procedure uses MaxCtrl. Alternatively, MaxGUI or direct REST API access could be used.

MaxScale Service Management

The systemctl command is used to start and stop the MaxScale service.

Operation

Command

Status

systemctl status maxscale

Start

systemctl start maxscale

Stop

systemctl stop maxscale

Restart

systemctl restart maxscale

Enable startup

systemctl enable maxscale

Disable startup

systemctl disable maxscale

View systemd journal

journalctl -u maxscale

Xpand Service Management

The clx command is used to start and stop the Xpand service.

This command is usually executed as the xpandm system user account. For additional information, see "MariaDB Xpand System User Accounts".

Operation

Command

Status

clx status

Start

clx dbstart

Stop

clx dbstop

Restart

clx dbrestart

Next Step

Navigation in the procedure "Deploy Xpand Storage Engine Topology":