Deploy Xpand Topology

Overview

This procedure describes the deployment of the Xpand Topology with MariaDB Xpand 5.3 and MariaDB MaxScale 2.5.

MariaDB products can be deployed to form other topologies to leverage advanced product capabilities or combine the capabilities of multiple topologies.

This procedure has 6 steps, which are executed in sequence.

This page provides an overview of the topology, requirements, and deployment procedure.

Please read and understand this procedure before executing.

License

A license key is required to use MariaDB Xpand.

Obtain an MariaDB Xpand license key:

  • Users who want to try MariaDB Xpand with a 45-day non-production trial can obtain a license key on the Xpand trial page.

  • Existing MariaDB Enterprise customers with an Xpand license for production use can obtain their license key from MariaDB Sales.

For additional information, see "Licensing for MariaDB Xpand".

Support

Support for this procedure is available in the MariaDB Xpand forum.

Production customers can obtain support in the MariaDB Xpand forum or by submitting a support case.

For additional information, see "Support".

Components

The following components are deployed during this procedure:

Component

Function

MariaDB MaxScale 2.5

An advanced database proxy, firewall, and query router.

MariaDB Xpand 5.3

Distributed SQL with fault tolerance, write scaling, and horizontal scale-out for transactional workloads.

Topology

Xpand Topology

The Xpand Topology consists of:

  • One or more MaxScale nodes

  • Three or more Xpand nodes

The MaxScale nodes:

  • Monitor the health and availability of each Xpand node using the Xpand Monitor (xpandmon)

  • Accept client and application connections

  • Route queries to Xpand nodes using the Read Connection (readconnroute) or Read/Write Split (readwritesplit) routers.

The Xpand nodes:

  • Receive queries from MaxScale

  • Store data in a distributed manner

  • Execute queries using parallel query evaluation

The Xpand Topology can also be deployed in a multi-zone environment:

  • Three or more zones

  • Each zone should be connected by low latency network connections

  • Each zone must have the same number of Xpand nodes

For additional information, see "MariaDB Xpand Topology".

MariaDB products can be deployed in many different topologies. The topology on this page is representative of basic product capabilities. MariaDB products can be deployed to form other topologies, leverage advanced product capabilities, or combine the capabilities of multiple topologies.

Requirements

These requirements are for the Xpand Topology when deployed with MariaDB Xpand 5.3 and MariaDB MaxScale 2.5.

Minimum System Requirements

  • Minimum of 3 servers required for a Xpand cluster

  • Operating System: RHEL or CentOS 7.4+

  • Between 8 and 32 CPU cores per server

  • SSDs for database storage

  • 64GB of RAM

Node Count

  • 1 or more MaxScale nodes are required.

  • 3 or more Xpand nodes are required.

To avoid problems establishing quorum during a network partition, it is recommended to deploy an odd number of Xpand nodes. If you deploy nodes in multiple zones, it is recommended to use an odd number of zones.

Production Xpand license keys define the maximum number of Xpand nodes which can be deployed. Up to 128 nodes are supported per Xpand cluster.

Node Sizing

Resource

MaxScale Node

Xpand Nodes

CPU Cores

8+

8-32

Memory

16 GiB

64 GiB

Production Xpand license keys define the maximum number of CPU cores which can be deployed.

CPU Recommendations

  • 8-32 physical CPU cores per node

  • 16 or 20 physical cores recommended

  • Hyper-threading enabled

Memory Recommendations

  • 64GB or more

AWS EC2 Instance Types

MariaDB Xpand 5.3 supports many different instance types in AWS EC2.

All instances in a Xpand cluster must be the same type.

For MariaDB Xpand 5.3, the following EC2 instance types are recommended:

  • C3

  • I2

  • I3

  • R3

  • X1

  • X1E

For MariaDB Xpand 5.3, the following EC2 instance types are not recommended, so the xpdnode_install.py script throws a warning during installation:

  • C4

  • D2

  • F1

  • G2

  • G3

  • M3

  • M4

  • P2

  • R4

For MariaDB Xpand 5.3, the following EC2 instance types are rejected, so the xpdnode_install.py script exits with an error during installation:

  • T2

GCP GCE Machine Types

  • Recommended:

    • n1-highmem-16

    • n1-standard-16

  • Not Recommended:

    • n1-highcpu-16 as it has the same CPU power as the highmem and standard, yet less RAM

Note: Currently Xpand recommends using 16-CPU VMs, and not using the 8-CPU VMs in GCE

Operating System

In alignment to the enterprise lifecycle, the Xpand Topology with MariaDB Xpand 5.3 and MariaDB MaxScale 2.5 is provided for:

  • CentOS 7 (x86_64)

  • Red Hat Enterprise Linux 7 (x86_64)

File System Requirements

Xpand requires the /data/clustrix directory to reside on the ext4 file system.

For additional information, see "Example File System Configuration for MariaDB Xpand".

Storage Recommendations

Each Xpand node should have 20+ GiB of SSD storage. Mechanical disks and network-attached storage lack sufficient performance to be used for Xpand data (i.e., /data mount point) and are not supported.

Xpand requires database files to be stored on a local SSD-based file system on a separate volume from the root volume containing the OS. This volume should be configured with RAID-0 for best performance and include all of the SSDs to be used for Xpand. For redundancy, Xpand already writes by default two copies of the data so in the event of a node failure or disk failure, the data is still available and don't require higher RAID volume types as that may impact performance. Xpand recommends using the default mount point of /data to simplify installation.

Solid state drives (SSDs) are available in various levels of quality. Xpand recommends using SSDs that have the following characteristics:

  • Enterprise-class SSDs (not consumer-class available from Amazon.com)

  • Contains a "Power Loss Protection" feature

  • Recommend using SSDs manufactured by Intel (every Intel SSD we've seen has Power Loss Protection)

Less performant disks may be used for logs.

The following storage types are recommended:

  • SATA or NVMe SSDs (for DB data) configured in RAID-0

  • SATA HDD (for OS, logs, etc)

See the following table for some recommendations:

Purpose

Disk Type

Mount as

Root Volume / OS

Any

/

Xpand data

SSD only

/data

Xpand logs

Mechanical OK

/data/xpand/log

AWS EC2 Storage Recommendations

  • Only Ephemeral SSD may be used for database data files

    • EBS attached storage is NOT supported for volumes containing database data files

  • EBS volumes may be used for volumes containing log files, or files managed outside of Xpand

GCP GCE Storage Recommendations

  • SSD Persistent Disk

  • Size of at least 1 TB per disk to achieve the maximum of 10,000 IOPS per disk

  • Multiple 1 TB disks may be used for larger storage requirements per VM

Firewall Considerations

Xpand requires a number of ports to be accessible by external applications and for inter-node communication.

One option is to place all Xpand nodes within a secure environment and allow access between all Xpand nodes on all ports.

Another option is to open up the required ports between Xpand nodes.

For additional information, see "TCP Ports for MariaDB Xpand".

Networking Recommendations

  • Separate front and back end networks

  • Back end network should be at least 10 Gbps

  • 10-Gigabit Ethernet switch with enough ports to connect each Xpand server (or 2x switches for redundancy)

AWS EC2 Networking Recommendations

  • Enhanced Networking must be enabled

  • Xpand instances must be in a Virtual Private Cloud (VPC)

  • Xpand instances within an Availability Zone should be in the same Placement Group for best performance

  • All Xpand instances must be in a Security Group that allows all TCP/UDP traffic within the Security Group (i.e., between Xpand instances)

  • Xpand clusters deployed across multiple Availability Zones must be within the same region and have <2ms network latency between all nodes

For additional information, see "AWS Security Groups for MariaDB Xpand".

GCP GCE Networking Recommendations

  • The VPC Network: Firewall rules must be configured to allow all TCP/UDP traffic between all Xpand VMs

  • Xpand clusters deployed across multiple Zones must be within the same region and have <2ms network latency between all nodes

System User Accounts

By default, Xpand and MaxScale use the following OS user accounts. These accounts are created during the installation process:

User Account

Purpose

maxscale

MaxScale process owner

xpand

Xpand process owner

xpandm

Used for administrative tasks, such as running the clx command

For additional information, see "Operating System User Accounts for MariaDB Xpand".

AWS EC2 Load Balancer

  • Xpand leverages a load balancer to present a single IP to the application servers, and spread DB connections equally across all Xpand instances

  • Using AWS's Elastic Load Balancer (ELB) is recommended due to its ease of operation

    • The Network Load Balancer offers the best performance, but requires considerations regarding how the Security Groups are configured

    • The Classic Load Balancer is easier to configure with Security Groups, but tends to add latency to traffic passing through it

  • Alternatively, a software load balancer (e.g., HAProxy) running on a separate EC2 instance, or directly on the application server is also suitable

  • The ELB should be created as an Internal Load Balancer:

GCP GCE Load Balancing

  • Xpand leverages a load balancer to present a single IP to the application servers, and spread DB connections equally across all Xpand instances

  • Google Load Balancer:

    • Select: TCP Load Balancing

    • Select: Internal facing

    • Select: Multiple Regions (or not sure yet)

  • Alternatively, a software load balancer (e.g., HAProxy) running on a separate GCE VM instance, or directly on the application server is also suitable

Example Xpand Server Configuration (Dell)

This is an example server specification using a Dell configuration. Other server vendors including HP and IBM offer servers with very similar specifications.

Dell PowerEdge R630 Server, No TPM

  • Chassis with up to 8, 2.5" Hard Drives, up to 2 PCIe Slots (With Optional Riser)

  • CPUs

    • Intel® Xeon® E5-2667 v4 3.2GHz,25M Cache,9.60GT/s QPI,Turbo,HT,8C/16T (135W) Max Mem 2400MHz

    • Upgrade to Two Intel® Xeon® E5-2667 v4 3.2GHz,25M Cache,9.60GT/s QPI,Turbo,HT,8C/16T (135W)

  • Memory

    • 4x 16GB RDIMM, 2400MT/s, Dual Rank, x8 Data Width

  • RAID Controller

    • PERC S130 RAID Controller

  • Disks

    • 2x 960GB Solid State Drive SATA Mix Use MLC 6Gpbs 2.5in Hot-plug Drive, SM863a

    • 2x 1TB 7.2K RPM SATA 6 Gbps 2.5in Hot-plug Hard Drive

  • Network

    • Embedded NIC: 4x 1 Gb, 2x 1 Gb + 2x 10 Gb, 4x 10 Gb

  • Other Notes:

    • No Operating System

    • Unconfigured RAID (deployment will use software-level RAID-0)

Quick Reference

MaxScale Configuration Management

MaxScale can be configured using several methods. These methods use the MaxScale REST API.

Method

Benefits

MaxCtrl

  • MaxCtrl is a command-line utility that can perform administrative tasks.

  • It supports many different commands.

  • See MaxCtrl Commands in 2.5 for a list of commands.

MaxGUI

  • MaxGUI is a graphical utility that can perform administrative tasks.

  • It was introduced in MaxScale 2.5.

REST API

  • The REST API can be used directly.

  • The curl utility can make REST API calls from the command-line.

  • Many programming languages also have libraries to interact with REST APIs.

This deployment procedure uses MaxCtrl. Alternatively, MaxGUI or direct REST API access could be used.

MaxScale Service Management

The systemctl command is used to start and stop the MaxScale service.

Operation

Command

Status

systemctl status maxscale

Start

systemctl start maxscale

Stop

systemctl stop maxscale

Restart

systemctl restart maxscale

Enable startup

systemctl enable maxscale

Disable startup

systemctl disable maxscale

View systemd journal

journalctl -u maxscale

Xpand Service Management

The clx command is used to start and stop the Xpand service.

This command is usually executed as the xpandm system user account. For additional information, see "MariaDB Xpand System User Accounts".

Operation

Command

Status

clx status

Start

clx dbstart

Stop

clx dbstop

Restart

clx dbrestart

Next Step

Navigation in the procedure "Deploy Xpand Topology":