Zones with MariaDB Xpand

Overview

MariaDB Xpand allows deployments to be deployed across multiple zones to provide fault tolerance during unplanned zone failure. A zone in Xpand is grouping of Xpand nodes, such as AWS Availability Zones within the same Region, different server racks, different network switches, different power sources, or even separate servers in different data centers. Xpand requires that network latency between zones not exceed 2ms.

When Xpand is configured for zones, the Rebalancer becomes zone-aware and ensures replicas are placed in different zones. Additionally, Paxos acceptors are distributed across multiple zones. The result is that Xpand is fault tolerant in the face of unexpected zone failure, regardless of how many Xpand nodes are in the zone.

The system variables, system tables, and ALTER CLUSTER statements referenced on this page are only available on the Xpand nodes. When using the Xpand Storage Engine topology, you need to connect to an Xpand node to use these features.

Prerequisites

Use of zones with Xpand has the following prerequisites:

  • Xpand supports configurations with a minimum of 3 and a maximum of 5 zones.

  • There must be a sufficient number of zones to maintain a quorum of Xpand nodes following a failure

  • Zones must be in the same geographical region with <2ms network latency between each zone.

  • A deployment deployed across zones must have the same number of Xpand nodes in each zone.

Fault Tolerance and Zones

The following examples illustrate how zones work with the default MAX_FAILURES = 1.

There must be the same number of Xpand nodes in each zone. Clustrix recommends deploying a 9 Xpand node deployment across 3 zones.

Xpand in 3 Zones

This deployment can tolerate one Xpand node failure:

1 Node Failure in Zoned Cluster

Or one zone failure:

1 Full Zone Failure

There must be the same number of Xpand nodes in each zone. The following configuration is not supported.

Unequal nodes per Zone

If you require a larger deployment, simply add the same number of Xpand nodes to each zone.

Expanded Zones

See MAX_FAILURES for additional information.

Configuring Zones

To configure zones, use ALTER CLUSTER ZONE to assign Xpand nodes to a zone. Zone ids must be greater than 0 and there must be an equal number of Xpand nodes in each zone. For example, a 9 Xpand node deployment deployed across 3 zones should have 3 Xpand nodes each in zones 1, 2, and 3.

Xpand recommends provisioning enough free disk space to reprotect the data after a zone failure. See Allocating Disk Space for Fault Tolerance and Availability.

Note

Xpand currently supports a minimum of 3 and a maximum of 5 zones.

Creating a New Cluster in Multiple Zones

To create a new deployment, follow these steps:

  1. Provision Xpand nodes in different zones, where zones are separate server racks, separate servers in different data centers, separate network switches, independent power sources, or AWS Availability Zones.

  2. Use ALTER CLUSTER ADD to build a deployment.

  3. Use ALTER CLUSTER ZONE to assign Xpand nodes to zones.

  4. Verify that zones have been configured using clx stat. All Xpand nodes should show a zone number of 1 - nn. Zone 0 should not appear in the results.

Migrating an Existing Cluster to Multiple Zones

If you would like to migrate your existing deployment, the high-level steps are:

  1. Temporarily disable the Rebalancer.

  2. Provision Xpand nodes in new zones and install Xpand on each node.

  3. Use the Scale-Out procedure to add the new Xpand nodes to the deployment.

  4. Configure zones using ALTER CLUSTER ZONE.

  5. Use the Scale-In procedure to remove surplus Xpand nodes from the deployment.

  6. Re-enable the Rebalancer.

Once this procedure is complete, you should have the same number of Xpand nodes in each zone and all Xpand nodes should have a non-zero zone assigned. Because the scale-out and scale-in operations can be time consuming, it is often simpler to migrate to a new deployment when moving from default to zone configurations.