Configure Multi-Region HA Cluster with MariaDB Xpand

Overview

Multi-region clusters provide full, zero RPO high availability in the face of a regional failure.

  • A single Xpand cluster can be deployed across multiple regions.

  • In the event of a complete region failure, the database remains completely available and able to serve traffic immediately.

  • No manual intervention is required to restore from backup since the fault tolerance and resilience is built into the cluster.

    • Xpand cluster is configured across three regions: a primary, a secondary, and an observer region.

    • If the primary region fails, failover to the secondary region happens automatically, with zero data loss.

    • The observer region serves as a tie-breaker for transaction resolution, should the primary or secondary region fail, but does not hold any ranked replicas.

    • The primary region is used to serve reads and writes, and all writes are synchronously and automatically replicated on the secondary region, ensuring full consistency.

  • Within each region, multiple zones can also be configured. In the event of a single zone failure within a region, the other zones within the same region will provide HA.

Compatibility

  • MariaDB Xpand 23.09

Requirements

  • All the nodes must have complete communication with each other.

  • The primary and secondary regions must be within 20ms of each other.

  • The observer region must be within 100ms of the primary and secondary regions.

  • All nodes in all regions must have the same number of cores and similar amounts of memory and storage.

  • The number of nodes in the primary and secondary regions must be the same. This is to insure that in the event of a primary region failure, the secondary region will have the full copy of the data set, and will be able to handle the full workload

  • When scaling out, node additions to the primary and secondary regions must be done symmetrically.

  • The observer region can have a single node only, but it must be the same size as the nodes in the primary and secondary regions.

Backups

Storing backups in both the primary and secondary regions is recommended since all backups are created by reading from the primary region.

Example Topologies

The observer region must have enough nodes so that if either the primary OR the secondary region goes down, the observer and the remaining zone represents a majority of nodes.

MariaDB products can be deployed in many different topologies. The topologies on this page are representative. MariaDB products can be deployed to form other topologies, leverage advanced product capabilities, or combine the capabilities of multiple topologies.

Multi-Region Configuration

Here is an example configuration consisting of 3 primary regions, 3 secondary regions, and 1 observer region (zones are optional):

Region

Nodes

Total number of nodes

Primary

[N1 N2 N3] + MaxScale

3 nodes

Secondary

[N4 N5 N6] + MaxScale

3 nodes

Observer

[N7]

1 node

Multi-Region Configuration with Zones

Here is an example configuration consisting of 3 primary regions of 3 zones each, mirrored by 3 secondary regions of 3 zones each, and 1 observer region:

Region

Nodes

Total number of nodes

Primary

Zone 1: [N1 N2 N3]
Zone 2: [N4 N5 N6]
Zone 3: [N7 N8 N9]

9 nodes

Secondary

Zone 1: [N10 N11 N12]
Zone 2: [N13 N14 N15]
Zone 3: [N16 N17 N18]

9 nodes

Observer

[N19]

1 node

Configuring Regions and Zones

When you have decided on your target topology for a multi-region Xpand cluster and have deployed your cluster, you can use the ALTER CLUSTER statement to specify region and zone configuration.

For example, let's create the topology illustrated here:

  1. From a single node, create regions:

    ALTER CLUSTER CREATE PRIMARY REGION `us-east-1`;
    
    ALTER CLUSTER CREATE SECONDARY REGION `us-east-2`;
    
    ALTER CLUSTER CREATE OBSERVER REGION `us-west-2`;
    
  2. Create zones:

    ALTER CLUSTER CREATE ZONE `us-east-1a`, `us-east-1b`, `us-east-1c`
       IN REGION `us-east-1`;
    
    ALTER CLUSTER CREATE ZONE `us-west-1a`, `us-west-1b`, `us-west-1c`
       IN REGION `us-west-2`;
    
    ALTER CLUSTER CREATE ZONE `us-east-2a`
       IN REGION `us-east-2`;
    

    Deploying across multiple zones for the observer regions is optional. If no zones are explicitly defined, Xpand will place nodes in a single, default zone within a region.

  3. Add nodes to the cluster, and specify their respective zones.

    • Get the zone ids:

      SELECT r.name region_name, r.name zone_name, z.zoneid
         FROM system.zones z, system.regions r
         WHERE z.region = r.region;
      
      +-------------+-----------+--------+
      | region_name | zone_name | zoneid |
      +-------------+-----------+--------+
      | us-east-2   | us-east-2 |      1 |
      | us-east-1   | us-east-1 |      2 |
      | us-east-1   | us-east-1 |      3 |
      | us-east-1   | us-east-1 |      4 |
      | us-west-2   | us-west-2 |      5 |
      | us-west-2   | us-west-2 |      6 |
      | us-west-2   | us-west-2 |      7 |
      +-------------+-----------+--------+
      
    • Get relevant node ids:

      SELECT * FROM system.nodes;
      
    • Add the nodes to zones using the following syntax:

      ALTER CLUSTER <node_id> ZONE <zone_id>;
      

      For example:

      ALTER CLUSTER 1 ZONE 1;
      
  4. Review your configuration by executing the following statement:

    SELECT r.name region_name,
       r.type,
       z.zoneid,
       z.name zone_name,
       n.nodeid,
       n.hostname
    FROM system.zones z,
       system.regions r,
       system.nodeinfo n
    WHERE z.region = r.region AND n.zone = z.zoneid
    ORDER BY r.type;
    

When you are satisfied with the configuration, execute the ALTER CLUSTER REFORM statement for the changes to take effect. This will result in a short disruption for a full group change.