ALTER CLUSTER ADD

Overview

Modifies an Xpand deployment, triggering group change to add a new Xpand Node to the deployment.

When using the Xpand Storage Engine topology, the details described here only apply when you connect to the Xpand nodes.

DETAILS

ALTER CLUSTER ADD ip[, ip...]

Using this statement, you can trigger a group change to add a new Xpand Node to the deployment. When you add Xpand Nodes to the deployment, the rebalancer process automatically works in the background to move data onto the new node, increasing the available storage and availability of the deployment.

Note

Xpand performs a group change during this operation. It is recommended that you only run it during non-peak periods or scheduled maintenance windows. There is a short disruption in service as the node is added and the potential for a slight degradation in performance while the Rebalancer moves data onto the new node.

Adding Nodes

To add a new Xpand Node to a deployment:

  1. Deploy a new Xpand node.

    When using zones, deploy a sufficient number of Xpand Nodes to ensure that the target configuration has an equal number of nodes in each zone.

    If you are using encrypted connections, ensure that certificates in keys are present in the same location with the correct permissions.

  2. Connect to an existing Xpand Node with the socket file used by the MariaDB Xpand service:

    $ mariadb --socket /data/clustrix/mysql.sock
    
  3. Use an ALTER CLUSTER ADD statement to add the new nodes to your deployment:

    ALTER CLUSTER ADD "192.0.2.1", "192.0.2.3", "192.0.2.3";
    

    When using zones, use an ALTER CLUSTER ZONE statement to add the nodes to the appropriate zones.

  4. Run a SELECT statement to verify scale-out:

    SELECT * FROM system.nodeinfo ORDER BY nodeid;
    

    You should see that all nodes appear OK on the display. If zones are in use, all nodes should have a non-zero zone assigned. If zones are not in use, all nodes will be assigned to zone 0.

    You may notice that the data distribution among your nodes is not yet balanced. Be patient. It will be, soon.

New Xpand Nodes do not yet contain data. The Xpand Rebalancer starts automatically in the background and begins to move data onto the new node. The deployment is fully functional and able to be used during this process.

ERRORS

While adding nodes to the deployment, Xpand performs checks to ensure the nodes have the same configuration.

SQL Errors

The following are errors you may encounter at the SQL prompt

  • This error occurs if you attempt to add a node that is already part of the deployment:

    ERROR 1 (HY000): [26633] Bad parameter.: Host "xxx" already in cluster
    
  • Node additions enter into a pending state due to:

    • A node is still in the process of being added

    • A node running a different version of ClustrixDB was added to the deployment

    • A node with a different multiport setting was added to the deployment

    ERROR 1 (HY000): [55300] Multiple nodes cannot be added when an existing addition is pending
    

    Additional information is available in, PROBLEM_NODES and PENDING_INVITES.

PENDING_INVITES

Xpand Nodes that are pending invites can be seen in the system.pending_invites table, available by connecting to the socket file used by the MariaDB Xpand service, which is /data/clustrix/mysql.sock by default.

The deployment periodically attempts to send invitations to nodes in system.pending_invites. For each invitation that is sent, an entry is added to the clustrix.log file. note

Note

If the same node is in system.pending_invites and system.problem_nodes, you may want to remove the node from system.pending_invites before resolving the issue with system.problem_nodes. Doing this will prevent the node from being automatically added to the deployment once the problem is resolved, causing a premature group change. Instead, you may prefer to complete the node addition during off-peak hours.

DELETE FROM system.pending_invites;

PROBLEM_NODES

Xpand Nodes that are problem nodes can be seen in the system.problem_nodes table, available by connecting to the socket file used by the MariaDB Xpand service, which is /data/clustrix/mysql.sock by default.

Reason

Resolution

Software binaries differ

Ensure that all Xpand Nodes use the same version of MariaDB Xpand.

Multiport settings mismatched.

Modify startup configuration options to enable or disable multiport.