ALTER CLUSTER ADD
Modifies an Xpand deployment, triggering group change to add a new Xpand Node to the deployment.
When using the Xpand Storage Engine topology, the details described here only apply when you connect to the Xpand nodes.
ALTER CLUSTER ADD ip[, ip...]
Using this statement, you can trigger a group change to add a new Xpand Node to the deployment. When you add Xpand Nodes to the deployment, the rebalancer process automatically works in the background to move data onto the new node, increasing the available storage and availability of the deployment.
Xpand performs a group change during this operation. It is recommended that you only run it during non-peak periods or scheduled maintenance windows. There is a short disruption in service as the node is added and the potential for a slight degradation in performance while the Rebalancer moves data onto the new node.
To add a new Xpand Node to a deployment:
Deploy a new Xpand node.
When using zones, deploy a sufficient number of Xpand Nodes to ensure that the target configuration has an equal number of nodes in each zone.
If you are using encrypted connections, ensure that certificates in keys are present in the same location with the correct permissions.
Connect to an existing Xpand Node with the socket file used by the MariaDB Xpand service:
$ mariadb --socket /data/clustrix/mysql.sock
ALTER CLUSTER ADDstatement to add the new nodes to your deployment:
ALTER CLUSTER ADD "192.0.2.1", "192.0.2.3", "192.0.2.3";
When using zones, use an ALTER CLUSTER ZONE statement to add the nodes to the appropriate zones.
Run a SELECT statement to verify scale-out:
SELECT * FROM system.nodeinfo ORDER BY nodeid;
You should see that all nodes appear OK on the display. If zones are in use, all nodes should have a non-zero zone assigned. If zones are not in use, all nodes will be assigned to zone 0.
You may notice that the data distribution among your nodes is not yet balanced. Be patient. It will be, soon.
New Xpand Nodes do not yet contain data. The Xpand Rebalancer starts automatically in the background and begins to move data onto the new node. The deployment is fully functional and able to be used during this process.
While adding nodes to the deployment, Xpand performs checks to ensure the nodes have the same configuration.
The following are errors you may encounter at the SQL prompt
This error occurs if you attempt to add a node that is already part of the deployment:
ERROR 1 (HY000):  Bad parameter.: Host "xxx" already in cluster
Node additions enter into a pending state due to:
A node is still in the process of being added
A node running a different version of ClustrixDB was added to the deployment
A node with a different multiport setting was added to the deployment
ERROR 1 (HY000):  Multiple nodes cannot be added when an existing addition is pending
Xpand Nodes that are pending invites can be seen in the
system.pending_invites table, available by connecting to the socket file used by the MariaDB Xpand service, which is
/data/clustrix/mysql.sock by default.
The deployment periodically attempts to send invitations to nodes in
system.pending_invites. For each invitation that is sent, an entry is added to the
If the same node is in
system.problem_nodes, you may want to remove the node from
system.pending_invites before resolving the issue with
system.problem_nodes. Doing this will prevent the node from being automatically added to the deployment once the problem is resolved, causing a premature group change. Instead, you may prefer to complete the node addition during off-peak hours.
DELETE FROM system.pending_invites;
Xpand Nodes that are problem nodes can be seen in the
system.problem_nodes table, available by connecting to the socket file used by the MariaDB Xpand service, which is
/data/clustrix/mysql.sock by default.
Software binaries differ
Ensure that all Xpand Nodes use the same version of MariaDB Xpand.
Multiport settings mismatched.
Modify startup configuration options to enable or disable multiport.