MariaDB Enterprise ColumnStore Adding a Node

Overview

To add a new node to Enterprise ColumnStore, perform the following procedure.

Deploy Enterprise ColumnStore

Before you can add a node to Enterprise ColumnStore, confirm that the Enterprise ColumnStore software has been deployed on the node in the desired topology.

For additional information, see "Deploy".

Backup MariaDB Data Directory on the Primary Server

Before the new node can be added, its MariaDB data directory must be consistent with the Primary Server. To ensure that it is consistent, take a backup of the Primary Server:

The instructions below show how to perform a backup using MariaDB Backup.

  1. On the Primary Server, take a full backup:

    $ sudo mariadb-backup --backup \
          --user=mariabackup_user \
          --password=mariabackup_passwd \
          --target-dir=/data/backup/replica_backup
    

    Confirm successful completion of the backup operation.

  2. On the Primary Server, prepare the backup:

    $ sudo mariadb-backup --prepare \
          --target-dir=/data/backup/replica_backup
    

    Confirm successful completion of the prepare operation.

Restore the Backup on the New Node

To make the new node consistent with the Primary Server, restore the new backup on the new node:

  1. On the Primary Server, copy the backup to the new node:

    $ sudo rsync -av /data/backup/replica_backup 192.0.2.3:/data/backup/
    
  2. On the new node, restore the backup using MariaDB Backup.

    $ sudo mariadb-backup --copy-back \
       --target-dir=/data/backup/replica_backup
    
  3. On the new node, fix the file permissions of the restored backup:

    $ sudo chown -R mysql:mysql /var/lib/mysql
    

Start the Enterprise ColumnStore Services

The Enterprise Server. Enterprise ColumnStore, and CMAPI services can be started using the systemctl command. In case the services were started during the installation process, use the restart command.

Perform the following procedure on the new node:

  1. Start and enable the MariaDB Enterprise Server service, so that it starts automatically upon reboot:

    $ sudo systemctl restart mariadb
    $ sudo systemctl enable mariadb
    
  2. Start and disable the MariaDB Enterprise ColumnStore service, so that it does not start automatically upon reboot:

    $ sudo systemctl restart mariadb-columnstore
    $ sudo systemctl disable mariadb-columnstore
    

    Note

    The Enterprise ColumnStore service should not be enabled in a multi-node deployment. The Enterprise ColumnStore service will be started as-needed by the CMAPI service, so it does not require starting automatically upon reboot.

  3. Start and enable the CMAPI service, so that it starts automatically upon reboot:

    $ sudo systemctl restart mariadb-columnstore-cmapi
    $ sudo systemctl enable mariadb-columnstore-cmapi
    

Configure MariaDB Replication

MariaDB Enterprise ColumnStore requires MariaDB Replication, which must be configured.

  1. Get the GTID position that corresponds to the restored backup.

    If the backup was taken with MariaDB Backup, this position will be located in xtrabackup_binlog_info:

    $ cat xtrabackup_binlog_info
    mariadb-bin.000096 568 0-1-2001,1-2-5139
    

    The GTID position from the above output is 0-1-2001,1-2-5139.

  2. Connect to the Replica Server using MariaDB Client using the root@localhost user account:

    $ sudo mariadb
    
  3. Set the gtid_slave_pos system variable to the GTID position:

    SET GLOBAL gtid_slave_pos='0-1-2001,1-2-5139';
    
  4. Execute the CHANGE MASTER TO statement to configure the new node to connect to the Primary Server at this position:

    CHANGE MASTER TO
       MASTER_USER = "repl",
       MASTER_HOST = "192.0.2.1",
       MASTER_PASSWORD = "repl_passwd",
       MASTER_USE_GTID=slave_pos;
    

    The above statement configures the Replica Server to connect to a Primary Server located at 192.0.2.1 using the repl user account.

  5. Start replication using the START SLAVE command:

    START SLAVE;
    

    The above statement configures the new node to connect to the Primary Server to retrieve new binary log events and replicate them into the local database.

Add the Node to Enterprise ColumnStore

The new node must be added to Enterprise ColumnStore using CMAPI:

For example, if the primary node's host name is mcs1 and the new node's IP address is 192.0.2.3:

  • In ES 10.5.10-7 and later:

    $ curl -k -s -X PUT https://mcs1:8640/cmapi/0.4.0/cluster/node \
       --header 'Content-Type:application/json' \
       --header 'x-api-key:93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd' \
       --data '{"timeout":20, "node": "192.0.2.3"}' \
       | jq .
    
  • In ES 10.5.9-6 and earlier:

    $ curl -k -s -X PUT https://mcs1:8640/cmapi/0.4.0/cluster/add-node \
       --header 'Content-Type:application/json' \
       --header 'x-api-key:93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd' \
       --data '{"timeout":20, "node": "192.0.2.3"}' \
       | jq .
    

Example output:

{
  "timestamp": "2020-10-28 00:39:14.672142",
  "node_id": "192.0.2.3"
}

Check Enterprise ColumnStore Status

To confirm that the node was properly added, the status of Enterprise ColumnStore should be checked using CMAPI:

  • Check the status using the status endpoint path

For example, if the primary node's host name is mcs1:

$ curl -k -s https://mcs1:8640/cmapi/0.4.0/cluster/status \
   --header 'Content-Type:application/json' \
   --header 'x-api-key:93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd' \
   | jq .

Example output:

{
  "timestamp": "2020-12-15 00:40:34.353574",
  "192.0.2.1": {
    "timestamp": "2020-12-15 00:40:34.362374",
    "uptime": 11467,
    "dbrm_mode": "master",
    "cluster_mode": "readwrite",
    "dbroots": [
      "1"
    ],
    "module_id": 1,
    "services": [
      {
        "name": "workernode",
        "pid": 19202
      },
      {
        "name": "controllernode",
        "pid": 19232
      },
      {
        "name": "PrimProc",
        "pid": 19254
      },
      {
        "name": "ExeMgr",
        "pid": 19292
      },
      {
        "name": "WriteEngine",
        "pid": 19316
      },
      {
        "name": "DMLProc",
        "pid": 19332
      },
      {
        "name": "DDLProc",
        "pid": 19366
      }
    ]
  },
  "192.0.2.2": {
    "timestamp": "2020-12-15 00:40:34.428554",
    "uptime": 11437,
    "dbrm_mode": "slave",
    "cluster_mode": "readonly",
    "dbroots": [
      "2"
    ],
    "module_id": 2,
    "services": [
      {
        "name": "workernode",
        "pid": 17789
      },
      {
        "name": "PrimProc",
        "pid": 17813
      },
      {
        "name": "ExeMgr",
        "pid": 17854
      },
      {
        "name": "WriteEngine",
        "pid": 17877
      }
    ]
  },
  "192.0.2.3": {
    "timestamp": "2020-12-15 00:40:34.428554",
    "uptime": 11437,
    "dbrm_mode": "slave",
    "cluster_mode": "readonly",
    "dbroots": [
      "2"
    ],
    "module_id": 2,
    "services": [
      {
        "name": "workernode",
        "pid": 17789
      },
      {
        "name": "PrimProc",
        "pid": 17813
      },
      {
        "name": "ExeMgr",
        "pid": 17854
      },
      {
        "name": "WriteEngine",
        "pid": 17877
      }
    ]
  },
  "num_nodes": 3
}

Add a Server to MaxScale

A server object for the new node must also be added to MaxScale using MaxScale's REST API:

  • Use MaxCtrl or another supported REST client

  • Add the server object using the create server command

  • As the first argument, provide a name for the server

  • As the second argument, provide the IP address for the node

For example:

$ maxctrl create server \
   mcs3 \
   192.0.2.3

Check the Server in MaxScale

To confirm that the server object was properly added, the server objects should be checked using MaxScale's REST API:

For example:

$ maxctrl show servers

Check the Monitor in MaxScale

To confirm that the server object was properly linked to the monitor, the monitor should be checked using MaxScale's REST API:

For example:

$ maxctrl show monitors

Check the Service in MaxScale

To confirm that the server object was properly linked to the service, the service should be checked using MaxScale's REST API:

For example:

$ maxctrl show services

Check Replication Status with MaxScale

MaxScale is capable of checking the status of MariaDB Replication using MaxScale's REST API:

For example:

$ maxctrl list servers

If the new node is properly replicating, then the State column will show Slave, Running.