Deploy ColumnStore Shared Local Storage Topology with MariaDB Enterprise Server 10.5

MariaDB Enterprise Server Components

MariaDB MaxScale Components

Node Count

• MaxScale nodes, 1 or more are required.

• Enterprise ColumnStore nodes, 3 or more are required for high availability. You should always have an odd number of nodes in a multi-node ColumnStore deployment to avoid split brain scenarios.

Operating System

In alignment to the enterprise lifecycle, the ColumnStore Shared Local Storage topology with MariaDB Enterprise Server 10.5, MariaDB Enterprise ColumnStore 5, and MariaDB MaxScale 2.5 is provided for:

• CentOS Linux 7 (x86_64)

• Debian 10 (x86_64)

• Red Hat Enterprise Linux 7 (x86_64)

• Red Hat Enterprise Linux 8 (x86_64)

• Ubuntu 18.04 LTS (x86_64)

• Ubuntu 20.04 LTS (x86_64)

Minimum Hardware Requirements

MariaDB Enterprise ColumnStore's minimum hardware requirements are not intended for production environments, but the minimum hardware requirements can be appropriate for development and test environments. For production environments, see the recommended hardware requirements instead.

The minimum hardware requirements are:

MariaDB Enterprise ColumnStore will refuse to start if the system has less than 3 GB of memory.

If Enterprise ColumnStore is started on a system with less memory, the following error message will be written to the ColumnStore system log called crit.log:

Apr 30 21:54:35 a1ebc96a2519 PrimProc[1004]: 35.668435 |0|0|0| C 28 CAL0000: Error total memory available is less than 3GB.

And the following error message will be raised to the client:

ERROR 1815 (HY000): Internal error: System is not ready yet. Please try again.

Recommended Hardware Requirements

MariaDB Enterprise ColumnStore's recommended hardware requirements are intended for production analytics.

The recommended hardware requirements are:

Storage Requirements

The ColumnStore Shared Local Storage topology requires the following storage types:

Shared Local Storage Directories

The ColumnStore Shared Local Storage topology uses shared local storage for the DB Root directories to store data and metadata.

The DB Root directories is located at the following path by default:

• /var/lib/columnstore/dataN

The N in dataN represents a range of integers that starts at 1 and stops at the number of nodes in the deployment. For example, with a 3-node Enterprise ColumnStore deployment, this would refer to the following directories:

• /var/lib/columnstore/data1

• /var/lib/columnstore/data2

• /var/lib/columnstore/data3

To enable high availability for the DB Root directories, each directory should be mounted on every ColumnStore node using shared local storage.

For the most common shared local storage options, see "Shared Local Storage Options".

Shared Local Storage Options

The most common shared local storage options for the ColumnStore Shared Local Storage topology are:

URL Endpoint Format for REST API

https://{server}:{port}/cmapi/{version}/{route}/{command}

For example:

• https://mcs1:8640/cmapi/0.4.0/cluster/shutdown

• https://mcs1:8640/cmapi/0.4.0/cluster/start

• https://mcs1:8640/cmapi/0.4.0/cluster/status

With CMAPI 1.4 and later:

• https://mcs1:8640/cmapi/0.4.0/cluster/node

With CMAPI 1.3 and earlier:

• https://mcs1:8640/cmapi/0.4.0/cluster/add-node

• https://mcs1:8640/cmapi/0.4.0/cluster/remove-node

Required Request Headers

• 'x-api-key': '93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd'

• 'Content-Type': 'application/json'

x-api-key can be set to any value of your choice during the first call to the server. Subsequent connections will require this same key.

Get Status

$ curl -k -s https://mcs1:8640/cmapi/0.4.0/cluster/status \
      --header 'Content-Type:application/json' \
      --header 'x-api-key:93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd' \
      | jq .

Start Cluster

$ curl -k -s -X PUT https://mcs1:8640/cmapi/0.4.0/cluster/start \
      --header 'Content-Type:application/json' \
      --header 'x-api-key:93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd' \
      --data '{"timeout":20}' \
      | jq .

Stop Cluster

$ curl -k -s -X PUT https://mcs1:8640/cmapi/0.4.0/cluster/shutdown \
      --header 'Content-Type:application/json' \
      --header 'x-api-key:93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd' \
      --data '{"timeout":20}' \
      | jq .

Add Node

• With CMAPI 1.4 and later:

  $ curl -k -s -X PUT https://mcs1:8640/cmapi/0.4.0/cluster/node \
        --header 'Content-Type:application/json' \
        --header 'x-api-key:93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd' \
        --data '{"timeout":20, "node": "192.0.2.2"}' \
        | jq .
  

• With CMAPI 1.3 and earlier:

  $ curl -k -s -X PUT https://mcs1:8640/cmapi/0.4.0/cluster/add-node \
        --header 'Content-Type:application/json' \
        --header 'x-api-key:93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd' \
        --data '{"timeout":20, "node": "192.0.2.2"}' \
        | jq .
  

Remove Node

• With CMAPI 1.4 and later:

  $ curl -k -s -X DELETE https://mcs1:8640/cmapi/0.4.0/cluster/node \
        --header 'Content-Type:application/json' \
        --header 'x-api-key:93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd' \
        --data '{"timeout":20, "node": "192.0.2.2"}' \
        | jq .
  

• With CMAPI 1.3 and earlier:

  $ curl -k -s -X PUT https://mcs1:8640/cmapi/0.4.0/cluster/remove-node \
        --header 'Content-Type:application/json' \
        --header 'x-api-key:93816fa66cc2d8c224e62275bd4f248234dd4947b68d4af2b29671dd7d5532dd' \
        --data '{"timeout":20, "node": "192.0.2.2"}' \
        | jq .
  

MariaDB Enterprise Server Configuration Management

MariaDB Enterprise Server packages are configured to read configuration files from different paths, depending on the operating system. Making custom changes to Enterprise Server default configuration files is not recommended because custom changes may be overwritten by other default configuration files that are loaded later.

To ensure that your custom changes will be read last, create a custom configuration file with the z- prefix in one of the include directories.

MariaDB Enterprise Server Service Management

The systemctl command is used to start and stop the MariaDB Enterprise Server service.

For additional information, see "Start and Stop Services".

MariaDB Enterprise Server Logs

MariaDB Enterprise Server produces log data that can be helpful in problem diagnosis.

Log filenames and locations may be overridden in the server configuration. The default location of logs is the data directory. The data directory is specified by the datadir system variable.

Enterprise ColumnStore Service Management

The systemctl command is used to start and stop the ColumnStore service.

In the ColumnStore Shared Local Storage topology, the mariadb-columnstore service should not be enabled. The CMAPI service restarts Enterprise ColumnStore as needed, so it does not need to start automatically upon reboot.

Enterprise ColumnStore CMAPI Service Management

The systemctl command is used to start and stop the CMAPI service.

For additional information on endpoints, see "CMAPI".

MaxScale Configuration Management

MaxScale can be configured using several methods. These methods make use of MaxScale's REST API.

The procedure on these pages configures MaxScale using MaxCtrl.

MaxScale Service Management

The systemctl command is used to start and stop the MaxScale service.

For additional information, see "Start and Stop Services".