Step 3: Start and Configure MariaDB Enterprise Server

Overview

This page details step 3 of the 4-step procedure "Deploy HTAP Topology".

This step starts and configures MariaDB Enterprise Server 10.6 and MariaDB Enterprise ColumnStore 6.

Interactive commands are detailed. Alternatively, the described operations can be performed using automation.

Configure Enterprise ColumnStore

Mandatory system variables and options for HTAP include:

System Variable/Option

Description

binlog_do_db

Set this to the name of the database to replicate from InnoDB to ColumnStore.

binlog_format

Set this to STATEMENT for HTAP.

character_set_server

Set this system variable to utf8

collation_server

Set this system variable to utf8_general_ci

columnstore_use_import_for_batchinsert

Set this system variable to ALWAYS to always use cpimport for LOAD DATA INFILE and INSERT...SELECT statements.

log_bin

Set this option to enable the Binary Log. This is a file path.

log_slave_updates

Set this system variable to ON.

replicate_rewrite_db

Set this to the names of the InnoDB and ColumnStore databases in the format <innodb database>-><columnstore database>.

replicate_wild_do_table

Set this to a pattern that matches the table names that you want to replicate from InnoDB to ColumnStore.

server_id

Sets the numeric Server ID for this MariaDB Enterprise Server. Must be unique for each cluster node in the deployment.

Example Configuration

[mariadb]
log_error                              = mariadbd.err
character_set_server                   = utf8
collation_server                       = utf8_general_ci
columnstore_use_import_for_batchinsert = ALWAYS

# Replication Configuration (HTAP Server)
server_id                     = 1
log_bin                       = mariadb-bin
binlog_format                 = STATEMENT
log_slave_updates             = OFF
columnstore_replication_slave = ON

# HTAP filtering rules

# Transactions replicate from same server
replicate_same_server_id = ON

# Only write queries that touch 'innodb_db' to the binary log
binlog_do_db = innodb_db

# Rewrite innodb_db to columnstore_db prior to applying transaction
replicate_rewrite_db = innodb_db->columnstore_db

# Only replicate tables that begin with "htap"
replicate_wild_do_table = columnstore_db.htap%

Configure the S3 Storage Manager

Configure Enterprise ColumnStore S3 Storage Manager to use S3-compatible storage by editing the /etc/columnstore/storagemanager.cnf configuration file:

[ObjectStorage]

service = S3

[S3]
bucket                = your_columnstore_bucket_name
endpoint              = your_s3_endpoint
aws_access_key_id     = your_s3_access_key_id
aws_secret_access_key = your_s3_secret_key
# iam_role_name       = your_iam_role
# sts_region          = your_sts_region
# sts_endpoint        = your_sts_endpoint
# ec2_iam_mode        = enabled
# port_number         = your_port_number

[Cache]
cache_size = your_local_cache_size
path       = your_local_cache_path

The S3-compatible object storage options are configured under [S3]:

  • The bucket option must be set to the name of the bucket that you created in "Create an S3 Bucket".

  • The endpoint option must be set to the endpoint for the S3-compatible object storage.

  • The aws_access_key_id and aws_secret_access_key options must be set to the access key ID and secret access key for the S3-compatible object storage.

  • To use a specific IAM role, you must uncomment and set iam_role_name, sts_region, and sts_endpoint.

  • To use the IAM role assigned to an EC2 instance, you must uncomment ec2_iam_mode=enabled.

  • To use a non-default port number, you must set port_number to the desired port.

The local cache options are configured under [Cache]:

  • The cache_size option is set to 2 GB by default.

  • The path option is set to /var/lib/columnstore/storagemanager/cache by default.

Ensure that the specified path has sufficient storage space for the specified cache size.

Start the Enterprise ColumnStore Services

  1. Start and enable the MariaDB Enterprise Server service, so that it starts automatically upon reboot:

    $ sudo systemctl restart mariadb
    
    $ sudo systemctl enable mariadb
    
  2. Start and enable the MariaDB Enterprise ColumnStore service, so that it starts automatically upon reboot:

    $ sudo systemctl restart mariadb-columnstore
    
    $ sudo systemctl enable mariadb-columnstore
    

Create User Accounts

The HTAP topology requires several user accounts.

Create the Utility User

Enterprise ColumnStore requires a mandatory utility user account. By default, it connects to the server using the root user with no password. MariaDB Enterprise Server 10.6 will reject this login attempt by default, so you will need to configure Enterprise ColumnStore to use a different user account and password and create this user account on Enterprise Server.

  1. On the Enterprise ColumnStore node, create the user account with the CREATE USER statement:

    CREATE USER 'util_user'@'127.0.0.1'
    IDENTIFIED BY 'util_user_passwd';
    
  2. On the Enterprise ColumnStore node, grant the user account SELECT privileges on all databases with the GRANT statement:

    GRANT SELECT, PROCESS ON *.*
    TO 'util_user'@'127.0.0.1';
    
  3. Configure Enterprise ColumnStore to use the utility user:

    $ sudo mcsSetConfig CrossEngineSupport Host 127.0.0.1
    
    $ sudo mcsSetConfig CrossEngineSupport Port 3306
    
    $ sudo mcsSetConfig CrossEngineSupport User util_user
    
  4. Generate the encryption key file:

    $ cskeys
    
  5. Generate an encrypted password:

    $ cspasswd util_user_passwd
    
  6. Set the encrypted password:

    $ sudo mcsSetConfig CrossEngineSupport Password util_user_encrypted_passwd
    

Passwords should meet your organization's password policies. If your MariaDB Enterprise Server instance has a password validation plugin installed, then the password should also meet the configured requirements.

Create the Replication User

Enterprise HTAP uses MariaDB Replication to replicate writes between InnoDB tables and ColumnStore tables.

Create a replication user and grant it the required privileges:

  1. Use the CREATE USER statement to create replication users for each replica server:

    CREATE USER 'repl'@'localhost' IDENTIFIED BY 'passwd';
    
  2. Grant the user account several global privileges with the GRANT statement.

    GRANT REPLICA MONITOR,
       REPLICATION REPLICA
    ON *.* TO 'repl'@'localhost';
    

Configure MariaDB Replication

  1. Set the GTID position by setting the gtid_slave_pos system variable.

    If this is a new deployment, then it would be set to the empty string:

    SET GLOBAL gtid_slave_pos='';
    
  2. Use the CHANGE MASTER TO statement to configure the server to replicate from itself starting from this position:

    CHANGE MASTER TO
       MASTER_HOST='localhost',
       MASTER_USER='htap_replication',
       MASTER_PASSWORD='passwd',
       MASTER_USE_GTID=slave_pos;
    
  3. Start replication using the START REPLICA statement:

    START REPLICA;
    
  4. Confirm that replication is working using the SHOW REPLICA STATUS statement:

    SHOW REPLICA STATUS;
    

Configure Linux Security Modules (LSM)

The specific steps to configure the security module depend on the operating system.

Configure SELinux (CentOS/RHEL)

Configure SELinux for Enterprise ColumnStore:

  1. To configure SELinux, you have to install the packages required for audit2allow.

    On CentOS 7 and RHEL 7, install the following:

    $ sudo yum install policycoreutils policycoreutils-python
    

    On CentOS 8 and RHEL 8, install the following:

    $ sudo yum install policycoreutils python3-policycoreutils policycoreutils-python-utils
    
  2. Allow the system to run under load for a while to generate SELinux audit events.

  3. After the system has taken some load, generate an SELinux policy from the audit events using audit2allow:

    $ sudo grep mysqld /var/log/audit/audit.log | audit2allow -M mariadb_local
    

    If no audit events were found, this will print the following:

    $ sudo grep mysqld /var/log/audit/audit.log | audit2allow -M mariadb_local
    
    Nothing to do
    
  4. If audit events were found, the new SELinux policy can be loaded using semodule:

    $ sudo semodule -i mariadb_local.pp
    
  5. Set SELinux to enforcing mode by setting SELINUX=enforcing in /etc/selinux/config.

    For example, the file will usually look like this after the change:

    # This file controls the state of SELinux on the system.
    # SELINUX= can take one of these three values:
    #     enforcing - SELinux security policy is enforced.
    #     permissive - SELinux prints warnings instead of enforcing.
    #     disabled - No SELinux policy is loaded.
    SELINUX=enforcing
    # SELINUXTYPE= can take one of three values:
    #     targeted - Targeted processes are protected,
    #     minimum - Modification of targeted policy. Only selected processes are protected.
    #     mls - Multi Level Security protection.
    SELINUXTYPE=targeted
    
  6. Set SELinux to enforcing mode:

    $ sudo setenforce enforcing
    

Configure AppArmor (Ubuntu)

For information on how to create a profile, see How to create an AppArmor Profile on ubuntu.com.

Next Step

Navigation in the procedure "Deploy HTAP Topology":

  • This page was step 3 of 4.