Release Notes for MariaDB Xpand 6.0.3

Overview

MariaDB Xpand is a distributed SQL database that is part of MariaDB Enterprise. This is the first General Availability (GA) release in the MariaDB Xpand 6 series.

MariaDB Xpand 6.0.3 was released on 2022-03-10.

Notable Changes

Columnar Indexes

  • MariaDB Xpand 6 provides support for columnar indexes, which can be used to speed up performance of OLAP-style queries.

  • Columnar indexes can improve performance of ad-hoc queries, where the fields in a query are not known in advance, because Columnar indexes provide efficient filtration capabilities that are not dependent on column order.

  • Columnar indexes can improve performance of operational analytics, where real-time analytics are performed on transactional data, because Columnar indexes outperform row-based secondary indexes for analytical queries that perform range scans and aggregations.

  • Columnar indexes are compressed automatically.

  • A single Columnar index can be created for each table.

  • Single column and composite (multi-column) Columnar indexes can be defined.

  • To add a Columnar index to an existing table, execute ALTER TABLE .. ADD COLUMNAR INDEX:

ALTER TABLE hq_sales.invoices
   ADD COLUMNAR INDEX cols (invoice_date, invoice_total);
  • For best performance, MariaDB recommends loading data into the table prior to adding a Columnar index.

  • The cost_read_column system variable is used for costing columnar reads. The default value of cost_read_column in Xpand 5.3 was 0.01. For deployments upgrading to Xpand 6 from previous versions, the value of cost_read_column should be set to 0.9.

  • For additional information, see "MariaDB Xpand Columnar Index Architecture".

Parallel Replication

  • In MariaDB Xpand 6, parallel replication is Generally Available (GA) and suitable for production.

  • Parallel replication minimizes latency and slave lag.

  • Parallel replication provides extremely scalable replication. When parallel replication is configured to use a specified number of parallel Slave Event Readers / Applicators, Xpand uses the consistent hashing algorithm to uniformly distribute the Slave Event Readers / Applicators between all of the nodes. The Slave Event Readers / Applicators on each node independently perform work in parallel. Each Slave Event Reader / Applicator independently streams and applies binary log events in parallel, which allows the full power of the cluster to be used without any choke points.

  • When parallel replication is used, MariaDB Xpand ensures transactional consistency and commits transactions in the same order as the replication primary.

  • Parallel replication requires row-based replication (RBR).

  • To configure parallel replication, execute CREATE SLAVE and set the options for parallel replication:

CREATE SLAVE slave_name PARALLEL_LOG = master_log_name, PARALLEL_POS = position, SLICES = num_slices, BATCH_SIZE_MS = batch_size
   [, MASTER_HOST = master_host]
   [, MASTER_USER = master_user]
   [, MASTER_PASSWORD =  master_password]
   [, MASTER_PORT = master_port];

Strict Mode

  • MariaDB Xpand 6 includes some improvements related to strict mode.

  • When sql_mode=STRICT_TRANS_TABLES is set, MariaDB Xpand 6 validates when values are out of range and performs type checks on input. Errors will be raised for invalid input for all data types except:

  • For compatibility, MariaDB Xpand 6 adds dummy support for MariaDB Server's innodb_strict_mode system variable.

Exclude Statements from Binlogs

  • MariaDB Xpand 6 provides a new system variable to exclude statements in the current session from being written to specific binlogs. (XPT-117)

  • The sql_log_skip_binlogs system variable can be set at the session-level and accepts a comma-separated list of binlog names.

  • To exclude the current session's statements from binlogA and binlogB, set the system variable by executing SET:

SET sql_log_skip_binlogs='binlogA,binlogB';
  • To re-enable writes to all binlogs for the current session, set the system variable back to DEFAULT:

SET sql_log_skip_binlogs=DEFAULT;

Static PDs (Probability Distributions)

  • MariaDB Xpand 6 provides the ability to statically collect PDs (Probability Distributions).

  • Probability Distributions (PDs) are statistics on data distribution used to generate optimal query execution plans.

  • Prior to this release, all Probability Distributions were dynamic.

  • With this release, Static PDs allow statistics to be gathered on-demand, removing the overhead of continual statistics collection. With Static PDs, statistics can be collected off-peak and persisted indefinitely.

ENGINE=XPAND

  • MariaDB Xpand 6 provides support for ENGINE=XPAND syntax.

  • With this release, when using the Xpand Performance Topology, CREATE TABLE .. ENGINE=XPAND and ALTER TABLE .. ENGINE=XPAND are accepted syntax.

  • Prior to this release, the CREATE TABLE .. ENGINE=XPAND and ALTER TABLE .. ENGINE=XPAND was only available when using the Xpand Storage Engine Topology to select Xpand rather than the default storage engine, and the syntax would cause a syntax error to be raised when using the Xpand Performance Topology.

Xpand Branding

  • MariaDB Xpand 6 includes some branding-related changes to directory names, file names, and messages.

  • As part of re-branding efforts, some Clustrix-related files and directories have been changed to reference Xpand:

    • Renamed /etc/clustrix to /etc/xpand

    • Renamed /etc/clustrix/clxnode.conf to /etc/xpand/xpdnode.conf

    • Renamed uninstall_clustrix to uninstall_xpand

  • Xpand's Alerter looks for noalerts in /etc/xpand.

  • Xpand's Upgrader uses /etc/xpand instead of /etc/clustrix.

    • If /etc/xpand doesn't exist, the upgrader will print instructions on how to create a symbolic link to from /etc/clustrix to /etc/xpand, and the upgrade will be aborted.

    • Once a symbolic link exists, the upgrade can be re-run.

  • Upgrade messages reference Xpand instead of Clustrix.

Other

  • A command line argument has been added to specify the node's hostname. (XPT-44)

  • rsync has been added to the list of RPM dependencies.

  • The fielddebug and wal_fs_dump utilities are part of the standard Xpand rpm.

  • Support for _utf8mb3 character set literals has been added.

  • OOM protection applies only when Xpand runs as root.

  • clx_import will not re-slice ALLNODES tables.

  • clx_import can prompt for password:

    • Prior to this release, clx_import required password to be specified on the command-line.

    • With this release, clx_import prompts for password when not specified on the command-line. (XPT-68)

  • When sierra_allow_right_deep_tree=1 is set, query plans are improved.

  • When a query contains a LEFT JOIN with an IN(..) list predicate, cardinality estimates are improved.

  • The softfail procedure has been updated to include additional steps when binlogs are present. (XPT-256)

Issues Fixed

Can result in data loss

  • After a partial rollback, corruption could occur when writing undo log records due to an issue with cache validation.

Can result in a hang or crash

  • When STOP SLAVE is executed, the slave node could crash.

  • When parallel replication is configured, an Xpand node could crash due to an out of memory condition. (XPT-135)

  • During query optimization, internal statistics could be erroneously re-used and cause a crash.

  • When multiple sessions concurrently execute ANALYZE TABLE on the same table, a crash could occur.

Can result in unexpected behavior

  • When the binlog indexer is behind, the performance of SHOW BINLOG FILES could be impacted. (XPT-113)

  • When the binlog indexer encounters a 0-size transaction, the binlog indexer could not proceed, and performance of SHOW BINLOG FILES and TRIM BINLOG could be impacted. (XPT-164)

  • Logging for an RBR replication slave does not correctly surface information about errors. (XPT-255)

  • REPAIR TABLE does not work with generated columns.

  • When a query containing an IN(..) list is executed as a prepared statement, the prepared statement could be translated multiple times.

  • Upgrades could re-set variables that were aliased, such as mysql_port and port. (XPT-162)

  • A group change could result in a container being leaked. (XPT-98)

  • Replication to or from MariaDB Server 10.6 does not work.

  • When a user tries to change their own password with SET PASSWORD, the SUPER privilege is incorrectly required. (XPT-148)

  • Logging should occur earlier in the group change process to optimize group change performance for environments where the HugeTLB is not enabled. (XPT-183)

  • When sql_mode=STRICT_TRANS_TABLES is set, a negative value inserted into an UNSIGNED FLOAT does not raise an error. (XPT-143)

  • When sql_mode=STRICT_TRANS_TABLES is set, validation is not performed for FLOAT, DOUBLE, and REAL UNSIGNED. (XPT-143)

  • Arguments to the current_timestamp(), current_time(), curtime(), now(), localtime(), localtimestamp(), utc_timestamp(), and utc_time() functions are not required to be in a valid range.

  • uninstall_clustrix is not replaced with uninstall_xpand when upgrading from previous Xpand versions.

  • Additional clx options could not be used in conjunction with debugdump.

  • The CLX tool can not parse Unicode characters.

  • Changes to MAX_REDO in clxnode.conf are not applied.

  • Executing SHOW COLUMNS or SHOW FIELDS executed against a non-existent table does not throw an error.

  • The USE INDEX hint does not work with DELETE statements.

Interface Changes

Added System Variables

System Variable

Default

Description

columnar_compression_algorithm

'xlz'

Compression algorithm to use for columnar containers

columnar_frame_size

64

Number of values per frame

columnar_rows_per_extent

2097152

Number of values to add to a columnar extent before creating a new one

innodb_strict_mode

true

Dummy variable for compatibility.

parallel_slave_unused_vrel_lifetime

100

How many batches unused slave_coord_relations will be kept for before eviction

sha256_password_private_key_path

RSA private key file

sha256_password_public_key_path

RSA public key file

sql_log_skip_binlogs

Do not log to the specified comma-separated binlogs

Changed System Variable Defaults

System Variable

Old Default

New Default

cost_read_column

0.01

0.9

mysql_relay_log_bytes

67108864

268435456

sierra_enable_static_pds

false

true

slave_slice_max_bytes

67108864

268435456

Renamed Files and Directories

  • Renamed /etc/clustrix to /etc/xpand

  • Renamed /etc/clustrix/clxnode.conf to /etc/xpand/xpdnode.conf

  • Renamed uninstall_clustrix to uninstall_xpand

Changes to statd and exporters

  • Added stats to statd and exporters for:

    • lockman.sessions_holding_locks

    • lockman.waiters histogram

    • sessions.node.X.limit and sessions.node.X.remaining

    • transactions.open.seconds

    • io.network.internode.latency.seconds

  • Renamed statd metric from lockman.num_locks to lockman.num_waiters (XPT-165)

Upgrade Instructions