Release Notes for MariaDB Xpand 6.1.0

Online

Starting with Xpand 6.1, when a node is added to a cluster, Xpand attempts to perform an online group change. When nodes are added as part of an online group change, the rest of the cluster is online and available for the duration of the group change.

When an online group change occurs:

• The new node(s) are marked with the "late" sub-state

• The Rebalancer copies data to the new node(s)

• Existing nodes continue processing transactions

• The new node(s) are fully able to service transactions, queries

• MaxScale can detect the new nodes and begin routing connections to the new nodes

• The new node(s) do not participate in some internal operations, such as the sequence manager and lock manager. The lockman_max_locks and lockman_max_transaction_locks system variables apply to non-late nodes only, and late nodes do not manage any locks.

Full

Full group changes are equivalent to the group changes from previous Xpand releases, but these types of group changes are known as full group changes going forward.

Full group changes cause the cluster to experience a brief period of unavailability during certain operations, such as the following:

• A node is removed from the cluster, due to a scale-in operation or an unexpected node or zone failure.

• If the number of nodes being added to the cluster is greater than or equal to the number of non-late nodes currently in the cluster.

• ALTER CLUSTER ADD .. COORDINATE is executed.

• The cluster is restarted via clx dbrestart

Scale-Out

As a part of this change in Xpand 6.1, the procedure to add nodes, which is known as the scale-out procedure, is overhauled to avoid a full group change when zones are not in use.

During scale-out, when zones are not in use, Xpand 6.1 adds the new nodes with the "late" sub-state. Administrators can join the system.nodeinfo and system.membership system tables or execute clx status to determine which nodes are "late".

For additional information about the scale-out procedure, see "Scale-Out with MariaDB Xpand 6.1".

For additional information about group changes, see "Group Changes with MariaDB Xpand 6.1".

AWS S3

• To use BACKUP with AWS S3:

  BACKUP *.* TO
     's3://AWS_BUCKET_NAME/OBJECT_NAME?access_key_id=AWS_ACCESS_KEY_ID&secret_access_key=AWS_SECRET_ACCESS_KEY&region=AWS_REGION&session_token=AWS_SESSION_TOKEN'"
  

• To use RESTORE with AWS S3:

  RESTORE *.* FROM
     's3://AWS_BUCKET_NAME/OBJECT_NAME?access_key_id=AWS_ACCESS_KEY_ID&secret_access_key=AWS_SECRET_ACCESS_KEY&region=AWS_REGION&session_token=AWS_SESSION_TOKEN'"
  

GCP Cloud Storage

• To use BACKUP with GCP Cloud Storage:

  BACKUP *.* TO
     'gs://GCP_BUCKET_NAME/OBJECT_NAME?credentials=GCP_CREDENTIALS_BASE64'
  

• To use RESTORE with GCP Cloud Storage:

  RESTORE *.* FROM
     'gs://GCP_BUCKET_NAME/OBJECT_NAME?credentials=GCP_CREDENTIALS_BASE64'
  

For additional information about cloud-based backup and restore, see "Fast Backup and Restore with MariaDB Xpand".

Create Roles

• To create one or more roles, use the CREATE ROLE statement:

  CREATE ROLE account_managers, customer_support, contractors;
  

Grant Privileges and Roles

• To grant privileges to one or more roles, use the GRANT statement:

  GRANT SELECT ON *.* TO account_managers, customer_support, contractors;
  

• To grant a role to one or more users, use the GRANT statement:

  GRANT account_managers TO alice, sofia;
  

Activate Roles

• To activate one or more roles for a user, use the SET ROLE statement:

  SET ROLE account_managers, customer_support;
  

• To activate all roles available for a user, use the SET ROLE ALL statement:

  SET ROLE ALL;
  

Default Roles

• To specify the default roles for a user, use the SET DEFAULT ROLE statement:

  SET DEFAULT ROLE customer_support FOR alice;
  

• To specify that all roles available for the user should be activated by default, use the SET DEFAULT ROLE ALL statement:

  SET DEFAULT ROLE ALL FOR alice;
  

For additional information about roles, see "Roles for MariaDB Xpand".

BINLOG ADMIN

• The BINLOG ADMIN privilege grants permissions for the following operations:

  • ALTER BINLOG

  • CREATE BINLOG

  • DROP BINLOG

  • SHOW BINLOG POSITION

  • SHOW BINLOG SEGMENTS

  • SHOW BINLOGS

  • TRIM BINLOG

CLUSTER ADMIN

• The CLUSTER ADMIN privilege grants permissions for the following operations:

  • ALTER CLUSTER

  • ALTER SLICE

  • ALTER REPLICA

CONNECTION ADMIN

• The CONNECTION ADMIN privilege grants permissions for the following operations:

  • KILL

REPAIR ADMIN

• The REPAIR ADMIN privilege grants permissions for the following operations:

  • REPAIR TABLE

REPLICATION ADMIN

• The REPLICATION ADMIN privilege grants permissions for the following operations:

  • CHANGE MASTER TO

  • CHANGE REPLICA

  • CHANGE SLAVE

  • CREATE REPLICA

  • CREATE SLAVE

  • DROP REPLICA

  • DROP SLAVE

  • REPLICA START

  • REPLICA STOP

  • SLAVE START

  • SLAVE STOP

  • START REPLICA

  • START SLAVE

  • STOP REPLICA

  • STOP SLAVE

SET USER

• The SET USER privilege grants permissions for the following operations:

  • CREATE DEFINER=[<user_name>] PROCEDURE

  • CREATE DEFINER=[<user_name>] TRIGGER

SYSTEM VARIABLES ADMIN

• The SYSTEM VARIABLES ADMIN privilege grants permissions for the following operations:

  • SELECT .. INTO @@global.<var_name>

  • SET GLOBAL <var_name>

Streaming Aggregations

In Xpand 6.1, streaming aggregation operations are enabled by default. (XPAND-36040)

The following system variables are enabled by default:

• enable_rule_scalar_stream_aggregate_impl

• enable_rule_scalar_stream_aggregate_over_partial

• enable_rule_groupby_stream_aggregate_impl

• enable_rule_dist_groupby_stream_aggregate_impl

In-Memory Skip-lists for Aggregations

• By default, Xpand 6.1 uses in-memory skip-lists for temporary data structures during execution of aggregation queries. (XPAND-35186)

• This change allows aggregation queries to be executed more efficiently.

• The default value of the sigma_skiplist system variable has changed from OFF to ON.

Set Threshold

• To set a threshold for the predictive governor, use the sierra_predictive_governor_threshold system variable:

  SET GLOBAL sierra_predictive_governor_threshold=<number_of_rows>;
  

Disable

• To disable the predictive governor, use the sierra_predictive_governor system variable:

  SET GLOBAL sierra_predictive_governor=OFF;
  

For additional information about the predictive governor, see "Configure the Predictive Governor with MariaDB Xpand".

Enable in Xpand

• To enable the proxy protocol in Xpand for a specific network, set the proxy_protocol_networks system variable to a space-separated list of IPv4 networks:

  SET GLOBAL proxy_protocol_networks='192.0.2.0/24 198.51.100.0/24';
  

• Xpand does not support a mix of proxy and non-proxy traffic on the same network.

Enable in MaxScale

• To enable the proxy protocol in MariaDB MaxScale, set the proxy_protocol parameter.

Disable in Xpand

• To disable the proxy protocol in Xpand, set the proxy_protocol_networks system variable to the empty string:

  SET GLOBAL proxy_protocol_networks='';
  

For additional information about the proxy protocol, see "Proxy Protocol".

clx_import

• The clx_import utility can detect and alert to non-retriable errors. (XPAND-35649)

• clx_import supports concurrent imports with the --allow-multi command-line option. (XPAND-34613)

  • The --allow-multi command-line option implies --no-globals, so that the concurrent imports don't leave system variables set incorrectly.

  • The first clx_import process can be run as usual without the --allow-multi command-line option, with the normal global system variable changes. The --allow-multi command-line option is only required for subsequent clx_import processes.

clx

• The clx utility is converted to Python 3. (XPAND-34219)

Character Sets and Collations

• Xpand 6.1 adds support for the cp850 character set and the cp850_general_ci collation. (XPAND-25915)

• Xpand 6.1 adds support for using prepared statements with the utf8mb4 character set. (XPAND-30737)

Data Types

• Xpand 6.1 adds support for the CLOB data type. (XPAND-36307, XPT-511)

• Xpand 6.1 adds support for the NVARCHAR data type. (XPAND-36306)

Strict Mode

• In Xpand 6.1, when strict mode is enabled using the STRICT_TRANS_TABLES or STRICT_ALL_TABLES SQL mode, data type and range checks are performed for more data types. (XPAND-894, XPAND-35266, XPAND-35362, XPAND-35491)

• Validation checks are added for the ENUM data type.

• Validation checks are added for the SET data type.

• Validation checks are added for the UNSIGNED INT data type.

DDL (Data Definition Language) Statements

• Xpand 6.1 adds support for IF EXISTS and IF NOT EXISTS to some ALTER TABLE statements:

  • ALTER TABLE .. ALTER COLUMN IF EXISTS (XPAND-35402)

  • ALTER TABLE .. ADD PARTITION IF NOT EXISTS (XPAND-35430)

  • ALTER TABLE .. CHANGE COLUMN IF EXISTS (XPAND-35402)

  • ALTER TABLE .. DROP COLUMN IF EXISTS (XPAND-35402)

  • ALTER TABLE .. DROP CONSTRAINT IF EXISTS (XPAND-34489)

  • ALTER TABLE .. DROP INDEX IF EXISTS (XPAND-34489)

  • ALTER TABLE .. DROP PARTITION IF EXISTS (XPAND-35430)

  • ALTER TABLE .. MODIFY COLUMN IF EXISTS (XPAND-35402)

• Xpand 6.1 adds support for a primary key that references the same column multiple times. (XPAND-32948)

• Xpand 6.1 adds an option to make transaction processing wait for ALTER TABLE operations to complete before continuing. (XPAND-13104)

  • To make transaction processing wait, set the global value of the alter_hold_until_checkpoint system variable to 1

  • This new setting is useful for ALTER TABLE operations that modify a column to be NOT NULL where inserts do not provide a value.

• Xpand 6.1 adds an option to control the default implicit locking behavior for ALTER TABLE operations. (XPAND-36409)

  • To control the implicit locking behavior, set the session or global value of the alter_lock system variable to NONE (the default), SHARED, or AUTO

  • The default value of NONE causes operations to fail when a lock is needed, which helps prevent accidental table locking.

  • To override the system variable for a single statement, set the LOCK clause:

    ALTER TABLE hq_sales.invoices
       DROP PRIMARY KEY, LOCK=SHARED;
    

• The REPAIR TABLE statement can repair tables that have corrupt unique indexes. (XPAND-35095)

Show Statements

• In Xpand 6.1, when the SHOW CREATE PROCEDURE statement is executed, the Create Procedure column is now a string instead of a blob. (XPAND-35696)

Information Schema

• In Xpand 6.1, the information_schema.COLUMNS table has been modified to be more similar to MySQL and MariaDB Enterprise Server. (XPAND-35574)

  • Starting with this release, the DATA_TYPE column is no longer suffixed with unsigned for columns using the FLOAT and DOUBLE data types.

• In Xpand 6.1, the information_schema.TABLES table has been modified to avoid returning inconsistent numbers of table rows on different cluster nodes. (XPAND-26352)

  • Starting with this release, the TABLE_ROWS column always reports 0

System Variables

• Xpand 6.1 adds dummy support for some system variables that are queried by MariaDB Connectors:

  • session_track_schema (XPAND-35400)

  • performance_schema (XPAND-35428)

Stored Functions

• Xpand 6.1 throws an error when a stored function parameter is declared as an UNSIGNED DOUBLE and a negative value is provided. (XPAND-35401)

Storage

• The sizes of individual partitions can be monitored using the system.partition_sizes view. (XPAND-33451)

• A histogram tracks I/O latency for each vdev in the system.vdev_io_latency_histogram table. (XPAND-34812)

statd

• When statd is used in Prometheus exporter mode, it removes cached stats for nodes that have been dropped. (XPAND-35943)

• When statd is used in Prometheus exporter mode, it can translate nanoseconds to seconds. (XPAND-36396)

• The total_busy metric is classified as a counter. (XPAND-36403)

• statd overrides the value of the session_log_slow_threshold_ms system variable, so that its queries are not logged to query.log. (XPAND-36418)

• The Com_commit and Com_rollback metrics can be used to monitor commits and rollbacks. (XPAND-24585)

Logging

• When the -hostname option is used to manually specify a hostname, Xpand writes the specified hostname to log files instead of the system hostname. (XPAND-35853)

Installer

• When the --storage-allocate option is specified to the installer, the wizard is skipped. (XPAND-35642)

• The installer decreases the TCP SYNACK retry count, so that the wait to detect a failed connection is reduced from 127 seconds to 15 seconds. (XPAND-35842)

• When installing Xpand 6.1, the installer tries to detect existing installations of Xpand or ClustrixDB. (XPAND-35957)

• The installer sets proper ownership of the pam_limits configuration file that it creates in the /etc/security/limits.d directory. (XPAND-35975)

• The installer is better at matching IP addresses to interfaces by parsing subnet masks. (XPAND-35967)

• The installer sets the correct permissions on the /etc/xpand/xpdnode.conf configuration file. (XPAND-35152)

Upgrade Script

• When the upgrade script fails, logs are left behind in the /tmp/xpand_upgrade_XXX directory, so that they can be inspected to diagnose the failure. (XPAND-30597)

• The upgrade script checks the MD5 checksum of the upgrade package earlier in the upgrade process, so that a corrupt package can be detected before the script tries to use the package. (XPAND-35935)

Character sets

• cp850 character set added

Collations

• cp850_general_ci collation added

Data Types

• CLOB data type added

• NVARCHAR data type added

Functions

• CAST_NULL_ON_ERR function added

Information Schema

• APPLICABLE_ROLES information schema table added

• ENABLED_ROLES information schema table added

Privileges

• BINLOG ADMIN Privilege user privileges added

• CLUSTER ADMIN Privilege user privileges added

• CONNECTION ADMIN Privilege user privileges added

• REPAIR ADMIN Privilege user privileges added

• REPLICATION ADMIN Privilege user privileges added

• SYSTEM VARIABLES ADMIN Privilege user privileges added

• SET USER Privilege user privileges added

SQL Statements

• CREATE ROLE SQL statement added

• DROP ROLE SQL statement added

• SET DEFAULT ROLE SQL statement added

• SET ROLE SQL statement added

Status Variables

• Com_commit status variable added

• Com_create_role status variable added

• Com_drop_role status variable added

• Com_grant_role status variable added

• Com_revoke_role status variable added

• Com_rollback status variable added

• qpc_recompile_total_factor status variable removed

System Tables

• columnar_containers system table added

• gmp system table added

• http_requests system table added

• http_socks system table added

• mysql_db_replication_policy system table removed

• roles system table added

• roles_mapping system table added

• session_acl system table added

• session_roles system table added

• session_routine_acl system table added

• session_variables system table added

• user_role_links system table added

• users_and_roles system table added

• vdev_io_latency_histogram system table added

System Variables

• alter_hold_until_checkpoint system variable added

• alter_lock system variable added

• barrier_row_cost system variable added

• barrier_setup_cost system variable added

• blocked_fibers_to_dump system variable added

• cost_sigma_wait system variable added

• dbstart_flip_empty_queues system variable added

• enable_rule_filter_isnull_from_or_removal system variable added

• http_idle_timeout_secs system variable added

• http_low_speed_limit system variable added

• http_low_speed_timeout_secs system variable added

• jdbcCompliantTruncation system variable removed

• join_swap_threshold system variable added

• performance_schema system variable added

• proxy_protocol_networks system variable added

• qpc_max_cost_factor system variable removed

• qpc_scale_tot_factor system variable removed

• row_limit_multiplier system variable added

• sierra_or_num_inputs_threshold system variable added

• sierra_predictive_governor system variable added

• sierra_predictive_governor_threshold system variable added

• sierra_remove_empty_filter_above_join system variable added

• sierra_skip_magic_formula_no_pd system variable added

• sierra_table_def_card system variable added

• straight_join_threshold system variable added

• task_tcp_arp_interval_ms system variable removed

• tx_sync_commit system variable removed

• union_all_rule_threshold system variable added

• url_redact system variable added

XpandGUI

• XpandGUI is deprecated and will be removed in a future release.

• As a replacement for XpandGUI, use the Remote Observability Service.