Using MariaDB GTIDs with MariaDB Galera Cluster

MariaDB's global transaction IDs (GTIDs) are very useful when used with MariaDB replication, which is primarily what that feature was developed for. Galera Cluster, on the other hand, was developed by Codership for all MySQL and MariaDB variants, and the initial development of the technology pre-dated MariaDB's GTID implementation. As a side effect, MariaDB Galera Cluster only partially supports MariaDB's GTID implementation.

GTID Support for Write Sets Replicated by Galera Cluster

Galera Cluster has its own certification-based replication method that is substantially different from MariaDB replication. However, it would still be beneficial if MariaDB Galera Cluster was able to associate a Galera Cluster write set with a GTID that is globally unique, but that is also consistent for that write set on each cluster node.

MariaDB Galera Cluster does not currently replicate the original GTID with the write set except in cases where the transaction was originally applied by a slave SQL thread. Each node independently generates its own GTID for each write set in most cases. See MDEV-10715 about that.

Wsrep GTID Mode

MariaDB starting with 10.1.4

MariaDB has supported wsrep_gtid_mode since version 10.1.4.

MariaDB 10.1 and above has a feature called wsrep GTID mode. When this mode is enabled, MariaDB uses some tricks to try to associate each Galera Cluster write set with a GTID that is globally unique, but that is also consistent for that write set on each cluster node. These tricks work in some cases, but GTIDs can still become inconsistent among cluster nodes.

Enabling Wsrep GTID Mode

Several things need to be configured for wsrep GTID mode to work, such as:

  • wsrep_gtid_domain_id needs to be set to the same value on all nodes in a given cluster, so that each cluster node uses the same domain when assigning GTIDs for Galera Cluster's write sets. When replicating between two clusters, each cluster should have this set to a different value, so that each cluster uses different domains when assigning GTIDs for their write sets.
  • log_bin needs to be set to the same path on all nodes in the cluster. See MDEV-9856 about that.

And as an extra safety measure:

  • gtid_domain_id should be set to a different value on all nodes in a given cluster, and each of these values should be different than the configured wsrep_gtid_domain_id value. This is to prevent a node from using the same domain used for Galera Cluster's write sets when assigning GTIDs for non-Galera transactions, such as DDL executed with wsrep_sst_method=RSU set or DML executed with wsrep_on=OFF set.

For information on setting server_id, see Using MariaDB Replication with MariaDB Galera Cluster: Setting server_id on Cluster Nodes.

Known Problems with Wsrep GTID Mode

Wsrep GTID mode is not a perfect fix that allows MariaDB's GTIDs to work perfectly with MariaDB Galera Cluster. There are definitely cases where GTIDs can become inconsistent across the cluster nodes.

Some known issues are:

  • Implicitly dropped temporary tables can make GTIDs inconsistent. See MDEV-14153 about that.

This is the list of known problems, but this does not necessarily imply that wsrep GTID mode works perfectly in all other situations. If you discover any other issues with it, please report a bug.

GTIDs for Transactions Applied by Slave Thread

If a Galera Cluster node is also a replication slave, then that node's slave SQL thread will be applying transactions that it replicates from its replication master. If the node has log_slave_updates=ON set, then each transaction that the slave SQL thread applies will also generate a Galera Cluster write set that is replicated to the rest of the nodes in the cluster.

In MariaDB 10.1.30 and earlier, the node acting as slave would apply the transaction with the original GTID that it received from the master, and the other Galera Cluster nodes would generate their own GTIDs for the transaction when they replicated the write set.

In MariaDB 10.1.31 and later, the node acting as slave will include the transaction's original Gtid_Log_Event in the replicated write set, so all nodes should associate the write set with its original GTID. See MDEV-13431 about that.

Comments

Comments loading...