Using MariaDB GTIDs with MariaDB Galera Cluster
MariaDB's global transaction IDs (GTIDs) are very useful when used with MariaDB replication, which is primarily what that feature was developed for. Galera Cluster, on the other hand, was developed by Codership for all MySQL and MariaDB variants, and the initial development of the technology pre-dated MariaDB's GTID implementation. As a side effect, MariaDB Galera Cluster (at least until MariaDB 10.5.1) only partially supports MariaDB's GTID implementation.
GTID Support for Write Sets Replicated by Galera Cluster
Galera Cluster has its own certification-based replication method that is substantially different from MariaDB replication. However, it would still be beneficial if MariaDB Galera Cluster was able to associate a Galera Cluster write set with a GTID that is globally unique, but that is also consistent for that write set on each cluster node.
Before MariaDB 10.5.1, MariaDB Galera Cluster did not replicate the original GTID with the write set except in cases where the transaction was originally applied by a slave SQL thread. Each node independently generated its own GTID for each write set in most cases. See MDEV-20720.
Wsrep GTID Mode
MariaDB starting with 10.1.4
MariaDB has supported
wsrep_gtid_mode since version 10.1.4.
MariaDB 10.1 and above has a feature called wsrep GTID mode. When this mode is enabled, MariaDB uses some tricks to try to associate each Galera Cluster write set with a GTID that is globally unique, but that is also consistent for that write set on each cluster node. These tricks work in some cases, but GTIDs can still become inconsistent among cluster nodes.
Enabling Wsrep GTID Mode
Several things need to be configured for wsrep GTID mode to work, such as:
wsrep_gtid_mode=ONneeds to be set on all nodes in the cluster.
wsrep_gtid_domain_idneeds to be set to the same value on all nodes in a given cluster, so that each cluster node uses the same domain when assigning GTIDs for Galera Cluster's write sets. When replicating between two clusters, each cluster should have this set to a different value, so that each cluster uses different domains when assigning GTIDs for their write sets.
log_slave_updatesneeds to be enabled on all nodes in the cluster. See MDEV-9855.
And as an extra safety measure:
gtid_domain_idshould be set to a different value on all nodes in a given cluster, and each of these values should be different than the configured
wsrep_gtid_domain_idvalue. This is to prevent a node from using the same domain used for Galera Cluster's write sets when assigning GTIDs for non-Galera transactions, such as DDL executed with
wsrep_sst_method=RSUset or DML executed with
For information on setting
server_id, see Using MariaDB Replication with MariaDB Galera Cluster: Setting server_id on Cluster Nodes.
Known Problems with Wsrep GTID Mode
Until MariaDB 10.5.1, there were known cases where GTIDs could become inconsistent across the cluster nodes.
A known issue (fixed in MariaDB 10.5.1) is:
- Implicitly dropped temporary tables can make GTIDs inconsistent. See MDEV-14153 and MDEV-20720.
This does not necessarily imply that wsrep GTID mode works perfectly in all other situations. If you discover any other issues with it, please report a bug.
GTIDs for Transactions Applied by Slave Thread
If a Galera Cluster node is also a replication slave, then that node's slave SQL thread will be applying transactions that it replicates from its replication master. If the node has
log_slave_updates=ON set, then each transaction that the slave SQL thread applies will also generate a Galera Cluster write set that is replicated to the rest of the nodes in the cluster.
In MariaDB 10.1.30 and earlier, the node acting as slave would apply the transaction with the original GTID that it received from the master, and the other Galera Cluster nodes would generate their own GTIDs for the transaction when they replicated the write set.
In MariaDB 10.1.31 and later, the node acting as slave will include the transaction's original
Gtid_Log_Event in the replicated write set, so all nodes should associate the write set with its original GTID. See MDEV-13431 about that.