All pages
Powered by GitBook
1 of 3

Loading...

Loading...

Loading...

Transaction Coordinator Log

Understand the Transaction Coordinator log in MariaDB Server. This section explains its role in distributed transactions, ensuring data consistency across multiple servers.

Transaction Coordinator Log Overview

The transaction coordinator log (tc_log) is used to coordinate transactions that affect multiple XA-capable storage engines. If you have two or more XA-capable storage engines enabled, then a transaction coordinator log must be available.

Types of Transaction Coordinator Logs

There are currently two implementations of the transaction coordinator log:

  • Binary log-based transaction coordinator log

  • Memory-mapped file-based transaction coordinator log

If the is enabled on a server, then the server will use the binary log-based transaction coordinator log. Otherwise, it will use the memory-mapped file-based transaction coordinator log.

Binary Log-Based Transaction Coordinator Log

This transaction coordinator uses the , which is enabled by the server option.

Memory-Mapped File-Based Transaction Coordinator Log

This transaction coordinator uses the memory-mapped file defined by the server option. The size is defined by the system variable.

Some facts about this log:

  • The log consists of a memory-mapped file that is divided into pages of 8KB size.

  • The usable size of the first page is smaller because of the log header. There is a PAGE control structure for each page.

  • Each page (or rather its PAGE control structure) can be in one of the three states - active, syncing, pool.

  • There could be only one page in the active or syncing state, but many in the pool state - pool is a fifo queue.

The result of such an architecture is a natural "commit grouping" - If commits are coming faster than the system can sync, they do not stall. Instead, all commits that came since the last sync are logged to the same "active" page, and they all are synced with the next - one - sync. Thus, thought individual commits are delayed, throughput is not decreasing.

When an xid is added to an active page, the thread of this xid waits for a page's condition until the page is synced. When a syncing slot becomes vacant one of these waiters is awakened to take care of syncing. It syncs the page and signals all waiters that the page is synced. The waiters are counted, and a page may never become active again until waiters==0, which means that is all waiters from the previous sync have noticed that the sync was completed.

Note that a page becomes "dirty" and has to be synced only when a new xid is added into it. Removing a xid from a page does not make it dirty - we don't sync xid removals to disk.

Monitoring the Memory-Mapped File-Based Transaction Coordinator Log

The memory-mapped transaction coordinator log can be monitored with the following status variables:

Heuristic Recovery with the Transaction Coordinator Log

One of the main purposes of the transaction coordinator log is in crash recovery. See for more information about that.

Known Issues

You must enable exactly N storage engines

Prior to , if you were using the memory-mapped file-based transaction coordinator log, and then if the server crashed and you changed the number of XA-capable storage engines that it loaded, then you could see errors like the following:

To recover from this error, delete the file defined by the server option, and then restart the server with the option set.

See for more information.

Bad magic header in tc log

If you are using the memory-mapped file-based transaction coordinator log, then it is possible to see errors like the following:

This means that the header of the memory-mapped file-based transaction coordinator log is corrupt. To recover from this error, delete the file defined by the --log-tc server option, and then restart the server with the --tc-heuristic-recover option set.

This issue is known to occur when using docker. In that case, the problem may be caused by using a MariaDB container version with a data directory from a different MariaDB or MySQL version. Therefore, some potential fixes are:

  • Pinning the docker instance to a specific MariaDB version in the docker compose file, so that it consistently uses the same version.

  • Running to ensure that the data directory is upgraded to match the server version.

See for more information.

MariaDB Galera Cluster

builds include a built-in plugin called wsrep. Prior to , this plugin was internally considered an . Consequently, these MariaDB Galera Cluster builds have multiple XA-capable storage engines by default, even if the only "real" storage engine that supports external enabled on these builds by default is . Therefore, when using one these builds MariaDB would be forced to use a transaction coordinator log by default, which could have performance implications.

For example, describes performance problems where MariaDB Galera Cluster actually performs better when the is enabled. It is possible that this is caused by the fact that MariaDB is forced to use the memory-mapped file-based transaction coordinator log in this case, which may not perform as well.

This became a bigger issue in when the patch that powers MariaDB Galera Cluster was enabled on most MariaDB builds on Linux by default. Consequently, this built-in wsrep plugin would exist on those MariaDB builds on Linux by default. Therefore, MariaDB users might pay a performance penalty, even if they never actually intended to use the MariaDB Galera Cluster features included in .

In and later, the built-in wsrep plugin has been changed to a replication plugin. Therefore, it is no longer considered an storage engine, so it no longer forces MariaDB to use a transaction coordinator log by default.

See for more information.

This page is licensed: CC BY-SA / Gnu FDL

The usual lifecycle of a page is pool->active->syncing->pool.

  • The "active" page is a page where new xid's are logged.

  • The page stays active as long as the syncing slot is taken.

  • The "syncing" page is being synced to disk. no new xid can be added to it.

  • When the syncing is done the page is moved to a pool and an active page becomes "syncing".

  • binary log
    binary log
    log_bin
    --log-tc
    log_tc_size
    Tc_log_max_pages_used
    Tc_log_page_size
    Tc_log_page_waits
    Heuristic Recovery with the Transaction Coordinator Log
    --log-tc
    --tc-heuristic-recover
    MDEV-9214
    mariadb-upgrade
    this docker issue
    XA-capable
    storage engine
    XA transactions
    InnoDB
    MDEV-16509
    binary log
    MySQL-wsrep
    XA-capable
    MDEV-16442
    2018-11-30 23:08:49 140046048638848 [Note] Recovering after a crash using tc.log          
    2018-11-30 23:08:49 140046048638848 [ERROR] Recovery failed! You must enable exactly 3 storage engines that support two-phase commit protocol
    2018-11-30 23:08:49 140046048638848 [ERROR] Crash recovery failed. Either correct the problem (if it's, for example, out of memory error) and restart, or delete tc log and start mysqld with --tc-heuristic-recover={commit|rollback}
    2018-11-30 23:08:49 140046048638848 [ERROR] Can't init tc log
    2018-11-30 23:08:49 140046048638848 [ERROR] Aborting
    2018-09-19  4:29:31 0 [Note] Recovering after a crash using tc.log                                                               
    2018-09-19  4:29:31 0 [ERROR] Bad magic header in tc log
    2018-09-19  4:29:31 0 [ERROR] Crash recovery failed. Either correct the problem (if it's, for example, out of memory error) and restart, or delete tc log and start mysqld with --tc-heuristic-recover={commit|rollback}                                           
    2018-09-19  4:29:31 0 [ERROR] Can't init tc log
    2018-09-19  4:29:31 0 [ERROR] Aborting

    Heuristic Recovery with the Transaction Coordinator Log

    The transaction coordinator log (tc_log) is used to coordinate transactions that affect multiple XA-capable storage engines. One of the main purposes of this log is in crash recovery.

    Modes of Crash Recovery

    There are two modes of crash recovery:

    • Automatic crash recovery.

    • Manual heuristic recovery when is set to some value other than OFF.

    Automatic Crash Recovery

    Automatic crash recovery occurs during startup when MariaDB needs to recover from a crash and is set to OFF, which is the default value.

    Automatic Crash Recovery with the Binary Log-Based Transaction Coordinator Log

    If MariaDB needs to perform automatic crash recovery and if the is enabled, then the will contain messages like this:

    Automatic Crash Recovery with the Memory-Mapped File-Based Transaction Coordinator Log

    If MariaDB needs to perform automatic crash recovery and if the is not enabled, then the will contain messages like this:

    Manual Heuristic Recovery

    Manual heuristic recovery occurs when is set to some value other than OFF. This might be needed if the server finds prepared transactions during crash recovery that are not in the transaction coordinator log. For example, the might contain an error like this:

    When manual heuristic recovery is initiated, MariaDB will ignore information about transactions in the transaction coordinator log during the recovery process. Prepared transactions that are encountered during the recovery process will either be rolled back or committed, depending on the value of --tc-heuristic-recover.

    When manual heuristic recovery is initiated, the will contain a message like this:

    Manual Heuristic Recovery with the Binary Log-Based Transaction Coordinator Log

    If is set to some value other than OFF and if the is enabled, then MariaDB will ignore information about transactions in the during the recovery process. Prepared transactions that are encountered during the recovery process will either be rolled back or committed, depending on the value of --tc-heuristic-recover.

    After the recovery process is complete, MariaDB will create a new empty file, so that the old corrupt ones can be ignored.

    Manual Heuristic Recovery with the Memory-Mapped File-Based Transaction Coordinator Log

    If is set to some value other than OFF and if the is not enabled, then MariaDB will ignore information about transactions in the memory-mapped file defined by the option during the recovery process. Prepared transactions that are encountered during the recovery process will either be rolled back or committed, depending on the value of --tc-heuristic-recover.

    This page is licensed: CC BY-SA / Gnu FDL

    --tc-heuristic-recover
    --tc-heuristic-recover
    binary log
    error log
    binary log
    error log
    --tc-heuristic-recover
    error log
    error log
    --tc-heuristic-recover
    binary log
    binary log
    binary log
    --tc-heuristic-recover
    binary log
    --log-tc
    [Note] Recovering after a crash using cmdb-mariadb-0-bin
    [Note] InnoDB: Buffer pool(s) load completed at 190313 11:24:29
    [Note] Starting crash recovery...
    [Note] Crash recovery finished.
    [Note] Recovering after a crash using tc.log
    [Note] InnoDB: Buffer pool(s) load completed at 190313 11:26:32
    [Note] Starting crash recovery...
    [Note] Crash recovery finished.
    [ERROR] Found 1 prepared transactions! It means that mysqld was not shut down properly last time and critical recovery information (last binlog or tc.log file) was manually deleted after a crash. You have to start mysqld with --tc-heuristic-recover switch to commit or rollback pending transactions.
    [Note] Heuristic crash recovery mode
    MariaDB Galera Cluster
    MariaDB 10.1.10
    MariaDB 10.4.3
    MariaDB 10.1
    MariaDB 10.1
    MariaDB 10.4.3