Debugging MariaDB With a Debugger

A technical guide on using standard debuggers like GDB to step through code and inspect the state of a MariaDB instance.

This page is focused on lower-level development: It contains technical "how-to" documentation for engineers who are actually modifying the MariaDB source code (e.g., compiling for debug, using GDB, and memory leak detection in the code).

If you have MariaDB compiled for debugging you can both use it in a debugger, like ddd or gdb, and get comprehensive trace files of the execution of MariaDB. The trace files allow you to both see the flow of the code and to see the differences in execution by comparing two trace files.

Core dumps are also much easier to investigate if they come from a debug binary.

Note that a binary compiled for debugging and tracing is about 10-20% slower than a normal binary. If you just compile a binary for debugging (option -g with gcc) the speed difference compared to a normal binary is negligible.

Checking That MariaDB is Compiled For Debugging

Execute:

mariadbd --debug --help

If you get an error unknown option '--debug, then MariaDB is not compiled for debugging and tracing.

Building MariaDB for Debugging

On Unix you need to pass -DCMAKE_BUILD_TYPE=Debug to cmake to compile with debug information.

Alternatively, use the scripts in the BUILD directory that will compile MariaDB with most common debug options and plugins, for example:

./BUILD/compile-pentium64-debug-max

For the most common configurations there exists a fine-tuned script in the BUILD directory.

If you want to use valgrind, a very good memory instrumentation tool and memory overrun checker, you should use

./BUILD/compile-pentium64-valgrind-max

Some recommended debugging scripts for Intel/AMD are:

BUILD/compile-pentium64-debug-max
BUILD/compile-pentium64-valgrind-max

This is an example of how to compile MariaDB for debugging in your home directory from a source tarball:

cd ~
mkdir mariadb
cd mariadb
tar xvf mariadb-11.4.8.tar.gz
ln -s mariadb-11.4.8 current
cd current
./BUILD/compile-pentium64-debug-max

The last command produces a debug version of sql/mariadbd.

Debugging MariaDB From the Source Directory

Creating the MariaDB Database Directory

The following example creates the MariaDB databases in /data.

./scripts/mariadb-install-db --srcdir=. --datadir=/data

Running MariaDB in a Debugger

The following example is using ddd, an excellent graphical debugger in Linux. If you don't have ddd installed, you can use gdb instead.

cd sql
ddd ./mariadbd &

In ddd or gdb

run --datadir=/data --language=./share/english --gdb

You can set the options in your /.my.cnf file so as not to have to repeat them on the run line.

If you run mariadbd with --debug, you get a trace file in /tmp/mariadbd.trace that shows what is happening.

Note that you can have different options in the configuration file for each MariaDB version (like having a specific language directory).

Debugging MariaDB Server with mariadb-test-run

If you get a crash while running mariadb-test-run you can debug this in a debugger by using one of the following options:

mariadb-test-run --gdb failing-test-name

or if you prefer the ddd debugger:

mariadb-test-run --ddd failing-test-name

Sample .my.cnf file to Make Debugging Easier

[client-server]
socket=/tmp/mysql-dbug.sock
port=3307

[mariadb]
datadir=/my/data
loose-innodb_file_per_table
server_id= 1
log-basename=master
loose-debug-mutex-deadlock-detector
max-connections=20
lc-messages=en_us

[mariadb-10.11]
lc-messages-dir=/my/maria-10.11/sql/share

[mariadb-11.4]
lc-messages-dir=/my/maria-11.4/sql/share

[mariadb-11.8]
lc-messages-dir=/my/maria-11.8/sql/share

[mariadb-12.0]
lc-messages-dir=/my/maria-12.0/sql/share

The above .my.cnf file:

Uses an explicit socket for both client and server.
Assumes the server source is in /my/maria-xxx. You should change this to point to where your sources are located.
Has a unique patch for each MariaDB version so that one doesn't have to specify --lc-messages-dir or --language even if one switches between debugging different MariaDB versions.