Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Get started quickly with MariaDB Server using these quickstart guides. Follow step-by-step instructions to install, configure, and begin using MariaDB for your projects.
Explore MariaDB Server, the powerful open-source relational database. This comprehensive documentation covers installation, deployment, usage, security, and advanced topics to help you master MariaDB.
Learn how to connect to MariaDB Server. This section details various methods and tools for establishing secure and efficient connections to your database from different applications and environments.
Optimize large tables in MariaDB Server with partitioning. Learn how to divide tables into smaller, manageable parts for improved performance, easier maintenance, and scalability.
Grasp the basics of using MariaDB Server. This section introduces fundamental concepts, common SQL commands, and essential operations to get you started with your database.
Learn effective data handling in MariaDB Server. This section covers data types, storage engines, data manipulation, and best practices for managing your information efficiently.
Explore different partitioning types for MariaDB Server tables. Understand range, list, hash, and key partitioning to optimize data management and improve query performance.
Automate tasks in MariaDB Server with stored routines. Learn to create and manage stored procedures and functions for enhanced database efficiency and code reusability.
Learn to back up and restore MariaDB Server databases. This section covers essential strategies and tools to ensure data safety and quick recovery from potential data loss.
Master stored procedures in MariaDB Server. This section covers creating, executing, and managing these powerful routines to encapsulate complex logic and improve application performance.
Utilize stored functions in MariaDB Server. This section details creating, using, and managing user-defined functions to extend SQL capabilities and streamline data manipulation.
Understand MariaDB Server's storage engines. Explore the features and use cases of InnoDB, Aria, MyISAM, and other engines to choose the best option for your specific data needs.
An index on Last_Name
organizes the records by surname, enhancing search efficiency without altering the original table order. Indices can be created for any column, such as ID or first name, to enable quick lookups based on different criteria.
Imagine you've created a table with the following rows:
+----+------------+-----------+-------------------------+---------------------------+--------------+
| ID | First_Name | Last_Name | Position | Home_Address | Home_Phone |
+----+------------+-----------+-------------------------+---------------------------+--------------+
| 1 | Mustapha | Mond | Chief Executive Officer | 692 Promiscuous Plaza | 326-555-3492 |
| 2 | Henry | Foster | Store Manager | 314 Savage Circle | 326-555-3847 |
| 3 | Bernard | Marx | Cashier | 1240 Ambient Avenue | 326-555-8456 |
| 4 | Lenina | Crowne | Cashier | 281 Bumblepuppy Boulevard | 328-555-2349 |
| 5 | Fanny | Crowne | Restocker | 1023 Bokanovsky Lane | 326-555-6329 |
| 6 | Helmholtz | Watson | Janitor | 944 Soma Court | 329-555-2478 |
+----+------------+-----------+-------------------------+---------------------------+--------------+
Now, imagine you've been asked to return the home phone of Fanny Crowne. Without indexes, the only way to do it is to go through every row until you find the matching first name and surname. Now imagine there are millions of records and you can see that, even for a speedy database server, this is highly inefficient.
The answer is to sort the records. If they were stored in alphabetical order by surname, even a human could quickly find a record amongst a large number. But we can't sort the entire record by surname. What if we want to also look a record by ID, or by first name? The answer is to create separate indexes for each column we wish to sort by. An index simply contains the sorted data (such as surname), and a link to the original record.
For example, an index on Last_Name:
+-----------+----+
| Last_Name | ID |
+-----------+----+
| Crowne | 4 |
| Crowne | 5 |
| Foster | 2 |
| Marx | 3 |
| Mond | 1 |
| Watson | 6 |
+-----------+----+
and an index on Position
+-------------------------+----+
| Position | ID |
+-------------------------+----+
| Cashier | 3 |
| Cashier | 4 |
| Chief Executive Officer | 1 |
| Janitor | 6 |
| Restocker | 5 |
| Store Manager | 2 |
+-------------------------+----+
would allow you to quickly find the phone numbers of all the cashiers, or the phone number of the employee with the surname Marx, very quickly.
Where possible, you should create an index for each column that you search for records by, to avoid having the server read every row of a table.
See CREATE INDEX and Getting Started with Indexes for more information.
This page is licensed: CC BY-SA / Gnu FDL
Learn how to effectively use MariaDB Server. This section covers SQL statements, built-in functions, client utilities, and best practices for daily database operations.
Manage tables in MariaDB Server. This section details creating, altering, and dropping tables, along with understanding data types and storage engines for optimal database design.
PARTITION BY LINEAR HASH (partitioning_expression)
[PARTITIONS(number_of_partitions)]
LINEAR HASH partitioning is a form of partitioning, similar to HASH partitioning, in which the server takes care of the partition in which to place the data, ensuring a relatively even distribution among the partitions.
LINEAR HASH partitioning makes use of a powers-of-two algorithm, while HASH partitioning uses the modulus of the hashing function's value. Adding, dropping, merging and splitting partitions is much faster than with the HASH partitioning type, however, data is less likely to be evenly distributed over the partitions.
CREATE OR REPLACE TABLE t1 (c1 INT, c2 DATETIME)
PARTITION BY LINEAR HASH(TO_DAYS(c2))
PARTITIONS 5;
This page is licensed: CC BY-SA / Gnu FDL
LINEAR PARTITION BY KEY ([column_names])
[PARTITIONS (number_of_partitions)]
LINEAR KEY partitioning is a form of partitioning, similar to KEY partitioning.
LINEAR KEY partitioning makes use of a powers-of-two algorithm, while KEY partitioning uses modulo arithmetic, to determine the partition number.
Adding, dropping, merging and splitting partitions is much faster than with the KEY partitioning type, however, data is less likely to be evenly distributed over the partitions.
CREATE OR REPLACE TABLE t1 (v1 INT)
PARTITION BY LINEAR KEY (v1)
PARTITIONS 2;
This page is licensed: CC BY-SA / Gnu FDL
The following limitations apply to partitioning in MariaDB:
Each table can contain a maximum of 8192 partitions. Until , the limit was 1024.
Queries are never parallelized, even when they involve multiple partitions.
A table can only be partitioned if the storage engine supports partitioning.
All partitions must use the same storage engine. For a workaround, see Using CONNECT - Partitioning and Sharding.
A partitioned table cannot contain, or be referenced by, foreign keys.
The query cache is not aware of partitioning and partition pruning. Modifying a partition will invalidate the entries related to the whole table.
Updates can run more slowly when binlog_format=ROW and a partitioned table is updated than an equivalent update of a non-partitioned table.
All columns used in the partitioning expression for a partitioned table must be part of every unique key that the table may have.
In versions prior to , it is not possible to create partitions on tables that contain GEOMETRY types.
INFORMATION_SCHEMA.PARTITIONS contains information about existing partitions.
Partition Maintenance for suggestions on using partitions
This page is licensed: CC BY-SA / Gnu FDL
ALTER PROCEDURE proc_name [characteristic ...]
characteristic:
{ CONTAINS SQL | NO SQL | READS SQL DATA | MODIFIES SQL DATA }
| SQL SECURITY { DEFINER | INVOKER }
| COMMENT 'string'
This statement can be used to change the characteristics of a stored
procedure. More than one change may be specified in an ALTER PROCEDURE
statement. However, you cannot change the parameters or body of a
stored procedure using this statement. To make such changes, you must
drop and re-create the procedure using either CREATE OR REPLACE PROCEDURE (since ) or DROP PROCEDURE and CREATE PROCEDURE (MariaDB 10.1.2 and before).
You must have the ALTER ROUTINE
privilege for the procedure. By default, that privilege is granted automatically to the procedure creator. See Stored Routine Privileges.
ALTER PROCEDURE simpleproc SQL SECURITY INVOKER;
This page is licensed: GPLv2, originally from fill_help_tables.sql
Get an overview of MariaDB Backup. This section introduces the hot physical backup tool, explaining its capabilities for efficient and consistent backups of your MariaDB Server.
When using mariadb-backup, you don't necessarily need to restore every table and/or partition that was backed up. Even if you're starting from a full backup, it is certainly possible to restore only certain tables and/or partitions from the backup, as long as the table or partition involved is in an InnoDB file-per-table tablespace. This page documents how to restore individual tables and partitions.
Before you can restore from a backup, you first need to prepare it to make the data files consistent. You can do so with the --prepare command option.
The ability to restore individual tables and partitions relies on InnoDB's transportable tablespaces. For MariaDB to import tablespaces like these, InnoDB looks for a file with a .cfg
extension. For mariadb-backup to create these files, you also need to add the --export option during the prepare step.
For example, you might execute the following command:
If this operation completes without error, then the backup is ready to be restored.
Note
mariadb-backup did not support the --export option to begin with. See about that. In earlier versions of MariaDB, this means that mariadb-backup could not create .cfg
files for InnoDB file-per-table tablespaces during the --prepare
stage. You can still import file-per-table tablespaces without the .cfg
files in many cases, so it may still be possible in those versions to restore partial backups or to restore individual tables and partitions with just the .ibd
files. If you have a full backup and you need to create .cfg
files for InnoDB file-per-table tablespaces, then you can do so by preparing the backup as usual without the --export
option, and then restoring the backup, and then starting the server. At that point, you can use the server's built-in features to copy the transportable tablespaces.
The restore process for restoring individual tables and/or partitions is quite different than the process for full backups.
Rather than using the --copy-back or the --move-back, each individual InnoDB file-per-table tablespace file will have to be manually imported into the target server. The process that is used to restore the backup will depend on whether partitioning is involved.
To restore individual non-partitioned tables from a backup, find the .ibd
and .cfg
files for the table in the backup, and then import them using the Importing Transportable Tablespaces for Non-partitioned Tables process.
To restore individual partitions or partitioned tables from a backup, find the .ibd
and .cfg
files for the partition(s) in the backup, and then import them using the Importing Transportable Tablespaces for Partitioned Tables process.
This page is licensed: CC BY-SA / Gnu FDL
mariadb-backup
backs up the files listed below.
mariadb-backup backs up the following InnoDB data files:
mariadb-backup
will back up tables that use the storage engine. This data is located in the directory defined by the system variable. mariadb-backup
backs this data up by performing a checkpoint using the system variable.
mariadb-backup
will back up tables that use the MyRocks storage engine.
mariadb-backup
also backs up files with the following extensions:
frm
isl
MYD
MYI
MAD
MAI
MRG
TRG
TRN
ARM
ARZ
CSM
CSV
opt
par
mariadb-backup
does not back up the files listed below.
This page is licensed: CC BY-SA / Gnu FDL
mariadb-backup supports streaming to stdout with the --stream=xbstream
option. This option allows easy integration with popular encryption and compression tools. Below are several examples.
The following example creates an AES-encrypted backup, protected with the password "mypass" and stores it in a file "backup.xb.enc":
To decrypt and unpack this backup into the current directory, the following command can be used:
This example compresses the backup without encrypting:
We can decompress and unpack the backup as follows:
This example adds a compression step before the encryption, otherwise looks almost identical to the previous example:
We can decrypt, decompress and unpack the backup as follow (note gzip -d
in the pipeline):
7zip archiver is a popular utility (especially on Windows) that supports reading from standard output, with the --si
option, and writing to stdout with the -so
option, and can thus be used together with mariadb-backup.
Compressing backup with the 7z command line utility works as follows:
Uncompress and unpack the archive with
7z also has builtin AES-256 encryption. To encrypt the backup from the previous example using password SECRET, add -pSECRET
to the 7z command line.
Compress
Decompress , unpack
Encryption
Decrypt, unpack
Most of the described tools also provide a way to enter a passphrase interactively (although 7zip does not seem to work well when reading input from stdin). Please consult documentation of the tools for more info.
By default files like xtrabackup_checkpoints are also written to the output stream only, and so would not be available for taking further incremental backups without prior extraction from the compressed or encrypted stream output file.
To avoid this these files can additionally be written to a directory that can then be used as input for further incremental backups using the --extra-lsndir=... option.
See also e.g: Combining incremental backups with streaming output
This page is licensed: CC BY-SA / Gnu FDL
can be used to support the strategy.
Replication alone is not sufficient for backup. It assists in protecting against hardware failure on the primary server, but does not protect against data loss. An accidental or malicious DROP DATABASE
or TRUNCATE TABLE
statement are replicated onto the replica as well. Care needs to be taken to prevent data getting out of sync between the primary and the replica.
Replication is most commonly used to support backups as follows:
A primary server replicates to a replica
Backups are then run off the replica without any impact on the primary.
Backups can have a significant effect on a server, and a high-availability primary may not be able to be stopped, locked or simply handle the extra load of a backup. Running the backup from a replica has the advantage of being able to shutdown or lock the replica and perform a backup without any impact on the primary server.
Note that when backing up off a replica server, it is important to ensure that the servers keep the data in sync. See for example for a situation when identical statements can result in different data on a replica and a primary.
This page is licensed: CC BY-SA / Gnu FDL
When a WHERE clause is related to the partitioning expression, the optimizer knows which partitions are relevant for the query. Other partitions will not be read. This optimization is called partition pruning.
can be used to know which partitions are read for a given query. A column called partitions
will contain a comma-separated list of the accessed partitions. For example:
Sometimes the WHERE clause does not contain the necessary information to use partition pruning, or the optimizer cannot infer this information. However, we may know which partitions are relevant for the query. Since , we can force MariaDB to only access the specified partitions by adding a PARTITION clause. This feature is called partition selection. For example:
The PARTITION clause is supported for all DML statements:
In general, partition pruning is applied to statements contained in .
However, note that if a BEFORE INSERT
or BEFORE UPDATE
trigger is defined on a table, MariaDB doesn't know in advance if the columns used in the partitioning expression are changed. For this reason, it is forced to lock all partitions.
This page is licensed: CC BY-SA / Gnu FDL
RANGE COLUMNS and LIST COLUMNS are variants of, respectively, and . With these partitioning types there is not a single partitioning expression; instead, a list of one or more columns is accepted. The following rules apply:
The list can contain one or more columns.
Columns can be of any , , , and types.
Only bare columns are permitted; no expressions.
All the specified columns are compared to the specified values to determine which partition should contain a specific row. See below for details.
The last part of a statement can be definition of the new table's partitions. In the case of RANGE COLUMNS partitioning, the syntax is the following:
The syntax for LIST COLUMNS is the following:
partition_name
is the name of a partition.
To determine which partition should contain a row, all specified columns are compared to each partition definition.
With LIST COLUMNS, a row matches a partition if all row values are identical to the specified values. At most one partition can match the row.
With RANGE COLUMNS, a row matches a partition if all row values are less than the specified values. The first partition that matches the row values are used.
The DEFAULT
partition catches all records which do not fit in other partitions. Only one DEFAULT
partition is allowed.
RANGE COLUMNS partition:
LIST COLUMNS partition:
This page is licensed: CC BY-SA / Gnu FDL
A partitioned table is stored in multiple files. By default, these files are stored in the MariaDB (or InnoDB) data directory. It is possible to keep them in different paths by specifying table options. This is useful to store different partitions on different devices.
Note that, if the server system variable is set to 0 at the time of the table creation, all partitions are stored in the system tablespace.
The following files exist for each partitioned tables:
For example, an InnoDB table with 4 partitions will have the following files:
If we convert the table to MyISAM, we will have these files:
This page is licensed: CC BY-SA / Gnu FDL
The table in the database contains information about partitions.
The statement contains a Create_options
column, that contains the string 'partitioned' for partitioned tables.
The statement returns the statement that can be used to re-create a table, including the partitions definition.
This page is licensed: CC BY-SA / Gnu FDL
The following restrictions apply to .
All of the restrictions listed in .
Any statements that return a result set are not permitted. For example, a regular is not permitted, but a is. A cursor and statement is permitted.
statements are not permitted.
Statements that perform explicit or implicit commits or rollbacks are not permitted
Cannot be used recursively.
Cannot make changes to a table that is already in use (reading or writing) by the statement invoking the stored function.
Cannot refer to a temporary table multiple times under different aliases, even in different statements.
ROLLBACK TO SAVEPOINT and RELEASE SAVEPOINT statement which are in a stored function cannot refer to a savepoint which has been defined out of the current function.
Prepared statements (, , ) cannot be used, and therefore nor can statements be constructed as strings and then executed.
This page is licensed: CC BY-SA / Gnu FDL
$ mariadb-backup --prepare --export \
--target-dir=/var/mariadb/backup/ \
--user=mariadb-backup --password=mypassword
mariadb-backup --user=root --backup --stream=xbstream | openssl enc -aes-256-cbc -k mypass > backup.xb.enc
openssl enc -d -aes-256-cbc -k mypass -in backup.xb.enc | mbstream -x
mariadb-backup --user=root --backup --stream=xbstream | gzip > backupstream.gz
gunzip -c backupstream.gz | mbstream -x
mariadb-backup --user=root --backup --stream=xbstream | gzip | openssl enc -aes-256-cbc -k mypass > backup.xb.gz.enc
openssl enc -d -aes-256-cbc -k mypass -in backup.xb.gz.enc |gzip -d| mbstream -x
mariadb-backup --user=root --backup --stream=xbstream | 7z a -si backup.xb.7z
7z e backup.xb.7z -so |mbstream -x
mariadb-backup --user=root --backup --stream=xbstream | zstd - -o backup.xb.zst -f -1
zstd -d backup.xbstream.zst -c | mbstream -x
mariadb-backup --user=root --backup --stream=xbstream | gpg -c --passphrase SECRET --batch --yes -o backup.xb.gpg
gpg --decrypt --passphrase SECRET --batch --yes backup.xb.gpg | mbstream -x
EXPLAIN PARTITIONS SELECT * FROM orders WHERE id < 15000000;
+------+-------------+--------+------------+-------+---------------+---------+---------+------+------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+--------+------------+-------+---------------+---------+---------+------+------+-------------+
| 1 | SIMPLE | orders | p0,p1 | range | PRIMARY | PRIMARY | 4 | NULL | 2 | Using where |
+------+-------------+--------+------------+-------+---------------+---------+---------+------+------+-------------+
SELECT * FROM orders PARTITION (p3) WHERE user_id = 50;
SELECT * FROM orders PARTITION (p2,p3) WHERE user_id >= 40;
PARTITION BY RANGE COLUMNS (col1, col2, ...)
(
PARTITION partition_name VALUES LESS THAN (value1, value2, ...),
[ PARTITION partition_name VALUES LESS THAN (value1, value2, ...), ... ]
)
PARTITION BY LIST COLUMNS (partitioning_expression)
(
PARTITION partition_name VALUES IN (value1, value2, ...),
[ PARTITION partition_name VALUES IN (value1, value2, ...), ... ]
[ PARTITION partititon_name DEFAULT ]
)
CREATE OR REPLACE TABLE t1 (
date1 DATE NOT NULL,
date2 DATE NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE COLUMNS (date1,date2) (
PARTITION p0 VALUES LESS THAN ('2013-01-01', '1994-12-01'),
PARTITION p1 VALUES LESS THAN ('2014-01-01', '1995-12-01'),
PARTITION p2 VALUES LESS THAN ('2015-01-01', '1996-12-01')
);
CREATE OR REPLACE TABLE t1 (
num TINYINT(1) NOT NULL
)
ENGINE = InnoDB
PARTITION BY LIST COLUMNS (num) (
PARTITION p0 VALUES IN (0,1),
PARTITION p1 VALUES IN (2,3),
PARTITION p2 DEFAULT
);
table_name.frm
Contains the table definition. Non-partitioned tables have this file, too.
table_name.par
Contains the partitions definitions.
table_name#P#partition_name.ext
Normal files created by the storage engine use this pattern for names. The extension depends on the storage engine.
orders.frm
orders.par
orders#P#p0.ibd
orders#P#p1.ibd
orders#P#p2.ibd
orders#P#p3.ibd
orders.frm
orders.par
orders#P#p0.MYD
orders#P#p0.MYI
orders#P#p1.MYD
orders#P#p1.MYI
orders#P#p2.MYD
orders#P#p2.MYI
orders#P#p3.MYD
orders#P#p3.MYI
When using mariadb-backup, you have the option of performing partial backups. Partial backups allow you to choose which databases or tables to backup, as long as the table or partition involved is in an InnoDB file-per-table tablespace.This page documents how to perform partial backups.
Just like with full backups, in order to back up the database, you need to run mariadb-backup with the --backup option to tell it to perform a backup and with the --target-dir option to tell it where to place the backup files. The target directory must be empty or not exist.
For a partial backup, there are a few other arguments that you can provide as well:
To tell it which databases to backup, you can provide the --databases option.
To tell it which databases to exclude from the backup, you can provide the --databases-exclude option.
To tell it to check a file for the databases to backup, you can provide the --databases-file option.
To tell it which tables to backup, you can use the --tables option.
To tell it which tables to exclude from the backup, you can provide the --tables-exclude option.
To tell it to check a file for specific tables to backup, you can provide the --tables-file option.
The non-file partial backup options support regex in the database and table names.
For example, to take a backup of any database that starts with the string app1_
and any table in those databases that start with the string tab_
, run the following command:
$ mariadb-backup --backup \
--target-dir=/var/mariadb/backup/ \
--databases='app1_*' --tables='tab_*' \
--user=mariadb-backup --password=mypassword
mariadb-backup cannot currently backup a subset of partitions from a partitioned table. Backing up a partitioned table is currently an all-or-nothing selection. See MDEV-17132 about that. If you need to backup a subset of partitions, then one possibility is that instead of using mariadb-backup, you can export the file-per-table tablespaces of the partitions.
The time the backup takes depends on the size of the databases or tables you're backing up. You can cancel the backup if you need to, as the backup process does not modify the database.
mariadb-backup writes the backup files to the target directory. If the target directory doesn't exist, then it creates it. If the target directory exists and contains files, then it raises an error and aborts.
Just like with full backups, the data files that mariadb-backup creates in the target directory are not point-in-time consistent, given that the data files are copied at different times during the backup operation. If you try to restore from these files, InnoDB notices the inconsistencies and crashes to protect you from corruption. In fact, for partial backups, the backup is not even a completely functional MariaDB data directory, so InnoDB would raise more errors than it would for full backups. This point will also be very important to keep in mind during the restore process.
Before you can restore from a backup, you first need to prepare it to make the data files consistent. You can do so with the --prepare command option.
Partial backups rely on InnoDB's transportable tablespaces. For MariaDB to import tablespaces like these, InnoDB looks for a file with a .cfg
extension. For mariadb-backup to create these files, you also need to add the --export option during the prepare step.
For example, you might execute the following command:
$ mariadb-backup --prepare --export \
--target-dir=/var/mariadb/backup/
If this operation completes without error, then the backup is ready to be restored.
mariadb-backup did not support the --export option. See MDEV-13466 about that. This means that mariadb-backup could not create .cfg
files for InnoDB file-per-table tablespaces during the --prepare
stage. You can still import file-per-table tablespaces without the .cfg
files in many cases, so it may still be possible in those versions to restore partial backups or to restore individual tables and partitions with just the .ibd
files. If you have a full backup and you need to create .cfg
files for InnoDB file-per-table tablespaces, then you can do so by preparing the backup as usual without the --export
option, and then restoring the backup, and then starting the server. At that point, you can use the server's built-in features to copy the transportable tablespaces.
The restore process for partial backups is quite different than the process for full backups. A partial backup is not a completely functional data directory. The data dictionary in the InnoDB system tablespace will still contain entries for the databases and tables that were not included in the backup.
Rather than using the --copy-back or the --move-back, each individual InnoDB file-per-table tablespace file will have to be manually imported into the target server. The process that is used to import the file will depend on whether partitioning is involved.
To restore individual non-partitioned tables from a backup, find the .ibd
and .cfg
files for the table in the backup, and then import them using the Importing Transportable Tablespaces for Non-partitioned Tables process.
To restore individual partitions or partitioned tables from a backup, find the .ibd
and .cfg
files for the partition(s) in the backup, and then import them using the Importing Transportable Tablespaces for Partitioned Tables process.
This page is licensed: CC BY-SA / Gnu FDL
A partitioning type determines how a partitioned table's rows are distributed across partitions. Some partition types require the user to specify a partitioning expression that determines in which partition a row are stored.
The size of individual partitions depends on the partitioning type. Read and write performance are affected by the partitioning expression. Therefore, these choices should be made carefully.
MariaDB supports the following partitioning types:
This page is licensed: CC BY-SA / Gnu FDL
PARTITION BY HASH (partitioning_expression)
[PARTITIONS(number_of_partitions)]
HASH partitioning is a form of partitioning in which the server takes care of the partition in which to place the data, ensuring an even distribution among the partitions.
It requires a column value, or an expression based on a column value, which is hashed, as well as the number of partitions into which to divide the table.
partitioning_expression needs to return a non-constant, deterministic integer. It is evaluated for each insert and update, so overly complex expressions can lead to performance issues. A hashing function operating on a single column, and where the value changes consistently with the column value, allows for easy pruning on ranges of partitions, and is usually a better choice. For this reason, using multiple columns in a hashing expression is not usually recommended.
number_of_partitions is a positive integer specifying the number of partitions into which to divide the table. If the PARTITIONS
clause is omitted, the default number of partitions is one.
To determine which partition to use, the following calculation is performed: MOD(partitioning_expression, number_of_partitions)
For example, if the expression is TO_DAYS(datetime_column) and the number of partitions is 5, inserting a datetime value of '2023-11-15' would determine the partition as follows:
TO_DAYS('2023-11-15') gives a value of 739204
MOD(739204,5) returns 4 so the 4th partition is used.
HASH partitioning making use of the modulus of the hashing function's value. The LINEAR HASH partitioning type is similar, using a powers-of-two algorithm. Data is more likely to be evenly distributed over the partitions than with the LINEAR HASH partitioning type, however, adding, dropping, merging and splitting partitions is much slower.
CREATE OR REPLACE TABLE t1 (c1 INT, c2 DATETIME)
PARTITION BY HASH(TO_DAYS(c2))
PARTITIONS 5;
Using the Information Schema PARTITIONS Table for more information:
INSERT INTO t1 VALUES (1,'2023-11-15');
SELECT PARTITION_NAME,TABLE_ROWS FROM INFORMATION_SCHEMA.PARTITIONS
WHERE TABLE_SCHEMA='test' AND TABLE_NAME='t1';
+----------------+------------+
| PARTITION_NAME | TABLE_ROWS |
+----------------+------------+
| p0 | 0 |
| p1 | 0 |
| p2 | 0 |
| p3 | 0 |
| p4 | 1 |
+----------------+------------+
Partition Maintenance for suggestions on using partitions
This page is licensed: CC BY-SA / Gnu FDL
PARTITION BY KEY ([column_names])
[PARTITIONS (number_of_partitions)]
Partitioning by key is a type of partitioning that is similar to and can be used in a similar way as partitioning by hash.
KEY takes an optional list of column_names, and the hashing function is given by the server.
Just like HASH partitioning, in KEY partitioning the server takes care of the partition and ensures an even distribution among the partitions. However, the largest difference is that KEY partitioning makes use of column_names, and cannot accept a partitioning_expression which is based on column_names, in contrast to HASH partitioning, which can.
If no column_names are specified, the table's primary key is used if present, or not null unique key if no primary key is present. If neither of these keys are present, not specifying any column_names will result in ERROR 1488 (HY000): Field in list of fields for partition function not found in table
Unlike other partitioning types, columns used for partitioning by KEY are not limited to integer or NULL values.
KEY partitions do not support column index prefixes. Any columns in the partitioning key that make use of column prefixes are not used (see also MDEV-32727).
CREATE OR REPLACE TABLE t1 (v1 INT)
PARTITION BY KEY (v1)
PARTITIONS 2;
CREATE OR REPLACE TABLE t1 (v1 INT, v2 INT)
PARTITION BY KEY (v1,v2)
PARTITIONS 2;
CREATE OR REPLACE TABLE t1 (
id INT NOT NULL PRIMARY KEY,
name VARCHAR(5)
)
PARTITION BY KEY()
PARTITIONS 2;
CREATE OR REPLACE TABLE t1 (
id INT NOT NULL UNIQUE KEY,
name VARCHAR(5)
)
PARTITION BY KEY()
PARTITIONS 2;
The unique key must be NOT NULL:
CREATE OR REPLACE TABLE t1 (
id INT NULL UNIQUE KEY,
name VARCHAR(5)
)
PARTITION BY KEY()
PARTITIONS 2;
ERROR 1488 (HY000): Field in list of fields for partition function not found in table
KEY requires column_values if no primary key or not null unique key is present:
CREATE OR REPLACE TABLE t1 (
id INT NULL UNIQUE KEY,
name VARCHAR(5)
)
PARTITION BY KEY()
PARTITIONS 2;
ERROR 1488 (HY000): Field in list of fields for partition function not found in table
CREATE OR REPLACE TABLE t1 (
id INT NULL UNIQUE KEY,
name VARCHAR(5)
)
PARTITION BY KEY(name)
PARTITIONS 2;
Primary key columns with index prefixes are silently ignored, so the following two queries are equivalent:
CREATE OR REPLACE TABLE t1 (
a VARCHAR(10),
b VARCHAR(10),
c VARCHAR(10),
PRIMARY KEY (a(5), b, c(5))
) PARTITION BY KEY() PARTITIONS 2;
CREATE OR REPLACE TABLE t1 (
a VARCHAR(10),
b VARCHAR(10),
c VARCHAR(10),
PRIMARY KEY (b)
) PARTITION BY KEY() PARTITIONS 2;
a(5)
and c(5)
are silently ignored in the former.
If all columns use index prefixes, the statement fails with a slightly misleading error:
CREATE OR REPLACE TABLE t1 (
a VARCHAR(10),
b VARCHAR(10),
c VARCHAR(10),
PRIMARY KEY (a(5), b(5), c(5))
) PARTITION BY KEY() PARTITIONS 2;
ERROR 1503 (HY000): A PRIMARY KEY must include all columns in the table's partitioning function
This page is licensed: CC BY-SA / Gnu FDL
LIST partitioning is conceptually similar to RANGE partitioning. In both cases you decide a partitioning expression (a column, or a slightly more complex calculation) and use it to determine which partitions will contain each row. However, with the RANGE type, partitioning is done by assigning a range of values to each partition. With the LIST type, we assign a set of values to each partition. This is usually preferred if the partitioning expression can return a limited set of values.
A variant of this partitioning method, LIST COLUMNS, allows us to use multiple columns and more datatypes.
The last part of a CREATE TABLE statement can be the definition of the new table's partitions. In the case of LIST partitioning, the syntax is the following:
PARTITION BY LIST (partitioning_expression)
(
PARTITION partition_name VALUES IN (value_list),
[ PARTITION partition_name VALUES IN (value_list), ... ]
[ PARTITION partition_name DEFAULT ]
)
PARTITION BY LIST indicates that the partitioning type is LIST.
The partitioning_expression
is an SQL expression that returns a value from each row. In the simplest cases, it is a column name. This value is used to determine which partition should contain a row.
partition_name
is the name of a partition.
value_list
is a list of values. If partitioning_expression
returns one of these values, the row are stored in this partition. If we try to insert something that does not belong to any of these value lists, the row are rejected with an error.
The DEFAULT
partition catches all records which do not fit into other partitions.
LIST partitioning can be useful when we have a column that can only contain a limited set of values. Even in that case, RANGE partitioning could be used instead; but LIST partitioning allows us to equally distribute the rows by assigning a proper set of values to each partition.
CREATE OR REPLACE TABLE t1 (
num TINYINT(1) NOT NULL
)
ENGINE = InnoDB
PARTITION BY LIST (num) (
PARTITION p0 VALUES IN (0,1),
PARTITION p1 VALUES IN (2,3),
PARTITION p2 DEFAULT
);
This page is licensed: CC BY-SA / Gnu FDL
DROP PROCEDURE [IF EXISTS] sp_name
This statement is used to drop a stored procedure. That is, the
specified routine is removed from the server along with all privileges specific to the procedure. You must have the ALTER ROUTINE
privilege for the routine. If the automatic_sp_privileges server system variable is set, that privilege and EXECUTE
are granted automatically to the routine creator - see Stored Routine Privileges.
The IF EXISTS
clause is a MySQL/MariaDB extension. It
prevents an error from occurring if the procedure or function does not exist. ANOTE
is produced that can be viewed with SHOW WARNINGS.
While this statement takes effect immediately, threads which are executing a procedure can continue execution.
DROP PROCEDURE simpleproc;
IF EXISTS:
DROP PROCEDURE simpleproc;
ERROR 1305 (42000): PROCEDURE test.simpleproc does not exist
DROP PROCEDURE IF EXISTS simpleproc;
Query OK, 0 rows affected, 1 warning (0.00 sec)
SHOW WARNINGS;
+-------+------+------------------------------------------+
| Level | Code | Message |
+-------+------+------------------------------------------+
| Note | 1305 | PROCEDURE test.simpleproc does not exist |
+-------+------+------------------------------------------+
This page is licensed: GPLv2, originally from fill_help_tables.sql
Quickstart Guide: Installing MariaDB Server
This guide provides a quick overview of how to install MariaDB Server on common operating systems. The specific steps may vary slightly depending on your Linux distribution or if you are installing on Windows.
The most common way to install MariaDB on Linux is through your system's package manager.
Steps:
Update Package List:
Before installing, it's a good practice to update your package index.
For Debian/Ubuntu:Bash
For Red Hat/CentOS/Fedora:Bash
Install MariaDB Server:
Install the MariaDB server and client packages.
For Debian/Ubuntu:Bash
For Red Hat/CentOS/Fedora:Bash
Secure the Installation:
After installation, run the security script to set a root password, remove anonymous users, and disable remote root login.
Follow the prompts to configure your security settings.
Start and Verify the Service:
MariaDB typically starts automatically after installation. You can check its status and manually start it if needed.
Check status:
Start service (if not running):Bash
Verify installation by connecting as root:Bash
Enter the root password you set during the secure installation.
For Windows, MariaDB provides an .msi
installer for a straightforward graphical installation.
Steps:
Download MariaDB:
Visit the MariaDB downloads page to get the latest .msi
installer.
Run the Installer:
Double-click the downloaded .msi
file to start the installation wizard.
Follow On-Screen Instructions:
The installer will guide you through the process, including:
Accepting the end-user license agreement.
Selecting features and the installation directory.
Setting a password for the root
user.
Configuring MariaDB as a service and setting the port (default is 3306).
Optionally, enabling UTF8 as the default server character set.
Firewall: Ensure your firewall is configured to allow connections to MariaDB on the appropriate port (default 3306) if you need remote access.
Root Password: Always set a strong root password during the secure installation step.
Further Configuration: For production environments, you may need to adjust further settings in the MariaDB configuration files (e.g., my.cnf
on Linux).
When using mariadb-backup, you have the option of performing a full or an incremental backup. Full backups create a complete backup of the database server in an empty directory while incremental backups update a previous backup with whatever changes to the data have occurred since the backup. This page documents how to perform full backups.
In order to back up the database, you need to run mariadb-backup with the --backup option to tell it to perform a backup and with the --target-dir option to tell it where to place the backup files. When taking a full backup, the target directory must be empty or it must not exist.
To take a backup, run the following command:
The time the backup takes depends on the size of the databases or tables you're backing up. You can cancel the backup if you need to, as the backup process does not modify the database.
Mariadb-backup writes the backup files the target directory. If the target directory doesn't exist, then it creates it. If the target directory exists and contains files, then it raises an error and aborts.
Here is an example backup directory:
The data files that mariadb-backup creates in the target directory are not point-in-time consistent, given that the data files are copied at different times during the backup operation. If you try to restore from these files, InnoDB notices the inconsistencies and crashes to protect you from corruption
Before you can restore from a backup, you first need to prepare it to make the data files consistent. You can do so with the --prepare option.
Run mariadb-backup --backup. You must use a version of mariadb-backup that is compatible with the server version you are planning to upgrade from. For instance, when upgrading from MariaDB 10.4 to 10.5, you must use the 10.4 version of mariadb-backup, Another example: When upgrading from MariaDB 10.6 to 10.11, you must use the 10.6 version of mariadb-backup.
Run mariadb-backup --prepare, again using a compatible version of mariadb-backup, as described in the previous step.
Once the backup is complete and you have prepared the backup for restoration (previous step), you can restore the backup using either the --copy-back or the --move-back options. The --copy-back option allows you to keep the original backup files. The --move-back option actually moves the backup files to the datadir, so the original backup files are lost.
First, stop the MariaDB Server process.
Then, ensure that the datadir is empty.
Then, run mariadb-backup with one of the options mentioned above:
Then, you may need to fix the file permissions.
When mariadb-backup restores a database, it preserves the file and directory privileges of the backup. However, it writes the files to disk as the user and group restoring the database. As such, after restoring a backup, you may need to adjust the owner of the data directory to match the user and group for the MariaDB Server, typically mysql
for both. For example, to recursively change ownership of the files to the mysql
user and group, you could execute:
Finally, start the MariaDB Server process.
Once a full backup is prepared, it is a fully functional MariaDB data directory. Therefore, as long as the MariaDB Server process is stopped on the target server, you can technically restore the backup using any file copying tool, such as cp
or rysnc
. For example, you could also execute the following to restore the backup:
This page is licensed: CC BY-SA / Gnu FDL
mariadb-backup makes it very easy to set up a replica using a full backup. This page documents how to set up a replica from a backup.
If you are using MariaDB Galera Cluster, then you may want to try one of the following pages instead:
The first step is to simply take and prepare a fresh full backup of a database server in the replication topology. If the source database server is the desired replication primary, then we do not need to add any additional options when taking the full backup. For example:
If the source database server is a replica of the desired primary, then we should add the --slave-info option, and possibly the --safe-slave-backup option. For example:
And then we would prepare the backup as you normally would. For example:
Once the backup is done and prepared, we can copy it to the new replica. For example:
At this point, we can restore the backup to the datadir, as you normally would. For example:
And adjusting file permissions, if necessary:
Before the new replica can begin replicating from the primary, we need to create a user account on the primary that the replica can use to connect, and we need to grant the user account the REPLICATION SLAVE privilege. For example:
Before we start the server on the new replica, we need to configure it. At the very least, we need to ensure that it has a unique server_id value. We also need to make sure other replication settings are what we want them to be, such as the various GTID system variables, if those apply in the specific environment.
Once configuration is done, we can on the new replica.
At this point, we need to get the replication coordinates of the primary from the original backup directory.
If we took the backup on the primary, then the coordinates are in the xtrabackup_binlog_info file. If we took the backup on another replica and if we provided the --slave-info option, then the coordinates are in the file xtrabackup_slave_info file.
mariadb-backup dumps replication coordinates in two forms: GTID coordinates and binary log file and position coordinates, like the ones you would normally see from SHOW MASTER STATUS output. We can choose which set of coordinates we would like to use to set up replication.
For example:
Regardless of the coordinates we use, we will have to set up the primary connection using CHANGE MASTER TO and then start the replication threads with START SLAVE.
If we want to use GTIDs, then we will have to first set gtid_slave_pos to the GTID coordinates that we pulled from either the xtrabackup_binlog_info file or the xtrabackup_slave_info file in the backup directory. For example:
And then we would set MASTER_USE_GTID=slave_pos
in the CHANGE MASTER TO command. For example:
If we want to use the binary log file and position coordinates, then we would set MASTER_LOG_FILE
and MASTER_LOG_POS
in the CHANGE MASTER TO command to the file and position coordinates that we pulled; either the xtrabackup_binlog_info file or the xtrabackup_slave_info file in the backup directory, depending on whether the backup was taken from the primary or from a replica of the primary. For example:
We should be done setting up the replica now, so we should check its status with SHOW SLAVE STATUS. For example:
This page is licensed: CC BY-SA / Gnu FDL
This is a description of the different stages in mariadb-backup, what they do and why they are needed.
Note that a few items are marked with TODO
; these are things we are working on and are in next version of mariadb-backup.
Connect to mysqld instance, find out important variables (datadir ,InnoDB pagesize, encryption keys, encryption plugin etc)
Scan the database directory, datadir
, looking for InnoDB tablespaces, load the tablespaces (basically, it is an “open” in InnoDB sense)
If --lock-ddl-per-table is used:
Do MDL locks, for InnoDB tablespaces that we want to copy. This is to ensure that there are no ALTER, RENAME , TRUNCATE or DROP TABLE on any of the tables that we want to copy.
This is implemented with:
If lock-ddl-per-table is not done, then mariadb-backup would have to know all tables that were created or altered during the backup. See .
Start a dedicated thread in mariadb-backup to copy InnoDB redo log (ib_logfile*
).
This is needed to record all changes done while the backup is running. (The redo log logically is a single circular file, split into innodb_log_files_in_group files.)
The log is also used to see detect if any truncate or online alter tables are used.
The assumption is that the copy thread are able to keep up with server. It should always be able keep up, if the redo log is big enough.
Copy all selected tablespaces, file by file, in dedicated threads in mariadb-backup without involving the mysqld server.
This is special “careful” copy, it looks for page-level consistency by checking the checksum.
The files are not point-in-time consistent as data may change during copy.
The idea is that InnoDB recovery would make it point-in-time consistent.
Copy Aria log files (TODO)
Execute FLUSH TABLE WITH READ LOCK. This is default, but may be omitted with the -–no-lock
parameter. The reason why FLUSH
is needed is to ensure that all tables are in a consistent state at the exact same point in time, independent of storage engine.
If --lock-ddl-per-table
is used and there is a user query waiting for MDL, the user query are killed to resolve a deadlock. Note that these are only queries of type ALTER, DROP, TRUNCATE or RENAME TABLE. ()
Copy .frm
, MyISAM
, Aria
and other storage engine files
If MyRocks
is used, create rocksdb checkpoint via "set rocksdb_create_checkpoint=$rocksdb_data_dir/mariadb-backup_rocksdb_checkpoint " command. The result of it is a directory with hardlinks to MyRocks files. Copy the checkpoint directory to the backup (or create hardlinks in backup directory is on the same partition as data directory). Remove the checkpoint directory.
Copy tables that were created while the backup was running and do rename files that were changed during backup (since )
Copy the rest of InnoDB redo log, stop redo-log-copy thread
Copy changes to Aria log files (They are append only, so this is easy to do) (TODO)
Write some metadata info (binlog position)
If FLUSH TABLE WITH READ LOCK was done:
execute: UNLOCK TABLES
If --lock-ddl-per-table
was done:
execute COMMIT
If log tables exists:
Take MDL lock for log tables
Copy part of log tables that wasn't copied before
Unlock log tables
If FLUSH TABLE WITH READ LOCK is not used, then only InnoDB tables are consistent (not the privilege tables in the mysql database or the binary log). The backup point depends on the content of the redo log within the backup itself.
This page is licensed: CC BY-SA / Gnu FDL
The DROP FUNCTION statement is used to drop a or a user-defined function (UDF). That is, the specified routine is removed from the server, along with all privileges specific to the function. You must have the ALTER ROUTINE
for the routine in order to drop it. If the server system variable is set, both the ALTER ROUTINE
and EXECUTE
privileges are granted automatically to the routine creator - see .
The IF EXISTS
clause is a MySQL/MariaDB extension. It
prevents an error from occurring if the function does not exist. ANOTE
is produced that can be viewed with .
For dropping a (UDF), see .
This page is licensed: GPLv2, originally from
The following SQL statements are not permitted inside any (, , or ).
; you can use instead.
and .
is permitted, but the statement is handled as a regular .
and .
References to within prepared statements inside a stored routine (use instead).
is treated as the beginning of a block, not a transaction, so needs to be used instead.
The number of permitted recursive calls is limited to . If this variable is 0 (default), recursivity is disabled. The limit does not apply to stored functions.
Most statements that are not permitted in prepared statements are not permitted in stored programs. See for a list of statements that can be used. , and are exceptions, and may be used in stored routines.
There are also further limitations specific to the kind of stored routine.
Note that, if a stored program calls another stored program, the latter will inherit the caller's limitations. So, for example, if a stored procedure is called by a stored function, that stored procedure will not be able to produce a result set, because stored functions can't do this.
This page is licensed: CC BY-SA / Gnu FDL
Core SQL Statements Guide
This guide provides a quick overview of essential SQL statements in MariaDB, categorized by their function in data definition, data manipulation, and transaction control. Find brief descriptions and links to detailed documentation for each statement, along with a simple illustrative example sequence.
(If you need a basic tutorial on how to use the MariaDB database server and execute simple commands, see A MariaDB Primer. Also see Essential Queries Guide for examples of commonly-used queries.)
CREATE DATABASE: Used to create a new, empty database.
DROP DATABASE: Used to completely destroy an existing database.
USE: Used to select a default database for subsequent statements.
CREATE TABLE: Used to create a new table, which is where your data is actually stored.
ALTER TABLE: Used to modify an existing table's definition (e.g., add/remove columns, change types).
DROP TABLE: Used to completely destroy an existing table and all its data.
DESCRIBE (or DESC
): Shows the structure of a table (columns, data types, etc.).
SELECT: Used when you want to read (or select) your data from one or more tables.
INSERT: Used when you want to add (or insert) new rows of data into a table.
UPDATE: Used when you want to change (or update) existing data in a table.
DELETE: Used when you want to remove (or delete) existing rows of data from a table.
REPLACE: Works like INSERT
, but if an old row in the table has the same value as a new row for a PRIMARY KEY
or a UNIQUE
index, the old row is deleted before the new row is inserted.
TRUNCATE TABLE: Used to quickly remove all data from a table, resetting any AUTO_INCREMENT
values. It is faster than DELETE
without a WHERE
clause for emptying a table.
START TRANSACTION (or BEGIN
): Used to begin a new transaction, allowing multiple SQL statements to be treated as a single atomic unit.
COMMIT: Used to save all changes made during the current transaction, making them permanent.
ROLLBACK: Used to discard all changes made during the current transaction, reverting the database to its state before the transaction began.
This example demonstrates several of the statements in action:
-- Create a new database
CREATE DATABASE mydb;
-- Select the new database to use
USE mydb;
-- Create a new table
CREATE TABLE mytable (
id INT PRIMARY KEY,
name VARCHAR(20)
);
-- Insert some data
INSERT INTO mytable VALUES (1, 'Will');
INSERT INTO mytable VALUES (2, 'Marry');
INSERT INTO mytable VALUES (3, 'Dean');
-- Select specific data
SELECT id, name FROM mytable WHERE id = 1;
-- Update existing data
UPDATE mytable SET name = 'Willy' WHERE id = 1;
-- Select all data to see changes
SELECT id, name FROM mytable;
-- Delete specific data
DELETE FROM mytable WHERE id = 1;
-- Select all data again
SELECT id, name FROM mytable;
-- Drop the database (removes the database and its tables)
DROP DATABASE mydb;
Common Query: Counting Rows
To count the number of records in a table:
SELECT COUNT(*) FROM mytable; -- Or SELECT COUNT(1) FROM mytable;
(Note: This query would typically be run on an existing table, for example, before it or its database is dropped.)
This page is licensed: CC BY-SA / Gnu FDL
Data Restoration Guide
This guide explains how to restore your MariaDB data from backup files created with mariadb-dump
. Learn the basic restoration process using the mariadb
client and a specific technique for selectively restoring a single table while minimizing data loss on other tables.
It's important to understand that mariadb-dump
is used for creating backup (dump) files, while the mariadb
client utility is used for restoring data from these files. The dump file contains SQL statements that, when executed, recreate the database structure and/or data.
To restore a dump file, you direct the mariadb
client to execute the SQL statements contained within the file.
mariadb --user your_username --password < /path/to/your/backupfile.sql
Replace your_username
with your MariaDB username and /path/to/your/backupfile.sql
with the actual path to your dump file.
You will be prompted for the password for your_username
.
The <
symbol is a standard input (STDIN) redirect, feeding the contents of backupfile.sql
to the mariadb
client.
Often, the dump file itself contains CREATE DATABASE IF NOT EXISTS
and USE database_name;
statements, so a specific database doesn't always need to be named on the command line during restore. If your dump file restores to a specific database, ensure that user has permissions to it. If the dump file does not specify a database, you might need to create the database first and then run:
mariadb --user your_username --password your_database_name < /path/to/your/backupfile.sql
Data Overwriting: Restoring a dump file will execute the SQL statements within it. If the dump file contains DROP TABLE
and CREATE TABLE
statements (common for full backups), existing tables with the same names will be dropped and recreated, leading to loss of any data added or changed since the backup was made.
Backup Age: If your dump file is several days old, restoring it entirely could revert all data in the affected tables/databases to that older state. This can be disastrous if only a small portion of data was lost and the rest has been actively updated.
Always ensure you understand the contents of the dump file and the potential impact before initiating a restore, especially on a production system. Consider testing the restore on a non-production environment first if possible.
If only one table has been lost or corrupted and your backup file contains an entire database (or multiple tables), a full restore might overwrite recent, valid data in other tables. Here’s a method to restore only a specific table using a temporary user with restricted privileges:
Create a Temporary User: Create a MariaDB user specifically for this restore operation.
Grant Limited Privileges:
Grant this temporary user the minimal privileges needed for the dump file to execute up to the point of restoring your target table. This might be SELECT
on all tables in the database if the dump file checks other tables, or simply the ability to USE
the database.
Then, grant ALL PRIVILEGES
(or specific necessary privileges like CREATE
, DROP
, INSERT
, SELECT
) only on the specific table you want to restore.
Example SQL to create a temporary user and grant permissions (replace placeholders):
-- Connect to MariaDB as an administrative user (e.g., root)
CREATE USER 'admin_restore_temp'@'localhost' IDENTIFIED BY 'its_very_secure_pwd';
-- Grant general SELECT on the database (might be needed if dump file structure requires it)
-- Or, if not needed, ensure the user can at least USE the database.
GRANT SELECT ON your_database_name.* TO 'admin_restore_temp'@'localhost';
-- Grant full privileges ONLY on the table to be restored
GRANT ALL PRIVILEGES ON your_database_name.table_to_restore TO 'admin_restore_temp'@'localhost';
FLUSH PRIVILEGES;
Restore Using the Temporary User and --force
:
Use the mariadb client with the temporary user and the --force option. The --force option tells MariaDB to continue executing statements in the dump file even if some SQL errors occur. Errors will occur for operations on tables where admin_restore_temp lacks permissions, but operations on table_to_restore (where permissions were granted) should succeed.
Bash
mariadb --user admin_restore_temp --password --force your_database_name < /path/to/your/fulldumpfile.sql
You will be prompted for the password of admin_restore_temp
.
Verify Restoration: Check that table_to_restore
has been correctly restored.
Clean Up: Drop the temporary user once the restoration is confirmed:
DROP USER 'admin_restore_temp'@'localhost';
This method helps to isolate the restore operation to the intended table, protecting other data from being inadvertently reverted to an older state.
Modifying Dates and Times Guide
This guide explores MariaDB functions for performing calculations and modifications on date and time values. Learn to use functions like DATE_ADD
, DATE_SUB
, TIME_TO_SEC
, and SEC_TO_TIME
to accurately add or subtract intervals and manage date/time changes that cross midnight or month/year boundaries.
(For foundational knowledge on date and time data types and basic retrieval, please refer to the "Date and Time Handling Guide".)
When adding hours to a TIME
value, calculations might exceed 24 hours. For example, if a task is entered at 23:00 and is promised 2 hours later, a simple addition can be problematic.
Consider an INSERT
statement for a tickets
table with entered
and promised
TIME
columns:
-- Example: Calculating a promised time 2 hours (7200 seconds) from current time
INSERT INTO tickets (client_id, urgency, trouble, ticket_date, entered, promised)
VALUES ('some_client', 'ASAP', 'Issue details',
CURDATE(), CURTIME(),
SEC_TO_TIME(TIME_TO_SEC(CURTIME()) + 7200));
TIME_TO_SEC(time)
converts a time value to seconds.
SEC_TO_TIME(seconds)
converts seconds back to a time format (HHH:MM:SS
).
If CURTIME()
is 23:00:00
(82,800 seconds), 82800 + 7200 = 90000
seconds. SEC_TO_TIME(90000)
would result in 25:00:00
. While MariaDB can store this, it doesn't represent a standard clock time for the next day.
Modulo Arithmetic for Time Rollover:
To handle time wrapping around the 24-hour clock (86,400 seconds in a day) for TIME columns, use the modulo operator (%):
-- Corrected calculation for 'promised' TIME, wraps around 24 hours
SEC_TO_TIME((TIME_TO_SEC(CURTIME()) + 7200) % 86400)
If current time is 23:00, (82800 + 7200) % 86400
becomes 90000 % 86400
, which is 3600
seconds. SEC_TO_TIME(3600)
correctly results in 01:00:00
.
DATETIME
The modulo arithmetic above gives the correct time of day but doesn't indicate if the promised time falls on the next calendar day. For calculations where the date might change, it's essential to use DATETIME
(or TIMESTAMP
) data types.
If your table initially used separate DATE
and TIME
columns (e.g., ticket_date
, entered_time
, promised_time
), you would typically alter the table to use DATETIME
columns (e.g., entered_datetime
, promised_datetime
) to store both date and time information accurately. This often involves:
Adding new DATETIME
columns.
Populating them by combining the old date and time columns (e.g., using CONCAT(ticket_date, ' ', entered_time)
).
Dropping the old separate date and time columns. (Always back up your data before such structural changes.)
With DATETIME
columns, NOW()
can be used to get the current date and time.
DATE_ADD
The DATE_ADD(date, INTERVAL expr unit)
function is the most robust way to add a duration to a date, time, or datetime value. It correctly handles rollovers across days, months, and years.
date
: A DATE
, DATETIME
, or TIME
value.
expr
: The value of the interval to add.
unit
: The unit of the interval (e.g., HOUR
, MINUTE
, DAY
, MONTH
, YEAR
, etc.).
Adding Hours (handles date change):
If entered and promised are DATETIME columns:
INSERT INTO tickets (client_id, urgency, trouble, entered, promised)
VALUES ('some_client', 'ASAP', 'Issue details',
NOW(),
DATE_ADD(NOW(), INTERVAL 2 HOUR));
If NOW()
is 2025-06-03 23:00:00
, promised
will correctly be 2025-06-04 01:00:00
.
Adding Combined Hours and Minutes:
Use HOUR_MINUTE as the unit. The expr is a string 'hours:minutes'.
-- Add 2 hours and 30 minutes
DATE_ADD(NOW(), INTERVAL '2:30' HOUR_MINUTE)
If NOW()
is 2025-06-03 23:00:00
, this results in 2025-06-04 01:30:00
.
DATE_ADD
DATE_ADD
also correctly handles date changes across month and year boundaries, including leap years.
Adding Days:
-- Add 5 days
DATE_ADD(NOW(), INTERVAL 5 DAY)
If NOW()
is 2025-02-27
, this would result in 2025-03-04
(assuming 2025 is not a leap year).
Adding Combined Days and Hours:
Use DAY_HOUR as the unit. The expr is a string 'days hours'.
-- Add 2 days and 6 hours
DATE_ADD(NOW(), INTERVAL '2 6' DAY_HOUR)
Adding Combined Years and Months:
Use YEAR_MONTH as the unit. The expr is a string 'years-months'.
-- Add 1 year and 2 months
DATE_ADD(NOW(), INTERVAL '1-2' YEAR_MONTH) -- Note: Original text used '1 2', '1-2' is common for YEAR_MONTH
If NOW()
is 2025-09-15 23:00:00
, this results in 2026-11-15 23:00:00
. This type of interval typically does not affect the day or time components directly, only the year and month.
Using DATE_ADD with a Negative Interval:
You can subtract durations by providing a negative value for expr.
-- Subtract 5 days
DATE_ADD(NOW(), INTERVAL -5 DAY)
Using DATE_SUB(date, INTERVAL expr unit):
This function is specifically for subtracting durations.
-- Subtract 5 days
DATE_SUB(NOW(), INTERVAL 5 DAY)
Note: With DATE_SUB
, expr
is positive for subtraction. A negative expr
would result in addition.
Data Backup with mariadb-dump Guide
This guide explains how to use the mariadb-dump
utility to create essential backup (dump) files of your MariaDB data. Learn to effectively back up all databases, specific databases, or individual tables, ensuring your data is protected and can be restored when needed.
mariadb-dump
mariadb-dump
is a command-line utility included with MariaDB for creating logical backups of your databases. It was previously known as mysqldump
, which often still works as a symbolic link.
Key Advantages:
No Server Shutdown: Backups can be performed while the MariaDB server is running.
SQL Output: It generates a .sql
file (a "dump file") containing SQL statements (CREATE TABLE
, INSERT
, etc.) necessary to reconstruct the databases and data.
All mariadb-dump
commands are executed from your system's command-line shell, not within the mariadb
client.
To export all databases managed by your MariaDB server:
mariadb-dump --user=admin_backup --password --lock-tables --all-databases > /data/backup/dbs_alldatabases.sql
--user=admin_backup
: Specifies the MariaDB user performing the backup (this user needs appropriate privileges, typically at least SELECT
and LOCK TABLES
).
--password
: Prompts for the user's password. For use in scripts where prompting is not possible, you can use --password=yourpassword
(note the absence of a space and the security implication of having the password in a script or command history).
--lock-tables
(or -x
): Locks all tables across all databases before starting the backup to ensure data consistency. The lock is released once the dump is complete for each table. For transactional tables like InnoDB, using --single-transaction
is often preferred as it provides a consistent snapshot without prolonged locking of all tables.
--all-databases
(or -A
): Specifies that all databases should be dumped.
>
/data/backup/dbs_alldatabases.sql
: Redirects the output (the SQL statements) to the specified file. Ensure the path exists and the user running the command has write permissions.
Commonly Used Option for Efficiency:
--extended-insert
(or -e
): Creates INSERT
statements that include multiple rows per statement. This generally results in a smaller dump file and faster restores. This option is often enabled by default but can be explicitly stated.
Example with long options and password in script:
mariadb-dump --user=admin_backup --password=yoursecurepassword --lock-tables --extended-insert --all-databases > /data/backup/dbs_alldatabases.sql
Backing up databases individually can result in smaller, more manageable dump files and allow for more flexible backup schedules.
mariadb-dump --user=admin_backup --password --lock-tables --extended-insert --databases your_database_name > /data/backup/your_database_name.sql
--databases
(or -B
): Followed by the name of the database to dump.
To back up multiple specific databases, list their names separated by spaces after the --databases
option:
mariadb-dump --user=admin_backup --password --lock-tables --extended-insert --databases db1_name db2_name > /data/backup/selected_databases.sql
For very large databases, or if only certain tables change frequently, you might back up individual tables.
mariadb-dump --user=admin_backup --password --lock-tables --extended-insert your_database_name table_name1 table_name2 > /data/backup/your_database_name_selected_tables.sql
First, specify the database name (your_database_name
).
Then, list one or more table names (table_name1
, table_name2
) separated by spaces.
Note that the --databases
option is not used when dumping specific tables in this manner.
User Privileges: The MariaDB user specified with --user
needs at least SELECT
privileges for the tables being dumped. LOCK TABLES
privilege is needed if using --lock-tables
. RELOAD
or FLUSH_TABLES
might be needed for options like --flush-logs
or --master-data
. For --single-transaction
, PROCESS
and RELOAD
might be required. A user with global SELECT
, LOCK TABLES
, SHOW VIEW
, EVENT
, and TRIGGER
privileges is often used for backups.
Consistency with InnoDB: For databases primarily using InnoDB tables, consider using the --single-transaction
option instead of --lock-tables
. This option starts a transaction before dumping and reads data from a consistent snapshot without locking the tables for extended periods, allowing concurrent reads and writes.Bash
mariadb-dump --user=admin_backup --password --single-transaction --extended-insert --databases your_innodb_database > /data/backup/your_innodb_database.sql
Practice Makes Perfect: mariadb-dump
is powerful but can have many options. Practice using it on a test database or server to become comfortable with its usage and to verify that your backup strategy works as expected.
Test Your Backups: Regularly test your backup files by restoring them to a non-production environment to ensure they are valid and can be used for recovery.
Restoration: To learn how to restore data from these dump files, see the "Data Restoration Guide".
Security: Store backup files in a secure location. If passwords are included in scripts, ensure the script files have restricted permissions.
This page is licensed: CC BY-SA / Gnu FDL
The strategy applied when implementing data backups depends on business needs.
Data backup strategy depends on business needs. Business needs can be evaluated by performing a data inventory, determining data recovery objectives, considering the replication environment, and considering encryption requirements. Also critical is a backup storage strategy and testing backup and recovery procedures.
Backup strategy requirements flow from the understanding you build by performing a data inventory. A data inventory is established by asking questions such as:
What data is housed in the databases?
What business purpose does this data serve?
How long does the data needed to be retained in order to meet this business purpose?
Are there any legal or regulatory requirements, which would limit the length of data retention?
Data recovery requirements are often defined in terms of Recovery Point Objective (RPO) and Recovery Time Objective (RTO). RTO and RPO are considered in the context of the data identified in the data inventory.
Recovery Point Objective (RPO) defines the maximum amount of data a business is willing to lose. For example, a business can define a RPO of 24 hours.
Recovery Time Objective (RTO) defines how quickly a business needs to restore service in the event of a fault. For example, a business can define a RTO of 8 hours.
Backup strategy plays a substantial role in achieving RPO and RTO.
RPO depends on completion of backups, which provide a viable recovery point. Since RPO is measured at backup completion, not backup initiation, backup jobs must be scheduled at an interval smaller than the RPO.
Techniques for achieving RPO include:
Frequent incremental backups and less frequent full backups.
Performing backups in conjunction with replication and clustering to eliminate impact on production workloads, allowing a higher backup frequency.
Automated monitoring of backup status.
Automated testing of backups.
The RTO window typically commences at the point when a decision is made by the business to recover from backups, not at the start of an incident.
Techniques for achieving RTO include:
Leveraging information produced during incident response, which can reduce the set of data to restore from backups, or identify specific data validation requirements dependent on the nature of the incident.
Having fast access to backup data. Performance requirements of backup infrastructure should be understood for both backup and restoration workloads.
Using delayed replication, either within the same data center or to a different data center, can provide shorter path to recovery. This is particularly true when coupled with robust application monitoring, which allows intervention before the window of delay elapses.
Applying written and tested recovery procedures, which designate the systems and commands to be used during recovery.
Performing drills and exercises that periodically test recovery procedures to confirm readiness.
MariaDB Enterprise Server supports several implementations of replication, which accurately duplicates data from one Server to one or more other Servers. The use of a dedicated replica as a source for backups can minimize workload impact.
MariaDB Enterprise Cluster implements virtually synchronous replication, where each Server instance contains a replica of all of the data for the Cluster. Backups can be performed from any node in the Cluster.
MariaDB Enterprise Server supports encryption on disk (data-at-rest encryption) and on the network (data-in-transit encryption).
MariaDB Enterprise Backup copies tablespaces from disk. When data-at-rest encryption is enabled, backups contain encrypted data.
MariaDB Enterprise Backup supports TLS encryption for communications with MariaDB Enterprise Server. To enable TLS encryption, set TLS options from the command-line or in the configuration file:
mariadb-backup --backup \
--target-dir=/data/backups/full \
--user=mariadb-backup \
--password=mbu_passwd \
--ssl-ca=/etc/my.cnf.d/certs/ca.pem \
--ssl-cert=/etc/my.cnf.d/certs/client-cert.pem \
--ssl-key=/etc/my.cnf.d/certs/client-key.pem
How backups are stored can impact backup viability. Backup storage also presents separate risks. These risks need to be carefully considered:
Backup data should always be stored separately from the system being backed up, and separate from the system used for recovery.
Backup data should be subject to equal or more controls than data in production databases. For example, backup data should generally be encrypted even where a decision has bee made that a production database will not use data-at-rest encryption.
Business requirements may define a need for offsite storage of backups as a means of guaranteeing delivery on RPO. In these cases you should also consider onsite storage of backups as a means of guaranteeing delivery on RTO.
Retention requirements and the run-rate of new data production can aid in capacity planning.
Testing has been identified as a critical success factor for the successful operation of data systems.
Backups should be tested. Recovery using backups and recovery procedures should be tested.
This page is: Copyright © 2025 MariaDB. All rights reserved.
Backup and restore implementations can help overcome specific technical challenges that would otherwise pose a barrier to meeting business requirements.
Each of these practices represents a trade-off. Understand risks before implementing any of these practices.
Technical challenge: restore time
Trade-off: increased ongoing overhead for backup processing
Backup data can be prepared for restore any time after it is produced and before it is used for restore. To expedite recovery, incremental backups can be pre-applied to the prior full backup to enable faster recovery. This may be done at the expense of recovery points, or at the expense of storage by maintaining copies of unmerged full and incremental backup directories.
Technical challenge: disk space limitations
Trade-off: modification of backup directory contents
Suggested method for moving restored data is to use --copy-back
as this method provides added safety. Where you might instead optimize for disk space savings, system resources, and time you may choose to instead use MariaDB Enterprise Backup's --move-back
option. Speed benefits are only present when backup files are on the same disk partition as the destination data directory.
The --move-back
option will result in the removal of all data files from the backup directory, so it is best to use this option only when you have an additional copy of your backup data in another location.
To restore from a backup by moving files, use the --move-back
option:
mariadb-backup --move-back --target-dir=/data/backups/full
Technical challenge:: CPU bottlenecks
Trade-off: Increased workload during backups
MariaDB Enterprise Backup is a multi-threaded application that by default runs on a single thread. In cases where you have a host with multiple cores available, you can specify the number of threads you want it to use for parallel data file transfers using the --parallel
option:
mariadb-backup --backup \
--target-dir=/data/backups/full \
--user=mariadb-backup \
--password=mbu_passwd \
--parallel=12
Technical challenge: Backup resource overhead, backup duration
Trade-off: Increased restore complexity, restore process duration
Under normal operation an incremental backup is taken against an existing full backup. This allows you to further shorten the amount of time MariaDB Enterprise Backup locks MariaDB Enterprise Server while copying tablespaces. You can then apply the changes in the increment to the full backup with a --prepare
operation at leisure, without disrupting database operations.
MariaDB Enterprise Backup also supports incrementing from an incremental backup. In this operation, the --incremental-basedir
option points not to the full backup directory but rather to the previous incremental backup.
mariadb-backup --backup \
--incremental-basedir=/data/backups/inc1 \
--target-dir=/data/backups/inc2 \
--user=mariadb-backup \
--password=mbu_passwd
In preparing a backup to restore the data directory, apply the chain of incremental backups to the full backup in order. That is, first inc1/, then inc2/
, and so on:
mariadb-backup --prepare \
--target-dir=/data/backups/full \
--incremental-dir=/data/backups/inc1
mariadb-backup --prepare \
--target-dir=/data/backups/full \
--incremental-dir=/data/backups/inc2
Continue to apply all the incremental changes until you have applied all available to the backup. Then restore as usual:
mariadb-backup --copy-back --target-dir=/data/backups/full
chown -R mysql:mysql /var/lib/mysql
Start MariaDB Enterprise Server on the restored data directory.
Technical challenge: Backup resource overhead, backup duration.
Trade-off: Limited to platforms with volume-level snapshots, may require crash recovery.
While MariaDB Enterprise Backups produces file-level backups, users on storage solutions may prefer to instead perform volume-level snapshots to minimize resource impact. This storage capability exists with some SAN, NAS, and volume manager platforms.
Snapshots occur point-in-time, so no preparation step is needed to ensure data is internally consistent. Snapshots occur while tablespaces are open, and a restored snapshot may need to undergo crash recovery.
Just as traditional full, incremental, and partial backups should be tested, so too should recovery from snapshots be tested on an ongoing basis.
MariaDB Server includes advanced backup functionality to reduce the impact of backup operations:
Connect with a client and issue a BACKUP STAGE START
statement and then a BACKUP STAGE BLOCK_COMMIT
statement.
Take the snapshot.
Issue a BACKUP STAGE END
statement.
Once the backup has been completed, remove all files which begin with the #sql prefix
. These files are generated when ALTER TABLE
occurs during a staged backup.
Retrieve, copy, or store the snapshot as is typical for your storage platform and as per business requirements to make the backup durable. This may require mounting the snapshot in some manner.
It is recommended to briefly prevent writes while snapshotting. Specific commands vary depending on storage platform, business requirements, and setup, but a general approach is to:
Connect with a client and issue a FLUSH TABLES WITH READ LOCK
statement, leaving the client connected.
Take the snapshot.
Issue an UNLOCK TABLES
statement, to remove the read lock.
Retrieve, copy, or store the snapshot as is typical for your storage platform and as per business requirements to make the backup durable. This may require mounting the snapshot in some manner.
This page is: Copyright © 2025 MariaDB. All rights reserved.
It's important to give careful thought to the privileges associated with stored functions and stored procedures. The following is an explanation of how they work.
To create a stored routine, the CREATE ROUTINE privilege is needed. The SUPER privilege is required if a DEFINER
is declared that's not the creator's account (see DEFINER clause below). The SUPER
privilege is also required if statement-based binary logging is used. See Binary Logging of Stored Routines for more details.
To make changes to, or drop, a stored routine, the ALTER ROUTINE privilege is needed. The creator of a routine is temporarily granted this privilege if they attempt to change or drop a routine they created, unless the automatic_sp_privileges variable is set to 0
(it defaults to 1).
The SUPER
privilege is also required if statement-based binary logging is used. See Binary Logging of Stored Routines for more details.
To run a stored routine, the EXECUTE privilege is needed. This is also temporarily granted to the creator if they attempt to run their routine unless the automatic_sp_privileges variable is set to 0
.
The SQL SECURITY clause (by default DEFINER
) specifies what privileges are used when a routine is called. If SQL SECURITY
is INVOKER
, the function body are evaluated using the privileges of the user calling the function. If SQL SECURITY
is DEFINER
, the function body is always evaluated using the privileges of the definer account. DEFINER
is the default. Thus, by default, users who can access the database associated with the stored routine can also run the routine, and potentially perform operations they wouldn't normally have permissions for.
The creator of a routine is the account that ran the CREATE FUNCTION or CREATE PROCEDURE statement, regardless of whether a DEFINER
is provided. The definer is by default the creator unless otherwise specified.
The server automatically changes the privileges in the mysql.proc table as required, but will not look out for manual changes.
If left out, the DEFINER
is treated as the account that created the stored routine or view. If the account creating the routine has the SUPER
privilege, another account can be specified as the DEFINER
.
This clause specifies the context the stored routine or view will run as. It can take two values - DEFINER
or INVOKER
. DEFINER
is the account specified as the DEFINER
when the stored routine or view was created (see the section above). INVOKER
is the account invoking the routine or view.
As an example, let's assume a routine, created by a superuser who's specified as the DEFINER
, deletes all records from a table. If SQL SECURITY=DEFINER
, anyone running the routine, regardless of whether they have delete privileges, are able to delete the records. If SQL SECURITY = INVOKER
, the routine will only delete the records if the account invoking the routine has permission to do so.
INVOKER
is usually less risky, as a user cannot perform any operations they're normally unable to. However, it's not uncommon for accounts to have relatively limited permissions, but be specifically granted access to routines, which are then invoked in the DEFINER
context.
All privileges that are specific to a stored routine are dropped when a DROP FUNCTION or DROP ROUTINE is run. However, if a CREATE OR REPLACE FUNCTION or CREATE OR REPLACE PROCEDURE is used to drop and replace and the routine, any privileges specific to that routine will not be dropped.
Changing the DEFINER of MySQL stored routines etc. - maria.com post on what to do after you've dropped a user, and now want to change the DEFINER on all database objects that currently have it set to this dropped user.
This page is licensed: CC BY-SA / Gnu FDL
sudo apt update
sudo yum update # For older systems
sudo dnf update # For newer systems
sudo apt install mariadb-server mariadb-client
sudo dnf install mariadb mariadb-server
sudo mariadb-secure-installation
sudo systemctl status mariadb
sudo systemctl start mariadb
mariadb -u root -p
DROP FUNCTION [IF EXISTS] f_name
DROP FUNCTION hello;
Query OK, 0 rows affected (0.042 sec)
DROP FUNCTION hello;
ERROR 1305 (42000): FUNCTION test.hello does not exist
DROP FUNCTION IF EXISTS hello;
Query OK, 0 rows affected, 1 warning (0.000 sec)
SHOW WARNINGS;
+-------+------+------------------------------------+
| Level | Code | Message |
+-------+------+------------------------------------+
| Note | 1305 | FUNCTION test.hello does not exist |
+-------+------+------------------------------------+
$ mariadb-backup --backup \
--target-dir=/var/mariadb/backup/ \
--user=mariadb-backup --password=mypassword
$ ls /var/mariadb/backup/
aria_log.0000001 mysql xtrabackup_checkpoints
aria_log_control performance_schema xtrabackup_info
backup-my.cnf test xtrabackup_logfile
ibdata1 xtrabackup_binlog_info
$ mariadb-backup --prepare \
--target-dir=/var/mariadb/backup/
$ mariadb-backup --copy-back \
--target-dir=/var/mariadb/backup/
$ chown -R mysql:mysql /var/lib/mysql/
$ rsync -avrP /var/mariadb/backup /var/lib/mysql/
$ chown -R mysql:mysql /var/lib/mysql/
$ mariadb-backup --backup \
--target-dir=/var/mariadb/backup/ \
--user=mariadb-backup --password=mypassword
$ mariadb-backup --backup \
--slave-info --safe-slave-backup \
--target-dir=/var/mariadb/backup/ \
--user=mariadb-backup --password=mypassword
$ mariadb-backup --prepare \
--target-dir=/var/mariadb/backup/
$ rsync -avP /var/mariadb/backup dbserver2:/var/mariadb/backup
$ mariadb-backup --copy-back \
--target-dir=/var/mariadb/backup/
$ chown -R mysql:mysql /var/lib/mysql/
CREATE USER 'repl'@'dbserver2' IDENTIFIED BY 'password';
GRANT REPLICATION SLAVE ON *.* TO 'repl'@'dbserver2';
mariadb-bin.000096 568 0-1-2
$ cat xtrabackup_binlog_info
mariadb-bin.000096 568 0-1-2
SET GLOBAL gtid_slave_pos = "0-1-2";
CHANGE MASTER TO
MASTER_HOST="dbserver1",
MASTER_PORT=3306,
MASTER_USER="repl",
MASTER_PASSWORD="password",
MASTER_USE_GTID=slave_pos;
START SLAVE;
CHANGE MASTER TO
MASTER_HOST="dbserver1",
MASTER_PORT=3306,
MASTER_USER="repl",
MASTER_PASSWORD="password",
MASTER_LOG_FILE='mariadb-bin.000096',
MASTER_LOG_POS=568;
START SLAVE;
SHOW SLAVE STATUS\G
BEGIN
FOR EACH affected TABLE
SELECT 1 FROM <TABLE> LIMIT 0
This page lists the most important SQL statements and contains links to their documentation pages. If you need a basic tutorial on how to use the MariaDB database server and how to execute simple commands, see A MariaDB Primer.
Also see Common MariaDB Queries for examples of commonly-used queries.
CREATE DATABASE is used to create a new, empty database.
DROP DATABASE is used to completely destroy an existing database.
USE is used to select a default database.
CREATE TABLE is used to create a new table, which is where your data is actually stored.
ALTER TABLE is used to modify an existing table's definition.
DROP TABLE is used to completely destroy an existing table.
DESCRIBE shows the structure of a table.
SELECT is used when you want to read (or select) your data.
INSERT is used when you want to add (or insert) new data.
UPDATE is used when you want to change (or update) existing data.
DELETE is used when you want to remove (or delete) existing data.
REPLACE is used when you want to add or change (or replace) new or existing data.
TRUNCATE is used when you want to empty (or delete) all data from the template.
START TRANSACTION is used to begin a transaction.
COMMIT is used to apply changes and end transaction.
ROLLBACK is used to discard changes and end transaction.
CREATE DATABASE mydb;
USE mydb;
CREATE TABLE mytable ( id INT PRIMARY KEY, name VARCHAR(20) );
INSERT INTO mytable VALUES ( 1, 'Will' );
INSERT INTO mytable VALUES ( 2, 'Marry' );
INSERT INTO mytable VALUES ( 3, 'Dean' );
SELECT id, name FROM mytable WHERE id = 1;
UPDATE mytable SET name = 'Willy' WHERE id = 1;
SELECT id, name FROM mytable;
DELETE FROM mytable WHERE id = 1;
SELECT id, name FROM mytable;
DROP DATABASE mydb;
SELECT COUNT(1) FROM mytable; gives the NUMBER OF records IN the TABLE
The first version of this article was copied, with permission, from Basic_SQL_Statements on 2012-10-05.
This page is licensed: CC BY-SA / Gnu FDL
When using mariadb-backup, you have the option of performing a full or incremental backup. Full backups create a complete copy in an empty directory while incremental backups update a previous backup with new data. This page documents incremental backups.
InnoDB pages contain log sequence numbers, or LSN's. Whenever you modify a row on any InnoDB table on the database, the storage engine increments this number. When performing an incremental backup, mariadb-backup checks the most recent LSN for the backup against the LSN's contained in the database. It then updates any of the backup files that have fallen behind.
In order to take an incremental backup, you first need to take a full backup. In order to back up the database, you need to run mariadb-backup with the --backup option to tell it to perform a backup and with the --target-dir option to tell it where to place the backup files. When taking a full backup, the target directory must be empty or it must not exist.
To take a backup, run the following command:
$ mariadb-backup --backup \
--target-dir=/var/mariadb/backup/ \
--user=mariadb-backup --password=mypassword
This backs up all databases into the target directory /var/mariadb/backup
. If you look in that directory at the xtrabackup_checkpoints file, you can see the LSN data provided by InnoDB.
For example:
backup_type = full-backuped
from_lsn = 0
to_lsn = 1635102
last_lsn = 1635102
recover_binlog_info = 0
Once you have created a full backup on your system, you can also back up the incremental changes as often as you would like.
In order to perform an incremental backup, you need to run mariadb-backup with the --backup option to tell it to perform a backup and with the --target-dir option to tell it where to place the incremental changes. The target directory must be empty. You also need to run it with the --incremental-basedir option to tell it the path to the full backup taken above. For example:
$ mariadb-backup --backup \
--target-dir=/var/mariadb/inc1/ \
--incremental-basedir=/var/mariadb/backup/ \
--user=mariadb-backup --password=mypassword
This command creates a series of delta files that store the incremental changes in /var/mariadb/inc1
. You can find a similar xtrabackup_checkpoints file in this directory, with the updated LSN values.
For example:
backup_type = incremental
from_lsn = 1635102
to_lsn = 1635114
last_lsn = 1635114
recover_binlog_info = 0
To perform additional incremental backups, you can then use the target directory of the previous incremental backup as the incremental base directory of the next incremental backup. For example:
$ mariadb-backup --backup \
--target-dir=/var/mariadb/inc2/ \
--incremental-basedir=/var/mariadb/inc1/ \
--user=mariadb-backup --password=mypassword
--stream
outputWhen using --stream, e.g for compression or encryption using external tools, the xtrabackup_checkpoints file containing the information where to continue from on the next incremental backup will also be part of the compressed/encrypted backup file, and so not directly accessible by default.
A directory containing an extra copy of the file can be created using the --extra-lsndir=... option though, and this directory can then be passed to the next incremental backup --incremental-basedir=..., for example:
# initial full backup
$ mariadb-backup --backup --stream=mbstream \
--user=mariadb-backup --password=mypassword \
--extra-lsndir=backup_base | gzip > backup_base.gz
# incremental backup
$ mariadb-backup --backup --stream=mbstream \
--incremental-basedir=backup_base \
--user=mariadb-backup --password=mypassword \
--extra-lsndir=backup_inc1 | gzip > backup-inc1.gz
Following the above steps, you have three backups in /var/mariadb
: The first is a full backup, the others are increments on this first backup. In order to restore a backup to the database, you first need to apply the incremental backups to the base full backup. This is done using the --prepare command option. In MariaDB 10.1, you would also have to use the --apply-log-only option.
In MariaDB 10.2 and later, perform the following process:
First, prepare the base backup:
$ mariadb-backup --prepare \
--target-dir=/var/mariadb/backup
Running this command brings the base full backup, that is, /var/mariadb/backup
, into sync with the changes contained in the InnoDB redo log collected while the backup was taken.
Then, apply the incremental changes to the base full backup:
$ mariadb-backup --prepare \
--target-dir=/var/mariadb/backup \
--incremental-dir=/var/mariadb/inc1
Running this command brings the base full backup, that is, /var/mariadb/backup
, into sync with the changes contained in the first incremental backup.
For each remaining incremental backup, repeat the last step to bring the base full backup into sync with the changes contained in that incremental backup.
Once you've applied all incremental backups to the base, you can restore the backup using either the --copy-back or the --move-back options. The --copy-back option allows you to keep the original backup files. The --move-back option actually moves the backup files to the datadir, so the original backup files are lost.
First, stop the MariaDB Server process.
Then, ensure that the datadir is empty.
Then, run mariadb-backup with one of the options mentioned above:
$ mariadb-backup --copy-back \
--target-dir=/var/mariadb/backup/
Then, you may need to fix the file permissions.
When mariadb-backup restores a database, it preserves the file and directory privileges of the backup. However, it writes the files to disk as the user and group restoring the database. As such, after restoring a backup, you may need to adjust the owner of the data directory to match the user and group for the MariaDB Server, typically mysql
for both. For example, to recursively change ownership of the files to the mysql
user and group, you could execute:
$ chown -R mysql:mysql /var/lib/mysql/
Finally, start the MariaDB Server process.
This page is licensed: CC BY-SA / Gnu FDL
This method is to solve a flaw with mariadb-backup; it cannot do single database restores from a full backup easily. There is a blog post that details a way to do this, but it's a manual process which is fine for a few tables but if you have hundreds or even thousands of tables then it would be impossible to do quickly.
We can't just move the data files to the datadir as the tables are not registered in the engines, so the database will error. Currently, the only effective method is to a do full restore in a test database and then dump the database that requires restoring or running a partial backup.
This has only been tested with InnoDB. Also, if you have stored procedures or triggers then these will need to be deleted and recreated.
Some of the issues that this method overcomes:
Tables not registered in the InnoDB engine so will error when you try to select from a table if you move the data files into the datadir
Tables with foreign keys need to be created without keys, otherwise it will error when you discard the tablespace
Below is the process to perform a single database restore.
Firstly, we will need the table structure from a mariadb-dump backup with the --no-data option. I recommend this is done at least once per day or every six hours via a cronjob. As it is just the structure, it are very fast.
mariadb-dump -u root -p --all-databases --no-data > nodata.sql
Using SED to return only the table structure we require, then use vim or another text editor to make sure nothing is left.
sed -n '/Current Database: `DATABASENAME`/, /Current Database:/p' nodata.sql > trimednodata.sql
vim trimednodata.sql
I won’t go over the backup process, as this is done earlier in other documents, such as full-backup-and-restore-with-mariadb-backup. Prepare the backup with any incremental-backup-and-restores that you have, and then run the following on the full backup folder using the --export option to generate files with .cfg extensions which InnoDB will look for.
mariadb-backup --prepare --export --target-dir=/media/backups/fullbackupfolder
Once we have done these steps, we can then import the table structure. If you have used the --all-databases option, then you will need to either use SED or open it in a text editor and export out tables that you require. You will also need to log in to the database and create the database if the dump file doesn't. Run the following command below:
Mysql -u root -p schema_name < nodata.sql
Once the structure is in the database, we have now registered the tables to the engine. Next, we will run the following statements in the information_schema database, to export statements to import/discard table spaces and drop and create foreign keys which we will use later. (edit the CONSTRAINT_SCHEMA and TABLE_SCHEMA WHERE clause to the database you are restoring. Also, add the following lines after your SELECT and before the FROM to have MariaDB export the files to the OS)
SELECT ...
INTO OUTFILE '/tmp/filename.SQL'
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
FROM ...
The following are the statements that we will need later.
USE information_schema;
SELECT concat("ALTER TABLE ",table_name," DISCARD TABLESPACE;") AS discard_tablespace
FROM information_schema.tables
WHERE TABLE_SCHEMA="DATABASENAME";
SELECT concat("ALTER TABLE ",table_name," IMPORT TABLESPACE;") AS import_tablespace
FROM information_schema.tables
WHERE TABLE_SCHEMA="DATABASENAME";
SELECT
CONCAT ("ALTER TABLE ", rc.CONSTRAINT_SCHEMA, ".",rc.TABLE_NAME," DROP FOREIGN KEY ", rc.CONSTRAINT_NAME,";") AS drop_keys
FROM REFERENTIAL_CONSTRAINTS AS rc
WHERE CONSTRAINT_SCHEMA = 'DATABASENAME';
SELECT
CONCAT ("ALTER TABLE ",
KCU.CONSTRAINT_SCHEMA, ".",
KCU.TABLE_NAME,"
ADD CONSTRAINT ",
KCU.CONSTRAINT_NAME, "
FOREIGN KEY ", "
(`",KCU.COLUMN_NAME,"`)", "
REFERENCES `",REFERENCED_TABLE_NAME,"`
(`",REFERENCED_COLUMN_NAME,"`)" ,"
ON UPDATE " ,(SELECT UPDATE_RULE FROM REFERENTIAL_CONSTRAINTS WHERE CONSTRAINT_NAME = KCU.CONSTRAINT_NAME AND CONSTRAINT_SCHEMA = KCU.CONSTRAINT_SCHEMA),"
ON DELETE ",(SELECT DELETE_RULE FROM REFERENTIAL_CONSTRAINTS WHERE CONSTRAINT_NAME = KCU.CONSTRAINT_NAME AND CONSTRAINT_SCHEMA = KCU.CONSTRAINT_SCHEMA),";") AS add_keys
FROM KEY_COLUMN_USAGE AS KCU
WHERE KCU.CONSTRAINT_SCHEMA = 'DATABASENAME'
AND KCU.POSITION_IN_UNIQUE_CONSTRAINT >= 0
AND KCU.CONSTRAINT_NAME NOT LIKE 'PRIMARY';
Once we have run those statements, and they have been exported to a Linux directory or copied from a GUI interface.
Run the ALTER DROP KEYS statements in the database
ALTER TABLE schemaname.tablename DROP FOREIGN KEY key_name;
...
Once completed, run the DROP TABLE SPACE statements in the database
ALTER TABLE test DISCARD TABLESPACE;
...
Exit out the database and change into the directory of the full backup location. Run the following commands to copy all the .cfg and .ibd files to the datadir such as /var/lib/mysql/testdatabase (Change the datadir location if needed). Learn more about files that mariadb-backup generates with files-created-by-mariadb-backup
cp *.cfg /var/lib/mysql
cp *.ibd /var/lib/mysql
After moving the files, it is very important that MySQL is the owner of the files, otherwise it won't have access to them and will error when we import the tablespaces.
sudo chown -R mysql:mysql /var/lib/mysql
Run the import table spaces statements in the database.
ALTER TABLE test IMPORT TABLESPACE;
...
Run the add key statements in the database
ALTER TABLE schmeaname.tablename ADD CONSTRAINT key_name FOREIGN KEY (`column_name`) REFERENCES `foreign_table` (`colum_name`) ON UPDATE NO ACTION ON DELETE NO ACTION;
...
We have successfully restored a single database. To test that this has worked, we can do a basic check on some tables.
USE DATABASE
SELECT * FROM test LIMIT 10;
If you have a primary-replica set up, it would be best to follow the sets above for the primary node and then either take a full mariadb-dump or take a new full mariadb-backup and restore this to the replica. You can find more information about restoring a replica with mariadb-backup in Setting up a Replica with mariadb-backup
After running the below command, copy to the replica and use the LESS linux command to grab the change master statement. Remember to follow this process: Stop replica > restore data > run CHANGE MASTER statement > start replica again.
mariadb-dump -u user -p --single-transaction --master-data=2 > fullbackup.sql
Please follow Setting up a Replica with mariadb-backup on restoring a replica with mariadb-backup
$ mariadb-backup --backup \
--slave-info --safe-slave-backup \
--target-dir=/var/mariadb/backup/ \
--user=mariadb-backup --password=mypassword
For this process to work with Galera cluster, we first need to understand that some statements are not replicated across Galera nodes. One of which is the DISCARD and IMPORT for ALTER TABLES statements, and these statements will need to be ran on all nodes. We also need to run the OS level steps on each server as seen below.
Run the ALTER DROP KEYS statements on ONE NODE as these are replicated.
ALTER TABLE schemaname.tablename DROP FOREIGN KEY key_name;
...
Once completed, run the DROP TABLE SPACE statements on EVERY NODE, as these are not replicated.
ALTER TABLE test DISCARD TABLESPACE;
...
Exit out the database and change into the directory of the full backup location. Run the following commands to copy all the .cfg and .ibd files to the datadir such as /var/lib/mysql/testdatabase (Change the datadir location if needed). Learn more about files that mariadb-backup generates with files-created-by-mariadb-backup. This step needs to be done on all nodes. You will need to copy the backup files to each node, we can use the same backup on all nodes.
cp *.cfg /var/lib/mysql
cp *.ibd /var/lib/mysql
After moving the files, it is very important that MySQL is the owner of the files, otherwise it won't have access to them and will error when we import the tablespaces.
sudo chown -R mysql:mysql /var/lib/mysql
Run the import table spaces statements on EVERY NODE.
ALTER TABLE test IMPORT TABLESPACE;
...
Run the add key statements on ONE NODE
ALTER TABLE schmeaname.tablename ADD CONSTRAINT key_name FOREIGN KEY (`column_name`) REFERENCES `foreign_table` (`colum_name`) ON UPDATE NO ACTION ON DELETE NO ACTION;
...
This page is licensed: CC BY-SA / Gnu FDL
Binary logging can be row-based, statement-based, or a mix of the two. See Binary Log Formats for more details on the formats. If logging is statement-based, it is possible that a statement will have different effects on the master and on the slave.
Stored routines are particularly prone to this, for two main reasons:
stored routines can be non-deterministic, in other words non-repeatable, and therefore have different results each time they are run.
the slave thread executing the stored routine on the slave holds full privileges, while this may not be the case when the routine was run on the master.
The problems with replication will only occur with statement-based logging. If row-based logging is used, since changes are made to rows based on the master's rows, there is no possibility of the slave and master getting out of sync.
By default, with row-based replication, triggers run on the master, and the effects of their executions are replicated to the slaves. However, starting from , it is possible to run triggers on the slaves. See Running triggers on the slave for Row-based events.
If the following criteria are met, then there are some limitations on whether stored routines can be created:
The binary log is enabled, and the binlog_format system variable is set to STATEMENT
. See Binary Log Formats for more information.
The log_bin_trust_function_creators is set to OFF
, which is the default value.
If the above criteria are met, then the following limitations apply:
When a stored function is created, it must be declared as either DETERMINISTIC
, NO SQL
or READS SQL DATA
, or else an error will occur. MariaDB cannot check whether a function is deterministic, and relies on the correct definition being used.
To create or modify a stored function, a user requires the SUPER
privilege as well as the regular privileges. See Stored Routine Privileges for these details.
Triggers can also update data. The slave uses the DEFINER attribute to determine which user is taken to have created the trigger.
Note that the above limitations do no apply to stored procedures or to events.
A deterministic function:
DELIMITER //
CREATE FUNCTION trust_me(x INT)
RETURNS INT
DETERMINISTIC
READS SQL DATA
BEGIN
RETURN (x);
END //
DELIMITER ;
A non-deterministic function, since it uses the UUID_SHORT function:
DELIMITER //
CREATE FUNCTION dont_trust_me()
RETURNS INT
BEGIN
RETURN UUID_SHORT();
END //
DELIMITER ;
This page is licensed: CC BY-SA / Gnu FDL
SQL Query Design Guide
This guide offers conventions and practical tips for designing SQL queries that are easier to read, understand, and debug. Learn about effectively using whitespace, choosing meaningful aliases, correctly placing JOIN
conditions, and strategies for identifying and resolving common syntax errors.
Following a few conventions makes finding errors in queries a lot easier, especially when asking for help from people who might know SQL but know nothing about your particular schema. A query easy to read is a query easy to debug.
A query that is hard to read is hard to debug. Whitespace is free; use new lines and indentation to make queries easy to read, particularly when constructing a query inside a scripting language where variables might be interspersed.
Example: Hard-to-read query with a syntax error
Can you find the error quickly in this?
Same query with better whitespace:
Code snippet
The missing closing parenthesis )
after team.teamId
in the second JOIN
condition is much easier to spot with clear formatting. The exact style (commas before or after selected items, tabs vs. spaces) is less important than overall legibility.
Aliases rename tables and fields within a query, useful for long names or when required (e.g., self-joins, some subqueries). Poorly chosen aliases, however, can hinder debugging. Aliases should ideally reflect the original table name.
Bad Example (arbitrary aliases):
Code snippet
As the query grows, it's hard to remember what a
, b
, or c
refer to without looking back.
Better Example (meaningful aliases):
To correct the SQL code, it should be properly formatted and structured:
This query selects all data from financial_report_Q_1
and joins sales_renderings
and sales_agents
using specified conditions. It filters for total sales greater than 10,000 and excludes records where the sales agent's ID matches
It's unclear how family
and relationships
are linked without parsing the entire WHERE
clause.
Better Example (clear separation):
The table relationship is obvious in the JOIN ... ON
clause. The WHERE
clause is reserved for filtering the result set. Always use explicit JOIN
keywords (INNER JOIN
, LEFT JOIN
, etc.) instead of commas for joining tables, and do not mix these styles.
MariaDB's error messages usually point to where the parser got confused. Check the query around the indicated point.
Interpreting the "Empty Error"
An error like ERROR 1064: ... syntax to use near '' at line 1 can be tricky. The empty ' ' often means the parser reached the end of the statement while expecting more tokens.
Check for missing closing characters like quotes '
or parentheses )
.SQL
Look for incomplete clauses, often indicated by a trailing comma.SQL
Checking for Reserved Keywords
If an identifier (table name, column name, alias) is a MariaDB reserved word, it must be enclosed in backticks (`) to avoid ambiguity.
A text editor with SQL syntax highlighting can help spot these. Common identifiers that are also keywords include:
DESC
(often for "description", but means "descending" in ORDER BY
)
DATE
, TIME
, TIMESTAMP
(data types)
ORDER
(used in sales contexts, but is an SQL clause)
It's a good practice to quote any identifier that is also a keyword, even if MariaDB might allow some unquoted in certain contexts.
Version-Specific Syntax
SQL syntax evolves. Features and syntax available in newer MariaDB versions might not work in older ones, and vice-versa (though less common).
New syntax in old versions: Web hosts may run older MariaDB versions. A query working on your newer local setup might fail in an older production environment.
Subqueries (e.g., WHERE someId IN (SELECT id FROM ...))
were added in MySQL 4.1.
Early JOIN
syntax did not always allow an ON
clause.
Old syntax in new versions: Sometimes, changes in operator precedence or syntax deprecation can cause issues. For example, the precedence of the comma operator relative to JOIN
changed in MySQL 5.0. A query like SELECT * FROM a, b JOIN c ON a.x = c.x;
that worked before might fail or produce different results.
Always check the MariaDB server version you are targeting and consult the manual for that version to ensure syntax compatibility. The manual usually indicates when specific syntax features became available.
This page is licensed: CC BY-SA / Gnu FDL
If you are completely new to MariaDB and relational databases, you may want to start with the . Also, make sure you understand the connection parameters discussed in the article.
There are a number of common problems that can occur when connecting to MariaDB.
If the error you get is something like:
or
the server is either not running, or not running on the specified port, socket or pipe. Make sure you are using the correct host, port, pipe, socket and protocol options, or alternatively, see , or .
The socket file can be in a non-standard path. In this case, the socket
option is probably written in the my.cnf file. Check that its value is identical in the [mysqld] and [client] sections; if not, the client will look for a socket in a wrong place.
If unsure where the Unix socket file is running, it's possible to find this out, for example:
Usually, the MariaDB server does not by default accept connections from a remote client or connecting with tcp and a hostname and has to be configured to permit these.
To solve this, see
The is enabled by default on Unix-like systems. This uses operating system credentials when connecting to MariaDB via the local Unix socket file. See for instructions on connecting and on switching to password-based authentication as well as for an overview.
Authentication is granted to a particular username/host combination. user1'@'localhost'
, for example, is not the same as user1'@'166.78.144.191'
. See the article for details on granting permissions.
Passwords are hashed with function. If you have set a password with the statement, the PASSWORD
function must be used at the same time. For example, SET PASSWORD FOR 'bob'@'%.loc.gov' = PASSWORD('newpass')
rather than just SET PASSWORD FOR 'bob'@'%.loc.gov' = 'newpass'
.
If you can run regular queries, but get an authentication error when running the , or statements, you do not have permission to write files to the server. This requires the FILE privilege. See the article.
If you can connect to the server, but not to a database, for example:
or can connect to a particular database, but not another, for examplemariadb -uname -p -u name db1
works but not mariadb -uname -p -u name db2
, you have not been granted permission for the particular database. See the article.
It's possible that option files or environment variables may be providing incorrect connection parameters. Check the values provided in any option files read by the client you are using (see and the documentation for the particular client you're using - see ).
Option files can usually be suppressed with no-defaults
option, for example:
If you are unable to connect to a server, for example because you have lost the root password, you can start the server without using the privilege tables by running the option, which gives users full access to all tables. You can then run to resume using the grant tables, followed by to change the password for an account.
You may have created a user with something like:
This creates a user with the '%' wildcard host.
However, you may still be failing to login from localhost. Some setups create anonymous users, including localhost. So the following records exist in the user table:
Since you are connecting from localhost, the anonymous credentials, rather than those for the 'melisa' user, are used. The solution is either to add a new user specific to localhost, or to remove the anonymous localhost user.
CC BY-SA / Gnu FDL
In the modern world, data importance is non-negotiable, and keeping data integrity and consistency is the top priority. Data stored in databases is vulnerable to system crashes, hardware problems, security breaches, and other failures causing data loss or corruption. To prevent database damage, it is important to back the data up regularly and implement the data restore policies. MariaDB, one of the most popular database management systems, provides several methods to configure routines for backing up and recovering data. The current guideline illustrates both processes performed with the help of dbForge Studio for MySQL which is also a fully-functional that has everything you need to accomplish the database-related tasks on MariaDB.
dbForge Studio for MySQL and MariaDB has a separate module dedicated to the data backing up and recovering jobs. Let us first look at how set the tool to create a MariaDB backup. Launch the Studio and go to Database > Backup and Restore > Backup Database. The Database Backup Wizard with several pages will appear. On the General page, specify the database in question and how to connect to it, then choose where to save the created backup file, and specify its name. There are additional optional settings – you can select to delete old files automatically, zip the output backup file, etc. When done, click Next.
On the Backup content page, select the objects to back up. Click Next.
The Options page. Here you can specify the details of the data backing up process. Plenty of available options allow you to configure this task precisely to meet the specific requirements. When done, click Next.
The Errors handling page. Here you configure how the Studio should handle the errors that might occur during the backing up process. Also, you can set the Studio to write the information about the errors it encountered into the log file.
You can save the project settings to apply them in the future. For this, in the left bottom corner of the Wizard, select one of the saving options: Save Project or Save Command Line. The latter allows saving settings as a backup script which you can execute from the command line at any time later.
The configuration process is complete. Click Backup to launch the data backing up.
Note: It is not obligatory to go through all the pages of the Wizard. The Backup button is available no matter on which page you are. Thus, you can launch the process of backing the data up whenever you have set everything you needed.
After you have clicked Backup, dbForge Studio for MySQL starts to create a MariaDB backup.
When this is done, you will see the confirmation message. Click Finish.
Backup and restore policies suggest creating regular backups on a daily, weekly, monthly, quarterly, and yearly basis. Besides, to minimize the consequences of possible data loss, it is highly recommended make a backup before making any changes to a database, such as upgrading, modifying data, redesigning the structure, etc. Simply speaking, you always need a fresh backup to restore the most up-to-date database version. To ensure regular backups on schedule, you can use a batch file created with the help of the Studio and Windows Task Scheduler, where you need to create and schedule the backup task.
This is an even faster task, done in half as many steps.
The process of data recovery from the backup file is simple. It only takes several clicks: Launch dbForge Studio for MySQL and go to Database > Backup and Restore > Restore Database. The Database Restore Wizard will appear. Specify the database name, its connection parameters, and the path to the backup file you want to restore. Then click Restore, and the process will start immediately.
When the process is complete, click Finish.
More information about this essential feature is available on the – it explores the routines performed on MySQL, but they fully apply to MariaDB backups. You can use the same IDE and the same workflow.
To test-drive this and other features of the Studio (the IDE includes all the tools necessary for the development, management, and administration of databases on MariaDB), . dbForge Studio for MySQL and MariaDB boasts truly advanced functionality that will help your teams deliver more value.
This page is licensed: CC BY-SA / Gnu FDL
The RANGE partitioning type is used to assign each partition a range of values generated by the partitioning expression. Ranges must be ordered, contiguous and non-overlapping. The minimum value is always included in the first range. The highest value may or may not be included in the last range.
A variant of this partitioning method, , allows us to use multiple columns and more datatypes.
The last part of a statement can be definition of the new table's partitions. In the case of RANGE partitioning, the syntax is the following:
PARTITION BY RANGE indicates that the partitioning type is RANGE.
The partitioning_expression
is an SQL expression that returns a value from each row. In the simplest cases, it is a column name. This value is used to determine which partition should contain a row.
partition_name
is the name of a partition.
value
indicates the upper bound for that partition. The values must be ascending. For the first partition, the lower limit is NULL. When trying to insert a row, if its value is higher than the upper limit of the last partition, the row are rejected (with an error, if the keyword is not used).
As a catchall, MAXVALUE can be specified as a value for the last partition. Note however that in order to append a new partition, it is not possible to use ; rather must be used.
A typical use case is when we want to partition a table whose rows refer to a moment or period in time; for example commercial transactions, blog posts, or events of some kind. We can partition the table by year, to keep all recent data in one partition and distribute historical data in big partitions that are stored on slower disks. Or, if our queries always read rows which refer to the same month or week, we can partition the table by month or year week (in this case, historical data and recent data are stored together).
values also represent a chronological order. So, these values can be used to store old data in separate partitions. However, partitioning by id is not the best choice if we usually query a table by date.
In the following example, we will partition a log table by year.
As an alternative, we can partition the table by both year and month:
As you can see, we used the function to accomplish the purpose. Also, the first two partitions cover longer periods of time (probably because the logged activities were less intensive).
In both cases, when our tables become huge and we don't need to store all historical data any more, we can drop the oldest partitions in this way:
We will still be able to drop a partition that does not contain the oldest data, but all rows stored in it will disappear.
Example of an error when inserting outside a defined partition range:
Unless the IGNORE keyword is used:
An alternative definition with MAXVALUE as a catchall:
This page is licensed: CC BY-SA / Gnu FDL
A Stored Procedure is a routine invoked with a statement. It may have input parameters, output parameters and parameters that are both input parameters and output parameters.
Here's a skeleton example to see a stored procedure in action:
First, the delimiter is changed, since the function definition will contain the regular semicolon delimiter. The procedure is named Reset_animal_count
. MODIFIES SQL DATA
indicates that the procedure will perform a write action of sorts, and modify data. It's for advisory purposes only. Finally, there's the actual SQL statement - an UPDATE.
A more complex example, with input parameters, from an actual procedure used by banks:
See for full syntax details.
Security is a key reason. Banks commonly use stored procedures so that applications and users don't have direct access to the tables. Stored procedures are also useful in an environment where multiple languages and clients are all used to perform the same operations.
To find which stored functions are running on the server, use .
or query the in the INFORMATION_SCHEMA database directly:
To find out what the stored procedure does, use .
To drop a stored procedure, use the statement.
To change the characteristics of a stored procedure, use . However, you cannot change the parameters or body of a stored procedure using this statement; to make such changes, you must drop and re-create the procedure using (which retains existing privileges), or DROP PROCEDURE followed CREATE PROCEDURE .
See the article .
This page is licensed: CC BY-SA / Gnu FDL
SELECT u.id, u.name, alliance.ally FROM users u JOIN alliance ON (u.id=alliance.userId) JOIN team ON (alliance.teamId=team.teamId WHERE team.teamName='Legionnaires' AND u.online=1 AND ((u.subscription='paid' AND u.paymentStatus='current') OR u.subscription='free') ORDER BY u.name;
SELECT
u.id,
u.name,
alliance.ally
FROM
users u
JOIN alliance ON (u.id = alliance.userId)
JOIN team ON (alliance.teamId = team.teamId -- Error: Missing ')'
WHERE
team.teamName = 'Legionnaires'
AND u.online = 1
AND (
(u.subscription = 'paid' AND u.paymentStatus = 'current')
OR
u.subscription = 'free'
)
ORDER BY
u.name;
SELECT *
FROM
financial_reportQ_1 AS a
JOIN sales_renderings AS b ON (a.salesGroup = b.groupId)
JOIN sales_agents AS c ON (b.groupId = c.group)
WHERE
b.totalSales > 10000
AND c.id != a.clientId;
SELECT *
FROM financial_report_Q_1 AS frq1
JOIN sales_renderings AS sr ON frq1.salesGroup = sr.groupId
JOIN sales_agents AS sa ON sr.groupId = sa.group
WHERE sr.totalSales > 10000
AND sa.id != frq1.clientId;
Using initials or recognizable abbreviations (e.g., `frq1` for `financial_report_Q_1`) makes the query more understandable.
## Placing JOIN Conditions
The `ON` clause of a `JOIN` should specify the conditions that link the tables. Avoid using it for filtering rows that belong in the `WHERE` clause. Conversely, avoid placing all join logic in the `WHERE` clause (as common with older, implicit join syntax).
**Bad Example (join condition mixed in `WHERE`):**
```sql
SELECT *
FROM
family,
relationships
WHERE
family.personId = relationships.personId -- Join condition
AND relationships.relation = 'father'; -- Filtering condition
SELECT *
FROM
family
JOIN relationships ON (family.personId = relationships.personId) -- Join condition
WHERE
relationships.relation = 'father'; -- Filtering condition
SELECT * FROM someTable WHERE field = 'value -- Missing closing quote
SELECT * FROM someTable WHERE field = 1 GROUP BY id, -- Incomplete GROUP BY
SELECT * FROM actionTable WHERE `DELETE` = 1; -- `DELETE` is a reserved word
PARTITION BY RANGE (partitioning_expression)
(
PARTITION partition_name VALUES LESS THAN (value),
[ PARTITION partition_name VALUES LESS THAN (value), ... ]
[ PARTITION partition_name VALUES LESS THAN MAXVALUE ]
)
CREATE TABLE log
(
id INT UNSIGNED NOT NULL AUTO_INCREMENT,
dt DATETIME NOT NULL,
user INT UNSIGNED,
PRIMARY KEY (id, dt)
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016)
);
CREATE TABLE log2
(
id INT UNSIGNED NOT NULL AUTO_INCREMENT,
ts TIMESTAMP NOT NULL,
user INT UNSIGNED,
PRIMARY KEY (id, ts)
)
ENGINE = InnoDB
PARTITION BY RANGE (UNIX_TIMESTAMP(ts))
(
PARTITION p0 VALUES LESS THAN (UNIX_TIMESTAMP('2014-08-01 00:00:00')),
PARTITION p1 VALUES LESS THAN (UNIX_TIMESTAMP('2014-11-01 00:00:00')),
PARTITION p2 VALUES LESS THAN (UNIX_TIMESTAMP('2015-01-01 00:00:00')),
PARTITION p3 VALUES LESS THAN (UNIX_TIMESTAMP('2015-02-01 00:00:00'))
);
ALTER TABLE log DROP PARTITION p0;
INSERT INTO log(id,dt) VALUES
(1, '2016-01-01 01:01:01'),
(2, '2015-01-01 01:01:01');
ERROR 1526 (HY000): Table has no partition for value 2016
INSERT IGNORE INTO log(id,dt) VALUES
(1, '2016-01-01 01:01:01'),
(2, '2015-01-01 01:01:01');
SELECT * FROM log;
+----+---------------------+------+
| id | timestamp | user |
+----+---------------------+------+
| 2 | 2015-01-01 01:01:01 | NULL |
+----+---------------------+------+
CREATE TABLE log
(
id INT UNSIGNED NOT NULL AUTO_INCREMENT,
dt DATETIME NOT NULL,
user INT UNSIGNED,
PRIMARY KEY (id, dt)
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016),
PARTITION p4 VALUES LESS THAN MAXVALUE
);
DELIMITER //
CREATE PROCEDURE Reset_animal_count()
MODIFIES SQL DATA
UPDATE animal_count SET animals = 0;
//
DELIMITER ;
SELECT * FROM animal_count;
+---------+
| animals |
+---------+
| 101 |
+---------+
CALL Reset_animal_count();
SELECT * FROM animal_count;
+---------+
| animals |
+---------+
| 0 |
+---------+
CREATE PROCEDURE
Withdraw /* Routine name */
(parameter_amount DECIMAL(6,2), /* Parameter list */
parameter_teller_id INTEGER,
parameter_customer_id INTEGER)
MODIFIES SQL DATA /* Data access clause */
BEGIN /* Routine body */
UPDATE Customers
SET balance = balance - parameter_amount
WHERE customer_id = parameter_customer_id;
UPDATE Tellers
SET cash_on_hand = cash_on_hand + parameter_amount
WHERE teller_id = parameter_teller_id;
INSERT INTO Transactions VALUES (
parameter_customer_id,
parameter_teller_id,
parameter_amount);
END;
SHOW PROCEDURE STATUS\G
*************************** 1. row ***************************
Db: test
Name: Reset_animal_count
Type: PROCEDURE
Definer: root@localhost
Modified: 2013-06-03 08:55:03
Created: 2013-06-03 08:55:03
Security_type: DEFINER
Comment:
character_set_client: utf8
collation_connection: utf8_general_ci
Database Collation: latin1_swedish_ci
SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.ROUTINES
WHERE ROUTINE_TYPE='PROCEDURE';
+--------------------+
| ROUTINE_NAME |
+--------------------+
| Reset_animal_count |
+--------------------+
SHOW CREATE PROCEDURE Reset_animal_count\G
*************************** 1. row ***************************
Procedure: Reset_animal_count
sql_mode:
Create Procedure: CREATE DEFINER=`root`@`localhost` PROCEDURE `Reset_animal_count`()
MODIFIES SQL DATA
UPDATE animal_count SET animals = 0
character_set_client: utf8
collation_connection: utf8_general_ci
Database Collation: latin1_swedish_ci
DROP PROCEDURE Reset_animal_count();
This article briefly discusses the main ways to backup MariaDB. For detailed descriptions and syntax, see the individual pages. More detail is in the process of being added.
Logical backups consist of the SQL statements necessary to restore the data, such as CREATE DATABASE, CREATE TABLE and INSERT.
Physical backups are performed by copying the individual data files or directories.
The main differences are as follows:
logical backups are more flexible, as the data can be restored on other hardware configurations, MariaDB versions or even on another DBMS, while physical backups cannot be imported on significantly different hardware, a different DBMS, or potentially even a different MariaDB version.
logical backups are larger in size than the equivalent physical backup.
logical backups takes more time to both backup and restore than the equivalent physical backup.
log files and configuration files are not part of a logical backup
mariadb-backup
The mariadb-backup program is a fork of Percona XtraBackup with added support for compression and data-at-rest encryption.
mariadb-dump
mariadb-dump (previously mysqldump) performs a logical backup. It is the most flexible way to perform a backup and restore, and a good choice when the data size is relatively small.
For large datasets, the backup file can be large, and the restore time lengthy.
mariadb-dump
dumps the data into SQL format (it can also dump into other formats, such as CSV or XML) which can then easily be imported into another database. The data can be imported into other versions of MariaDB, MySQL, or even another DBMS entirely, assuming there are no version or DBMS-specific statements in the dump.
mariadb-dump dumps triggers along with tables, as these are part of the table definition. However, stored procedures, views, and events are not, and need extra parameters to be recreated explicitly (for example, --routines
and --events
). Procedures and functions are however also part of the system tables (for example mysql.proc).
InnoDB uses the buffer pool, which stores data and indexes from its tables in memory. This buffer is very important for performance. If InnoDB data doesn't fit the memory, it is important that the buffer contains the most frequently accessed data. However, last accessed data is candidate for insertion into the buffer pool. If not properly configured, when a table scan happens, InnoDB may copy the whole contents of a table into the buffer pool. The problem with logical backups is that they always imply full table scans.
An easy way to avoid this is by increasing the value of the innodb_old_blocks_time system variable. It represents the number of milliseconds that must pass before a recently accessed page can be put into the "new" sublist in the buffer pool. Data which is accessed only once should remain in the "old" sublist. This means that they will soon be evicted from the buffer pool. Since during the backup process the "old" sublist is likely to store data that is not useful, one could also consider resizing it by changing the value of the innodb_old_blocks_pct system variable.
It is also possible to explicitly dump the buffer pool on disk before starting a logical backup, and restore it after the process. This will undo any negative change to the buffer pool which happens during the backup. To dump the buffer pool, the innodb_buffer_pool_dump_now system variable can be set to ON
. To restore it, the innodb_buffer_pool_load_now system variable can be set to ON
.
mariadb-dump
ExamplesBacking up a single database
mariadb-dump db_name > backup-file.sql
Restoring or loading the database
mariadb db_name < backup-file.sql
See the mariadb-dump page for detailed syntax and examples.
mariadb-hotcopy
mariadb-hotcopy performs a physical backup, and works only for backing up MyISAM and ARCHIVE tables. It can only be run on the same machine as the location of the database directories.
mariadb-hotcopy
Examplesmariadb-hotcopy db_name [/path/to/new_directory]
mariadb-hotcopy db_name_1 ... db_name_n /path/to/new_directory
Percona XtraBackup is not supported in MariaDB. mariadb-backup is the recommended backup method to use instead of Percona XtraBackup. See Percona XtraBackup Overview: Compatibility with MariaDB for more information.
Percona XtraBackup is a tool for performing fast, hot backups. It was designed specifically for XtraDB/InnoDB databases, but can be used with any storage engine (although not with encryption and compression). It is not included with MariaDB.
Some filesystems, like Veritas, support snapshots. During the snapshot, the table must be locked. The proper steps to obtain a snapshot are:
From the mariadb client, execute FLUSH TABLES WITH READ LOCK. The client must remain open.
From a shell, execute mount vxfs snapshot
The client can execute UNLOCK TABLES.
Copy the snapshot files.
From a shell, unmount the snapshot with umount snapshot
.
Widely-used physical backup method, using a Perl script as a wrapper. See http://www.lenzg.net/mylvmbackup/ for more information.
For details, see:
Besides the system utilities, it is possible to use third-party GUI tools to perform backup and restore operations. In this context, it is worth mentioning dbForge Studio for MySQL, a feature-rich database IDE that is fully compatible with MariaDB and delivers extensive backup functionality.
The backup and restore module of the Studio allows precise configuration and management of full and partial backups up to particular database objects. The feature of scheduling regular backups offers specific settings to handle errors and keep a log of them. Additionally, settings and configurations can be saved for later reuse.
These operations are wizard-aided allowing users to set up all tasks in a visual mode.
Streaming MariaDB backups in the cloud (mariadb.com blog)
This page is licensed: CC BY-SA / Gnu FDL
mariadb -uname -p -uname -p
ERROR 2002 (HY000): Can't connect to local MySQL server through
socket '/var/run/mysqld/mysqld.sock' (2 "No such file or directory")
mariadb -uname -p --port=3307 --protocol=tcp
ERROR 2003 (HY000): Can't connect to MySQL server on 'localhost'
(111 "Connection refused")
netstat -ln | grep mysqld
unix 2 [ ACC ] STREAM LISTENING 33209505 /var/run/mysqld/mysqld.sock
(/my/maria-10.4) ./client/mysql --host=myhost --protocol=tcp --port=3306 test
ERROR 2002 (HY000): Can't connect to MySQL server on 'myhost' (115)
(/my/maria-10.4) telnet myhost 3306
Trying 192.168.0.11...
telnet: connect to address 192.168.0.11: Connection refused
(/my/maria-10.4) perror 115
OS error code 115: Operation now in progress
USE test;
ERROR 1044 (42000): Access denied for user 'ian'@'localhost' to database 'test'
mariadb-import --no-defaults ...
CREATE USER melisa IDENTIFIED BY 'password';
SELECT user,host FROM mysql.user WHERE user='melisa';
+--------+------+
| user | host |
+--------+------+
| melisa | % |
+--------+------+
SELECT user,host FROM mysql.user WHERE user='melisa' OR user='';
+--------+-----------+
| user | host |
+--------+-----------+
| melisa | % |
| | localhost |
+--------+-----------+
Following a few conventions makes finding errors in queries a lot easier, especially when you ask for help from people who might know SQL, but know nothing about your particular schema. A query easy to read is a query easy to debug. Use whitespace to group clauses within the query. Choose good table and field aliases to add clarity, not confusion. Choose the syntax that supports the query's meaning.
A query hard to read is a query hard to debug. White space is free. New lines and indentation make queries easy to read, particularly when constructing a query inside a scripting language, where variables are interspersed throughout the query.
There is a syntax error in the following. How fast can you find it?
SELECT u.id, u.name, alliance.ally FROM users u JOIN alliance ON
(u.id=alliance.userId) JOIN team ON (alliance.teamId=team.teamId
WHERE team.teamName='Legionnaires' AND u.online=1 AND ((u.subscription='paid'
AND u.paymentStatus='current') OR u.subscription='free') ORDER BY u.name;
Here's the same query, with correct use of whitespace. Can you find the error faster?
SELECT
u.id
, u.name
, alliance.ally
FROM
users u
JOIN alliance ON (u.id = alliance.userId)
JOIN team ON (alliance.teamId = team.teamId
WHERE
team.teamName = 'Legionnaires'
AND u.online = 1
AND (
(u.subscription = 'paid' AND u.paymentStatus = 'current')
OR
u.subscription = 'free'
)
ORDER BY
u.name;
Even if you don't know SQL, you might still have caught the missing ')' following team.teamId.
The exact formatting style you use isn't so important. You might like commas in the select list to follow expressions, rather than precede them. You might indent with tabs or with spaces. Adherence to some particular form is not important. Legibility is the only goal.
Aliases allow you to rename tables and fields for use within a query. This can be handy when the original names are very long, and is required for self joins and certain subqueries. However, poorly chosen aliases can make a query harder to debug, rather than easier. Aliases should reflect the original table name, not an arbitrary string.
Bad:
SELECT *
FROM
financial_reportQ_1 AS a
JOIN sales_renderings AS b ON (a.salesGroup = b.groupId)
JOIN sales_agents AS c ON (b.groupId = c.group)
WHERE
b.totalSales > 10000
AND c.id != a.clientId
As the list of joined tables and the WHERE
clause grow, it becomes necessary to repeatedly look back to the top of the query to see to which table any given alias refers.
Better:
SELECT *
FROM
financial_report_Q_1 AS frq1
JOIN sales_renderings AS sr ON (frq1.salesGroup = sr.groupId)
JOIN sales_agents AS sa ON (sr.groupId = sa.group)
WHERE
sr.totalSales > 10000
AND sa.id != frq1.clientId
Each alias is just a little longer, but the table initials give enough clues that anyone familiar with the database only need see the full table name once, and can generally remember which table goes with which alias while reading the rest of the query.
The manual warns against using the JOIN
condition (that is, the ON clause) for restricting rows. Some queries, particularly those using implicit joins, take the opposite extreme - all join conditions are moved to the WHERE
clause. In consequence, the table relationships are mixed with the business
logic.
Bad:
SELECT *
FROM
family,
relationships
WHERE
family.personId = relationships.personId
AND relationships.relation = 'father'
Without digging through the WHERE clause, it is impossible to say what links the two tables.
Better:
SELECT *
FROM
family
JOIN relationships ON (family.personId = relationships.personId)
WHERE
relationships.relation = 'father'
The relation between the tables is immediately obvious. The WHERE clause is left to limit rows in the result set.
Compliance with such a restriction negates the use of the comma operator to join tables. It is a small price to pay. Queries should be written using the explicit JOIN keyword anyway, and the two should never be mixed (unless you like rewriting all your queries every time a new version changes operator precedence).
Syntax errors are among the easiest problems to solve. MariaDB provides an error message showing the exact point where the parser became confused. Check the query, including a few words before the phrase shown in the error message. Most syntax and parsing errors are obvious after a second look, but some are more elusive, especially when the error text seems empty, points to a valid keyword, or seems to error on syntax that appears exactly correct.
Most syntax errors are easy to interpret. The error generally details the exact source of the trouble. A careful look at the query, with the error message in mind, often reveals an obvious mistake, such as misspelled field names, a missing 'AND
', or an extra closing parenthesis. Sometimes the error is a little
less helpful. A frequent, less-than-helpful message:
ERROR 1064: You have an error in your SQL syntax; check the manual that corresponds to your
MariaDB server version for the right syntax to use near ' ' at line 1
The empty ' ' can be disheartening. Clearly there is an error, but where? A good place to look is at the end of the query. The ' ' suggests that the parser reached the end of the statement while still expecting some syntax token to appear.
Check for missing closers, such as ' and ):
SELECT * FROM someTable WHERE field = 'value
Look for incomplete clauses, often indicated by an exposed comma:
SELECT * FROM someTable WHERE field = 1 GROUP BY id,
MariaDB allows table and field names and aliases that are also reserved words. To prevent ambiguity, such names must be enclosed in backticks (`):
SELECT * FROM actionTable WHERE `DELETE` = 1;
If the syntax error is shown near one of your identifiers, check if it appears on the reserved word list.
A text editor with color highlighting for SQL syntax helps to find these errors. When you enter a field name, and it shows up in the same color as the SELECT keyword, you know something is amiss. Some common culprits:
DESC is a common abbreviation for "description" fields. It means "descending" in a MariaDB ORDER clause.
DATE, TIME, and TIMESTAMP are all common field names. They are also field types.
ORDER appears in sales applications. MariaDB uses it to specify sorting for results.
Some keywords are so common that MariaDB makes a special allowance to use them unquoted. My advice: don't. If it's a keyword, quote it.
As MariaDB adds new features, the syntax must change to support them. Most of the time, old syntax will work in newer versions of MariaDB. One notable exception is the change in precedence of the comma operator relative to the JOIN keyword in version 5.0. A query that used to work, such as
SELECT * FROM a, b JOIN c ON a.x = c.x;
will now fail.
More common, however, is an attempt to use new syntax in an old version. Web hosting companies are notoriously slow to upgrade MariaDB, and you may find yourself using a version several years out of date. The result can be very frustrating when a query that executes flawlessly on your own workstation, running a recent installation, fails completely in your production environment.
This query fails in any version of MySQL prior to 4.1, when subqueries were added to the server:
SELECT * FROM someTable WHERE someId IN (SELECT id FROM someLookupTable);
This query fails in some early versions of MySQL, because the JOIN syntax did not originally allow an ON clause:
SELECT * FROM tableA JOIN tableB ON tableA.x = tableB.y;
Always check the installed version of MariaDB, and read the section of the manual relevant for that version. The manual usually indicates exactly when particular syntax became available for use.
The initial version of this article was copied, with permission, from Basic_Debugging on 2012-10-05.
This page is licensed: CC BY-SA / Gnu FDL
The BACKUP STAGE commands are a set of commands to make it possible to make an efficient external backup tool. How mariadb-backup uses these commands depends on whether you are using the version that is bundled with MariaDB Community Server or the version that is bundled with MariaDB Enterprise Server.
BACKUP STAGE
Commands in MariaDB Community ServerThe BACKUP STAGE commands are supported. However, the version of mariadb-backup
that is bundled with MariaDB Community Server does not yet use the BACKUP STAGE
commands in the most efficient way. mariadb-backup simply executes the following BACKUP STAGE
commands to lock the database:
BACKUP STAGE START;
BACKUP STAGE BLOCK_COMMIT;
When the backup is complete, it executes the following BACKUP STAGE
command to unlock the database:
BACKUP STAGE END;
If you would like to use a version of mariadb-backup
that uses the BACKUP STAGE commands in the most efficient way, then your best option is to use MariaDB Backup that is bundled with MariaDB Enterprise Server.
BACKUP STAGE
in MariaDB Community ServerCopy some transactional tables.
InnoDB (i.e. ibdataN
and file extensions .ibd
and .isl
)
Copy the tail of some transaction logs.
The tail of the InnoDB redo log (i.e. ib_logfileN
files) are copied for InnoDB tables.
BACKUP STAGE START
in MariaDB Community Servermariadb-backup from MariaDB Community Server does not currently perform any tasks in the START
stage.
BACKUP STAGE FLUSH
in MariaDB Community Servermariadb-backup from MariaDB Community Server does not currently perform any tasks in the FLUSH
stage.
BACKUP STAGE BLOCK_DDL
in MariaDB Community Servermariadb-backup from MariaDB Community Server does not currently perform any tasks in the BLOCK_DDL
stage.
BACKUP STAGE BLOCK_COMMIT
in MariaDB Community Servermariadb-backup
from MariaDB Community Server performs the following tasks in the BLOCK_COMMIT
stage:
Copy other files.
i.e. file extensions .frm
, .isl
, .TRG
, .TRN
, .opt
, .par
Copy some transactional tables.
Aria (i.e. aria_log_control
and file extensions .MAD
and .MAI
)
Copy the non-transactional tables.
MyISAM (i.e. file extensions .MYD
and .MYI
)
MERGE (i.e. file extensions .MRG
)
ARCHIVE (i.e. file extensions .ARM
and .ARZ
)
CSV (i.e. file extensions .CSM
and .CSV
)
Create a MyRocks checkpoint using the rocksdb_create_checkpoint system variable.
Copy the tail of some transaction logs.
The tail of the InnoDB redo log (i.e. ib_logfileN
files) are copied for InnoDB tables.
Save the binary log position to xtrabackup_binlog_info.
Save the Galera Cluster state information to xtrabackup_galera_info.
BACKUP STAGE END
in MariaDB Community Servermariadb-backup from MariaDB Community Server performs the following tasks in the END
stage:
Copy the MyRocks checkpoint into the backup.
BACKUP STAGE
Commands in MariaDB Enterprise ServerThe following sections describe how the MariaDB Backup version of mariadb-backup that is bundled with MariaDB Enterprise Server uses each BACKUP STAGE command in an efficient way.
BACKUP STAGE START
in MariaDB Enterprise Servermariadb-backup from MariaDB Enterprise Server performs the following tasks in the START
stage:
Copy all transactional tables.
InnoDB (i.e. ibdataN
and file extensions .ibd
and .isl
)
Aria (i.e. aria_log_control
and file extensions .MAD
and .MAI
)
Copy the tail of all transaction logs.
The tail of the InnoDB redo log (i.e. ib_logfileN
files) are copied for InnoDB tables.
The tail of the Aria redo log (i.e. aria_log.N
files) are copied for Aria tables.
BACKUP STAGE FLUSH
in MariaDB Enterprise Servermariadb-backup from MariaDB Enterprise Server performs the following tasks in the FLUSH
stage:
Copy all non-transactional tables that are not in use. This list of used tables is found with SHOW OPEN TABLES.
MyISAM (i.e. file extensions .MYD
and .MYI
)
MERGE (i.e. file extensions .MRG
)
ARCHIVE (i.e. file extensions .ARM
and .ARZ
)
CSV (i.e. file extensions .CSM
and .CSV
)
Copy the tail of all transaction logs.
The tail of the InnoDB redo log (i.e. ib_logfileN
files) are copied for InnoDB tables.
The tail of the Aria redo log (i.e. aria_log.N
files) are copied for Aria tables.
BACKUP STAGE BLOCK_DDL
in MariaDB Enterprise Servermariadb-backup from MariaDB Enterprise Server performs the following tasks in the BLOCK_DDL
stage:
Copy other files.
i.e. file extensions .frm
, .isl
, .TRG
, .TRN
, .opt
, .par
Copy the non-transactional tables that were in use during BACKUP STAGE FLUSH
.
MyISAM (i.e. file extensions .MYD
and .MYI
)
MERGE (i.e. file extensions .MRG
)
ARCHIVE (i.e. file extensions .ARM
and .ARZ
)
CSV (i.e. file extensions .CSM
and .CSV
)
Check ddl.log
for DDL executed before the BLOCK DDL
stage.
The file names of newly created tables can be read from ddl.log
.
The file names of dropped tables can also be read from ddl.log
.
The file names of renamed tables can also be read from ddl.log
, so the files can be renamed instead of re-copying them.
Copy changes to system log tables.
mysql.general_log
mysql.slow_log
This is easy as these are append only.
Copy the tail of all transaction logs.
The tail of the InnoDB redo log (i.e. ib_logfileN
files) are copied for InnoDB tables.
The tail of the Aria redo log (i.e. aria_log.N
files) are copied for Aria tables.
BACKUP STAGE BLOCK_COMMIT
in MariaDB Enterprise Servermariadb-backup from MariaDB Enterprise Server performs the following tasks in the BLOCK_COMMIT
stage:
Create a MyRocks checkpoint using the rocksdb_create_checkpoint system variable.
Copy changes to system log tables.
mysql.general_log
mysql.slow_log
This is easy as these are append only.
Copy changes to statistics tables.
mysql.table_stats
mysql.column_stats
mysql.index_stats
Copy the tail of all transaction logs.
The tail of the InnoDB redo log (i.e. ib_logfileN
files) are copied for InnoDB tables.
The tail of the Aria redo log (i.e. aria_log.N
files) are copied for Aria tables.
Save the binary log position to xtrabackup_binlog_info.
Save the Galera Cluster state information to xtrabackup_galera_info.
BACKUP STAGE END
in MariaDB Enterprise Servermariadb-backup from MariaDB Enterprise Server performs the following tasks in the END
stage:
Copy the MyRocks checkpoint into the backup.
This page is licensed: CC BY-SA / Gnu FDL
With MariaDB it's very easy to copy tables between different MariaDB databases and different MariaDB servers. This works for tables created with the Archive, Aria, CSV, InnoDB, MyISAM, MERGE, and XtraDB engines.
The normal procedures to copy a table is:
FLUSH TABLES db_name.table_name FOR EXPORT
# Copy the relevant files associated with the table
UNLOCK TABLES;
The table files can be found in datadir/databasename (you can executeSELECT @@datadir
to find the correct directory).
When copying the files, you should copy all files with the same
table_name + various extensions. For example, for an Aria table of
name foo, you will have files foo.frm, foo.MAI, foo.MAD and possibly
foo.TRG if you have triggers.
If one wants to distribute a table to a user that doesn't need write access to the table and one wants to minimize the storage size of the table, the recommended engine to use is Aria or MyISAM as one can pack the table with aria_pack or myisampack respectively to make it notablly smaller. MyISAM is the most portable format as it's not dependent on whether the server settings are different. Aria and InnoDB require the same block size on both servers.
The following storage engines support export without FLUSH TABLES ... FOR EXPORT
, assuming the source server is down and the receiving server is not accessing the files during the copy.
Requires clean shutdown. Table will automatically be fixed on the receiving server if aria_chk --zerofill was not run. If aria_chk --zerofill is run, then the table is immediately usable without any delays
.MRG files can be copied even while server is running as the file only contains a list of tables that are part of merge.
For all of the above storage engines (Archive, Aria, CSV, MyISAM and MERGE), one can copy tables even from a live server under the following circumstances:
You have done a FLUSH TABLES
or FLUSH TABLE table_name
for the specific table.
The server is not accessing the tables during the copy process.
The advantage of FLUSH TABLES table_name FOR EXPORT is that the table is read locked until UNLOCK TABLES is executed.
Warning: If you do the above live copy, you are doing this on your own risk as if you do something wrong, the copied table is very likely to be corrupted. The original table will of course be fine.
If you want to give a user access to some data in a table for the user to use in their MariaDB server, you can do the following:
First let's create the table we want to export. To speed up things, we
create this without any indexes. We use TRANSACTIONAL=0 ROW_FORMAT=DYNAMIC
for Aria to use the smallest possible row format.
CREATE TABLE new_table ... ENGINE=ARIA TRANSACTIONAL=0;
ALTER TABLE new_table DISABLE_KEYS;
# Fill the table with data:
INSERT INTO new_table SELECT * ...
FLUSH TABLE new_table WITH READ LOCK;
# Copy table data to some external location, like /tmp with something
# like cp /my/data/test/new_table.* /tmp/
UNLOCK TABLES;
Then we pack it and generate the indexes. We use a big sort buffer to speed up generating the index.
> ls -l /tmp/new_table.*
-rw-rw---- 1 mysql my 42396148 Sep 21 17:58 /tmp/new_table.MAD
-rw-rw---- 1 mysql my 8192 Sep 21 17:58 /tmp/new_table.MAI
-rw-rw---- 1 mysql my 1039 Sep 21 17:58 /tmp/new_table.frm
> aria_pack /tmp/new_table
Compressing /tmp/new_table.MAD: (922666 records)
- Calculating statistics
- Compressing file
46.07%
> aria_chk -rq --ignore-control-file --sort_buffer_size=1G /tmp/new_table
Recreating table '/tmp/new_table'
- check record delete-chain
- recovering (with sort) Aria-table '/tmp/new_table'
Data records: 922666
- Fixing index 1
State updated
> ls -l /tmp/new_table.*
-rw-rw---- 1 mysql my 26271608 Sep 21 17:58 /tmp/new_table.MAD
-rw-rw---- 1 mysql my 10207232 Sep 21 17:58 /tmp/new_table.MAI
-rw-rw---- 1 mysql my 1039 Sep 21 17:58 /tmp/new_table.frm
The procedure for MyISAM tables is identical, except that myisamchk doesn't have the --ignore-control-file
option.
InnoDB's file-per-table tablespaces are transportable, which means that you can copy a file-per-table tablespace from one MariaDB Server to another server. See Copying Transportable Tablespaces for more information.
Tables that use most storage engines are immediately usable when their files are copied to the new datadir.
However, this is not true for tables that use InnoDB. InnoDB tables have to be imported with ALTER TABLE ... IMPORT TABLESPACE. See Copying Transportable Tablespaces for more information.
myisampack - Compressing the MyISAM data file for easier distribution.
aria_pack - Compressing the Aria data file for easier distribution
mariadb-dump - Copying tables to other SQL servers. You can use the --tab
to create a CSV file of your table content.
This page is licensed: CC BY-SA / Gnu FDL
A Stored Function is a defined function that is called from within an SQL statement like a regular function, and returns a single value.
Here's a skeleton example to see a stored function in action:
DELIMITER //
CREATE FUNCTION FortyTwo() RETURNS TINYINT DETERMINISTIC
BEGIN
DECLARE x TINYINT;
SET x = 42;
RETURN x;
END
//
DELIMITER ;
First, the delimiter is changed, since the function definition will contain the regular semicolon delimiter. See Delimiters in the mariadb client for more. Then the function is named FortyTwo
and defined to return a tinyin
. The DETERMINISTIC
keyword is not necessary in all cases (although if binary logging is on, leaving it out will throw an error), and is to help the query optimizer choose a query plan. A deterministic function is one that, given the same arguments, will always return the same result.
Next, the function body is placed between BEGIN and END statements. It declares a tinyint, X
, which is simply set to 42, and this is the result returned.
SELECT FortyTwo();
+------------+
| FortyTwo() |
+------------+
| 42 |
+------------+
Of course, a function that doesn't take any arguments is of little use. Here's a more complex example:
DELIMITER //
CREATE FUNCTION VatCents(price DECIMAL(10,2)) RETURNS INT DETERMINISTIC
BEGIN
DECLARE x INT;
SET x = price * 114;
RETURN x;
END //
Query OK, 0 rows affected (0.04 sec)
DELIMITER ;
This function takes an argument, price
which is defined as a DECIMAL, and returns an INT.
Take a look at the CREATE FUNCTION page for more details.
From , it is also possible to create stored aggregate functions.
To find which stored functions are running on the server, use SHOW FUNCTION STATUS.
SHOW FUNCTION STATUS\G
*************************** 1. row ***************************
Db: test
Name: VatCents
Type: FUNCTION
Definer: root@localhost
Modified: 2013-06-01 12:40:31
Created: 2013-06-01 12:40:31
Security_type: DEFINER
Comment:
character_set_client: utf8
collation_connection: utf8_general_ci
Database Collation: latin1_swedish_ci
1 row in set (0.00 sec)
or query the routines table in the INFORMATION_SCHEMA database directly:
SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.ROUTINES WHERE
ROUTINE_TYPE='FUNCTION';
+--------------+
| ROUTINE_NAME |
+--------------+
| VatCents |
+--------------+
To find out what the stored function does, use SHOW CREATE FUNCTION.
SHOW CREATE FUNCTION VatCents\G
*************************** 1. row ***************************
Function: VatCents
sql_mode:
Create Function: CREATE DEFINER=`root`@`localhost` FUNCTION `VatCents`(price DECIMAL(10,2)) RETURNS int(11)
DETERMINISTIC
BEGIN
DECLARE x INT;
SET x = price * 114;
RETURN x;
END
character_set_client: utf8
collation_connection: utf8_general_ci
Database Collation: latin1_swedish_ci
To drop a stored function, use the DROP FUNCTION statement.
DROP FUNCTION FortyTwo;
To change the characteristics of a stored function, use ALTER FUNCTION. Note that you cannot change the parameters or body of a stored function using this statement; to make such changes, you must drop and re-create the function using DROP FUNCTION and CREATE FUNCTION.
See the article Stored Routine Privileges.
This page is licensed: CC BY-SA / Gnu FDL
Aggregate functions are functions that are computed over a sequence of rows and return one result for the sequence of rows.
Creating a custom aggregate function is done using the CREATE FUNCTION statement with two main differences:
The addition of the AGGREGATE keyword, so CREATE AGGREGATE FUNCTION
The FETCH GROUP NEXT ROW
instruction inside the loop
Oracle PL/SQL compatibility using SQL/PL is provided
CREATE AGGREGATE FUNCTION function_name (parameters) RETURNS return_type
BEGIN
ALL types of declarations
DECLARE CONTINUE HANDLER FOR NOT FOUND RETURN return_val;
LOOP
FETCH GROUP NEXT ROW; // fetches next row FROM TABLE
other instructions
END LOOP;
END
Stored aggregate functions were a project by Varun Gupta.
SET sql_mode=Oracle;
DELIMITER //
CREATE AGGREGATE FUNCTION function_name (parameters) RETURN return_type
declarations
BEGIN
LOOP
FETCH GROUP NEXT ROW; -- fetches next row from table
-- other instructions
END LOOP;
EXCEPTION
WHEN NO_DATA_FOUND THEN
RETURN return_val;
END //
DELIMITER ;
First a simplified example:
CREATE TABLE marks(stud_id INT, grade_count INT);
INSERT INTO marks VALUES (1,6), (2,4), (3,7), (4,5), (5,8);
SELECT * FROM marks;
+---------+-------------+
| stud_id | grade_count |
+---------+-------------+
| 1 | 6 |
| 2 | 4 |
| 3 | 7 |
| 4 | 5 |
| 5 | 8 |
+---------+-------------+
DELIMITER //
CREATE AGGREGATE FUNCTION IF NOT EXISTS aggregate_count(x INT) RETURNS INT
BEGIN
DECLARE count_students INT DEFAULT 0;
DECLARE CONTINUE HANDLER FOR NOT FOUND
RETURN count_students;
LOOP
FETCH GROUP NEXT ROW;
IF x THEN
SET count_students = count_students+1;
END IF;
END LOOP;
END //
DELIMITER ;
A non-trivial example that cannot easily be rewritten using existing functions:
DELIMITER //
CREATE AGGREGATE FUNCTION medi_int(x INT) RETURNS DOUBLE
BEGIN
DECLARE CONTINUE HANDLER FOR NOT FOUND
BEGIN
DECLARE res DOUBLE;
DECLARE cnt INT DEFAULT (SELECT COUNT(*) FROM tt);
DECLARE lim INT DEFAULT (cnt-1) DIV 2;
IF cnt % 2 = 0 THEN
SET res = (SELECT AVG(a) FROM (SELECT a FROM tt ORDER BY a LIMIT lim,2) ttt);
ELSE
SET res = (SELECT a FROM tt ORDER BY a LIMIT lim,1);
END IF;
DROP TEMPORARY TABLE tt;
RETURN res;
END;
CREATE TEMPORARY TABLE tt (a INT);
LOOP
FETCH GROUP NEXT ROW;
INSERT INTO tt VALUES (x);
END LOOP;
END //
DELIMITER ;
This uses the same marks table as created above.
SET sql_mode=Oracle;
DELIMITER //
CREATE AGGREGATE FUNCTION aggregate_count(x INT) RETURN INT AS count_students INT DEFAULT 0;
BEGIN
LOOP
FETCH GROUP NEXT ROW;
IF x THEN
SET count_students := count_students+1;
END IF;
END LOOP;
EXCEPTION
WHEN NO_DATA_FOUND THEN
RETURN count_students;
END aggregate_count //
DELIMITER ;
SELECT aggregate_count(stud_id) FROM marks;
This page is licensed: CC BY-SA / Gnu FDL
The quickstart guide walks you through connecting to a MariaDB server, creating your initial database and table structures, and performing fundamental data operations. It's designed for new users or anyone needing a quick refresher on essential MariaDB commands and basic syntax.
To interact with the MariaDB server, use a client program. The default command-line client is mariadb
.
Connect to MariaDB in monitor mode from the Linux command-line:
Common options:
-u username
: Specifies the MariaDB user (e.g., root
). This is not the OS user.
-p
: Prompts for the password. If no password is set, press [Enter].
-h hostname_or_IP
: Specifies the server's hostname or IP address if the client is on a different machine than the server. Often not needed if connecting locally.
If logged into Linux as root
, you might only need:
To exit the mariadb
monitor, type quit
or exit
and press [Enter].
First, create and select a database.
This creates a database named bookstore
and sets it as the default for subsequent operations.
Next, create tables to hold data.
This statement creates a books
table with six columns:
isbn
: CHAR(20)
, the primary key for unique identification.
title
: VARCHAR(50)
, a variable-width string for the book title.
author_id
, publisher_id
: INT
, for storing numeric IDs.
year_pub
: CHAR(4)
, a fixed-width string for the publication year.
description
: TEXT
, for longer descriptive text (up to 65,535 bytes).
To view the structure of a created table:
To modify an existing table, use the ALTER TABLE
statement (see ). To delete a table and all its data (irreversibly), use DROP TABLE table_name;
(see ).
Example of another table, authors
, using AUTO_INCREMENT
for the primary key:
The author_id
will automatically generate a unique number for each new author.
SQL statements typically end with a semicolon (;
) or \G
.
Statements can span multiple lines; execution occurs after the terminating character and [Enter].
To cancel a partially typed statement in the mariadb
client, enter \c
and press [Enter].
SQL reserved words (e.g., CREATE
, SELECT
) are often written in uppercase for readability but are case-insensitive in MariaDB.
Database and table names are case-sensitive on Linux systems (as they map to directories and files) but generally not on Windows. Column names are case-insensitive.
Using lowercase for table and column names is a common convention.
Use the INSERT
statement (see ) to add new rows to a table.
Since author_id
in the authors
table is AUTO_INCREMENT
(see ), its value is assigned automatically. If not all columns are being supplied with data, the column names must be listed, followed by their corresponding values in the VALUES
clause.
To insert data for a book, referencing author_id
1
(assuming Kafka's author_id
became 1
):
Multiple rows can be inserted with a single INSERT
statement:
Use the SELECT statement (see SELECT documentation) to query data from tables.
To retrieve all book titles:
To limit the number of rows returned (e.g., to 5) using LIMIT
(see ):
To retrieve data from multiple tables, use a JOIN
(see ). This example lists book titles and author last names by joining books
and authors
on their common author_id
column:
To filter results, use the WHERE
clause. This example finds books by 'Kafka' and renames the title
column1 in the output to 'Kafka Books' using AS
(an alias):
To modify existing data, use the UPDATE
statement (see ). Always use a WHERE
clause to specify which rows to update.
This changes the title
for the book with the specified isbn
. Multiple columns can be updated by separating column = value
assignments with commas within the SET
clause.
To remove rows from a table, use the DELETE
statement (see ). Use WHERE
to specify which rows to delete.
This deletes all books associated with author_id
'2034'.
This page is licensed: CC BY-SA / Gnu FDL
This guide details the parameters for connecting to a MariaDB server using client programs like mariadb
. Learn about default connection behaviors and how to use various command-line options to customize your connection, including secure TLS configurations.
While the examples focus on the mariadb
command-line client, the concepts apply to other clients like graphical interfaces or backup utilities (e.g., mariadb-dump
). If you are completely new to MariaDB, refer to first.
When a connection parameter is not explicitly provided, a default value is used. To connect using only default values with the mariadb
client:
In this scenario, the following defaults typically apply:
Host name: localhost
User name: Your Unix login name (on Unix-like systems) or ODBC
(on Windows).
Password: No password is sent.
Database: The client connects to the server but not to a specific database by default.
Socket: The default socket file is used for connection.
You can override these defaults by specifying parameters. For example:
In this example:
-h 166.78.144.191
: Specifies the host IP address instead of localhost
.
-u username
: Specifies username
as the MariaDB user.
-ppassword
: Specifies password
as the password.
Note: For passwords, there must be no space between -p
and the password value.
Security Warning: Providing a password directly on the command line is insecure as it can be visible to other users on the system. It's more secure to use -p
without the password value, which will prompt you to enter it.
database_name
: This is the name of the database to connect to, provided as the first argument after all options.
The connection will use the default TCP/IP port (usually 3306).
The following are common connection parameters:
--host=name
-h name
Connects to the MariaDB server on the given host.
Default: localhost.
MariaDB typically does not permit remote logins by default; see Configuring MariaDB for Remote Client Access.
--password[=passwd]
-p[passwd]
Specifies the password for the MariaDB account.
Security Best Practice: For improved security, use the -p
or --password
option without providing the password value directly on the command line. You will be prompted to enter it, preventing it from being visible in command history or process lists.
--pipe
-W
(Windows only) Connects to the server using a named pipe, if the server was started with the --enable-named-pipe
option.
--port=num
-P num
Specifies the TCP/IP port number for the connection.
Default: 3306.
--protocol=name
Specifies the connection protocol. Possible values (case-insensitive): TCP
, SOCKET
, PIPE
, MEMORY
. The default protocol is typically the most efficient for the operating system (e.g., SOCKET
on Unix).
TCP: TCP/IP connection (local or remote). Available on all OS.
SOCKET: Unix socket file connection (local server on Unix systems only). If --socket
is not specified, the default is /tmp/mysql.sock
.
PIPE: Named-pipe connection (local or remote). Windows only.
MEMORY: Shared-memory connection (local server on Windows systems only).
--shared-memory-base-name=name
(Windows only) Specifies the shared-memory name for connecting to a local server started with the --shared-memory option. The value is case-sensitive.
Default: MARIADB.
--socket=name
-S name
For connections to localhost
:
On Unix: Specifies the Unix socket file to use. Default: /tmp/mysql.sock
.
On Windows: Specifies the name (case-insensitive) of the named pipe if the server was started with --enable-named-pipe
. Default: MARIADB
.
--user=name
-u name
Specifies the MariaDB user name for the connection.
Default: Your Unix login name (on Unix-like systems) or ODBC (on Windows).
See the GRANT command for information on creating MariaDB user accounts.
These options control the use of TLS (Transport Layer Security) for secure connections. For comprehensive details, see and .
--ssl
: Enable TLS for the connection. Automatically enabled if other --ssl-*
flags are used. Disable with --skip-ssl
.
--ssl-ca=name
: CA (Certificate Authority) file in PEM format. (Implies --ssl
).
--ssl-capath=name
: Directory containing CA certificates in PEM format. (Implies --ssl
).
--ssl-cert=name
: Client X.509 certificate in PEM format. (Implies --ssl
).
--ssl-cipher=name
: Specific TLS cipher(s) to use for the connection. (Implies --ssl
).
--ssl-key=name
: Client X.509 private key in PEM format. (Implies --ssl
).
--ssl-crl=name
: Certificate Revocation List (CRL) file in PEM format. (Implies --ssl
).
--ssl-crlpath=name
: Directory containing CRL files. (Implies --ssl
).
--ssl-verify-server-cert
: Verifies the server's certificate "Common Name" against the hostname used for connecting. Disabled by default.
Connection parameters and other options can also be set in option files (configuration files), which most MariaDB clients read upon startup. To see which option files a client reads and the option groups it recognizes, typically run the client with the --help
option.
Basic Joins Guide
This guide offers a simple, hands-on introduction to three basic JOIN
types in MariaDB: INNER JOIN
, CROSS JOIN
, and LEFT JOIN
. Use these examples to understand how different joins combine data from multiple tables based on specified conditions.
First, create and populate two simple tables, t1
and t2
, to use in the JOIN
examples:
Below are examples of different JOIN
types using the tables t1
and t2
.
An INNER JOIN
produces a result set containing only rows that have a match in both tables for the specified join condition(s).
Output:
Explanation: Only the row where t1.a
(value 2) matches t2.b
(value 2) is returned.
A CROSS JOIN
produces a result set in which every row from the first table is joined to every row in the second table. This is also known as a Cartesian product.
Output:
Explanation: Each of the 3 rows in t1
is combined with each of the 2 rows in t2
, resulting in 3 * 2 = 6 rows. Note: In MariaDB, the CROSS
keyword can often be omitted if no ON
clause is present (e.g., SELECT * FROM t1 JOIN t2;
or SELECT * FROM t1, t2;
would also produce a Cartesian product).
A LEFT JOIN
(or LEFT OUTER JOIN
) produces a result set with all rows from the "left" table (t1
in this case). If a match is found in the "right" table (t2
), the corresponding columns from the right table are included. If no match is found, these columns are filled with NULL
.
Output:
Explanation: All rows from t1
are present. For t1.a = 1
and t1.a = 3
, there are no matching t2.b
values, so b
is NULL
. For t1.a = 2
, a match is found (t2.b = 2
), so b
is 2
.
A RIGHT JOIN
(or RIGHT OUTER JOIN
) produces a result set with all rows from the "right" table (t2
in this case). If a match is found in the "left" table (t1
), the corresponding columns from the left table are included. If no match is found, these columns are filled with NULL
.
Output:
This example uses a LEFT JOIN
but with t2
as the left table. This effectively demonstrates how a RIGHT JOIN
would behave if t1
were the left table and t2
the right. A RIGHT JOIN
includes all rows from the "right" table and NULL
s for non-matching "left" table columns.
Output:
Explanation: All rows from t2
are present. For t2.b = 2
, a match is found (t1.a = 2
), so a
is 2
. For t2.b = 4
, there is no matching t1.a
value, so a
is NULL
.
The first two SELECT
statements (INNER JOIN
and CROSS JOIN
) are sometimes written using an older, implicit join syntax:
Implicit INNER JOIN:
This is equivalent to SELECT * FROM t1 INNER JOIN t2 ON t1.a = t2.b;
.
Implicit CROSS JOIN (Cartesian Product):
This is equivalent to SELECT * FROM t1 CROSS JOIN t2;
.
While this syntax works, the explicit JOIN
syntax (INNER JOIN
, LEFT JOIN
, etc.) with an ON
clause is generally preferred for clarity and to better distinguish join conditions from filtering conditions (WHERE
clause).
INNER JOIN
: Returns rows only when there is a match in both tables based on the join condition.
CROSS JOIN
: Returns the Cartesian product of the two tables (all possible combinations of rows).
LEFT JOIN
(Outer Join): Returns all rows from the left table, and the matched rows from the right table. If there is no match, NULL
is returned for columns from the right table.
RIGHT JOIN
(Outer Join): Returns all rows from the right table, and the matched rows from the left table. If there is no match, NULL
is returned for columns from the left table. (The example SELECT * FROM t2 LEFT JOIN t1 ...
shows this behavior from t1
's perspective).
JOIN
clauses can be concatenated (chained) to retrieve results from three or more tables by progressively joining them.
The initial version of this article was copied, with permission, from on 2012-10-05.
This page is licensed: CC BY-SA / Gnu FDL
MariaDB Primer
This primer offers a quick jump-start for beginners using an existing MariaDB database via the mariadb
command-line client. Learn how to log in, understand basic database concepts, and perform essential SQL operations like creating tables, inserting data, and retrieving or modifying records.
To begin, log into your MariaDB server from your system's command-line:
Replace user_name
with your MariaDB username.
Replace ip_address
with the hostname or IP address of your MariaDB server. If you are accessing MariaDB from the same server you're logged into (i.e., locally), you can usually omit the -h ip_address
part.
Replace db_name
with the name of the database you wish to access (e.g., test
). Some setups may have a test
database by default; others might not, or it might have been removed (e.g., by mariadb-secure-installation
). If unsure, or if you want to connect without selecting a specific database initially, you can omit db_name
.
You will be prompted to enter your password. If your login is successful, you will see a prompt similar to this:
The "MariaDB" indicates you are connected to a MariaDB server. The name within the brackets (e.g., test
) is your current default database. If no database was specified or successfully connected to, it might show [(none)]
.
SQL (Structured Query Language): This is the language used to interact with MariaDB. An SQL statement that requests data is called a query.
Tables: Databases store information in tables, which are structured like spreadsheets with rows and columns, but are much more efficient for data management.
Example Setup:
If the test database is empty or doesn't exist, you can run the following SQL statements to create and populate tables for the examples in this primer. Copy and paste these into the mariadb client prompt.
Semicolons (;
): The mariadb
client allows complex SQL statements over multiple lines. It sends the statement to the server for execution only after you type a semicolon (;
) and press [Enter].
Listing Tables:
To see the tables in your current database:
Output (example):
Describing a Table:
To get information about the columns in a table (like their names and types):
Output (example):
The Field
column lists the column names, which you'll need to retrieve specific data.
To retrieve data from a table, use the SELECT
statement.
The asterisk (*
) is a wildcard meaning "all columns." Output (example):
To add new rows to a table, use the INSERT
statement.
After INSERT INTO table_name
, list the columns you are providing data for in parentheses.
The VALUES
keyword is followed by a list of values in parentheses, in the same order as the listed columns. Output:
You can run SELECT * FROM books;
again to see the newly added row.
To change existing data in a table, use the UPDATE
statement. Let's correct the spelling of "The Hobbbit".
SET Title = "The Hobbit"
specifies the column to change and its new value.
WHERE BookID = 7
is crucial; it specifies which row(s) to update. Without a WHERE
clause, UPDATE
would change all rows in the table. Output:
Run SELECT * FROM books WHERE BookID = 7;
to see the correction.
Using MariaDB involves understanding SQL syntax. It doesn't allow for typing mistakes or clauses in the wrong order, but with practice, it becomes straightforward.
There are several ways to add and to change data in MariaDB. There are a few SQL statements that you can use, each with a few options. Additionally, there are twists that you can do by mixing SQL statements together with various clauses. In this article, we will explore the ways in which data can be added and changed in MariaDB.
To add data to a table in MariaDB, you will need to use the statement. Its basic, minimal syntax is the command INSERT
followed by the table name and then the keyword VALUES
with a comma separated list of values contained in parentheses:
In this example, text is added to a table called table1, which contains only three columns—the same number of values that we're inserting. The number of columns must match. If you don't want to insert data into all of the columns of a table, though, you could name the columns desired:
Notice that the keyword INTO
was added here. This is optional and has no effect on MariaDB. It's only a matter of grammatical preference. In this example we not only name the columns, but we list them in a different order. This is acceptable to MariaDB. Just be sure to list the values in the same order. If you're going to insert data into a table and want to specify all of the values except one (say the key column since it's an auto-incremented one), then you could just give a value of DEFAULT
to keep from having to list the columns. Incidentally, you can give the column names even if you're naming all of them. It's just unnecessary unless you're going to reorder them as we did in this last example.
When you have many rows of data to insert into the same table, it can be more efficient to insert all of the rows in one SQL statement. Multiple row insertions can be done like so:
Notice that the keyword VALUES
is used only once and each row is contained in its own set of parentheses and each set is separated by commas. We've added an intentional mistake to this example: We are attempting to insert three rows of data into table2 for which the first column happens to be a UNIQUE
key field. The third row entered here has the same identification number for the key column as the second row. This would normally result in an error and none of the three rows would be inserted. However, since the statement has an IGNORE
flag, duplicates are ignored and not inserted, but the other rows will still be inserted. So, the first and second rows above are inserted and the third one won't.
An INSERT
statement takes priority over read statements (i.e., statements). An INSERT
will lock the table and force other clients to wait until it's finished. On a busy MariaDB server that has many simultaneous requests for data, this could cause users to experience delays when you run a script that performs a series of INSERT
statements. If you don't want user requests to be put on hold and you can wait to insert the data, you could use the LOW_PRIORITY
flag:
The LOW_PRIORITY
flag will put the INSERT
statement in queue, waiting for all current and pending requests to be completed before it's performed. If new requests are made while a low priority statement is waiting, then they are put ahead of it in the queue. MariaDB does not begin to execute a low priority statement until there are no other requests waiting. Once the transaction begins, though, the table is locked and any other requests for data from the table that come in after it starts must wait until it's completed. Because it locks the table, low priority statements will prevent simultaneous insertions from other clients even if you're dealing with a MyISAM table. Incidentally, notice that the LOW_PRIORITY
flag comes before the INTO
.
One potential inconvenience with an INSERT LOW_PRIORITY
statement is that the client are tied up waiting for the statement to be completed successfully. So if you're inserting data into a busy server with a low priority setting using the mariadb client, your client could be locked up for minutes, maybe hours depending on how busy your server is at the time. As an alternative either to making other clients with read requests wait or to having your client wait, you can use the DELAYED flag instead of the LOW_PRIORITY
flag:
MariaDB will take the request as a low priority one and put it on its list of tasks to perform when it has a break. However, it will immediately release the client so that the client can go on to enter other SQL statements or even exit. Another advantage of this method is that multiple INSERT DELAYED
requests are batched together for block insertion when there is a gap, making the process potentially faster than INSERT LOW_PRIORITY
. The flaw in this choice, however, is that the client is never told if a delayed insertion is successfully made or not. The client is informed of error messages when the statement is entered—the statement has to be valid before it are queued—but it's not told of problems that occur after it's accepted. This brings up another flaw: delayed insertions are stored in the server's memory. So if the MariaDB daemon (mariadbd) dies or is manually killed, then the transactions are lost and the client is not notified of the failure. So is not always a good alternative.
As an added twist to INSERT
, you can combine it with a SELECT
statement. Suppose that you have a table called employees which contains employee information for your company. Suppose further that you have a column to indicate whether an employee is on the company's softball team. However, you one day decide to create a separate database and table for the softball team's data that someone else will administer. To get the database ready for the new administrator, you have to copy some data for team members to the new table. Here's one way you can accomplish this task:
In this SQL statement the columns in which data is to be inserted into are listed, then the complete SELECT
statement follows with the appropriate WHERE
clause to determine if an employee is on the softball team. Since we're executing this statement from the new database and since the table employees is in a separate database called company, we have to specify it as you see here. By the way, statements cannot be performed on the same table.
When you're adding massive amounts of data to a table that has a key field, as mentioned earlier, you can use the IGNORE
flag to prevent duplicates from being inserted, but still allow unique rows to be entered. However, there may be times when you actually want to replace the rows with the same key fields with the new ones. In such a situation, instead of using INSERT
you can use a statement:
Notice that the syntax is the same as an INSERT
statement. The flags all have the same effect, as well. Also, multiple rows may be inserted, but there's no need for the IGNORE
flag since duplicates won't happen—the originals are just overwritten. Actually, when a row is replaced, it's first deleted completely and the new row is then inserted. Any columns without values in the new row are given the default values for the columns. None of the values of the old row are kept. Incidentally, REPLACE will also allow you to combine it with a SELECT
statement as we saw with the INSERT statement earlier.
If you want to change the data contained in existing records, but only for certain columns, then you would need to use an statement. The syntax for UPDATE is a little bit different from the syntax shown before for and statements:
In the SQL statement here, we are changing the value of the two columns named individually using the SET
clause. Incidentally, the SET
clause optionally can be used in INSERT
and REPLACE
statements, but it eliminates the multiple row option. In the statement above, we're also using a WHERE
clause to determine which records are changed: only rows with an id that has a value less than 100 are updated. Notice that the LOW_PRIORITY
flag can be used with this statement, too. The IGNORE
flag can be used, as well.
A useful feature of the UPDATE
statement is that it allows the use of the current value of a column to update the same column. For instance, suppose you want to add one day to the value of a date column where the date is a Sunday. You could do the following:
For rows where the day of the week is Sunday, the function will take the value of col_date
before it's updated and add one day to it. MariaDB will then take this sum and set col_date to it.
There are a couple more twists that you can now do with the UPDATE
statement: if you want to update the rows in a specific order, you can add an clause. You can also limit the number of rows that are updated with a clause. Below is an example of both of these clauses:
The ordering can be descending as indicated here by the DESC
flag, or ascending with either the ASC flag or by just leaving it out, as ascending is the default. The LIMIT
clause, of course, limits the number of rows affected to ten here.
If you want to refer to multiple tables in one UPDATE
statement, you can do so like this:
Here we see a join between the two tables named. In table3, the value of col1 is set to the value of the same column in table4 where the values of id from each match. We're not updating both tables here; we're just accessing both. We must specify the table name for each column to prevent an ambiguity error. Incidentally, ORDER BY
and LIMIT
clauses aren't allowed with multiple table updates.
There's another combination that you can do with the INSERT
statement that we didn't mention earlier. It involves the UPDATE
statement. When inserting multiple rows of data, if you want to note which rows had potentially duplicate entries and which ones are new, you could add a column called status and change it's value accordingly with a statement like this one:
Because of the IGNORE
flag, errors will not be generated, duplicates won't be inserted or replaced, but the rest are added. Because of the , the column status of the original row are set to old when there are duplicate entry attempts. The rest are inserted and their status set to new.
As you can see from some of these SQL statements, MariaDB offers you quite a few ways to add and to change data. In addition to these methods, there are also some bulk methods of adding and changing data in a table. You could use the statement and the command-line utility. These methods are covered in another article on .
This page is licensed: CC BY-SA / Gnu FDL
CREATE TABLE t1 ( a INT );
CREATE TABLE t2 ( b INT );
INSERT INTO t1 VALUES (1), (2), (3);
INSERT INTO t2 VALUES (2), (4);
SELECT * FROM t1 INNER JOIN t2 ON t1.a = t2.b;
+------+------+
| a | b |
+------+------+
| 2 | 2 |
+------+------+
1 row in set (0.00 sec)
SELECT * FROM t1 CROSS JOIN t2;
+------+------+
| a | b |
+------+------+
| 1 | 2 |
| 2 | 2 |
| 3 | 2 |
| 1 | 4 |
| 2 | 4 |
| 3 | 4 |
+------+------+
6 rows in set (0.00 sec)
SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.b;
+------+------+
| a | b |
+------+------+
| 1 | NULL |
| 2 | 2 |
| 3 | NULL |
+------+------+
3 rows in set (0.00 sec)
SELECT * FROM t1 RIGHT JOIN t2 ON t1.a = t2.b;
+------+------+
| a | b |
+------+------+
| 2 | 2 |
| NULL | 4 |
+------+------+
2 rows in set (0.00 sec)
SELECT * FROM t2 LEFT JOIN t1 ON t1.a = t2.b;
+------+------+
| b | a |
+------+------+
| 2 | 2 |
| 4 | NULL |
+------+------+
2 rows in set (0.00 sec)
SELECT * FROM t1, t2 WHERE t1.a = t2.b;
SELECT * FROM t1, t2;
INSERT table1
VALUES('text1','text2','text3');
INSERT INTO table1
(col3, col1)
VALUES('text3','text1');
INSERT IGNORE
INTO table2
VALUES('id1','text','text'),
('id2','text','text'),
('id2','text','text');
INSERT LOW_PRIORITY
INTO table1
VALUES('text1','text2','text3');
INSERT DELAYED
INTO table1
VALUES('text1','text2','text3');
INSERT INTO softball_team
(last, first, telephone)
SELECT name_last, name_first, tel_home
FROM company.employees
WHERE softball='Y';
REPLACE LOW_PRIORITY
INTO table2 (id, col1, col2)
VALUES('id1','text','text'),
('id2','text','text'),
('id3','text','text');
UPDATE LOW_PRIORITY table3
SET col1 = 'text-a', col2='text-b'
WHERE id < 100;
UPDATE table5
SET col_date = DATE_ADD(col_date, INTERVAL 1 DAY)
WHERE DAYOFWEEK(col_date) = 1;
UPDATE LOW_PRIORITY table3
SET col1='text-a', col2='text-b'
WHERE id < 100
ORDER BY col3 DESC
LIMIT 10;
UPDATE table3, table4
SET table3.col1 = table4.col1
WHERE table3.id = table4.id;
INSERT IGNORE INTO table1
(id, col1, col2, status)
VALUES('1012','text','text','new'),
('1025,'text','text','new'),
('1030,'text','text','new')
ON DUPLICATE KEY
UPDATE status = 'old';
Adding and Modifying Data Guide
This guide explains how to add new data and modify existing data in MariaDB using INSERT
, REPLACE
, and UPDATE
statements. Learn about various options for handling duplicates, managing statement priorities, inserting data from other tables, and performing conditional updates.
INSERT
The INSERT
statement is used to add new rows to a table.
Basic Syntax:
If providing values for all columns in their defined order:
INSERT table1 VALUES('value1','value2','value3');
The number of values must match the number of columns in table1
.
Specifying Columns:
It's good practice to specify the columns you are inserting data into, which also allows you to insert columns in any order or omit columns that have default values or allow NULL.
INSERT INTO table1 (col3, col1) VALUES('value_for_col3', 'value_for_col1');
The INTO
keyword is optional but commonly used for readability.
If a column is not listed and is an AUTO_INCREMENT
key, its value will be generated. For other omitted columns, their DEFAULT
value will be used, or NULL
if allowed. You can explicitly insert a default value using the DEFAULT
keyword in the VALUES
list for a specific column.
Multiple Row Inserts:
Insert multiple rows in a single statement for efficiency:
INSERT INTO table2 (id_col, data_col1, data_col2) VALUES
('id1', 'text_a', 'text_b'),
('id2', 'text_c', 'text_d'),
('id3', 'text_e', 'text_f');
The VALUES
keyword is used only once, with each row's values enclosed in parentheses and separated by commas.
Handling Duplicates with INSERT IGNORE:
If you attempt to insert a row that would cause a duplicate value in a PRIMARY KEY or UNIQUE index, an error normally occurs, and the row (and potentially subsequent rows in a multi-row insert) might not be inserted.
Using IGNORE tells MariaDB to discard the duplicate row(s) and continue inserting any other valid rows without generating an error.
INSERT IGNORE INTO table2 (unique_id_col, data_col) VALUES
('id1', 'some_data'), -- Will be inserted if new
('id2', 'other_data'), -- Will be inserted if new
('id1', 'duplicate_data'); -- Will be ignored if 'id1' already exists or was just inserted
INSERT
Priority and BehaviorLOW_PRIORITY:
An INSERT statement normally takes priority over SELECT statements, potentially locking the table and making other clients wait. LOW_PRIORITY makes the INSERT wait until no other clients are reading from the table.
INSERT LOW_PRIORITY INTO table1 VALUES('value1','value2','value3');
Once the LOW_PRIORITY
insert begins, it will lock the table as usual. New read requests that arrive while it's waiting will be processed before it.
DELAYED:
(Note: INSERT DELAYED is a feature that was primarily associated with the MyISAM storage engine. It is deprecated in older MariaDB/MySQL versions and removed in modern MariaDB versions (e.g., from MariaDB 10.5). Check your MariaDB version for support and consider alternatives if using a recent version.)
INSERT DELAYED
allowed the server to queue the insert request and return control to the client immediately. Data was written when the table was not in use. Multiple DELAYED
inserts were batched.
-- Syntax for historical reference; may not be supported
INSERT DELAYED INTO table1 VALUES('value1','value2','value3');
Flaws included no confirmation of successful insertion and potential data loss if the server crashed before data was written from memory.
INSERT...SELECT
)You can insert rows into a table based on data retrieved from another table (or tables) using INSERT ... SELECT
.
INSERT INTO softball_team (last_name, first_name, telephone)
SELECT name_last, name_first, tel_home
FROM company_database.employees
WHERE is_on_softball_team = 'Y';
The columns in the INSERT INTO softball_team (...)
list must correspond in number and general data type compatibility to the columns in the SELECT
list.
INSERT...SELECT
statements generally cannot operate on the exact same table as both the target and the source directly without mechanisms like temporary tables or certain subquery structures.
REPLACE
The REPLACE
statement works like INSERT
, but if a new row has the same value as an existing row for a PRIMARY KEY
or a UNIQUE
index, the existing row is deleted before the new row is inserted. If no such conflict exists, it acts like a normal INSERT
.
REPLACE LOW_PRIORITY INTO table2 (id_col, data_col1, data_col2) VALUES
('id1', 'new_text_a', 'new_text_b'), -- If 'id1' exists, old row is deleted, this is inserted
('id2', 'new_text_c', 'new_text_d'), -- If 'id2' doesn't exist, this is inserted
('id3', 'new_text_e', 'new_text_f');
Flags like LOW_PRIORITY
work similarly to INSERT
.
REPLACE
also supports the REPLACE ... SELECT
syntax.
Because REPLACE
performs a delete then an insert, any columns in the table not specified in the REPLACE
statement will receive their default values for the newly inserted row, not values from the old row.
UPDATE
Use the UPDATE
statement to change data in existing rows.
Basic Syntax:
UPDATE table3
SET col1 = 'new_value_a', col2 = 'new_value_b'
WHERE id_column < 100;
The SET
clause specifies which columns to modify and their new values.
The WHERE
clause is crucial; it determines which rows are updated. Without a WHERE
clause, all rows in the table will be updated.
LOW_PRIORITY
and IGNORE
(to ignore errors like unique key violations during update, allowing other valid row updates to proceed) can also be used with UPDATE
.
Using Current Column Values in an Update:
You can use a column's current value in the calculation for its new value.
UPDATE table5
SET event_date = DATE_ADD(event_date, INTERVAL 1 DAY)
WHERE DAYOFWEEK(event_date) = 1; -- Example: Add 1 day if event_date is a Sunday
ORDER BY and LIMIT with UPDATE:
You can control the order in which rows are updated and limit the number of rows affected (for single-table updates).
UPDATE LOW_PRIORITY table3
SET col1 = 'updated_text_a', col2 = 'updated_text_b'
WHERE status_column = 'pending'
ORDER BY creation_date DESC
LIMIT 10;
This updates the 10 most recently created 'pending' rows.
Multi-Table UPDATE:
You can update rows in one table based on values from another table by joining them.
UPDATE products p
JOIN stock_levels s ON p.product_id = s.product_id
SET p.stock_count = s.current_stock
WHERE s.warehouse_id = 'WHA';
Here, products.stock_count
is updated using values from stock_levels
.
ORDER BY
and LIMIT
are generally not allowed with multi-table UPDATE
statements in this direct join form.
INSERT ... ON DUPLICATE KEY UPDATE
)This powerful feature allows you to INSERT
a new row, but if a duplicate key (Primary or Unique) conflict occurs, it performs an UPDATE
on the existing row instead.
INSERT INTO table1 (id, col1, col2, status_column)
VALUES ('1012', 'some_text', 'other_text', 'new')
ON DUPLICATE KEY UPDATE status_column = 'old', col2 = VALUES(col2);
If id
'1012' does not exist, the row is inserted with status_column = 'new'
.
If id
'1012' already exists, the existing row is updated: status_column
is set to 'old', and col2
is updated with the value that would have been inserted for col2
(using VALUES(col2)
).
The IGNORE
keyword can be used with INSERT ... ON DUPLICATE KEY UPDATE
to ignore errors that might occur during the UPDATE
part if the update itself causes a problem (though this is less common). If IGNORE
is used with INSERT
and ON DUPLICATE KEY UPDATE
is also present, IGNORE
only applies to the INSERT
part, not the UPDATE
part.
Beyond these SQL statements, MariaDB offers bulk methods for adding data, such as:
LOAD DATA INFILE
: For importing data from text files.
mariadb-import
utility: A command-line tool that uses LOAD DATA INFILE
. These are covered in "Bulk Data Importing Guide").
CC BY-SA / Gnu FDL
Remote Access Configuration Guide
This guide explains how to configure your MariaDB server to accept connections from remote hosts. Learn to adjust crucial network settings like bind-address
, grant appropriate user privileges for remote connections, and configure essential firewall rules.
Two main configuration directives control MariaDB's network accessibility:
skip-networking
: If this directive is enabled, MariaDB will not listen for TCP/IP connections at all. All interaction must be through local mechanisms like Unix sockets or named pipes.
bind-address
: This directive specifies the IP address the server listens on.
By default, for security, many MariaDB packages bind to 127.0.0.1
(localhost). This means the server will only accept connections originating from the server machine itself via the loopback interface. Remote connections will fail.
If bind-address
is set to 127.0.0.1
, attempting to connect from another host, or even from the same host using a non-loopback IP address, will result in errors like:
ERROR 2002 (HY000): Can't connect to MySQL server on 'myhost' (115)
A telnet myhost 3306
test would likely show "Connection refused."
To allow connections from other hosts, you must either comment out the bind-address
directive (making MariaDB listen on all available network interfaces, i.e., 0.0.0.0
for IPv4), or set it to a specific public IP address of the server.
MariaDB 10.11 and later: bind-address
can accept multiple comma-separated IP addresses, allowing the server to listen on specific interfaces while excluding others.
Connecting via localhost
typically works even if bind-address
is 127.0.0.1
(using the loopback interface):
./client/mariadb --host=localhost --protocol=tcp --port=3306 test
Locating the MariaDB Configuration File
To change these network settings, you need to edit MariaDB's configuration file (often named my.cnf
or my.ini
).
See Configuring MariaDB with my.cnf for comprehensive details.
Common Locations:
/etc/my.cnf
(Unix/Linux/BSD)
/etc/mysql/my.cnf
(Common on Debian/Ubuntu)
$MYSQL_HOME/my.cnf
(Unix/Linux/BSD, where $MYSQL_HOME
is MariaDB's base directory)
SYSCONFDIR/my.cnf
(Compile-time specified system configuration directory)
DATADIR\my.ini
(Windows, in the data directory)
~/.my.cnf
(User-specific configuration file)
Identifying Loaded Files: To see which configuration files your mariadbd
server instance reads and in what order, execute:
Bash
mariadbd --help --verbose
Look for a line similar to: Default options are read from the following files in the given order: /etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf
Open the File: Use a text editor to open the primary configuration file identified (e.g., /etc/mysql/my.cnf
).
Locate [mysqld]
Section: Find the section starting with [mysqld]
.
Adjust Directives:
If skip-networking
is present and enabled (not commented out with #
), comment it out or set it to 0
:Ini, TOML
#skip-networking
orIni, TOML
skip-networking=0
If bind-address = 127.0.0.1
(or another loopback/specific IP that's too restrictive) is present:
To listen on all available IPv4 interfaces: Comment it out entirely (#bind-address = 127.0.0.1
) or set bind-address = 0.0.0.0
.
To listen on a specific public IP address of your server: bind-address = <your_server_public_ip>
.
Alternatively, to effectively disable binding to a specific address and listen on all, you can add skip-bind-address
. Example changes:
[mysqld]
...
#skip-networking
#bind-address = 127.0.0.1
...
Or, to be explicit for listening on all interfaces if bind-address
was previously restrictive:
[mysqld]
bind-address = 0.0.0.0
Save and Restart: Save the configuration file and restart the MariaDB server service.
See Starting and Stopping MariaDB for instructions.
Verify Settings (Optional): You can check the options mariadbd
is effectively using by running:
./sql/mariadbd --print-defaults # Adjust path to mariadbd if necessary
Look for the effective bind-address
value or the absence of skip-networking
. If multiple [mysqld]
sections or skip-bind-address
are used, the last specified prevailing value is typically what counts.
Configuring the server to listen for remote connections is only the first step. You must also grant privileges to user accounts to connect from specific remote hosts. MariaDB user accounts are defined as 'username'@'hostname'
.
Connect to MariaDB:
mariadb -u root -p
View Existing Remote Users (Optional):SQL
SELECT User, Host FROM mysql.user
WHERE Host <> 'localhost' AND Host <> '127.0.0.1' AND Host <> '::1';
Grant Privileges: Use the GRANT
statement to allow a user to connect from a remote host or a range of hosts.
Syntax Elements:
Privileges (e.g., ALL PRIVILEGES
, SELECT, INSERT, UPDATE
)
Database/tables (e.g., database_name.*
for all tables in a database, *.*
for all databases)
Username
Host (IP address, hostname, or subnet with wildcards like %
)
Password (using IDENTIFIED BY 'password'
)
Example: Grant root
-like access from a specific LAN subnet: It's highly discouraged to allow root
access from all hosts ('root'@'%'
) directly to the internet. Instead, restrict it to trusted networks if necessary.
SQL
GRANT ALL PRIVILEGES ON *.* TO 'root'@'192.168.100.%'
IDENTIFIED BY 'my-very-strong-password' WITH GRANT OPTION;
FLUSH PRIVILEGES;
This allows the root
user to connect from any IP address in the 192.168.100.x
subnet. Replace 'my-very-strong-password'
with a strong, unique password.
For creating less privileged users or more granular permissions, see the GRANT documentation.
Even if MariaDB is configured for remote access, a firewall on the server (software or hardware) might block incoming connections on MariaDB's port (default is 3306
).
RHEL/CentOS 7 Example (using firewall-cmd
):
sudo firewall-cmd --add-port=3306/tcp --permanent
sudo firewall-cmd --reload
The first command adds the rule, the second makes it persist after reboots and applies the changes. Consult your OS/firewall documentation for specific commands.
Security: Opening MariaDB to remote connections, especially to the internet, increases security risks. Always use strong passwords, grant minimal necessary privileges, and restrict host access as much as possible. Consider using TLS/SSL for encrypted connections (see Secure Connections Overview).
Reverting: To disable remote access and revert to a more secure local-only setup:
Edit your MariaDB configuration file.
Ensure skip-networking
is not enabled (or is 0
).
Set bind-address = 127.0.0.1
explicitly, or remove any skip-bind-address
directive if you previously added it to listen on all interfaces. The goal is to have bind-address=127.0.0.1
as the effective setting.
Restart the MariaDB server.
Review and revoke any unnecessary remote GRANT
privileges.
The initial version of this article was copied, with permission, from Remote_Clients_Cannot_Connect on 2012-10-30.
This page is licensed: CC BY-SA / Gnu FDL
String Functions Guide
This guide explores a variety of MariaDB's built-in string functions essential for effective data manipulation. Learn how to format text for display, extract specific substrings, replace content, and utilize various expression aids to enhance your string operations in SQL queries.
Several functions are available for formatting text and numbers for display or processing.
Concatenating Strings:
CONCAT(str1, str2, ...)
: Joins two or more strings together.
SQL
SELECT CONCAT(name_first, ' ', name_last) AS Name FROM contacts;
This displays a full name by combining name_first
, a space, and name_last
.
CONCAT_WS(separator, str1, str2, ...)
: Joins strings with a specified separator between each.
SQL
SELECT CONCAT_WS('|', col1, col2, col3) FROM table1;
This creates a pipe-delimited string from col1
, col2
, and col3
.
Formatting Numbers:
FORMAT(number, decimal_places)
: Formats a number with commas every three digits and a specified number of decimal places.SQL
SELECT CONCAT('$', FORMAT(col5, 2)) AS Price FROM table3;
This prepends a dollar sign to a number formatted with commas and two decimal places (e.g., $100,000.00
).
Changing Case:
UCASE(str)
or UPPER(str)
: Converts a string to all upper-case letters.
LCASE(str)
or LOWER(str)
: Converts a string to all lower-case letters.SQL
SELECT UCASE(col1) AS Upper_Col1, LCASE(col2) AS Lower_Col2 FROM table4;
Padding Strings:
LPAD(str, len, padstr)
: Left-pads str
with padstr
until it is len
characters long.
RPAD(str, len, padstr)
: Right-pads str
with padstr
until it is len
characters long.SQL
SELECT RPAD(part_nbr, 8, '.') AS 'Part Nbr.', LPAD(description, 15, '_') AS Description FROM catalog;
Example: RPAD('H200', 8, '.')
might produce H200....
. LPAD('hinge', 15, '_')
might produce __________hinge
.
Trimming Strings:
LTRIM(str)
: Removes leading spaces.
RTRIM(str)
: Removes trailing spaces.
TRIM([{BOTH | LEADING | TRAILING} [remstr] FROM] str)
: Removes leading, trailing, or both occurrences of remstr
(or spaces if remstr
is not given). BOTH
is the default if no specifier is given before remstr
. If only str
is provided, trims leading and trailing spaces.
SELECT
TRIM(LEADING '.' FROM col1) AS Trimmed_Leading_Dots,
TRIM(TRAILING FROM col2) AS Trimmed_Trailing_Spaces, -- Trims spaces
TRIM(BOTH '_' FROM col3) AS Trimmed_Both_Underscores,
TRIM(col4) AS Trimmed_Spaces -- Trims leading and trailing spaces
FROM table5;
These functions help extract specific parts of a string.
LEFT(str, len)
: Returns the leftmost len
characters from str
.
RIGHT(str, len)
: Returns the rightmost len
characters from str
.
SELECT LEFT(telephone, 3) AS area_code, RIGHT(telephone, 7) AS tel_nbr
FROM contacts
ORDER BY area_code;
This extracts the first 3 characters as area_code
and the last 7 as tel_nbr
.
SUBSTRING(str, pos, [len])
or MID(str, pos, [len])
: Returns a substring len
characters long from str
, starting at position pos
. MID()
is a synonym for SUBSTRING()
. If len
is omitted, returns the rest of the string from pos
.
SELECT CONCAT('(', LEFT(telephone, 3), ') ',
SUBSTRING(telephone, 4, 3), '-',
MID(telephone, 7)) AS 'Telephone Number'
FROM contacts
ORDER BY LEFT(telephone, 3);
This formats a 10-digit phone number like (504) 555-1234
.
Functions for changing or generating strings.
REPLACE(str, from_str, to_str)
: Replaces all occurrences of from_str
within str
with to_str
.
SELECT CONCAT(REPLACE(title, 'Mrs.', 'Ms.'), ' ', name_first, ' ', name_last) AS Name
FROM contacts;
This replaces "Mrs." with "Ms." in the title
column.
INSERT(str, pos, len, newstr)
: Replaces the substring in str
starting at pos
and len
characters long with newstr
. If len
is 0, newstr
is inserted at pos
without overwriting.
LOCATE(substr, str, [pos])
: Returns the starting position of the first occurrence of substr
within str
. An optional pos
specifies where to start searching. Returns 0 if substr
is not found.
-- Example: Change 'Mrs.' to 'Ms.' where title is embedded in a 'name' column
SELECT INSERT(name, LOCATE('Mrs.', name), LENGTH('Mrs.'), 'Ms.')
FROM contacts
WHERE name LIKE '%Mrs.%';
This finds 'Mrs.' in the name
string, and replaces it with 'Ms.'. LENGTH('Mrs.')
(which is 4) is used for len
. If LOCATE()
returns 0, INSERT()
with a position of 0 typically returns the original string unchanged.
REVERSE(str)
: Reverses the characters in str
.
SELECT REVERSE('MariaDB'); -- Output: BDeiraM
REPEAT(str, count)
: Repeats str
count
times.
SELECT REPEAT('Ha', 3); -- Output: HaHaHa
Functions that provide information about strings or assist in specific comparisons/conversions.
CHAR_LENGTH(str)
or CHARACTER_LENGTH(str)
: Returns the length of str
in characters.
SELECT COUNT(school_id) AS 'Number of Students'
FROM table8
WHERE CHAR_LENGTH(school_id) = 8;
This counts rows where school_id
has exactly 8 characters.
INET_ATON(ip_address_str)
: Converts an IPv4 address string (e.g., '10.0.1.1') into a numeric representation suitable for numeric sorting.
INET_NTOA(numeric_ip_representation): Converts the numeric representation back to an IPv4 address string.
To correctly sort IP addresses numerically instead of lexically:
SELECT ip_address
FROM computers
WHERE server = 'Y'
ORDER BY INET_ATON(ip_address)
LIMIT 3;
Lexical sort of 10.0.1.1, 10.0.11.1, 10.0.2.1 might be 10.0.1.1, 10.0.11.1, 10.0.2.1.
Numeric sort (using INET_ATON) would correctly be 10.0.1.1, 10.0.2.1, 10.0.11.1.
STRCMP(str1, str2)
: Performs a case-sensitive comparison of str1
and str2
.
Returns 0
if strings are identical.
Returns -1
if str1
is alphabetically before str2
.
Returns 1
if str1
is alphabetically after str2
.
SELECT col1, col2
FROM table6
WHERE STRCMP(col3, 'text') = 0; -- Finds exact case-sensitive match for 'text'
SUBSTRING_INDEX(str, delim, count)
: Returns a substring from str
before or after count
occurrences of the delimiter delim
.
If count
is positive, returns everything to the left of the count
-th delimiter (from the left).
If count
is negative, returns everything to the right of the abs(count)
-th delimiter (from the right).
-- Get the first two elements from a pipe-delimited string
SELECT SUBSTRING_INDEX('elem1|elem2|elem3|elem4', '|', 2); -- Output: elem1|elem2
-- Get the last two elements
SELECT SUBSTRING_INDEX('elem1|elem2|elem3|elem4', '|', -2); -- Output: elem3|elem4
This page is licensed: CC BY-SA / Gnu FDL
mariadb -u root -p -h localhost
mariadb -p
CREATE DATABASE bookstore;
USE bookstore;
CREATE TABLE books (
isbn CHAR(20) PRIMARY KEY,
title VARCHAR(50),
author_id INT,
publisher_id INT,
year_pub CHAR(4),
description TEXT
);
DESCRIBE books;
+--------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+-------+
| isbn | char(20) | NO | PRI | NULL | |
| title | varchar(50) | YES | | NULL | |
| author_id | int(11) | YES | | NULL | |
| publisher_id | int(11) | YES | | NULL | |
| year_pub | char(4) | YES | | NULL | |
| description | text | YES | | NULL | |
+--------------+-------------+------+-----+---------+-------+
CREATE TABLE authors (
author_id INT AUTO_INCREMENT PRIMARY KEY,
name_last VARCHAR(50),
name_first VARCHAR(50),
country VARCHAR(50)
);
INSERT INTO authors (name_last, name_first, country)
VALUES('Kafka', 'Franz', 'Czech Republic');
INSERT INTO books (title, author_id, isbn, year_pub)
VALUES('The Castle', '1', '0805211063', '1998');
INSERT INTO books (title, author_id, isbn, year_pub)
VALUES('The Trial', '1', '0805210407', '1995'),
('The Metamorphosis', '1', '0553213695', '1995'),
('America', '1', '0805210644', '1995');
SELECT title FROM books;
SELECT title FROM books LIMIT 5;
SELECT title, name_last
FROM books
JOIN authors USING (author_id);
SELECT title AS 'Kafka Books'
FROM books
JOIN authors USING (author_id)
WHERE name_last = 'Kafka';
+-------------------+
| Kafka Books |
+-------------------+
| The Castle |
| The Trial |
| The Metamorphosis |
| America |
+-------------------+
UPDATE books
SET title = 'Amerika'
WHERE isbn = '0805210644';
DELETE FROM books
WHERE author_id = '2034'; -- Assuming '2034' is the author_id to be deleted
mariadb
mariadb -h 166.78.144.191 -u username -ppassword database_name
mariadb -u user_name -p -h ip_address db_name
MariaDB [test]>
CREATE DATABASE IF NOT EXISTS test;
USE test;
CREATE TABLE IF NOT EXISTS books (
BookID INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
Title VARCHAR(100) NOT NULL,
SeriesID INT,
AuthorID INT
);
CREATE TABLE IF NOT EXISTS authors (
id INT NOT NULL PRIMARY KEY AUTO_INCREMENT
-- You would typically add more columns like name, etc.
);
CREATE TABLE IF NOT EXISTS series (
id INT NOT NULL PRIMARY KEY AUTO_INCREMENT
-- You would typically add more columns like series_name, etc.
);
INSERT INTO books (Title, SeriesID, AuthorID) VALUES
('The Fellowship of the Ring', 1, 1),
('The Two Towers', 1, 1),
('The Return of the King', 1, 1),
('The Sum of All Men', 2, 2),
('Brotherhood of the Wolf', 2, 2),
('Wizardborn', 2, 2),
('The Hobbbit', 0, 1); -- Note: "Hobbbit" is intentionally misspelled for a later example
SHOW TABLES;
+----------------+
| Tables_in_test |
+----------------+
| authors |
| books |
| series |
+----------------+
3 rows in set (0.00 sec)
DESCRIBE books;
+----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+----------------+
| BookID | int(11) | NO | PRI | NULL | auto_increment |
| Title | varchar(100) | NO | | NULL | |
| SeriesID | int(11) | YES | | NULL | |
| AuthorID | int(11) | YES | | NULL | |
+----------+--------------+------+-----+---------+----------------+
SELECT * FROM books;
+--------+----------------------------+----------+----------+
| BookID | Title | SeriesID | AuthorID |
+--------+----------------------------+----------+----------+
| 1 | The Fellowship of the Ring | 1 | 1 |
| 2 | The Two Towers | 1 | 1 |
| 3 | The Return of the King | 1 | 1 |
| 4 | The Sum of All Men | 2 | 2 |
| 5 | Brotherhood of the Wolf | 2 | 2 |
| 6 | Wizardborn | 2 | 2 |
| 7 | The Hobbbit | 0 | 1 |
+--------+----------------------------+----------+----------+
7 rows in set (0.00 sec)
INSERT INTO books (Title, SeriesID, AuthorID)
VALUES ("Lair of Bones", 2, 2);
Query OK, 1 row affected (0.00 sec)
UPDATE books
SET Title = "The Hobbit"
WHERE BookID = 7;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
This primer is designed to teach you the basics of getting information into and out of an existing MariaDB database using the mariadb command-line client program. It's not a complete reference and will not touch on any advanced topics. It is just a quick jump-start into using MariaDB.
Log into your MariaDB server from the command-line like so:
mariadb -u user_name -p -h ip_address db_name
Replace user_name with your database username. Replace ip_address with the host name or address of your server. If you're accessing MariaDB from the same server you're logged into, then don't include -h
and the ip_address. Replace db_name with the name of the database you want to access (such as test, which sometimes comes already created for testing purposes - note that Windows does not create this database, and some setups may also have removed the test
database by running mariadb-secure-installation, in which case you can leave the db_name out).
When prompted to enter your password, enter it. If your login is successful you should see something that looks similar to this:
MariaDB [test]>
This is where you will enter in all of your SQL statements. More about those later. For now, let's look at the components of the prompt: The "MariaDB" part means you that you are connected to a MariaDB database server. The word between the brackets is the name of your default database, the test database in this example.
To make changes to a database or to retrieve data, you will need to enter an SQL statement. SQL stands for Structured Query Language. An SQL statement that requests data is called a query. Databases store information in tables. They're are similar to spreadsheets, but much more efficient at managing data.
Note that the test database may not contain any data yet. If you want to follow along with the primer, copy and paste the following into the mariadb client. This will create the tables we will use and add some data to them. Don't worry about understanding them yet; we'll get to that later.
CREATE DATABASE IF NOT EXISTS test;
USE test;
CREATE TABLE IF NOT EXISTS books (
BookID INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
Title VARCHAR(100) NOT NULL,
SeriesID INT, AuthorID INT);
CREATE TABLE IF NOT EXISTS authors
(id INT NOT NULL PRIMARY KEY AUTO_INCREMENT);
CREATE TABLE IF NOT EXISTS series
(id INT NOT NULL PRIMARY KEY AUTO_INCREMENT);
INSERT INTO books (Title,SeriesID,AuthorID)
VALUES('The Fellowship of the Ring',1,1),
('The Two Towers',1,1), ('The Return of the King',1,1),
('The Sum of All Men',2,2), ('Brotherhood of the Wolf',2,2),
('Wizardborn',2,2), ('The Hobbbit',0,1);
Notice the semi-colons used above. The mariadb client lets you enter very complex SQL statements over multiple lines. It won't send an SQL statement until you type a semi-colon and hit [Enter].
Let's look at what you've done so far. Enter the following:
SHOW TABLES;
+----------------+
| Tables_in_test |
+----------------+
| authors |
| books |
| series |
+----------------+
3 rows in set (0.00 sec)
Notice that this displays a list of the tables in the database. If you didn't already have tables in your test
database, your results should look the same as above. Let's now enter the following to get information about one of these tables:
DESCRIBE books;
+----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+----------------+
| BookID | int(11) | NO | PRI | NULL | auto_increment |
| Title | varchar(100) | NO | | NULL | |
| SeriesID | int(11) | YES | | NULL | |
| AuthorID | int(11) | YES | | NULL | |
+----------+--------------+------+-----+---------+----------------+
The main bit of information of interest to us is the Field column. The other columns provide useful information about the structure and type of data in the database, but the Field column gives us the names, which is needed to retrieve data from the table.
Let's retrieve data from the books
table. We'll do so by executing a SELECT statement like so:
SELECT * FROM books;
+--------+----------------------------+----------+----------+
| BookID | Title | SeriesID | AuthorID |
+--------+----------------------------+----------+----------+
| 1 | The Fellowship of the Ring | 1 | 1 |
| 2 | The Two Towers | 1 | 1 |
| 3 | The Return of the King | 1 | 1 |
| 4 | The Sum of All Men | 2 | 2 |
| 5 | Brotherhood of the Wolf | 2 | 2 |
| 6 | Wizardborn | 2 | 2 |
| 7 | The Hobbbit | 0 | 1 |
+--------+----------------------------+----------+----------+
7 rows in set (0.00 sec)
This SQL statement or query asks the database to show us all of the data in the books
table. The wildcard ('*
') character indicates to select all columns.
Suppose now that we want to add another book to this table. We'll add the book, Lair of Bones. To insert data into a table, you would use the INSERT statement. To insert information on a book, we would enter something like this:
INSERT INTO books (Title, SeriesID, AuthorID)
VALUES ("Lair of Bones", 2, 2);
Query OK, 1 row affected (0.00 sec)
Notice that we put a list of columns in parentheses after the name of the table, then we enter the keyword VALUES
followed by a list of values in parentheses--in the same order as the columns were listed. We could put the columns in a different order, as long as the values are in the same order as we list the columns. Notice the message that was returned indicates that the execution of the SQL statement went fine and one row was entered.
Execute the following SQL statement again and see what results are returned:
SELECT * FROM books;
You should see the data you just entered on the last row of the results. In looking at the data for the other books, suppose we notice that the title of the seventh book is spelled wrong. It should be spelled The Hobbit, not The Hobbbit. We will need to update the data for that row.
To change data in a table, you will use the UPDATE statement. Let's change the spelling of the book mentioned above. To do this, enter the following:
UPDATE books
SET Title = "The Hobbit"
WHERE BookID = 7;
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
Notice the syntax of this SQL statement. The SET
clause is where you list the columns and the values to set them. The WHERE
clause says that you want to update only rows in which the BookID
column has a value of 7
, of which there are only one. You can see from the message it returned that one row matched the WHERE
clause and one row was changed. There are no warnings because everything went fine. Execute the SELECT from earlier to see that the data changed.
As you can see, using MariaDB isn't very difficult. You just have to understand the syntax of SQL since it doesn't allow for typing mistakes or things in the wrong order or other deviations.
This article covers connecting to MariaDB and the basic connection parameters. If you are completely new to MariaDB, take a look at A MariaDB Primer first.
In order to connect to the MariaDB server, the client software must provide the correct connection parameters. The client software will most often be the mariadb client, used for entering statements from the command line, but the same concepts apply to any client, such as a graphical client, a client to run backups such as mariadb-dump, etc. The rest of this article assumes that the mariadb command line client is used.
If a connection parameter is not provided, it will revert to a default value.
For example, to connect to MariaDB using only default values with the mariadb client, enter the following from the command line:
mariadb
In this case, the following defaults apply:
The host name is localhost
.
The user name is either your Unix login name, or ODBC
on Windows.
No password is sent.
The client will connect to the server with the default socket, but not any particular database on the server.
These defaults can be overridden by specifying a particular parameter to use. For example:
mariadb -h 166.78.144.191 -u username -ppassword database_name
In this case:
-h
specifies a host. Instead of using localhost
, the IP 166.78.144.191
is used.
-u
specifies a user name, in this case username
-p
specifies a password, password
. Note that for passwords, unlike the other parameters, there cannot be a space between the option (-p
) and the value (password
). It is also not secure to use a password in this way, as other users on the system can see it as part of the command that has been run. If you include the -p
option, but leave out the password, you are prompted for it, which is more secure.
The database name is provided as the first argument after all the options, in this case database_name
.
It will connect with the default tcp_ip port, 3306
--host=name
-h name
Connect to the MariaDB server on the given host. The default host is localhost
. By default, MariaDB does not permit remote logins - see Configuring MariaDB for Remote Client Access.
--password[=passwd]
-p[passwd]
The password of the MariaDB account. It is generally not secure to enter the password on the command line, as other users on the system can see it as part of the command that has been run. If you include the -p
or --password
option, but leave out the password, you are prompted for it, which is more secure.
--pipe
-W
On Windows systems that have been started with the --enable-named-pipe
option, use this option to connect to the server using a named pipe.
--port=num
-P num
The TCP/IP port number to use for the connection. The default is 3306
.
--protocol=name
Specifies the protocol to be used for the connection for the connection. It can be one of TCP
, SOCKET
, PIPE
or MEMORY
(case-insensitive). Usually you would not want to change this from the default. For example on Unix, a Unix socket file (SOCKET
) is the default protocol, and usually results in the quickest connection.
TCP
: A TCP/IP connection to a server (either local or remote). Available on all operating systems.
SOCKET
: A Unix socket file connection, available to the local server on Unix systems only. If socket is not specified with --socket, in a config file or with the environment variable MYSQL_UNIX_PORT
then the default /tmp/mysql.sock
are used.
PIPE
. A named-pipe connection (either local or remote). Available on Windows only.
MEMORY
. Shared-memory connection to the local server on Windows systems only.
--shared-memory-base-name=name
Only available on Windows systems in which the server has been started with the --shared-memory
option, this specifies the shared-memory name to use for connecting to a local server. The value is case-sensitive, and defaults to MARIADB
.
--socket=name
-S name
For connections to localhost, this specifies either the Unix socket file to use (default /tmp/mysql.sock
), or, on Windows where the server has been started with the --enable-named-pipe
option, the name (case-insensitive) of the named pipe to use (default MARIADB
).
A brief listing is provided below. See Secure Connections Overview and TLS System Variables for more detail.
--ssl
Enable TLS for connection (automatically enabled with other TLS flags). Disable with '--skip-ssl
'
--ssl-ca=name
CA file in PEM format (check OpenSSL docs, implies --ssl
).
--ssl-capath=name
CA directory (check OpenSSL docs, implies --ssl).
--ssl-cert=name
X509 cert in PEM format (implies --ssl
).
--ssl-cipher=name
TLS cipher to use (implies --ssl
).
--ssl-key=name
X509 key in PEM format (implies --ssl
).
--ssl-crl=name
Certificate revocation list (implies --ssl
).
--ssl-crlpath=name
Certificate revocation list path (implies --ssl
).
--ssl-verify-server-cert
Verify server's "Common Name" in its cert against hostname used when connecting. This option is disabled by default.
--user=name
-u name
The MariaDB user name to use when connecting to the server. The default is either your Unix login name, or ODBC
on Windows. See the GRANT command for details on creating MariaDB user accounts.
It's also possible to use option files (or configuration files) to set these options. Most clients read option files. Usually, starting a client with the --help
option will display which files it looks for as well as which option groups it recognizes.
--skip-grant-tables allows you to start MariaDB without GRANT
. This is useful if you lost your root password.
This page is licensed: CC BY-SA / Gnu FDL
mariadb-backup
creates the following files:
backup-my.cnf
During the backup, any server options relevant to mariadb-backup
are written to the backup-my.cnf
option file, so that they can be re-read later during the --prepare
stage.
ib_logfile0
In and later, mariadb-backup creates an empty InnoDB redo log file called ib_logfile0
as part of the --prepare stage. This file has 3 roles:
In the source server, ib_logfile0
is the first (and possibly the only) InnoDB redo log file.
In the non-prepared backup, ib_logfile0
contains all of the InnoDB redo log copied during the backup. Previous versions of mariadb-backup would use a file called xtrabackup_logfile for this.
During the --prepare stage, ib_logfile0
would previously be deleted. Now during the --prepare
stage, ib_logfile0
is initialized as an empty InnoDB redo log file. That way, if the backup is manually restored, any pre-existing InnoDB redo log files would get overwritten by the empty one. This helps to prevent certain kinds of known issues. For example, see mariadb-backup Overview: Manual Restore with Pre-existing InnoDB Redo Log files.
xtrabackup_logfile
In and before, mariadb-backup creates xtrabackup_logfile
to store the InnoDB redo log, In later versions, ib_logfile0 is created instead.
xtrabackup_binlog_info
This file stores the binary log file name and position that corresponds to the backup.
This file also stores the value of the gtid_current_pos system variable that correspond to the backup.
For example:
mariadb-bin.000096 568 0-1-2
The values in this file are only guaranteed to be consistent with the backup if the --no-lock option was not provided when the backup was taken.
xtrabackup_binlog_pos_innodb
This file is created by mariadb-backup to provide the binary log file name and position when the --no-lock option is used. It can be used instead of the file "xtrabackup_binlog_info" to obtain transactionally consistent binlog coordinates from the backup of a master server with the --no-lock option to minimize the impact on a running server.
Whenever a transaction is committed inside InnoDB when the binary log is enabled, the corresponding binlog coordinates are written to the InnoDB redo log along with the transaction commit. This allows one to restore the binlog coordinates corresponding to the last commit done by InnoDB along with a backup.
The limitation of using "xtrabackup_binlog_pos_innodb" with the "--no-lock" option is that no DDL or modification of non-transactional tables should be done during the backup. If the last event in the binlog is a DDL/non-transactional update, the coordinates in the file "xtrabackup_binlog_pos_innodb" are too old. But as long as only InnoDB updates are done during the backup, the coordinates are correct.
xtrabackup_checkpoints
The xtrabackup_checkpoints
file contains metadata about the backup.
For example:
backup_type = full-backuped
from_lsn = 0
to_lsn = 1635102
last_lsn = 1635102
recover_binlog_info = 0
See below for a description of the fields.
If the --extra-lsndir
option is provided, then an extra copy of this file are saved in that directory.
backup_type
If the backup is a non-prepared full backup or a non-prepared partial backup, then backup_type
is set to full-backuped
.
If the backup is a non-prepared incremental backup, then backup_type
is set to incremental
.
If the backup has already been prepared, then backup_type
is set to log-applied
.
from_lsn
If backup_type
is full-backuped
, then from_lsn
has the value of 0
.
If backup_type
is incremental
, then from_lsn
has the value of the log sequence number (LSN) at which the backup started reading from the InnoDB redo log. This is internally used by mariadb-backup when preparing incremental backups.
This value can be manually set during an incremental backup with the --incremental-lsn option. However, it is generally better to let mariadb-backup figure out the from_lsn
automatically by specifying a parent backup with the --incremental-basedir option.
to_lsn
to_lsn
has the value of the log sequence number (LSN) of the last checkpoint in the InnoDB redo log. This is internally used by mariadb-backup when preparing incremental backups.
last_lsn
last_lsn
has the value of the last log sequence number (LSN) read from the InnoDB redo log. This is internally used by mariadb-backup when preparing incremental backups.
xtrabackup_info
The xtrabackup_info
file contains information about the backup. The fields in this file are listed below.
If the --extra-lsndir option is provided, then an extra copy of this file are saved in that directory.
uuid
If a UUID was provided by the --incremental-history-uuid option, then it are saved here. Otherwise, this is the empty string.
name
If a name was provided by the --history or the ---incremental-history-name options, then it are saved here. Otherwise, this is the empty string.
tool_name
The name of the mariadb-backup executable that performed the backup. This is generally mariadb-backup
.
tool_command
The arguments that were provided to mariadb-backup when it performed the backup.
tool_version
The version of mariadb-backup that performed the backup.
ibbackup_version
The version of mariadb-backup that performed the backup.
server_version
The version of MariaDB Server that was backed up.
start_time
The time that the backup started.
end_time
The time that the backup ended.
lock_time
The amount of time that mariadb-backup held its locks.
binlog_pos
This field stores the binary log file name and position that corresponds to the backup.
This field also stores the value of the gtid_current_pos system variable that correspond to the backup.
The values in this field are only guaranteed to be consistent with the backup if the --no-lock option was not provided when the backup was taken.
innodb_from_lsn
This is identical to from_lsn
in xtrabackup_checkpoints.
If the backup is a full backup, then innodb_from_lsn
has the value of 0
.
If the backup is an incremental backup, then innodb_from_lsn
has the value of the log sequence number (LSN) at which the backup started reading from the InnoDB redo log.
innodb_to_lsn
This is identical to to_lsn
in xtrabackup_checkpoints.
innodb_to_lsn
has the value of the log sequence number (LSN) of the last checkpoint in the InnoDB redo log.
partial
If the backup is a partial backup, then this value are Y
.
Otherwise, this value are N
.
incremental
If the backup is an incremental backup, then this value are Y
.
Otherwise, this value are N
.
format
This field's value is the format of the backup.
If the --stream option was set to xbstream
, then this value are xbstream
.
If the --stream option was not provided, then this value are file
.
compressed
If the --compress option was provided, then this value are compressed
.
Otherwise, this value are N
.
xtrabackup_slave_info
If the --slave-info option is provided, then this file contains the CHANGE MASTER command that can be used to set up a new server as a slave of the original server's master after the backup has been restored.
mariadb-backup does not check if GTIDs are being used in replication. It takes a shortcut and assumes that if the gtid_slave_pos system variable is non-empty, then it writes the CHANGE MASTER command with the MASTER_USE_GTID option set to slave_pos
. Otherwise, it writes the CHANGE MASTER command with the MASTER_LOG_FILE and MASTER_LOG_POS options using the master's binary log file and position. See MDEV-19264 for more information.
xtrabackup_galera_info
If the --galera-info option is provided, then this file contains information about a Galera Cluster node's state.
The file contains the values of the and status variables.
The values are written in the following format:
wsrep_local_state_uuid:wsrep_last_committed
For example:
d38587ce-246c-11e5-bcce-6bbd0831cc0f:1352215
<table>.delta
If the backup is an incremental backup, then this file contains changed pages for the table.
<table>.delta.meta
If the backup is an incremental backup, then this file contains metadata about <table>.delta
files. The fields in this file are listed below.
page_size
This field contains either the value of innodb_page_size or the value of the KEY_BLOCK_SIZE table option for the table if the ROW_FORMAT table option for the table is set to COMPRESSED.
zip_size
If the ROW_FORMAT table option for this table is set to COMPRESSED, then this field contains the value of the compressed page size.
space_id
This field contains the value of the table's space_id
.
This page is licensed: CC BY-SA / Gnu FDL
A high-level overview of the main reasons for choosing a particular storage engine:
InnoDB is a good general transaction storage engine, and the best choice in most cases. It is the default storage engine.
MyISAM has a small footprint and allows for easy copying between systems. MyISAM is MySQL's oldest storage engine. There is usually little reason to use it except for legacy purposes. Aria is MariaDB's more modern improvement.
XtraDB is no longer available. It was a performance-enhanced fork of InnoDB and was MariaDB's default engine until .
When you want to split your database load on several servers or optimize for scaling. We also suggest looking at , a synchronous multi-master cluster.
Spider uses partitioning to provide data sharding through multiple servers.
utilizes a massively parallel distributed data architecture and is designed for big data scaling to process petabytes of data.
MyRocks enables greater compression than InnoDB, as well as less write amplification giving better endurance of flash storage and improving overall throughput.
The Archive storage engine is, unsurprisingly, best used for archiving.
When you want to use data not stored in a MariaDB database.
The CSV storage engine can read and append to files stored in CSV (comma-separated-values) format. However, since MariaDB 10.0, CONNECT is a better choice and is more flexibly able to read and write such files.
Search engines optimized for search.
SphinxSE is used as a proxy to run statements on a remote Sphinx database server (mainly useful for advanced fulltext searches).
Mroonga provides fast CJK-ready full text searching using column store.
S3 Storage Engine is a read-only storage engine that stores its data in Amazon S3.
Sequence allows the creation of ascending or descending sequences of numbers (positive integers) with a given starting value, ending value and increment, creating virtual, ephemeral tables automatically when you need them.
The BLACKHOLE storage engine accepts data but does not store it and always returns an empty result. This can be useful in replication environments, for example, if you want to run complex filtering rules on a slave without incurring any overhead on a master.
OQGRAPH allows you to handle hierarchies (tree structures) and complex graphs (nodes having many connections in several directions).
The Archive storage engine is, unsurprisingly, best used for archiving.
Aria, MariaDB's more modern improvement on MyISAM, has a small footprint and allows for easy copy between systems.
The BLACKHOLE storage engine accepts data but does not store it and always returns an empty result. This can be useful in replication environments, for example, if you want to run complex filtering rules on a slave without incurring any overhead on a master.
utilizes a massively parallel distributed data architecture and is designed for big data scaling to process petabytes of data.
CONNECT allows access to different kinds of text files and remote resources as if they were regular MariaDB tables.
The CSV storage engine can read and append to files stored in CSV (comma-separated-values) format. However, since MariaDB 10.0, CONNECT is a better choice and is more flexibly able to read and write such files.
InnoDB is a good general transaction storage engine, and the best choice in most cases. It is the default storage engine.
The MERGE storage engine is a collection of identical MyISAM tables that can be used as one. "Identical" means that all tables have identical column and index information.
Mroonga provides fast CJK-ready full text searching using column store.
MyISAM has a small footprint and allows for easy copying between systems. MyISAM is MySQL's oldest storage engine. There is usually little reason to use it except for legacy purposes. Aria is MariaDB's more modern improvement.
MyRocks enables greater compression than InnoDB, as well as less write amplification giving better endurance of flash storage and improving overall throughput.
OQGRAPH allows you to handle hierarchies (tree structures) and complex graphs (nodes having many connections in several directions).
S3 Storage Engine is a read-only storage engine that stores its data in Amazon S3.
Sequence allows the creation of ascending or descending sequences of numbers (positive integers) with a given starting value, ending value and increment, creating virtual, ephemeral tables automatically when you need them.
SphinxSE is used as a proxy to run statements on a remote Sphinx database server (mainly useful for advanced fulltext searches).
Spider uses partitioning to provide data sharding through multiple servers.
This page is licensed: CC BY-SA / Gnu FDL
This guide provides essential instructions for modifying existing table structures. Learn how to add, drop, and change columns, manage indexes and default values, and rename tables, along with key precautions for these operations when working with your database.
Before making any structural changes to a table, especially if it contains data, always create a backup. The mariadb-dump
utility is a common and effective tool for this.
Example: Backing up a single table
Suppose you have a database db1 and a table clients. Its initial structure is:
To back up the clients
table from the command-line:
Replace 'your_username'
and 'your_password'
with your actual MariaDB credentials.
--add-locks
: Locks the table during the backup and unlocks it afterward.
db1 clients
: Specifies the database and then the table.
> clients.sql
: Redirects the output to a file named clients.sql
.
Restoring from a backup
If you need to restore the table:
This command uses the mariadb
client to execute the SQL in clients.sql
, which will typically drop the existing table (if it exists) and recreate it from the backup. Ensure no critical data has been added to the live table since the backup if you intend to overwrite it.
For the examples that follow, we'll assume structural changes are being made, sometimes on an empty table for simplicity, but the backup step is always recommended for tables with data.
Use the ALTER TABLE
statement with the ADD COLUMN
clause.
Add a column to the end of the table:
To add a status column with a fixed width of two characters:
Add a column after a specific existing column:
To add address2 (varchar 25) after the address column:
Add a column to the beginning of the table:
(Assuming new_first_column
is the one to be added at the beginning).
After additions, the table structure might look like (excluding new_first_column
for consistency with original example flow):
Use ALTER TABLE
with CHANGE
or MODIFY
.
Change column type (e.g., to ENUM):
The status column name is specified twice even if not changing the name itself when using CHANGE.
Change column name and keep type:
To change status to active while keeping the ENUM definition:
When using CHANGE
, the current column name is followed by the new column name and the complete type definition.
Modify column type or attributes without renaming:
Use MODIFY if you are only changing the data type or attributes, not the name.
Complex Changes (e.g., ENUM migration with data):
Changing ENUM values in a table with existing data requires careful steps to prevent data loss. This typically involves:
Temporarily modifying the ENUM to include both old and new values.
Updating existing rows to use the new values.
Modifying the ENUM again to remove the old values.
Example of changing address
to address1
(40 chars) and preparing active
ENUM for new values 'yes','no' from 'AC','IA':
Then, update the data:
Finally, restrict the ENUM to new values:
To remove a column and its data (this action is permanent and irreversible without a backup):
Set a default value for a column:
If most clients are in 'LA', set it as the default for the state column:
Remove a default value from a column:
This reverts the default to its standard (e.g., NULL if nullable, or determined by data type).
This DROP DEFAULT
does not delete existing data in the column.
Indexes are separate objects from columns. Modifying an indexed column often requires managing its index.
View existing indexes on a table:
The \G displays results in a vertical format, which can be easier to read for wide output.
Example output:
Changing an indexed column (e.g., Primary Key):
Attempting to CHANGE a column that is part of a PRIMARY KEY without addressing the key might result in an error like "Multiple primary key defined". The index must be dropped first, then the column changed, and the key re-added.
The order is important: DROP PRIMARY KEY
first.
Changing a column with another index type (e.g., UNIQUE):
If cust_id had a UNIQUE index named cust_id_unique_idx (Key_name from SHOW INDEX):
If the Key_name
is the same as the Column_name
(e.g. for a single column UNIQUE
key defined on cust_id
where cust_id
is also its Key_name
):
Changing index type and handling duplicates (e.g., INDEX to UNIQUE):
If changing from an index type that allows duplicates (like a plain INDEX) to one that doesn't (UNIQUE), and duplicate data exists, the operation will fail. To force the change and remove duplicates (use with extreme caution):
The IGNORE
keyword causes rows with duplicate key values (for the new UNIQUE
key) to be deleted. Only the first encountered row is kept.
Rename a table:
To change the name of clients to client_addresses:
Move a table to another database (can be combined with renaming):
To move client_addresses to a database named db2:
Re-sort data within a table (MyRocks/Aria, not typically InnoDB):
For some storage engines (excluding InnoDB where tables are ordered by the primary key), you can physically reorder rows. This does not usually apply to InnoDB unless the ORDER BY columns form the primary key.
After this, SELECT * FROM client_addresses
(without an ORDER BY
clause) might return rows in this new physical order, until further data modifications occur.
Backup First: Always back up tables before making structural alterations, especially on production systems.
Data Integrity: Be mindful of how changes (e.g., type changes, ENUM modifications, dropping columns) can affect existing data. Test changes in a development environment.
Irreversible Actions: Operations like DROP COLUMN
or DROP TABLE
are generally irreversible without restoring from a backup. There's typically no confirmation prompt.
Indexes: Understand that indexes are distinct from columns. Modifying indexed columns often requires separate steps to manage the associated indexes.
Performance: ALTER TABLE
operations on large tables can be time-consuming and resource-intensive, potentially locking the table and impacting application performance. Plan these operations during maintenance windows if possible.
CC BY-SA / Gnu FDL
The guide helps diagnose and resolve common issues encountered when connecting to a MariaDB server. Identify causes for errors like 'Can't connect to local server' or access denied messages, and learn steps to effectively troubleshoot these connection problems.
If you are completely new to MariaDB and relational databases, you may want to start with . Also, ensure you understand the connection parameters discussed in the .
Symptoms:
You receive errors similar to:
or
Causes & Solutions:
Server Not Running: The MariaDB server process may not be running.
Incorrect Parameters: The server is running, but not on the specified host, port, socket, pipe, or protocol. Verify your connection parameters.
Socket File Mismatch (Unix): The socket file path might be non-standard or inconsistent between server and client configurations.
Check your my.cnf
(or my.ini
) configuration file. Ensure the socket
option has the identical value in both the [mysqld]
(server) section and the [client]
(or [mysql]
) section.
To find the running Unix socket file, you can try commands like:
Example output:
See also: .
Symptoms:
You can connect locally, but not from a remote machine, possibly seeing errors like:
You can use telnet
(if available) to test basic network connectivity to the port:
A "Connection refused" message from telnet indicates a network or firewall issue, or that MariaDB is not listening for TCP/IP connections or on that specific interface/port.
The perror utility can interpret OS error codes:
Example output:
Causes & Solutions:
By default, MariaDB often does not accept remote TCP/IP connections or is bound only to localhost
(127.0.0.1
).
Solution: See for detailed instructions on how to enable remote connections by adjusting the bind-address
server variable and ensuring user accounts are configured correctly for remote hosts.
Symptoms:
Connection is established, but authentication fails (e.g., "Access denied for user...").
Causes & Solutions:
Unix Socket Authentication (MariaDB 10.4.3+): On Unix-like systems, the unix_socket
authentication plugin is enabled by default for local connections via the Unix socket file. This plugin uses operating system user credentials.
See the for connection instructions and how to switch to password-based authentication if needed.
For an overview of authentication changes in MariaDB 10.4, see .
Incorrect Username/Host Combination: Authentication is specific to a username@host
combination. For example, 'user1'@'localhost'
is distinct from 'user1'@'166.78.144.191'
. Ensure the user account exists for the host from which you are connecting.
See the article for details on granting permissions.
Password Hashing: When setting or changing passwords using SET PASSWORD
, ensure the PASSWORD()
function is used if the server expects hashed passwords.
Example: SET PASSWORD FOR 'bob'@'%.loc.gov' = PASSWORD('newpass');
Rather than: SET PASSWORD FOR 'bob'@'%.loc.gov' = 'newpass';
(which might store the password as plain text, potentially leading to issues depending on the authentication plugin).
Symptoms:
You can run regular queries but get authentication or permission errors when using SELECT ... INTO OUTFILE, SELECT ... INTO DUMPFILE, or LOAD DATA INFILE.
Causes & Solutions:
These operations require the FILE
privilege on the server.
Solution: Grant the necessary FILE
privilege to the user. See the article.
Symptoms:
You can connect to the MariaDB server, but attempting to USE or query a specific database results in an error:
Or, connecting with mariadb -u user -p db1
works, but mariadb -u user -p db2
fails for db2
.
Causes & Solutions:
The user account has not been granted sufficient privileges for that particular database.
Solution: Grant the required privileges (e.g., SELECT
, INSERT
, etc.) on the specific database to the user. See the GRANT article.
Symptoms:
Unexpected connection behavior or parameter usage that you didn't explicitly provide on the command line.
Causes & Solutions:
Option files (e.g., my.cnf
, .my.cnf
) or environment variables (e.g., MYSQL_HOST
) might be supplying incorrect or overriding connection parameters.
Troubleshooting:
Check the values in any option files read by your client. See and the documentation for the specific client you are using (listed under ).
You can often suppress the reading of default option files by using a --no-defaults
option (if supported by the client):Bash
Symptoms:
You cannot connect to a running server, often because the root (or other administrative) password is lost or unknown.
Causes & Solutions:
Solution: You can start the MariaDB server with the --skip-grant-tables
option. This bypasses the privilege system, granting full access. Use this with extreme caution and only temporarily.
Stop the MariaDB server.
Restart the server manually from the command line, adding the --skip-grant-tables
option.
Connect to the server (no password will be required for root@localhost
).
Execute FLUSH PRIVILEGES;
to reload the grant tables (they are now active again).
Change the password for the necessary account, e.g.:
Stop the server and restart it normally (without --skip-grant-tables
).
localhost
vs. %
Wildcard Host IssuesSymptoms:
You've created a user like 'melisa'@'%' but cannot log in as melisa when connecting from localhost.
Example output showing the problem:
Causes & Solutions:
MariaDB's user authentication prioritizes more specific host matches. If an anonymous user (''@'localhost'
) exists, it can take precedence over 'melisa'@'%'
when connecting from localhost
.
Solutions:
Create a specific user for localhost:SQL
Remove the anonymous user for localhost (use with caution):SQL
Ensure this doesn't break other intended anonymous access if any.
This page is licensed: CC BY-SA / Gnu FDL
Date and Time Handling Guide
This guide covers effective ways to work with date and time information in MariaDB. Learn about temporal data types, essential functions for recording current date/time, extracting specific parts, and formatting your date/time values for display or analysis.
While dates and times can be stored as character strings, using specific temporal data types allows you to leverage MariaDB's built-in functions for manipulation and formatting.
DATE
: For dates only. Format: YYYY-MM-DD
.
TIME
: For time only. Format: HHH:MM:SS
(hours can range beyond 24).
DATETIME
: For combined date and time. Format: YYYY-MM-DD HH:MM:SS
.
TIMESTAMP
: Similar to DATETIME
, but with a more limited range and automatic update capabilities (not covered here). Range typically from 1970-01-01 00:00:01
UTC to 2038-01-19 03:14:07
UTC. From MariaDB 11.5 (64-bit), this range extends to 2106-02-07
.
YEAR
: For years only. Format: YY
or YYYY
.
MariaDB provides several functions to get the current date and time.
Current Date:
Use CURRENT_DATE
(no parentheses) or CURDATE()
(with parentheses).
To see the ID of the last inserted row (if the primary key is AUTO_INCREMENT
):
Current Time:
Use CURRENT_TIME
or CURTIME()
.
Current Date and Time (Timestamp):
Use CURRENT_TIMESTAMP
, NOW()
, or SYSDATE()
. These functions return the current date and time in YYYY-MM-DD HH:MM:SS
format, suitable for DATETIME
or TIMESTAMP
columns.
Extracting from DATE
types:
YEAR(date_column)
: Extracts the year.
MONTH(date_column)
: Extracts the month number (1-12).
DAYOFMONTH(date_column)
: Extracts the day of the month (1-31). Also DAY()
.
(The AS
keyword is used to provide an alias for the output column name.)
Day of the Week:
DAYOFWEEK(date_column)
: Returns the weekday index (1=Sunday, 2=Monday, ..., 7=Saturday).
WEEKDAY(date_column)
: Returns the weekday index (0=Monday, 1=Tuesday, ..., 6=Sunday).
Example using IF()
to determine a billing rate based on the day of the week (Saturday = day 7 for DAYOFWEEK
):
The IF(condition, value_if_true, value_if_false)
function allows conditional logic.
Other Date Part Functions:
DAYOFYEAR(date_column)
: Returns the day of the year (1-366).
QUARTER(date_column)
: Returns the quarter of the year (1-4).
Example: Selecting sessions in a specific quarter (e.g., Q2):
User variables can be used for dynamic queries:
Extracting from TIME
types:
HOUR(time_column)
: Extracts the hour.
MINUTE(time_column)
: Extracts the minute.
SECOND(time_column)
: Extracts the second.
Using EXTRACT()
for DATETIME
or TIMESTAMP
types:
The EXTRACT(unit FROM datetime_column)
function extracts a specified unit
from a date/time value.
Common units: YEAR
, MONTH
, DAY
, HOUR
, MINUTE
, SECOND
.
Combined units: YEAR_MONTH
, DAY_HOUR
, HOUR_MINUTE
, etc.
(For details on joining tables, refer to relevant SQL documentation or a guide like "Essential Queries Guide".)
Using a combined unit:
Output for HOUR_MINUTE
might be like 1303
(for 13:03).
Wordier Date Formats:
MONTHNAME(date_column)
: Returns the full name of the month (e.g., 'May').
DAYNAME(date_column)
: Returns the full name of the day (e.g., 'Wednesday').
Example using CONCAT()
to combine parts:
Using DATE_FORMAT(datetime_column, format_string)
:
This function provides extensive formatting options.
Syntax: DATE_FORMAT(date_value, 'format_options_and_literals')
.
Common format specifiers:
%W
: Full weekday name
%M
: Full month name
%e
: Day of the month, numeric (1-31)
%d
: Day of the month, 2 digits (01-31)
%Y
: Year, 4 digits
%y
: Year, 2 digits
%c
: Month, numeric (1-12)
%r
: Time in 12-hour format (hh:mm:ss AM/PM)
%T
: Time in 24-hour format (hh:mm:ss)
%H
: Hour (00-23)
%h
or %I
: Hour (01-12)
%i
: Minutes (00-59)
%s
or %S
: Seconds (00-59)
%p
: AM or PM
Example with time:
For a complete list of options, see the official .
Using TIME_FORMAT(time_column, format_string)
:
Similar to DATE_FORMAT()
, but uses only time-related format options.
Here, %l
is hour (1-12) and %p
adds AM/PM.
Use Appropriate Data Types: Choose temporal data types (DATE
, TIME
, DATETIME
, TIMESTAMP
, YEAR
) over string types for date/time data to leverage built-in functions and ensure data integrity.
Leverage Built-in Functions: MariaDB offers a rich set of functions for date/time manipulation. Use them within your SQL queries to avoid complex logic in your application code.
Test Queries: When dealing with complex date/time logic or formatting, test your SQL statements directly in a MariaDB client (like the mariadb
command-line tool) to verify results before embedding them in applications.
Be Aware of Time Zones: TIMESTAMP
values are stored in UTC and converted to/from the session's time zone, while DATETIME
values are stored "as is" without time zone conversion. Understand how your server and session time zones are configured if working with data across different regions. (Time zone handling is a more advanced topic not fully covered here).
This page is licensed: CC BY-SA / Gnu FDL
Bulk Data Importing Guide
This guide introduces methods and tools for efficiently importing bulk data into MariaDB. Learn to prepare your data, use LOAD DATA INFILE
and the mariadb-import
utility, handle common data import challenges, and manage potential constraints.
The most common approach for bulk importing is to use a delimited text file.
Export Source Data: Load your data in its original software (e.g., MS Excel, MS Access) and export it as a delimited text file.
Delimiter: Use a character not commonly found in your data to separate fields. The pipe symbol (|
) is often a good choice. Tab () is also common.
Record Separator: Use line feeds () to separate records.
Align Columns (Recommended for Simplicity): Ideally, the order and number of columns in your text file should match the target MariaDB table.
If the table has extra columns not in your file, they will be filled with their default values (or NULL
).
If your file has extra columns not in the table, you'll need to specify which file columns to load (see "Mapping File Columns to Table Columns" below) or remove them from the text file.
Clean Data: Remove any header rows or footer information from the text file unless you plan to skip them during import (see IGNORE N LINES
below).
Upload File: Transfer the text file to a location accessible by the MariaDB server.
Use ASCII mode for FTP transfers to ensure correct line endings.
For security, upload data files to non-public directories on the server.
LOAD DATA INFILE
The LOAD DATA INFILE
statement is a powerful SQL command for importing data from text files. Ensure the MariaDB user has the FILE
privilege.
Basic Syntax:
First, connect to MariaDB using the mariadb client and select your target database:
Then, load the data:
Replace /tmp/prospects.txt
with the actual path to your data file on the server. On Windows, paths use forward slashes (e.g., 'C:/tmp/prospects.txt'
).
prospect_contact
is the target table. You can also specify database_name.table_name
.
FIELDS TERMINATED BY '|'
specifies the field delimiter. For tab-delimited, use '\t'
.
The default record delimiter is the line feed ().
Specifying Line Terminators and Enclosing Characters:
If your file has custom line endings or fields enclosed by characters (e.g., quotes):
ENCLOSED BY '"'
: Specifies that fields are enclosed in double quotes.
LINES STARTING BY '"'
: Indicates each line starts with a double quote.
TERMINATED BY '"\r\n'
: Indicates each line ends with a double quote followed by a Windows-style carriage return and line feed.
To specify a single quote as an enclosing character, you can escape it or put it within double quotes: ENCLOSED BY '\''
or ENCLOSED BY "'"
.
When importing data, you might encounter records with primary key values that already exist in the target table.
Default Behavior: MariaDB attempts to import all rows. If duplicates are found and the table has a primary or unique key that would be violated, an error occurs, and subsequent rows may not be imported.
REPLACE
: If you want new data from the file to overwrite existing rows with the same primary key:SQL
IGNORE
: If you want to keep existing rows and skip importing duplicate records from the file:SQL
If the target table is actively being used, importing data can lock it, preventing access.
LOW_PRIORITY
: To allow other users to read from the table while the load operation is pending, use LOW_PRIORITY
. The load will wait until no other clients are reading the table.SQL
Without LOW_PRIORITY
or CONCURRENT
, the table is typically locked for the duration of the import.
LOAD DATA INFILE
OptionsBinary Line Endings:
If your file has Windows CRLF line endings and was uploaded in binary mode, you can specify the hexadecimal value:
Note: No quotes around the hexadecimal value.
Skipping Header Lines:
To ignore a certain number of lines at the beginning of the file (e.g., a header row):
SQL
Handling Escaped Characters:
If fields are enclosed by quotes and contain embedded quotes that are escaped by a special character (e.g., # instead of the default backslash \):
Mapping File Columns to Table Columns:
If the order or number of columns in your text file differs from the target table, you can specify the column mapping at the end of the LOAD DATA INFILE statement.
Assume prospect_contact table has: (row_id INT AUTO_INCREMENT, name_first VARCHAR, name_last VARCHAR, telephone VARCHAR).
And prospects.txt has columns in order: Last Name, First Name, Telephone.
MariaDB will map data from the file's first column to name_last
, second to name_first
, and third to telephone
.
The row_id
column in the table, not being specified in the list, will be filled by its default mechanism (e.g., AUTO_INCREMENT
or DEFAULT
value, or NULL
).
mariadb-import
UtilityThe mariadb-import
utility (known as mysqlimport
before MariaDB 10.5) is a command-line program that acts as a wrapper for LOAD DATA INFILE
. It's useful for scripting imports.
Syntax:
This command is run from the system shell, not within the mariadb
client.
Lines are continued with \
for readability here; it can be a single line.
--password
: If the password value is omitted, you'll be prompted.
The database name (sales_dept
) is specified before the file path.
File Naming: mariadb-import
expects the text file's name (without extension) to match the target table name (e.g., prospect_contact.txt
for table prospect_contact
). If your file is prospects.txt
and table is prospect_contact
, you might need to rename the file or import into a temporary table named prospects
first.
--verbose
: Shows progress information.
You can list multiple text files to import into correspondingly named tables.
Some web hosts disable LOAD DATA INFILE
or mariadb-import
for security reasons. A workaround involves using mariadb-dump
:
Prepare Data Locally: Prepare your delimited text file (e.g., prospects.txt
).
Local Import: If you have a local MariaDB server, import the text file into a local table (e.g., local_db.prospect_contact
) using LOAD DATA INFILE
as described above.
Local Export with mariadb-dump
: Export the data from your local table into an SQL file containing INSERT
statements.
--no-create-info
(or -t
): Prevents the CREATE TABLE
statement from being included, outputting only INSERT
statements. This is useful if the table already exists on the remote server.
Upload SQL File: Upload the generated .sql
file (e.g., prospects.sql
) to your web server (in ASCII mode).
Remote Import of SQL File: Log into your remote server's shell and import the SQL file using the mariadb
client:
Handling Duplicates with mariadb-dump Output:
mariadb-dump does not have a REPLACE flag like LOAD DATA INFILE. If the target table might contain duplicates:
Open the .sql
file generated by mariadb-dump
in a text editor.
Perform a search and replace operation to change all occurrences of INSERT INTO
to REPLACE INTO
. The syntax for INSERT
and REPLACE
(for the data values part) is similar enough that this often works. Test thoroughly.
Flexibility: MariaDB provides powerful and flexible options for data importing. Understanding the details of LOAD DATA INFILE
and mariadb-import
can save significant effort.
Data Validation: While these tools are efficient for bulk loading, they may not perform extensive data validation beyond basic type compatibility. Cleanse and validate your data as much as possible before importing.
Character Sets: Ensure your data file's character set is compatible with the target table's character set to avoid data corruption. You can specify character sets in LOAD DATA INFILE
.
Other Tools/Methods: For very complex transformations or ETL (Extract, Transform, Load) processes, dedicated ETL tools or scripting languages (e.g., Python, Perl with database modules) might be more suitable, though they are beyond the scope of this guide.
This page is licensed: CC BY-SA / Gnu FDL
Introduction to Views Guide
This guide introduces SQL Views in MariaDB, virtual tables based on the result-set of a stored query. Learn how views simplify complex queries, enhance data security by restricting access, and provide an abstraction layer over your database tables through practical examples.
A basic understanding of SQL, particularly JOIN
operations. (You may want to refer to guides like "Basic Joins Guide" or "More Advanced Joins" if available.)
Access to a MariaDB database.
Privileges to CREATE TABLE
and CREATE VIEW
.
First, we'll create and populate two tables, Employees
and Hours
, to use in our examples. If you have already completed a tutorial using this database structure (e.g., from a "More Advanced Joins" guide), you might be able to skip this setup.
Employees Table:
Hours Table:
Let's say Human Resources needs a report on employees who are late (clock in after 7:00:59 AM) and do not make up the time at the end of their shift (work less than 10 hours and 1 minute).
Initial Query (Helmholtz's Lateness):
This query finds instances where Helmholtz was late within a specific week:
Output:
Refined Query (Policy Violators):
This query identifies all employees who were late and whose shift duration was less than 10 hours and 1 minute (601 minutes).
Output of Refined Query (example structure):
The refined query is becoming complex. Storing this query logic in application code makes it harder to manage and means changes to table structures require application code changes. Views can simplify this.
A view is a virtual table based on the result-set of a stored query.
Creating the Employee_Tardiness View:
We use the refined query to create a view. SQL SECURITY INVOKER means the view runs with the permissions of the user querying it.
Querying the View:
Now, retrieving the tardiness data is much simpler:
This will produce the same results as the complex "Refined Query" above.
You can also apply further conditions when querying the view:
Output (example structure, showing those at least 5 minutes short):
Simplifying Complex Queries: As demonstrated, views hide complex joins and calculations.
Restricting Data Access (Column-Level Security): Views can expose only a subset of columns from underlying tables, preventing users or applications from seeing sensitive information (e.g., Home_Address
, Home_Phone
were not included in our Employee_Tardiness
view).
Implementing Row-Level Security: A view can include a WHERE
clause that filters rows based on the user querying it or other criteria, effectively providing row-level access control. For updatable views, defining them with WITH CHECK OPTION
(or the similar effect of a CASCADE
clause mentioned in original text, usually WITH CASCADED CHECK OPTION
) can ensure that INSERT
s or UPDATE
s through the view adhere to the view's WHERE
clause conditions.
Pre-emptive Optimization: Complex, frequently used queries can be defined as views with optimal join strategies and indexing considerations. Other users or applications query the already optimized view, reducing the risk of running inefficient ad-hoc queries.
Abstracting Table Structures: Views provide a consistent interface to applications even if the underlying table structures change (e.g., tables are normalized, split, or merged). The view definition can be updated to map to the new structure, while applications continue to query the unchanged view.
Views offer a powerful way to:
Simplify data access: Make complex queries easier to write and understand.
Abstract database logic: Separate application code from the complexities of the database schema.
Enhance security: Control access to specific rows and columns.
Improve maintainability: Changes to underlying tables can often be managed by updating the view definition without altering application queries.
This page is licensed: CC BY-SA / Gnu FDL
Creates a . By default, a routine is associated with the default database. To associate the routine explicitly with a given database, specify the name as db_name.sp_name when you create it.
When the routine is invoked, an implicit USE db_name is performed (and undone when the routine terminates). The causes the routine to have the given default database while it executes. USE statements within stored routines are disallowed.
When a stored procedure has been created, you invoke it by
using the CALL
statement (see ).
To execute the CREATE PROCEDURE
statement, it is
necessary to have the CREATE ROUTINE
privilege. By default, MariaDB
automatically grants the ALTER ROUTINE
and EXECUTE
privileges to the
routine creator. See also .
The DEFINER
and SQL SECURITY clauses specify the security context to
be used when checking access privileges at routine execution time, as
described . Requires the privilege, or, from , the privilege.
If the routine name is the same as the name of a built-in SQL function, you must use a space between the name and the following parenthesis when defining the routine, or a syntax error occurs. This is also true when you invoke the routine later. For this reason, we suggest that it is better to avoid re-using the names of existing SQL functions for your own stored routines.
The IGNORE_SPACE SQL mode applies to built-in functions, not to stored routines. It is always allowable to have spaces after a routine name, regardless of whether IGNORE_SPACE is enabled.
The parameter list enclosed within parentheses must always be present. If there are no parameters, an empty parameter list of () should be used. Parameter names are not case sensitive.
Each parameter can be declared to use any valid data type, except that the COLLATE attribute cannot be used.
For valid identifiers to use as procedure names, see .
One can't use OR REPLACE
together with IF EXISTS
.
If the IF NOT EXISTS
clause is used, then the procedure will only be created if a procedure with the same name does not already exist. If the procedure already exists, then a warning are triggered by default.
Each parameter is an IN
parameter by default. To specify otherwise for
a parameter, use the keyword OUT
or INOUT
before the parameter name.
An IN
parameter passes a value into a procedure. The procedure might
modify the value, but the modification is not visible to the caller
when the procedure returns. An OUT
parameter passes a value from the
procedure back to the caller. Its initial value is NULL within the
procedure, and its value is visible to the caller when the procedure
returns. An INOUT
parameter is initialized by the caller, can be
modified by the procedure, and any change made by the procedure is
visible to the caller when the procedure returns.
For each OUT
or INOUT
parameter, pass a user-defined variable in theCALL
statement that invokes the procedure so that you can obtain its
value when the procedure returns. If you are calling the procedure
from within another stored procedure or function, you can also pass a
routine parameter or local routine variable as an IN
or INOUT
parameter.
DETERMINISTIC
and NOT DETERMINISTIC
apply only to . Specifying DETERMINISTC
or NON-DETERMINISTIC
in procedures has no effect. The default value is NOT DETERMINISTIC
. Functions are DETERMINISTIC
when they always return the same value for the same input. For example, a truncate or substring function. Any function involving data, therefore, is always NOT DETERMINISTIC
.
CONTAINS SQL
, NO SQL
, READS SQL DATA
, and MODIFIES SQL DATA
are informative clauses that tell the server what the function does. MariaDB does not check in any way whether the specified clause is correct. If none of these clauses are specified, CONTAINS SQL
is used by default.
MODIFIES SQL DATA
means that the function contains statements that may modify data stored in databases. This happens if the function contains statements like , , , or DDL.
READS SQL DATA
means that the function reads data stored in databases, but does not modify any data. This happens if statements are used, but there no write operations are executed.
CONTAINS SQL
means that the function contains at least one SQL statement, but it does not read or write any data stored in a database. Examples include or .
NO SQL
means nothing, because MariaDB does not currently support any language other than SQL.
The routine_body consists of a valid SQL procedure statement. This can be a simple statement such as or , or it can be a compound statement written using . Compound statements can contain declarations, loops, and other control structure statements. See for syntax details.
MariaDB allows routines to contain DDL statements, such as CREATE
and
DROP. MariaDB also allows (but not )
to contain SQL transaction statements such as COMMIT
.
For additional information about statements that are not allowed in stored routines, see .
For information about invoking from within programs written in a language that has a MariaDB/MySQL interface, see .
If the optional OR REPLACE
clause is used, it acts as a shortcut for:
with the exception that any existing for the procedure are not dropped.
MariaDB stores the system variable setting that is in effect at the time a routine is created, and always executes the routine with this setting in force, regardless of the server in effect when the routine is invoked.
Procedure parameters can be declared with any character set/collation. If the character set and collation are not specifically set, the database defaults at the time of creation are used. If the database defaults change at a later stage, the stored procedure character set/collation will not be changed at the same time; the stored procedure needs to be dropped and recreated to ensure the same character set/collation as the database is used.
A subset of Oracle's PL/SQL language is supported in addition to the traditional SQL/PSM-based MariaDB syntax. See Oracle mode for details on changes when running Oracle mode.
The following example shows a simple stored procedure that uses an OUT
parameter. It uses the DELIMITER command to set a new delimiter for the duration of the process — see .
Character set and collation:
CREATE OR REPLACE:
This page is licensed: CC BY-SA / Gnu FDL
MariaDB features pluggable storage engines to allow per-table workload optimization.
A storage engine is a type of for MariaDB:
Different storage engines may be optimized for different workloads, such as transactional workloads, analytical workloads, or high throughput workloads.
Different storage engines may be designed for different use cases, such as federated table access, table sharding, and table archiving in the cloud.
Different tables on the same server may use different storage engines.
Identify the server's global default storage engine by using to query the system variable:
Identify the session's default storage engine by using :
Global default storage engine:
Session default storage engine supersedes global default during this session:
Storage engine is specified at time of table creation using a ENGINE = parameter.
Standard MariaDB storage engines are used for System Table storage:
Yes, different tables can use different storage engines on the same server.
To create a table with a specific storage engine, specify the ENGINE table option to the statement.
Yes, a single query can reference tables that use multiple storage engines.
In some cases, special configuration may be required. For example, ColumnStore requires cross engine joins to be configured.
is the recommended storage engine for transactional or OLTP workloads.
is the recommended storage engine for analytical or OLAP workloads.
An application that performs both transactional and analytical queries is known as .
HTAP can be implemented with MariaDB by using for transactional queries and for analytical queries.
.
, which shows available storage engines.
, which shows storage engine by table.
This page is: Copyright © 2025 MariaDB. All rights reserved.
DESCRIBE clients;
+-------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+-------------+------+-----+---------+-------+
| cust_id | int(11) | | PRI | 0 | |
| name | varchar(25) | YES | | NULL | |
| address | varchar(25) | YES | | NULL | |
| city | varchar(25) | YES | | NULL | |
| state | char(2) | YES | | NULL | |
| zip | varchar(10) | YES | | NULL | |
| client_type | varchar(4) | YES | | NULL | |
+-------------+-------------+------+-----+---------+-------+
mariadb-dump --user='your_username' --password='your_password' --add-locks db1 clients > clients.sql
mariadb --user='your_username' --password='your_password' db1 < clients.sql
ALTER TABLE clients
ADD COLUMN status CHAR(2);
ALTER TABLE clients
ADD COLUMN address2 VARCHAR(25) AFTER address;
ALTER TABLE clients
ADD COLUMN new_first_column VARCHAR(50) FIRST;
DESCRIBE clients;
+-------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+-------------+------+-----+---------+-------+
| cust_id | int(11) | | PRI | 0 | |
| name | varchar(25) | YES | | NULL | |
| address | varchar(25) | YES | | NULL | |
| address2 | varchar(25) | YES | | NULL | |
| city | varchar(25) | YES | | NULL | |
| state | char(2) | YES | | NULL | |
| zip | varchar(10) | YES | | NULL | |
| client_type | varchar(4) | YES | | NULL | |
| status | char(2) | YES | | NULL | |
+-------------+-------------+------+-----+---------+-------+
ALTER TABLE clients
CHANGE status status ENUM('AC','IA');
ALTER TABLE clients
CHANGE status active ENUM('AC','IA');
ALTER TABLE clients
MODIFY address1 VARCHAR(40); -- Assuming 'address1' is an existing column
ALTER TABLE clients
CHANGE address address1 VARCHAR(40),
MODIFY active ENUM('yes','no','AC','IA'); -- Temporarily include all
UPDATE clients
SET active = 'yes'
WHERE active = 'AC';
UPDATE clients
SET active = 'no'
WHERE active = 'IA';
ALTER TABLE clients
MODIFY active ENUM('yes','no');
ALTER TABLE clients
DROP COLUMN client_type;
ALTER TABLE clients
ALTER state SET DEFAULT 'LA';
ALTER TABLE clients
ALTER state DROP DEFAULT;
SHOW INDEX FROM clients\G
*************************** 1. row ***************************
Table: clients
Non_unique: 0
Key_name: PRIMARY
Seq_in_index: 1
Column_name: cust_id
Collation: A
Cardinality: 0
Sub_part: NULL
Packed: NULL
Comment:
ALTER TABLE clients
DROP PRIMARY KEY,
CHANGE cust_id client_id INT PRIMARY KEY;
ALTER TABLE clients
DROP INDEX cust_id_unique_idx, -- Use the actual Key_name
CHANGE cust_id client_id INT UNIQUE;
ALTER TABLE clients
DROP INDEX cust_id, -- If cust_id is the Key_name
CHANGE cust_id client_id INT UNIQUE;
ALTER IGNORE TABLE clients
DROP INDEX cust_id_idx, -- Assuming cust_id_idx is the name of the old INDEX
CHANGE cust_id client_id INT UNIQUE;
RENAME TABLE clients TO client_addresses;
RENAME TABLE client_addresses TO db2.client_addresses;
ALTER TABLE client_addresses
ORDER BY city, name;
ERROR 2002 (HY000): Can't connect to local MySQL server through
socket '/var/run/mysqld/mysqld.sock' (2 "No such file or directory")
mariadb -u someuser -p --port=3307 --protocol=tcp
ERROR 2003 (HY000): Can't connect to MySQL server on 'localhost'
(111 "Connection refused")
netstat -ln | grep mysqld
unix 2 [ ACC ] STREAM LISTENING 33209505 /var/run/mysqld/mysqld.sock
./client/mysql --host=myhost --protocol=tcp --port=3306 test
ERROR 2002 (HY000): Can't connect to MySQL server on 'myhost' (115)
telnet myhost 3306
perror 115
OS error code 115: Operation now in progress
USE test;
ERROR 1044 (42000): Access denied for user 'youruser'@'yourhost' to database 'test'
mariadb-import --no-defaults ...
SET PASSWORD FOR 'root'@'localhost' = PASSWORD('your_new_strong_password');
-- User created with '%' host
CREATE USER 'melisa'@'%' IDENTIFIED BY 'password';
-- Checking users in mysql.user table
SELECT user, host FROM mysql.user WHERE user='melisa' OR user='';
+--------+-----------+
| user | host |
+--------+-----------+
| melisa | % |
| | localhost | -- An anonymous user for localhost
+--------+-----------+
CREATE USER 'melisa'@'localhost' IDENTIFIED BY 'password_for_melisa_localhost';
GRANT ALL PRIVILEGES ON yourdatabase.* TO 'melisa'@'localhost'; -- Grant necessary privileges
FLUSH PRIVILEGES;
DROP USER ''@'localhost';
FLUSH PRIVILEGES;
INSERT INTO billable_work (doctor_id, patient_id, session_date)
VALUES ('1021', '1256', CURRENT_DATE);
SELECT rec_id, doctor_id, patient_id, session_date
FROM billable_work
WHERE rec_id = LAST_INSERT_ID();
+--------+-----------+------------+--------------+
| rec_id | doctor_id | patient_id | session_date |
+--------+-----------+------------+--------------+
| 2462 | 1021 | 1256 | 2025-05-28 | -- Example date
+--------+-----------+------------+--------------+
UPDATE billable_work
SET session_time = CURTIME()
WHERE rec_id = '2462';
SELECT patient_id, session_date, session_time
FROM billable_work
WHERE rec_id = '2462';
+------------+--------------+--------------+
| patient_id | session_date | session_time |
+------------+--------------+--------------+
| 1256 | 2025-05-28 | 13:03:22 | -- Example time
+------------+--------------+--------------+
SELECT
MONTH(session_date) AS Month,
DAYOFMONTH(session_date) AS Day,
YEAR(session_date) AS Year
FROM billable_work
WHERE rec_id = '2462';
+-------+------+------+
| Month | Day | Year |
+-------+------+------+
| 5 | 28 | 2025 | -- Example output
+-------+------+------+
SELECT
patient_id AS 'Patient ID',
session_date AS 'Date of Session',
IF(DAYOFWEEK(session_date) = 7, 1.5, 1.0) AS 'Billing Rate'
FROM billable_work
WHERE rec_id = '2462';
SELECT patient_id, session_date
FROM billable_work
WHERE QUARTER(session_date) = 2;
SET @target_quarter := 2;
SELECT patient_id, COUNT(*) AS num_sessions
FROM billable_work
WHERE QUARTER(session_date) = @target_quarter AND doctor_id = '1021'
GROUP BY patient_id;
SELECT
HOUR(session_time) AS Hour,
MINUTE(session_time) AS Minute,
SECOND(session_time) AS Second
FROM billable_work
WHERE rec_id = '2462';
+------+--------+--------+
| Hour | Minute | Second |
+------+--------+--------+
| 13 | 03 | 22 | -- Example output
+------+--------+--------+
SELECT
patient_name AS Patient,
EXTRACT(HOUR FROM appointment) AS Hour,
EXTRACT(MINUTE FROM appointment) AS Minute
FROM billable_work
JOIN patients ON billable_work.patient_id = patients.patient_id
WHERE doctor_id = '1021'
AND EXTRACT(MONTH FROM appointment) = 5
AND EXTRACT(DAY FROM appointment) = 28;
SELECT
patient_name AS Patient,
EXTRACT(HOUR_MINUTE FROM appointment) AS AppointmentHM
FROM billable_work
JOIN patients ON billable_work.patient_id = patients.patient_id
WHERE doctor_id = '1021';
SELECT
patient_name AS Patient,
CONCAT(
DAYNAME(appointment), ' - ',
MONTHNAME(appointment), ' ',
DAYOFMONTH(appointment), ', ',
YEAR(appointment)
) AS Appointment
FROM billable_work
JOIN patients ON billable_work.patient_id = patients.patient_id
WHERE doctor_id = '1021' AND DATE(appointment) = '2025-05-28'
LIMIT 1;
+-------------------+------------------------------+
| Patient | Appointment |
+-------------------+------------------------------+
| Michael Zabalaoui | Wednesday - May 28, 2025 | -- Example
+-------------------+------------------------------+
SELECT
patient_name AS Patient,
DATE_FORMAT(appointment, '%W - %M %e, %Y') AS Appointment
FROM billable_work
JOIN patients ON billable_work.patient_id = patients.patient_id
WHERE doctor_id = '1021' AND DATE_FORMAT(appointment, '%c') = 5 -- Filter by month 5 (May)
LIMIT 1;
SELECT
DATE_FORMAT(appointment, '%W - %M %e, %Y at %r') AS Appointment
FROM billable_work
LIMIT 1;
+-------------------------------------------------+
| Appointment |
+-------------------------------------------------+
| Wednesday - May 28, 2025 at 01:03:22 PM | -- Example
+-------------------------------------------------+
SELECT
patient_name AS Patient,
TIME_FORMAT(appointment, '%l:%i %p') AS AppointmentTime
FROM billable_work
JOIN patients ON billable_work.patient_id = patients.patient_id
WHERE doctor_id = '1021'
AND DATE(appointment) = CURDATE();
+-------------------+-----------------+
| Patient | AppointmentTime |
+-------------------+-----------------+
| Michael Zabalaoui | 1:03 PM | -- Example
+-------------------+-----------------+
USE sales_dept; -- Or your database name
LOAD DATA INFILE '/tmp/prospects.txt'
INTO TABLE prospect_contact
FIELDS TERMINATED BY '|';
LOAD DATA INFILE '/tmp/prospects.txt'
INTO TABLE prospect_contact
FIELDS TERMINATED BY '|' ENCLOSED BY '"'
LINES STARTING BY '"' TERMINATED BY '"\r\n';
LOAD DATA INFILE '/tmp/prospects.txt'
REPLACE INTO TABLE prospect_contact
FIELDS TERMINATED BY '|';
LOAD DATA INFILE '/tmp/prospects.txt'
IGNORE INTO TABLE prospect_contact
FIELDS TERMINATED BY '|';
LOAD DATA LOW_PRIORITY INFILE '/tmp/prospects.txt'
INTO TABLE prospect_contact
FIELDS TERMINATED BY '|';
LOAD DATA INFILE '/tmp/prospects.txt'
INTO TABLE prospect_contact
FIELDS TERMINATED BY '|'
LINES TERMINATED BY 0x0d0a; -- 0x0d is carriage return, 0x0a is line feed
LOAD DATA INFILE '/tmp/prospects.txt'
INTO TABLE prospect_contact
FIELDS TERMINATED BY '|'
IGNORE 1 LINES; -- Skips the first line
LOAD DATA INFILE '/tmp/prospects.txt'
INTO TABLE prospect_contact
FIELDS TERMINATED BY '|'
ENCLOSED BY '"'
ESCAPED BY '#'
IGNORE 1 LINES;
LOAD DATA INFILE '/tmp/prospects.txt'
INTO TABLE prospect_contact
FIELDS TERMINATED BY '|' -- Or your actual delimiter, e.g., 0x09 for tab
ENCLOSED BY '"'
ESCAPED BY '#'
IGNORE 1 LINES
(name_last, name_first, telephone);
mariadb-import --user='your_username' --password='your_password' \
--fields-terminated-by='|' --lines-terminated-by='\r\n' \
--replace --low-priority --fields-enclosed-by='"' \
--fields-escaped-by='#' --ignore-lines='1' --verbose \
--columns='name_last,name_first,telephone' \
sales_dept '/tmp/prospect_contact.txt'
mariadb-dump --user='local_user' --password='local_pass' --no-create-info local_db prospect_contact > /tmp/prospects.sql
mariadb --user='remote_user' --password='remote_pass' remote_sales_dept < /tmp/prospects.sql
CREATE TABLE `Employees` (
`ID` TINYINT(3) UNSIGNED NOT NULL AUTO_INCREMENT,
`First_Name` VARCHAR(25) NOT NULL,
`Last_Name` VARCHAR(25) NOT NULL,
`Position` VARCHAR(25) NOT NULL,
`Home_Address` VARCHAR(50) NOT NULL,
`Home_Phone` VARCHAR(12) NOT NULL,
PRIMARY KEY (`ID`)
) ENGINE=MyISAM;
INSERT INTO `Employees` (`First_Name`, `Last_Name`, `Position`, `Home_Address`, `Home_Phone`)
VALUES
('Mustapha', 'Mond', 'Chief Executive Officer', '692 Promiscuous Plaza', '326-555-3492'),
('Henry', 'Foster', 'Store Manager', '314 Savage Circle', '326-555-3847'),
('Bernard', 'Marx', 'Cashier', '1240 Ambient Avenue', '326-555-8456'),
('Lenina', 'Crowne', 'Cashier', '281 Bumblepuppy Boulevard', '328-555-2349'),
('Fanny', 'Crowne', 'Restocker', '1023 Bokanovsky Lane', '326-555-6329'),
('Helmholtz', 'Watson', 'Janitor', '944 Soma Court', '329-555-2478');
CREATE TABLE `Hours` (
`ID` TINYINT(3) UNSIGNED NOT NULL,
`Clock_In` DATETIME NOT NULL,
`Clock_Out` DATETIME NOT NULL
) ENGINE=MyISAM;
INSERT INTO `Hours`
VALUES ('1', '2005-08-08 07:00:42', '2005-08-08 17:01:36'),
('1', '2005-08-09 07:01:34', '2005-08-09 17:10:11'),
('1', '2005-08-10 06:59:56', '2005-08-10 17:09:29'),
('1', '2005-08-11 07:00:17', '2005-08-11 17:00:47'),
('1', '2005-08-12 07:02:29', '2005-08-12 16:59:12'),
('2', '2005-08-08 07:00:25', '2005-08-08 17:03:13'),
('2', '2005-08-09 07:00:57', '2005-08-09 17:05:09'),
('2', '2005-08-10 06:58:43', '2005-08-10 16:58:24'),
('2', '2005-08-11 07:01:58', '2005-08-11 17:00:45'),
('2', '2005-08-12 07:02:12', '2005-08-12 16:58:57'),
('3', '2005-08-08 07:00:12', '2005-08-08 17:01:32'),
('3', '2005-08-09 07:01:10', '2005-08-09 17:00:26'),
('3', '2005-08-10 06:59:53', '2005-08-10 17:02:53'),
('3', '2005-08-11 07:01:15', '2005-08-11 17:04:23'),
('3', '2005-08-12 07:00:51', '2005-08-12 16:57:52'),
('4', '2005-08-08 06:54:37', '2005-08-08 17:01:23'),
('4', '2005-08-09 06:58:23', '2005-08-09 17:00:54'),
('4', '2005-08-10 06:59:14', '2005-08-10 17:00:12'),
('4', '2005-08-11 07:00:49', '2005-08-11 17:00:34'),
('4', '2005-08-12 07:01:09', '2005-08-12 16:58:29'),
('5', '2005-08-08 07:00:04', '2005-08-08 17:01:43'),
('5', '2005-08-09 07:02:12', '2005-08-09 17:02:13'),
('5', '2005-08-10 06:59:39', '2005-08-10 17:03:37'),
('5', '2005-08-11 07:01:26', '2005-08-11 17:00:03'),
('5', '2005-08-12 07:02:15', '2005-08-12 16:59:02'),
('6', '2005-08-08 07:00:12', '2005-08-08 17:01:02'),
('6', '2005-08-09 07:03:44', '2005-08-09 17:00:00'),
('6', '2005-08-10 06:54:19', '2005-08-10 17:03:31'),
('6', '2005-08-11 07:00:05', '2005-08-11 17:02:57'),
('6', '2005-08-12 07:02:07', '2005-08-12 16:58:23');
SELECT
`Employees`.`First_Name`,
`Employees`.`Last_Name`,
`Hours`.`Clock_In`,
`Hours`.`Clock_Out`
FROM `Employees`
INNER JOIN `Hours` ON `Employees`.`ID` = `Hours`.`ID`
WHERE `Employees`.`First_Name` = 'Helmholtz'
AND DATE_FORMAT(`Hours`.`Clock_In`, '%Y-%m-%d') >= '2005-08-08'
AND DATE_FORMAT(`Hours`.`Clock_In`, '%Y-%m-%d') <= '2005-08-12'
AND DATE_FORMAT(`Hours`.`Clock_In`, '%H:%i:%S') > '07:00:59';
+------------+-----------+---------------------+---------------------+
| First_Name | Last_Name | Clock_In | Clock_Out |
+------------+-----------+---------------------+---------------------+
| Helmholtz | Watson | 2005-08-09 07:03:44 | 2005-08-09 17:00:00 |
| Helmholtz | Watson | 2005-08-12 07:02:07 | 2005-08-12 16:58:23 |
+------------+-----------+---------------------+---------------------+
SELECT
`Employees`.`First_Name`,
`Employees`.`Last_Name`,
`Hours`.`Clock_In`,
`Hours`.`Clock_Out`,
(601 - TIMESTAMPDIFF(MINUTE, `Hours`.`Clock_In`, `Hours`.`Clock_Out`)) AS Difference -- Corrected Difference Calculation
FROM `Employees`
INNER JOIN `Hours` USING (`ID`) -- Simplified JOIN condition
WHERE DATE_FORMAT(`Hours`.`Clock_In`, '%Y-%m-%d') BETWEEN '2005-08-08' AND '2005-08-12'
AND TIME(`Hours`.`Clock_In`) > '07:00:59'
AND TIMESTAMPDIFF(MINUTE, `Hours`.`Clock_In`, `Hours`.`Clock_Out`) < 601;
+------------+-----------+---------------------+---------------------+------------+
| First_Name | Last_Name | Clock_In | Clock_Out | Difference |
+------------+-----------+---------------------+---------------------+------------+
| Mustapha | Mond | 2005-08-12 07:02:29 | 2005-08-12 16:59:12 | 4 |
... (other rows matching the criteria)
+------------+-----------+---------------------+---------------------+------------+
CREATE SQL SECURITY INVOKER VIEW Employee_Tardiness AS
SELECT
`Employees`.`First_Name`,
`Employees`.`Last_Name`,
`Hours`.`Clock_In`,
`Hours`.`Clock_Out`,
(601 - TIMESTAMPDIFF(MINUTE, `Hours`.`Clock_In`, `Hours`.`Clock_Out`)) AS Difference
FROM `Employees`
INNER JOIN `Hours` USING (`ID`)
WHERE DATE_FORMAT(`Hours`.`Clock_In`, '%Y-%m-%d') BETWEEN '2005-08-08' AND '2005-08-12'
AND TIME(`Hours`.`Clock_In`) > '07:00:59'
AND TIMESTAMPDIFF(MINUTE, `Hours`.`Clock_In`, `Hours`.`Clock_Out`) < 601;
SELECT * FROM Employee_Tardiness;
SELECT * FROM Employee_Tardiness WHERE Difference >= 5;
+------------+-----------+---------------------+---------------------+------------+
| First_Name | Last_Name | Clock_In | Clock_Out | Difference |
+------------+-----------+---------------------+---------------------+------------+
| Mustapha | Mond | 2005-08-12 07:02:29 | 2005-08-12 16:59:12 | 5 |
... (other rows where Difference >= 5)
+------------+-----------+---------------------+---------------------+------------+
The Essential Queries Guide offers a concise collection of commonly-used SQL queries. It's designed to help developers and database administrators quickly find syntax and examples for typical database operations, from table creation and data insertion to effective data retrieval and manipulation.
To create new tables:
CREATE TABLE t1 ( a INT );
CREATE TABLE t2 ( b INT );
CREATE TABLE student_tests (
name CHAR(10), test CHAR(10),
score TINYINT, test_date DATE
);
For more details, see the official CREATE TABLE documentation.
To add data into your tables:
INSERT INTO t1 VALUES (1), (2), (3);
INSERT INTO t2 VALUES (2), (4);
INSERT INTO student_tests
(name, test, score, test_date) VALUES
('Chun', 'SQL', 75, '2012-11-05'),
('Chun', 'Tuning', 73, '2013-06-14'),
('Esben', 'SQL', 43, '2014-02-11'),
('Esben', 'Tuning', 31, '2014-02-09'),
('Kaolin', 'SQL', 56, '2014-01-01'),
('Kaolin', 'Tuning', 88, '2013-12-29'),
('Tatiana', 'SQL', 87, '2012-04-28'),
('Tatiana', 'Tuning', 83, '2013-09-30');
For more information, see the official INSERT documentation.
The AUTO_INCREMENT
attribute automatically generates a unique identity for new rows.
Create a table with an AUTO_INCREMENT
column:
CREATE TABLE student_details (
id INT NOT NULL AUTO_INCREMENT, name CHAR(10),
date_of_birth DATE, PRIMARY KEY (id)
);
When inserting, omit the id
field; it will be automatically generated:
INSERT INTO student_details (name,date_of_birth) VALUES
('Chun', '1993-12-31'),
('Esben','1946-01-01'),
('Kaolin','1996-07-16'),
('Tatiana', '1988-04-13');
Verify the inserted records:
SELECT * FROM student_details;
+----+---------+---------------+
| id | name | date_of_birth |
+----+---------+---------------+
| 1 | Chun | 1993-12-31 |
| 2 | Esben | 1946-01-01 |
| 3 | Kaolin | 1996-07-16 |
| 4 | Tatiana | 1988-04-13 |
+----+---------+---------------+
For more details, see the AUTO_INCREMENT documentation.
To combine rows from two tables based on a related column:
SELECT * FROM t1 INNER JOIN t2 ON t1.a = t2.b;
This type of query is a join. For more details, consult the documentation on JOINS.
To find the maximum value in a column:
SELECT MAX(a) FROM t1;
+--------+
| MAX(a) |
+--------+
| 3 |
+--------+
See the MAX() function documentation. For a grouped example, refer to Finding the Maximum Value and Grouping the Results below.
To find the minimum value in a column:
SELECT MIN(a) FROM t1;
+--------+
| MIN(a) |
+--------+
| 1 |
+--------+
See the MIN() function documentation.
To calculate the average value of a column:
SELECT AVG(a) FROM t1;
+--------+
| AVG(a) |
+--------+
| 2.0000 |
+--------+
See the AVG() function documentation.
To find the maximum value within groups:
SELECT name, MAX(score) FROM student_tests GROUP BY name;
+---------+------------+
| name | MAX(score) |
+---------+------------+
| Chun | 75 |
| Esben | 43 |
| Kaolin | 88 |
| Tatiana | 87 |
+---------+------------+
Further details are available in the MAX() function documentation.
To sort your query results (e.g., in descending order):
SELECT name, test, score FROM student_tests
ORDER BY score DESC; -- Use ASC for ascending order
+---------+--------+-------+
| name | test | score |
+---------+--------+-------+
| Kaolin | Tuning | 88 |
| Tatiana | SQL | 87 |
| Tatiana | Tuning | 83 |
| Chun | SQL | 75 |
| Chun | Tuning | 73 |
| Kaolin | SQL | 56 |
| Esben | SQL | 43 |
| Esben | Tuning | 31 |
+---------+--------+-------+
For more options, see the ORDER BY documentation.
To find the entire row containing the minimum value of a specific column across all records:
SELECT name, test, score FROM student_tests
WHERE score = (SELECT MIN(score) FROM student_tests);
+-------+--------+-------+
| name | test | score |
+-------+--------+-------+
| Esben | Tuning | 31 |
+-------+--------+-------+
To retrieve the full record for the maximum value within each group (e.g., highest score per student):
SELECT name, test, score FROM student_tests st1
WHERE score = (SELECT MAX(st2.score) FROM student_tests st2 WHERE st1.name = st2.name);
+---------+--------+-------+
| name | test | score |
+---------+--------+-------+
| Chun | SQL | 75 |
| Esben | SQL | 43 |
| Kaolin | Tuning | 88 |
| Tatiana | SQL | 87 |
+---------+--------+-------+
Use the TIMESTAMPDIFF
function to calculate age from a birth date.
To see the current date (optional, for reference):
SELECT CURDATE() AS today;
+------------+
| today |
+------------+
| 2014-02-17 | -- Example output; actual date will vary
+------------+
To calculate age as of a specific date (e.g., '2014-08-02'):
SELECT name, date_of_birth, TIMESTAMPDIFF(YEAR, date_of_birth, '2014-08-02') AS age
FROM student_details;
+---------+---------------+------+
| name | date_of_birth | age |
+---------+---------------+------+
| Chun | 1993-12-31 | 20 |
| Esben | 1946-01-01 | 68 |
| Kaolin | 1996-07-16 | 18 |
| Tatiana | 1988-04-13 | 26 |
+---------+---------------+------+
To calculate current age, replace the specific date string (e.g., '2014-08-02') with CURDATE().
See the TIMESTAMPDIFF() documentation for more.
User-defined variables can store values for use in subsequent queries within the same session.
Example: Set a variable for the average score and use it to filter results.
SELECT @avg_score := AVG(score) FROM student_tests;
+-------------------------+
| @avg_score:= AVG(score) |
+-------------------------+
| 67.000000000 |
+-------------------------+
SELECT * FROM student_tests WHERE score > @avg_score;
+---------+--------+-------+------------+
| name | test | score | test_date |
+---------+--------+-------+------------+
| Chun | SQL | 75 | 2012-11-05 |
| Chun | Tuning | 73 | 2013-06-14 |
| Kaolin | Tuning | 88 | 2013-12-29 |
| Tatiana | SQL | 87 | 2012-04-28 |
| Tatiana | Tuning | 83 | 2013-09-30 |
+---------+--------+-------+------------+
Example: Add an incremental counter to a result set.
SET @count = 0;
SELECT @count := @count + 1 AS counter, name, date_of_birth FROM student_details;
+---------+---------+---------------+
| counter | name | date_of_birth |
+---------+---------+---------------+
| 1 | Chun | 1993-12-31 |
| 2 | Esben | 1946-01-01 |
| 3 | Kaolin | 1996-07-16 |
| 4 | Tatiana | 1988-04-13 |
+---------+---------+---------------+
See User-defined Variables for more.
To list all tables in the current database, ordered by their size (data + index) in megabytes:
SELECT table_schema AS `DB`, table_name AS `TABLE`,
ROUND(((data_length + index_length) / 1024 / 1024), 2) `Size (MB)`
FROM information_schema.TABLES
WHERE table_schema = DATABASE() -- This clause restricts results to the current database
ORDER BY (data_length + index_length) DESC;
+--------------------+---------------------------------------+-----------+
| DB | Table | Size (MB) | -- Example Output
+--------------------+---------------------------------------+-----------+
| your_db_name | some_large_table | 7.05 |
| your_db_name | another_table | 6.59 |
...
+--------------------+---------------------------------------+-----------+
To remove duplicate rows based on specific column values, while keeping one instance (e.g., the instance with the highest id
).
This example assumes id
is a unique primary key and duplicates are identified by the values in column f1
. It keeps the row with the maximum id
for each distinct f1
value.
Setup sample table and data:
CREATE TABLE t (id INT, f1 VARCHAR(2));
INSERT INTO t VALUES (1,'a'), (2,'a'), (3,'b'), (4,'a');
To delete duplicate rows, keeping the one with the highest id
for each group of f1
values:
DELETE t_del FROM t AS t_del
INNER JOIN (
SELECT f1, MAX(id) AS max_id
FROM t
GROUP BY f1
HAVING COUNT(*) > 1 -- Identify groups with actual duplicates
) AS t_keep ON t_del.f1 = t_keep.f1 AND t_del.id < t_keep.max_id;
This query targets rows for deletion (t_del
) where their f1
value matches an f1
in a subquery (t_keep
) that has duplicates, and their id
is less than the maximum id
found for that f1
group.
Verify results after deletion:
SELECT * FROM t;
+------+------+
| id | f1 |
+------+------+
| 3 | b |
| 4 | a |
+------+------+
This page is licensed: CC BY-SA / Gnu FDL
SELECT Statement Guide
This guide explains how to retrieve data from MariaDB using the SELECT
statement, progressing from basic syntax to more involved queries. Learn to select specific columns, limit results, filter with WHERE
, sort with ORDER BY
, join tables, and use various helpful options and functions.
To follow the examples, first create and populate the books
and authors
tables:
CREATE OR REPLACE TABLE books (
isbn CHAR(20) PRIMARY KEY,
title VARCHAR(50),
author_id INT,
publisher_id INT,
year_pub CHAR(4),
description TEXT
);
CREATE OR REPLACE TABLE authors (
author_id INT AUTO_INCREMENT PRIMARY KEY,
name_last VARCHAR(50),
name_first VARCHAR(50),
country VARCHAR(50)
);
INSERT INTO authors (name_last, name_first, country) VALUES
('Kafka', 'Franz', 'Czech Republic'),
('Dostoevsky', 'Fyodor', 'Russia');
INSERT INTO books (title, author_id, isbn, year_pub) VALUES
('The Trial', 1, '0805210407', '1995'),
('The Metamorphosis', 1, '0553213695', '1995'),
('America', 2, '0805210644', '1995'), -- Note: Original data had author_id 2 for 'America', Dostoevsky is author_id 2.
('Brothers Karamozov', 2, '0553212168', ''),
('Crime & Punishment', 2, '0679420290', ''),
('Crime & Punishment', 2, '0553211757', ''),
('Idiot', 2, '0192834118', ''),
('Notes from Underground', 2, '067973452X', '');
Selecting All Columns:
Use * to select all columns from a table.
SELECT * FROM books;
Output (example):
+------------+------------------------+-----------+--------------+----------+-------------+
| isbn | title | author_id | publisher_id | year_pub | description |
+------------+------------------------+-----------+--------------+----------+-------------+
| 0192834118 | Idiot | 2 | NULL | | NULL |
| 0553211757 | Crime & Punishment | 2 | NULL | | NULL |
... (other rows)
| 0805210644 | America | 2 | NULL | 1995 | NULL |
+------------+------------------------+-----------+--------------+----------+-------------+
8 rows in set (0.001 sec)
Selecting Specific Columns:
List the column names separated by commas.
SELECT isbn, title, author_id FROM books;
Output (example):
+------------+------------------------+-----------+
| isbn | title | author_id |
+------------+------------------------+-----------+
| 0192834118 | Idiot | 2 |
| 0553211757 | Crime & Punishment | 2 |
... (other rows)
+------------+------------------------+-----------+
8 rows in set (0.001 sec)
Limiting the Number of Rows with LIMIT
:
To get the first N
rows:
SELECT isbn, title, author_id FROM books LIMIT 5;
Output (example):
+------------+--------------------+-----------+
| isbn | title | author_id |
+------------+--------------------+-----------+
| 0192834118 | Idiot | 2 |
| 0553211757 | Crime & Punishment | 2 |
| 0553212168 | Brothers Karamozov | 2 |
| 0553213695 | The Metamorphosis | 1 |
| 0679420290 | Crime & Punishment | 2 |
+------------+--------------------+-----------+
5 rows in set (0.001 sec)
To get N
rows starting from an offset (offset is 0-indexed):SQL
SELECT isbn, title, author_id FROM books LIMIT 5, 10; -- Skip 5 rows, show next 10 (or fewer if less remain)
Output (example, assuming only 3 more rows exist after offset 5):
+------------+------------------------+-----------+
| isbn | title | author_id |
+------------+------------------------+-----------+
| 067973452X | Notes from Underground | 2 |
| 0805210407 | The Trial | 1 |
| 0805210644 | America | 2 |
+------------+------------------------+-----------+
3 rows in set (0.001 sec)
Filtering with WHERE
:
Use the WHERE
clause to specify conditions for row selection.
SELECT isbn, title
FROM books
WHERE author_id = 2
LIMIT 5;
Output (example):
+------------+------------------------+
| isbn | title |
+------------+------------------------+
| 0192834118 | Idiot |
| 0553211757 | Crime & Punishment |
| 0553212168 | Brothers Karamozov |
| 0679420290 | Crime & Punishment |
| 067973452X | Notes from Underground |
+------------+------------------------+
5 rows in set (0.000 sec)
Ordering with ORDER BY
:
Use ORDER BY column_name [ASC|DESC]
to sort the result set.
SELECT isbn, title
FROM books
WHERE author_id = 2
ORDER BY title ASC
LIMIT 5;
Output (example):
+------------+--------------------+
| isbn | title |
+------------+--------------------+
| 0805210644 | America |
| 0553212168 | Brothers Karamozov |
| 0553211757 | Crime & Punishment |
| 0679420290 | Crime & Punishment |
| 0192834118 | Idiot |
+------------+--------------------+
5 rows in set (0.001 sec)
ASC
(ascending) is the default order. DESC
is for descending order.
You can order by multiple columns: ORDER BY col1 ASC, col2 DESC
.
Clause Order: SELECT ... FROM ... WHERE ... ORDER BY ... LIMIT ...
. MariaDB generally processes WHERE
, then ORDER BY
, then LIMIT
.
Joining Tables:
Use JOIN to combine rows from two or more tables based on a related column.
SELECT isbn, title, CONCAT(name_first, ' ', name_last) AS author
FROM books
JOIN authors USING (author_id) -- Assumes 'author_id' column exists in both tables
WHERE name_last = 'Dostoevsky'
ORDER BY title ASC
LIMIT 5;
Output (example):
+------------+--------------------+-------------------+
| isbn | title | author |
+------------+--------------------+-------------------+
| 0805210644 | America | Fyodor Dostoevsky |
| 0553212168 | Brothers Karamozov | Fyodor Dostoevsky |
| 0553211757 | Crime & Punishment | Fyodor Dostoevsky |
| 0679420290 | Crime & Punishment | Fyodor Dostoevsky |
| 0192834118 | Idiot | Fyodor Dostoevsky |
+------------+--------------------+-------------------+
5 rows in set (0.00 sec)
Alternative JOIN
syntax: ... JOIN authors ON books.author_id = authors.author_id ...
. For more on joins, see the JOIN Syntax documentation or a "Basic Joins Guide".
CONCAT(str1, str2, ...)
: Concatenates strings.
AS alias_name
: Assigns an alias to an output column.
Pattern Matching with LIKE:
Use LIKE in the WHERE clause for pattern matching. % is a wildcard for zero or more characters.
SELECT isbn, title, CONCAT(name_first, ' ', name_last) AS author
FROM books
JOIN authors USING (author_id)
WHERE name_last LIKE 'Dostoevsk%'
ORDER BY title ASC
LIMIT 5;
Output (example, same as above if only Dostoevsky matches):
+------------+--------------------+-------------------+
| isbn | title | author |
+------------+--------------------+-------------------+
| 0805210644 | America | Fyodor Dostoevsky |
| 0553212168 | Brothers Karamozov | Fyodor Dostoevsky |
| 0553211757 | Crime & Punishment | Fyodor Dostoevsky |
| 0679420290 | Crime & Punishment | Fyodor Dostoevsky |
| 0192834118 | Idiot | Fyodor Dostoevsky |
+------------+--------------------+-------------------+
5 rows in set (0.001 sec)
Place these modifiers immediately after the SELECT
keyword.
ALL
vs DISTINCT
:
ALL
(default): Returns all rows that meet the criteria.
DISTINCT
: Returns only unique rows for the selected columns. If multiple identical rows are found for the specified columns, only the first one is displayed.
SELECT DISTINCT title
FROM books
JOIN authors USING (author_id)
WHERE name_last = 'Dostoevsky'
ORDER BY title;
Output (example, showing one "Crime & Punishment"):
+------------------------+
| title |
+------------------------+
| America |
| Brothers Karamozov |
| Crime & Punishment |
| Idiot |
| Notes from Underground |
+------------------------+
HIGH_PRIORITY
:
Gives the SELECT
statement higher priority over concurrent data modification statements (use with caution as it can impact write performance).
SQL
SELECT DISTINCT HIGH_PRIORITY title
FROM books
JOIN authors USING (author_id)
WHERE name_last = 'Dostoevsky'
ORDER BY title;
SQL_CALC_FOUND_ROWS
and FOUND_ROWS():
To find out how many rows a query would have returned without a LIMIT
clause, use SQL_CALC_FOUND_ROWS
in your SELECT
statement, and then execute SELECT
FOUND_ROWS()
; immediately after.
SELECT SQL_CALC_FOUND_ROWS isbn, title
FROM books
JOIN authors USING (author_id)
WHERE name_last = 'Dostoevsky'
ORDER BY title -- Order before limit to ensure consistent FOUND_ROWS() for a given query logic
LIMIT 5;
Output (example for the first query):
+------------+------------------------+
| isbn | title |
+------------+------------------------+
| 0805210644 | America |
| 0553212168 | Brothers Karamozov |
| 0553211757 | Crime & Punishment |
| 0679420290 | Crime & Punishment |
| 0192834118 | Idiot |
+------------+------------------------+
5 rows in set (0.001 sec)
Then, to get the total count:
SELECT FOUND_ROWS();
Output (example, if 6 Dostoevsky books in total):
+--------------+
| FOUND_ROWS() |
+--------------+
| 6 |
+--------------+
1 row in set (0.000 sec)
The value from FOUND_ROWS()
is temporary and specific to the current session.
This page is licensed: CC BY-SA / Gnu FDL
Some MariaDB packages bind MariaDB to 127.0.0.1
(the loopback IP address) by default as a security measure using the bind-address configuration directive. Old MySQL packages sometimes disabled TCP/IP networking altogether using the skip-networking directive. Before going in to how to configure these, let's explain what each of them actually does:
skip-networking is fairly simple. It just tells MariaDB to run without any of the TCP/IP networking options.
bind-address requires a little bit of background information. A given server usually has at least two networking interfaces (although this is not required) and can easily have more. The two most common are a Loopback network device and a physical Network Interface Card (NIC) which allows
you to communicate with the network. MariaDB is bound to the loopback interface by default because it makes it impossible to connect to the TCP port on the server from a remote host (the bind-address must refer to a local IP address, or you will receive a fatal error and MariaDB will not start). This of course is not desirable if you want to use the TCP port from a remote host, so you must remove this bind-address directive or replace it either 0.0.0.0
to listen on all interfaces, or the address of a specific public interface.
Multiple comma-separated addresses can be given to bind_address
to allow the server to listen on more than one specific interface while not listening on others.
If bind-address is bound to 127.0.0.1 (localhost), one can't connect to the MariaDB server from other hosts or from the same host over TCP/IP on a different interface than the loopback (127.0.0.1). This for example will not work (connecting with a hostname that points to a local IP of the host):
(/my/maria-10.11) ./client/mariadb --host=myhost --protocol=tcp --port=3306 test
ERROR 2002 (HY000): Can't connect to MySQL server on 'myhost' (115)
(/my/maria-10.11) telnet myhost 3306
Trying 192.168.0.11...
telnet: connect to address 192.168.0.11: Connection refused
Using 'localhost' works when binding with bind_address
:
(my/maria-10.11) ./client/mariadb --host=localhost --protocol=tcp --port=3306 test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Welcome to the MariaDB monitor. Commands end with ; or \g.
...
Multiple comma-separated addresses cannot be given to bind_address
. Use a single address.
To enable MariaDB to listen to remote connections, you need to edit your defaults file. See Configuring MariaDB with my.cnf for more detail.
Common locations for defaults files:
* /etc/my.cnf (*nix/BSD)
* $MYSQL_HOME/my.cnf (*nix/BSD) *Most Notably /etc/mysql/my.cnf
* SYSCONFDIR/my.cnf (*nix/BSD)
* DATADIR\my.ini (Windows)
You can see which defaults files are read and in which order by executing:
shell> mariadbd --help --verbose
mariadbd Ver 10.11.5-MariaDB for linux-systemd on x86_64 (MariaDB Server)
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Starts the MariaDB database server.
Usage: ./mariadbd [OPTIONS]
Default options are read from the following files in the given order:
/etc/my.cnf /etc/mysql/my.cnf ~/.my.cnf
The last line shows which defaults files are read.
Once you have located the defaults file, use a text editor to open the file and try to find lines like this under the [mysqld]
section:
[mysqld]
...
skip-networking
...
bind-address = <some ip-address>
...
The lines may not be in this particular order, but the order doesn't matter.
If you are able to locate these lines, make sure they are both commented out (prefaced with hash (#) characters), so that they look like this:
[mysqld]
...
#skip-networking
...
#bind-address = <some ip-address>
...
Again, the order of these lines don't matter.
Alternatively, just add the following lines at the end of your .my.cnf
(notice that the file name starts with a dot) file in your home directory or alternative last in your /etc/my.cnf
file.
[mysqld]
skip-networking=0
skip-bind-address
This works as one can have any number of [mysqld] sections.
Save the file and restart the mariadbd daemon or service (see Starting and Stopping MariaDB).
You can check the options mariadbd is using by executing:
shell> ./sql/mariadbd --print-defaults
./sql/mariadbd would have been started with the following arguments:
--bind-address=127.0.0.1 --innodb_file_per_table=ON --server-id=1 --skip-bind-address ...
It doesn't matter if you have the original --bind-address
left as the later --skip-bind-address
will overwrite it.
Now that your MariaDB server installation is setup to accept connections from remote hosts, we have to add a user that is allowed to connect from something other than 'localhost' (Users in MariaDB are defined as 'user'@'host', so 'chadmaynard'@'localhost
' and 'chadmaynard'@'1.1.1.1
' (or 'chadmaynard'@'server.domain.local') are different users that can have different permissions and/or passwords.
To create a new user:
Log into the mariadb command line client (or your favorite graphical client if you wish):
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 36
Server version: 5.5.28-MariaDB-mariadb1~lucid mariadb.org binary distribution
Copyright (c) 2000, 2012, Oracle, Monty Program Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]>
if you are interested in viewing any existing remote users, issue the following SQL statement on the mysql.user table:
SELECT User, Host FROM mysql.user WHERE Host <> 'localhost';
+--------+-----------+
| User | Host |
+--------+-----------+
| daniel | % |
| root | 127.0.0.1 |
| root | ::1 |
| root | gandalf |
+--------+-----------+
4 rows in set (0.00 sec)
(If you have a fresh install, it is normal for no rows to be returned)
Now you have some decisions to make. At the heart of every grant statement you have these things:
list of allowed privileges
what database/tables these privileges apply to
username
host this user can connect from
and optionally a password
It is common for people to want to create a "root" user that can connect from anywhere, so as an example, we'll do just that, but to improve on it we'll create a root user that can connect from anywhere on my local area network (LAN), which has addresses in the subnet 192.168.100.0/24
. This is an improvement because opening a MariaDB server up to the Internet and granting access to all hosts is bad practice.
GRANT ALL PRIVILEGES ON *.* TO 'root'@'192.168.100.%'
IDENTIFIED BY 'my-new-password' WITH GRANT OPTION;
%
is a wildcard.
For more information about how to use GRANT, please see the GRANT page.
At this point, we have accomplished our goal and we have a user 'root' that can connect from anywhere on the 192.168.100.0/24
LAN.
One more point to consider whether the firewall is configured to allow incoming request from remote clients:
On RHEL and CentOS 7, it may be necessary to configure the firewall to allow TCP access to MariaDB from remote hosts. To do so, execute both of these commands:
firewall-cmd --add-port=3306/tcp
firewall-cmd --permanent --add-port=3306/tcp
If your system is running a software firewall (or behind a hardware firewall or NAT) you must allow connections destined to TCP port that MariaDB runs on (by default and almost always 3306).
To undo this change and not allow remote access anymore, simply remove the skip-bind-address
line or uncomment the bind-address line in your defaults file. The end result should be that you should have in the output from ./sql/mariadbd --print-defaults
the option --bind-address=127.0.0.1
and no --skip-bind-address
.
The initial version of this article was copied, with permission, from Remote_Clients_Cannot_Connect on 2012-10-30.
This page is licensed: CC BY-SA / Gnu FDL
CREATE
[OR REPLACE]
[DEFINER = { user | CURRENT_USER | role | CURRENT_ROLE }]
PROCEDURE [IF NOT EXISTS] sp_name ([proc_parameter[,...]])
[characteristic ...] routine_body
proc_parameter:
[ IN | OUT | INOUT ] param_name type
type:
Any valid MariaDB data type
characteristic:
LANGUAGE SQL
| [NOT] DETERMINISTIC
| { CONTAINS SQL | NO SQL | READS SQL DATA | MODIFIES SQL DATA }
| SQL SECURITY { DEFINER | INVOKER }
| COMMENT 'string'
routine_body:
Valid SQL procedure statement
DROP PROCEDURE IF EXISTS name;
CREATE PROCEDURE name ...;
DELIMITER //
CREATE PROCEDURE simpleproc (OUT param1 INT)
BEGIN
SELECT COUNT(*) INTO param1 FROM t;
END;
//
DELIMITER ;
CALL simpleproc(@a);
SELECT @a;
+------+
| @a |
+------+
| 1 |
+------+
DELIMITER //
CREATE PROCEDURE simpleproc2 (
OUT param1 CHAR(10) CHARACTER SET 'utf8' COLLATE 'utf8_bin'
)
BEGIN
SELECT CONCAT('a'),f1 INTO param1 FROM t;
END;
//
DELIMITER ;
DELIMITER //
CREATE PROCEDURE simpleproc2 (
OUT param1 CHAR(10) CHARACTER SET 'utf8' COLLATE 'utf8_bin'
)
BEGIN
SELECT CONCAT('a'),f1 INTO param1 FROM t;
END;
//
ERROR 1304 (42000): PROCEDURE simpleproc2 already exists
DELIMITER ;
DELIMITER //
CREATE OR REPLACE PROCEDURE simpleproc2 (
OUT param1 CHAR(10) CHARACTER SET 'utf8' COLLATE 'utf8_bin'
)
BEGIN
SELECT CONCAT('a'),f1 INTO param1 FROM t;
END;
//
ERROR 1304 (42000): PROCEDURE simpleproc2 already exists
DELIMITER ;
Query OK, 0 rows affected (0.03 sec)
Read-Heavy
Reads
ES 10.5+
Analytics, HTAP
Big Data, Analytical
ES 10.5+
General Purpose
Mixed Read/Write
ES 10.5+
Cache, Temp
Temporary Data
ES 10.5+
Reads
Reads
ES 10.5+
Write-Heavy
I/O Reduction, SSD
ES 10.5+
Cloud
Read-Only
ES 10.5+
Federation
Sharding, Interlink
ES 10.5+
SHOW GLOBAL VARIABLES LIKE 'default_storage_engine';
+------------------------+--------+
| Variable_name | Value |
+------------------------+--------+
| default_storage_engine | InnoDB |
+------------------------+--------+
SHOW SESSION VARIABLES LIKE 'default_storage_engine';
+------------------------+--------+
| Variable_name | Value |
+------------------------+--------+
| default_storage_engine | InnoDB |
+------------------------+--------+
SET GLOBAL default_storage_engine='MyRocks';
SET SESSION default_storage_engine='MyRocks';
[mariadb]
...
default_storage_engine=MyRocks
SHOW ENGINES;
CREATE TABLE accounts.messages (
id INT PRIMARY KEY AUTO_INCREMENT,
sender_id INT,
receiver_id INT,
message TEXT
) ENGINE = MyRocks;
MariaDB is a database system, a database server. To interface with the MariaDB server, you can use a client program, or you can write a program or script with one of the popular programming languages (e.g., PHP) using an API (Application Programming Interface) to interface with the MariaDB server. For the purposes of this article, we will focus on using the default client that comes with MariaDB called mariadb
. With this client, you can either enter queries from the command-line, or you can switch to a terminal, that is to say, monitor mode. To start, we'll use the latter.
From the Linux command-line, you would enter the following to log in as the root user and to enter monitor mode:
mariadb -u root -p -h localhost
The -u
option is for specifying the user name. You would replace root
here if you want to use a different user name. This is the MariaDB user name, not the Linux user name. The password for the MariaDB user root
will probably be different from the Linux user root
. Incidentally, it's not a good security practice to use the root
user unless you have a specific administrative task to perform for which only root
has the needed privileges.
The -p
option above instructs the mariadb
client to prompt you for the password. If the password for the root
user hasn't been set yet, then the password is blank and you would just hit [Enter] when prompted. The -h
option is for specifying the host name or the IP address of the server. This would be necessary if the client is running on a different machine than the server. If you've secure-shelled into the server machine, you probably won't need to use the host option. In fact, if you're logged into Linux as root
, you won't need the user option—the -p
is all you'll need. Once you've entered the line above along with the password when prompted, you are logged into MariaDB through the client. To exit, type quit or exit and press [Enter].
In order to be able to add and to manipulate data, you first have to create a database structure. Creating a database is simple. You would enter something like the following from within the mariadb client:
CREATE DATABASE bookstore;
USE bookstore;
This very minimal, first SQL statement will create a sub-directory called bookstore on the Linux filesystem in the directory which holds your MariaDB data files. It won't create any data, obviously. It'll just set up a place to add tables, which will in turn hold data. The second SQL statement above will set this new database as the default database. It will remain your default until you change it to a different one or until you log out of MariaDB.
The next step is to begin creating tables. This is only a little more complicated. To create a simple table that will hold basic data on books, we could enter something like the following:
CREATE TABLE books (
isbn CHAR(20) PRIMARY KEY,
title VARCHAR(50),
author_id INT,
publisher_id INT,
year_pub CHAR(4),
description TEXT );
This SQL statement creates the table books with six fields, or rather columns. The first column (isbn) is an identification number for each row—this name relates to the unique identifier used in the book publishing business. It has a fixed-width character type of 20 characters. It are the primary key column on which data are indexed. The column data type for the book title is a variable width character column of fifty characters at most. The third and fourth columns are used for identification numbers for the author and the publisher. They are integer data types. The fifth column is used for the publication year of each book. The last column is for entering a description of each book. It's a TEXT data type, which means that it's a variable width column and it can hold up to 65535 bytes of data for each row. There are several other data types that may be used for columns, but this gives you a good sampling.
To see how the table we created looks, enter the following SQL statement:
DESCRIBE books;
+--------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+-------+
| isbn | char(20) | NO | PRI | NULL | |
| title | varchar(50) | YES | | NULL | |
| author_id | int(11) | YES | | NULL | |
| publisher_id | int(11) | YES | | NULL | |
| year_pub | char(4) | YES | | NULL | |
| description | text | YES | | NULL | |
+--------------+-------------+------+-----+---------+-------+
To change the settings of a table, you can use the ALTER TABLE statement. I'll cover that statement in another article. To delete a table completely (including its data), you can use the DROP TABLE statement, followed by the table name. Be careful with this statement since it's not reversible.
The next table we'll create for our examples is the authors table to hold author information. This table will save us from having to enter the author's name and other related data for each book written by each author. It also helps to ensure consistency of data: there's less chance of inadvertent spelling deviations.
CREATE TABLE authors
(author_id INT AUTO_INCREMENT PRIMARY KEY,
name_last VARCHAR(50),
name_first VARCHAR(50),
country VARCHAR(50) );
We'll join this table to the books table as needed. For instance, we would use it when we want a list of books along with their corresponding authors' names. For a real bookstore's database, both of these tables would probably have more columns. There would also be several more tables. For the examples that follow, these two tables as they are are enough.
Before moving on to the next step of adding data to the tables, let me point out a few minor items that I've omitted mentioning. SQL statements end with a semi-colon (or a \G). You can spread an SQL statement over multiple lines. However, it won't be passed to the server by the client until you terminate it with a semi-colon and hit [Enter]. To cancel an SQL statement once you've started typing it, enter \c
and press [Enter].
As a basic convention, reserved words are printed in all capital letters. This isn't necessary, though. MariaDB is case-insensitive with regards to reserved words. Database and table names, however, are case-sensitive on Linux. This is because they reference the related directories and files on the filesystem. Column names aren't case sensitive since they're not affected by the filesystem, per se. As another convention, we use lower-case letters for structural names (e.g., table names). It's a matter of preference for deciding on names.
The primary method for entering data into a table is to use the INSERT statement. As an example, let's enter some information about an author into the authors table. We'll do that like so:
INSERT INTO authors
(name_last, name_first, country)
VALUES('Kafka', 'Franz', 'Czech Republic');
This will add the name and country of the author Franz Kafka to the authors table. We don't need to give a value for the author_id since that column was created with the AUTO_INCREMENT option. MariaDB will automatically assign an identification number. You can manually assign one, especially if you want to start the count at a higher number than 1 (e.g., 1000). Since we are not providing data for all of the columns in the table, we have to list the columns for which we are giving data and in the order that the data is given in the set following the VALUES keyword. This means that we could give the data in a different order.
For an actual database, we would probably enter data for many authors. We'll assume that we've done that and move on to entering data for some books. Below is an entry for one of Kafka's books:
INSERT INTO books
(title, author_id, isbn, year_pub)
VALUES('The Castle', '1', '0805211063', '1998');
This adds a record for Kafka's book, The Castle. Notice that we mixed up the order of the columns, but it still works because both sets agree. We indicate that the author is Kafka by giving a value of 1 for the author_id. This is the value that was assigned by MariaDB when we entered the row for Kafka earlier. Let's enter a few more books for Kafka, but by a different method:
INSERT INTO books
(title, author_id, isbn, year_pub)
VALUES('The Trial', '1', '0805210407', '1995'),
('The Metamorphosis', '1', '0553213695', '1995'),
('America', '1', '0805210644', '1995');
In this example, we've added three books in one statement. This allows us to give the list of column names once. We also give the keyword VALUES
only once, followed by a separate set of values for each book, each contained in parentheses and separated by commas. This cuts down on typing and speeds up the process. Either method is fine and both have their advantages. To be able to continue with our examples, let's assume that data on thousands of books has been entered. With that behind us, let's look at how to retrieve data from tables.
The primary method of retrieving data from tables is to use a SELECT statement. There are many options available with the SELECT statement, but you can start simply. As an example, let's retrieve a list of book titles from the books table:
SELECT title
FROM books;
This will display all of the rows of books in the table. If the table has thousands of rows, MariaDB will display thousands. To limit the number of rows retrieved, we could add a LIMIT clause to the SELECT statement like so:
SELECT title
FROM books
LIMIT 5;
This will limit the number of rows displayed to five. To be able to list the author's name for each book along with the title, you will have to join the books table with the authors table. To do this, we can use the JOIN clause like so:
SELECT title, name_last
FROM books
JOIN authors USING (author_id);
Notice that the primary table from which we're drawing data is given in the FROM
clause. The table to which we're joining is given in the JOIN clause along with the commonly named column (i.e., author_id) that we're using for the join.
To retrieve the titles of only books written by Kafka based on his name (not the author_id), we would use the WHERE
clause with the SELECT statement. This would be entered like the following:
SELECT title AS 'Kafka Books'
FROM books
JOIN authors USING (author_id)
WHERE name_last = 'Kafka';
+-------------------+
| Kafka Books |
+-------------------+
| The Castle |
| The Trial |
| The Metamorphosis |
| America |
+-------------------+
This statement will list the titles of Kafka books stored in the database. Notice that I've added the AS
parameter next to the column name title to change the column heading in the results set to Kafka Books. This is known as an alias. Looking at the results here, we can see that the title for one of Kafka's books is incorrect. His book Amerika is spelled above with a "c" in the table instead of a "k". This leads to the next section on changing data.
In order to change existing data, a common method is to use the UPDATE statement. When changing data, though, we need to be sure that we change the correct rows. In our example, there could be another book with the title America written by a different author. Since the key column isbn has only unique numbers and we know the ISBN number for the book that we want to change, we can use it to specify the row.
UPDATE books
SET title = 'Amerika'
WHERE isbn = '0805210644';
This will change the value of the title column for the row specified. We could change the value of other columns for the same row by giving the column = value for each, separated by commas.
If we want to delete a row of data, we can use the DELETE statement. For instance, suppose that our fictitious bookstore has decided no longer to carry books by John Grisham. By first running a SELECT statement, we determine the identification number for the author to be 2034. Using this author identification number, we could enter the following:
DELETE FROM books
WHERE author_id = '2034';
This statement will delete all rows from the table books for the author_id given. To do a clean job of it, we'll have to do the same for the authors table. We would just replace the table name in the statement above; everything else would be the same.
This is a very basic primer for using MariaDB. Hopefully, it gives you the idea of how to get started with MariaDB. Each of the SQL statements mentioned here have several more options and clauses each. We will cover these statements and others in greater detail in other articles. For now, though, you can learn more about them from experimenting and by further reading of the documentation online documentation.
This page is licensed: CC BY-SA / Gnu FDL
Up-front warning: This is the beginning of a very basic tutorial on views, based on my experimentation with them. This tutorial assumes that you've read the appropriate tutorials up to and including More Advanced Joins (or that you understand the concepts behind them). This page is intended to give you a general idea of how views work and what they do, as well as some examples of when you could use them.
In order to perform the SQL statements in this tutorial, you will need access to a MariaDB database and you will need the CREATE TABLE and CREATE VIEW privileges on this table.
First, we need some data we can perform our optimizations on, so we'll recreate the tables from the More Advanced Joins tutorial, to provide us with a starting point. If you have already completed that tutorial and have this database already, you can skip ahead.
First, we create the table that will hold all of the employees and their contact information:
CREATE TABLE `Employees` (
`ID` TINYINT(3) UNSIGNED NOT NULL AUTO_INCREMENT,
`First_Name` VARCHAR(25) NOT NULL,
`Last_Name` VARCHAR(25) NOT NULL,
`Position` VARCHAR(25) NOT NULL,
`Home_Address` VARCHAR(50) NOT NULL,
`Home_Phone` VARCHAR(12) NOT NULL,
PRIMARY KEY (`ID`)
) ENGINE=MyISAM;
Next, we add a few employees to the table:
INSERT INTO `Employees` (`First_Name`, `Last_Name`, `Position`, `Home_Address`, `Home_Phone`)
VALUES
('Mustapha', 'Mond', 'Chief Executive Officer', '692 Promiscuous Plaza', '326-555-3492'),
('Henry', 'Foster', 'Store Manager', '314 Savage Circle', '326-555-3847'),
('Bernard', 'Marx', 'Cashier', '1240 Ambient Avenue', '326-555-8456'),
('Lenina', 'Crowne', 'Cashier', '281 Bumblepuppy Boulevard', '328-555-2349'),
('Fanny', 'Crowne', 'Restocker', '1023 Bokanovsky Lane', '326-555-6329'),
('Helmholtz', 'Watson', 'Janitor', '944 Soma Court', '329-555-2478');
Now, we create a second table, containing the hours which each employee clocked in and out during the week:
CREATE TABLE `Hours` (
`ID` TINYINT(3) UNSIGNED NOT NULL,
`Clock_In` DATETIME NOT NULL,
`Clock_Out` DATETIME NOT NULL
) ENGINE=MyISAM;
Finally, although it is a lot of information, we add a full week of hours for each of the employees into the second table that we created:
INSERT INTO `Hours`
VALUES ('1', '2005-08-08 07:00:42', '2005-08-08 17:01:36'),
('1', '2005-08-09 07:01:34', '2005-08-09 17:10:11'),
('1', '2005-08-10 06:59:56', '2005-08-10 17:09:29'),
('1', '2005-08-11 07:00:17', '2005-08-11 17:00:47'),
('1', '2005-08-12 07:02:29', '2005-08-12 16:59:12'),
('2', '2005-08-08 07:00:25', '2005-08-08 17:03:13'),
('2', '2005-08-09 07:00:57', '2005-08-09 17:05:09'),
('2', '2005-08-10 06:58:43', '2005-08-10 16:58:24'),
('2', '2005-08-11 07:01:58', '2005-08-11 17:00:45'),
('2', '2005-08-12 07:02:12', '2005-08-12 16:58:57'),
('3', '2005-08-08 07:00:12', '2005-08-08 17:01:32'),
('3', '2005-08-09 07:01:10', '2005-08-09 17:00:26'),
('3', '2005-08-10 06:59:53', '2005-08-10 17:02:53'),
('3', '2005-08-11 07:01:15', '2005-08-11 17:04:23'),
('3', '2005-08-12 07:00:51', '2005-08-12 16:57:52'),
('4', '2005-08-08 06:54:37', '2005-08-08 17:01:23'),
('4', '2005-08-09 06:58:23', '2005-08-09 17:00:54'),
('4', '2005-08-10 06:59:14', '2005-08-10 17:00:12'),
('4', '2005-08-11 07:00:49', '2005-08-11 17:00:34'),
('4', '2005-08-12 07:01:09', '2005-08-12 16:58:29'),
('5', '2005-08-08 07:00:04', '2005-08-08 17:01:43'),
('5', '2005-08-09 07:02:12', '2005-08-09 17:02:13'),
('5', '2005-08-10 06:59:39', '2005-08-10 17:03:37'),
('5', '2005-08-11 07:01:26', '2005-08-11 17:00:03'),
('5', '2005-08-12 07:02:15', '2005-08-12 16:59:02'),
('6', '2005-08-08 07:00:12', '2005-08-08 17:01:02'),
('6', '2005-08-09 07:03:44', '2005-08-09 17:00:00'),
('6', '2005-08-10 06:54:19', '2005-08-10 17:03:31'),
('6', '2005-08-11 07:00:05', '2005-08-11 17:02:57'),
('6', '2005-08-12 07:02:07', '2005-08-12 16:58:23');
In this example, we are going to assist Human Resources by simplifying the queries that their applications need to perform. At the same time, it's going to enable us to abstract their queries from the database, which allows us more flexibility in maintaining it.
In the previous tutorial, we looked at a JOIN query that displayed all of the lateness instances for a particular employee. In this tutorial, we are going to abstract that query somewhat to provide us with all lateness occurrences for all employees, and then standardize that query by making it into a view.
Our previous query looked like this:
SELECT
`Employees`.`First_Name`,
`Employees`.`Last_Name`,
`Hours`.`Clock_In`,
`Hours`.`Clock_Out`
FROM `Employees`
INNER JOIN `Hours` ON `Employees`.`ID` = `Hours`.`ID`
WHERE `Employees`.`First_Name` = 'Helmholtz'
AND DATE_FORMAT(`Hours`.`Clock_In`, '%Y-%m-%d') >= '2005-08-08'
AND DATE_FORMAT(`Hours`.`Clock_In`, '%Y-%m-%d') <= '2005-08-12'
AND DATE_FORMAT(`Hours`.`Clock_In`, '%H:%i:%S') > '07:00:59';
The result:
+------------+-----------+---------------------+---------------------+
| First_Name | Last_Name | Clock_In | Clock_Out |
+------------+-----------+---------------------+---------------------+
| Helmholtz | Watson | 2005-08-09 07:03:44 | 2005-08-09 17:00:00 |
| Helmholtz | Watson | 2005-08-12 07:02:07 | 2005-08-12 16:58:23 |
+------------+-----------+---------------------+---------------------+
The previous example displays to us all of Heimholtz's punch-in times that were after seven AM. We can see here that Heimholz has been late twice within this reporting period, and we can also see that in both instances, he either left exactly on time or he left early. Our company policy, however, dictates that late instances must be made up at the end of one's shift, so we want to exclude from our report anyone whose clock-out time was greater than 10 hours and one minute after their clock-in time.
SELECT
`Employees`.`First_Name`,
`Employees`.`Last_Name`,
`Hours`.`Clock_In`,
`Hours`.`Clock_Out`,
(TIMESTAMPDIFF(MINUTE,`Hours`.`Clock_Out`,`Hours`.`Clock_In`) + 601) AS Difference
FROM `Employees`
INNER JOIN `Hours` USING (`ID`)
WHERE DATE_FORMAT(`Hours`.`Clock_In`, '%Y-%m-%d') >= '2005-08-08'
AND DATE_FORMAT(`Hours`.`Clock_In`, '%Y-%m-%d') <= '2005-08-12'
AND DATE_FORMAT(`Hours`.`Clock_In`, '%H:%i:%S') > '07:00:59'
AND TIMESTAMPDIFF(MINUTE,`Hours`.`Clock_Out`,`Hours`.`Clock_In`) > -601;
This gives us the following list of people who have violated our attendance policy:
+------------+-----------+---------------------+---------------------+------------+
| First_Name | Last_Name | Clock_In | Clock_Out | Difference |
+------------+-----------+---------------------+---------------------+------------+
| Mustapha | Mond | 2005-08-12 07:02:29 | 2005-08-12 16:59:12 | 4 |
| Henry | Foster | 2005-08-11 07:01:58 | 2005-08-11 17:00:45 | 2 |
| Henry | Foster | 2005-08-12 07:02:12 | 2005-08-12 16:58:57 | 4 |
| Bernard | Marx | 2005-08-09 07:01:10 | 2005-08-09 17:00:26 | 1 |
| Lenina | Crowne | 2005-08-12 07:01:09 | 2005-08-12 16:58:29 | 3 |
| Fanny | Crowne | 2005-08-11 07:01:26 | 2005-08-11 17:00:03 | 2 |
| Fanny | Crowne | 2005-08-12 07:02:15 | 2005-08-12 16:59:02 | 4 |
| Helmholtz | Watson | 2005-08-09 07:03:44 | 2005-08-09 17:00:00 | 4 |
| Helmholtz | Watson | 2005-08-12 07:02:07 | 2005-08-12 16:58:23 | 4 |
+------------+-----------+---------------------+---------------------+------------+
We can see in the previous example that there have been several instances of employees coming in late and leaving early. Unfortunately, we can also see that this query is getting needlessly complex. Having all of this SQL in our application not only creates more complex application code, but also means that if we ever change the structure of this table we're going to have to change what is becoming a somewhat messy query. This is where views begin to show their usefulness.
Creating a view is almost exactly the same as creating a SELECT statement, so we can use our previous SELECT statement in the creation of our new view:
CREATE SQL SECURITY INVOKER VIEW Employee_Tardiness AS
SELECT
`Employees`.`First_Name`,
`Employees`.`Last_Name`,
`Hours`.`Clock_In`,
`Hours`.`Clock_Out`,
(TIMESTAMPDIFF(MINUTE,`Hours`.`Clock_Out`,`Hours`.`Clock_In`) + 601) as Difference
FROM `Employees`
INNER JOIN `Hours` USING (`ID`)
WHERE DATE_FORMAT(`Hours`.`Clock_In`, '%Y-%m-%d') >= '2005-08-08'
AND DATE_FORMAT(`Hours`.`Clock_In`, '%Y-%m-%d') <= '2005-08-12'
AND DATE_FORMAT(`Hours`.`Clock_In`, '%H:%i:%S') > '07:00:59'
AND TIMESTAMPDIFF(MINUTE,`Hours`.`Clock_Out`,`Hours`.`Clock_In`) > -601;
Note that the first line of our query contains the statement 'SQL SECURITY INVOKER' - this means that when the view is accessed, it runs with the same privileges that the person accessing the view has. Thus, if someone without access to our Employees table tries to access this view, they will get an error.
Other than the security parameter, the rest of the query is fairly self explanatory. We simply run 'CREATE VIEW AS' and then append any valid SELECT statement, and our view is created. Now if we do a SELECT from the view, we can see we get the same results as before, with much less SQL:
SELECT * FROM Employee_Tardiness;
+------------+-----------+---------------------+---------------------+------------+
| First_Name | Last_Name | Clock_In | Clock_Out | Difference |
+------------+-----------+---------------------+---------------------+------------+
| Mustapha | Mond | 2005-08-12 07:02:29 | 2005-08-12 16:59:12 | 5 |
| Henry | Foster | 2005-08-11 07:01:58 | 2005-08-11 17:00:45 | 3 |
| Henry | Foster | 2005-08-12 07:02:12 | 2005-08-12 16:58:57 | 5 |
| Bernard | Marx | 2005-08-09 07:01:10 | 2005-08-09 17:00:26 | 2 |
| Lenina | Crowne | 2005-08-12 07:01:09 | 2005-08-12 16:58:29 | 4 |
| Fanny | Crowne | 2005-08-09 07:02:12 | 2005-08-09 17:02:13 | 1 |
| Fanny | Crowne | 2005-08-11 07:01:26 | 2005-08-11 17:00:03 | 3 |
| Fanny | Crowne | 2005-08-12 07:02:15 | 2005-08-12 16:59:02 | 5 |
| Helmholtz | Watson | 2005-08-09 07:03:44 | 2005-08-09 17:00:00 | 5 |
| Helmholtz | Watson | 2005-08-12 07:02:07 | 2005-08-12 16:58:23 | 5 |
+------------+-----------+---------------------+---------------------+------------+
Now we can even perform operations on the table, such as limiting our results to just those with a Difference of at least five minutes:
SELECT * FROM Employee_Tardiness WHERE Difference >=5;
+------------+-----------+---------------------+---------------------+------------+
| First_Name | Last_Name | Clock_In | Clock_Out | Difference |
+------------+-----------+---------------------+---------------------+------------+
| Mustapha | Mond | 2005-08-12 07:02:29 | 2005-08-12 16:59:12 | 5 |
| Henry | Foster | 2005-08-12 07:02:12 | 2005-08-12 16:58:57 | 5 |
| Fanny | Crowne | 2005-08-12 07:02:15 | 2005-08-12 16:59:02 | 5 |
| Helmholtz | Watson | 2005-08-09 07:03:44 | 2005-08-09 17:00:00 | 5 |
| Helmholtz | Watson | 2005-08-12 07:02:07 | 2005-08-12 16:58:23 | 5 |
+------------+-----------+---------------------+---------------------+------------+
Aside from just simplifying our application's SQL queries, there are also other benefits that views can provide, some of which are only possible by using views.
For example, even though our Employees database contains fields for Position, Home Address, and Home Phone, our query does not allow for these fields to be shown. This means that in the case of a security issue in the application (for example, an SQL injection attack, or even a malicious programmer), there is no risk of disclosing an employee's personal information.
We can also define separate views to include a specific WHERE clause for security; for example, if we wanted to restrict a department head's access to only the staff that report to him, we could specify his identity in the view's CREATE statement, and he would then be unable to see any other department's employees, despite them all being in the same table. If this view is writeable and it is defined with the CASCADE clause, this restriction will also apply to writes. This is actually the only way to implement row-level security in MariaDB, so views play an important part in that area as well.
We can also define our views in such a way as to force the use of indexes, so that other, less-experienced developers don't run the risk of running un-optimized queries or JOINs that result in full-table scans and extended locks. Expensive queries, queries that SELECT *, and poorly thought-out JOINs can not only slow down the database entirely, but can cause inserts to fail, clients to time out, and reports to error out. By creating a view that is already optimized and letting users perform their queries on that, you can ensure that they won't cause a significant performance hit unnecessarily.
When we re-engineer our application, we sometimes need to change the database to optimize or accommodate new or removed features. We may, for example, want to our tables when they start getting too large and queries start taking too long. Alternately, we may be installing a new application with different requirements alongside a legacy application. Unfortunately, database redesign will tend to break backwards-compatibility with previous applications, which can cause obvious problems.
Using views, we can change the format of the underlying tables while still presenting the same table format to the legacy application. Thus, an application which demands username, hostname, and access time in string format can access the same data as an application which requires firstname, lastname, user@host, and access time in Unix timestamp format.
Views are an SQL feature that can provide a lot of versatility in larger applications, and can even simplify smaller applications further. Just as stored procedures can help us abstract out our database logic, views can simplify the way we access data in the database, and can help un-complicate our queries to make application debugging easier and more efficient.
The initial version of this article was copied, with permission, from Views_(Basic on 2012-10-05.
This page is licensed: CC BY-SA / Gnu FDL
Indexing Guide
This guide explains the different types of indexes in MariaDB, their characteristics, and how they are used. Learn to create and manage Primary Keys, Unique Indexes, and Plain Indexes, along with key considerations for choosing and maintaining effective indexes for optimal query performance.
In MariaDB, the terms KEY
and INDEX
are generally used interchangeably in SQL statements.
There are four main kinds of indexes:
Primary Keys: Unique and not NULL.
Unique Indexes: Must be unique but can contain NULL values.
Plain Indexes (or Regular Indexes): Not necessarily unique.
Full-Text Indexes: Used for full-text searching capabilities.
A primary key uniquely identifies each record in a table. Its values must be unique, and it cannot contain NULL
values. Each table can have only one primary key.
InnoDB Considerations:
In InnoDB tables, the primary key is included as a suffix in all other indexes. Therefore, keeping the primary key compact (e.g., using an appropriate integer type) is important for performance and storage efficiency.
If a table has no explicitly defined primary key and no UNIQUE
indexes, InnoDB automatically creates an invisible 6-byte clustered index.
Using AUTO_INCREMENT
: The AUTO_INCREMENT
attribute is commonly used with numeric primary keys to automatically generate a unique ID for each new row.
Note: The column defined as a primary key (or part of it) must be explicitly declared as NOT NULL
.
Adding a Primary Key to an Existing Table: Use ALTER TABLE
. You cannot create a primary key with CREATE INDEX
.
Finding Tables Without Primary Keys: This query uses the information_schema
database to find tables lacking primary keys:
A unique index ensures that all values in the indexed column (or combination of columns) are unique. However, unlike a primary key, columns in a unique index can store NULL
values.
Each key value uniquely identifies a row, but not every row needs to be represented if NULL
s are allowed.
Behavior (MariaDB 10.5+):
If the index type is not specified, UNIQUE
typically creates a BTREE index, usable by the optimizer.
If a key exceeds the maximum length for the storage engine and the engine supports long unique indexes, a HASH key might be created to enforce uniqueness.
Creating Unique Indexes: During table creation:
After table creation using ALTER TABLE
:
After table creation using CREATE UNIQUE INDEX
:
Multi-Column Unique Indexes: An index can span multiple columns. MariaDB can use the leftmost part(s) of such an index if it cannot use the whole index (except for HASH indexes).
NULL
Values in Unique Indexes: A UNIQUE
constraint allows multiple NULL
values because in SQL, NULL
is never equal to another NULL
.
Verification:
Conditional Uniqueness with Virtual Columns: You can enforce uniqueness over a subset of rows using unique indexes on . This example ensures user_name
is unique for 'Active' or 'On-Hold' users, but allows duplicate names for 'Deleted' users:
Trailing Pad Characters: If a unique index is on a column where trailing pad characters are stripped or ignored (e.g., CHAR
vs VARCHAR
behavior), inserts where values differ only by the number of trailing pad characters can result in duplicate-key errors.
Long Keys and HASH Indexes (MariaDB 10.5+): For engines like InnoDB, UNIQUE
can be used with various column types and numbers. If a key's length exceeds the engine's maximum, a HASH key may be created.
Example output snippet showing USING HASH
:
Plain indexes do not enforce uniqueness; they are primarily used to speed up data retrieval.
Full-text indexes are used for performing full-text searches on text data. For details, see the documentation.
Index for Queries: Add indexes that match the WHERE
clauses, JOIN
conditions, and ORDER BY
clauses of your application's queries.
Avoid Over-Indexing: Extra indexes consume storage and can slow down INSERT
, UPDATE
, and DELETE
operations.
Impact of Table Size: Indexes provide more significant speed-ups on large tables (larger than buffer sizes) than on very small tables.
Use EXPLAIN
: Analyze your queries with the EXPLAIN
statement to determine if indexes are being used effectively and identify columns that might benefit from indexing.
LIKE '%word%'
: Queries using a leading wildcard in a LIKE
clause (e.g., LIKE '%word%'
) typically cannot use standard BTREE indexes effectively and may result in full table scans unless a full-text index is used.
Delayed Writes: For tables with many reads and writes, consider storage engine options or server configurations related to delayed writes to potentially improve performance by batching disk I/O. (This is an advanced topic.)
Creating Indexes on Existing Tables: Use CREATE INDEX index_name ON table_name (column_list);
Large Tables: For very large tables, it's often faster to load data into the table first and then create indexes, rather than creating indexes on an empty table and then loading data.
SHOW INDEX FROM table_name;
: Displays information about all indexes on a table.SQL
SHOW CREATE TABLE table_name;
: Shows the CREATE TABLE
statement, which includes definitions for all indexes.SQL
Remove an index if:
It is rarely or never used. Unused indexes still incur overhead during data modification operations.
Identifying Unused Indexes:
If are enabled, query the information_schema.INDEX_STATISTICS
table.
If the is enabled and the log_queries_not_using_indexes
is ON
, queries performing full table scans will be logged, which can indicate missing or ineffective indexes.
This page is licensed: CC BY-SA / Gnu FDL
This page is intended to be a quick reference of commonly-used and/or useful queries in MariaDB.
See for more.
See for more.
The AUTO_INCREMENT attribute is used to automatically generate a unique identity for new rows.
When inserting, the id field can be omitted, and is automatically created.
See for more.
This kind of query is called a join - see for more.
See the for more, as well as below for a more practical example.
See the for more.
See the for more.
See the for more.
See for more.
In this example, we want to find the lowest test score for any student.
This example returns the best test results of each student:
The function can be used to calculate someone's age:
See for more.
This example sets a with the average test score, and then uses it in a later query to return all results above the average.
User-defined variables can also be used to add an incremental counter to a resultset:
See for more.
Returns a list of all tables in the database, ordered by size:
This example assumes there's a unique ID, but that all other fields are identical. In the example below, there are 4 records, 3 of which are duplicates, so two of the three duplicates need to be removed. The intermediate SELECT is not necessary, but demonstrates what is being returned.
CC BY-SA / Gnu FDL
When a MariaDB developer first creates a MariaDB database for a client, often times the client has already accumulated data in other, simpler applications. Being able to convert data easily to MariaDB is critical. In the previous two articles of this MariaDB series, we explored how to set up a database and how to query one. In this third installment, we will introduce some methods and tools for bulk importing of data into MariaDB. This isn't an overly difficult task, but the processing of large amounts of data can be intimidating for a newcomer and as a result it can be a barrier to getting started with MariaDB. Additionally, for intermediate developers, there are many nuances to consider for a clean import, which is especially important for automating regularly scheduled imports. There are also restraints to deal with that may be imposed on a developer when using a web hosting company.
Clients sometimes give developers raw data in formats created by simple database programs like MS Access ®. Since non-technical clients don't typically understand database concepts, new clients often give me their initial data in Excel spreadsheets. Let's first look at a simple method for importing data. The simplest way to deal with incompatible data in any format is to load it up in its original software and to export it out to a delimited text file. Most applications have the ability to export data to a text format and will allow the user to set the delimiters. We like to use the bar (i.e., |
, a.k.a. pipe) to separate fields and the line-feed to separate records.
For the examples in this article, we will assume that a fictitious client's data was in Excel and that the exported text file are named prospects.txt
. It contains contact information about prospective customers for the client's sales department, located on the client's intranet site. The data is to be imported into a MariaDB table called prospect_contact
, in a database called sales_dept
. To make the process simpler, the order and number of columns in MS Excel ® (the format of the data provided by the client) should be the same as the table into which the data is going to be imported. So if prospect_contact has columns that are not included in the spreadsheet, one would make a copy of the spreadsheet and add the missing columns and leave them blank. If there are columns in the spreadsheet that aren't in prospect_contact
, one would either add them to the MariaDB table, or, if they're not to be imported, one would delete the extra columns from the spreadsheet. One should also delete any headings and footnotes from the spreadsheet. After this is completed then the data can be exported. Since this is Unix Review, we'll skip how one would export data in Excel and assume that the task was accomplished easily enough using its export wizard.
The next step is to upload the data text file to the client's web site by FTP. It should be uploaded in ASCII mode. Binary mode may send binary hard-returns for row-endings. Also, it's a good security habit to upload data files to non-public directories. Many web hosting companies provide virtual domains with a directory like /public_html
, which is the document root for the Apache web server; it typically contains the site's web pages. In such a situation, /
is a virtual root containing logs and other files that are inaccessible to the public. We usually create a directory called tmp
in the virtual root directory to hold data files temporarily for importing into MariaDB. Once that's done, all that's required is to log into MariaDB with the client as an administrative user (if not root, then a user with FILE
privileges), and run the proper SQL statement to import the data.
The statement is the easiest way to import data from a plain text file into MariaDB. Below is what one would enter in the client to load the data in the file called prospects.txt into the table prospect_contact
:
Before entering the statement above, the MariaDB session would, of course, be switched to the sales_dept database with a statement. It is possible, though, to specify the database along with the table name (e.g., sales_dept.prospect_contact
). If the server is running Windows, the forward slashes are still used for the text file's path, but a drive may need to be specified at the beginning of the path: 'c:/tmp/prospects.txt
'. Notice that the SQL statement above has |
as the field delimiter. If the delimiter was [TAB]—which is common—then one would replace |
with here. A line-feed () isn't specified as the record delimiter since it's assumed. If the rows start and end with something else, though, then they will need to be stated. For instance, suppose the rows in the text file start with a double-quote and end with a double-quote and a Windows hard-return (i.e., a return and a line-feed). The statement would need to read like this:
Notice that the starting double-quote is inside of single-quotes. If one needs to specify a single-quote as the start of a line, one could either put the one single-quote within double-quotes or one could escape the inner single-quote with a back-slash, thus telling MariaDB that the single-quote that follows is to be taken literally and is not part of the statement, per se:
If the table prospect_contact already contains some of the records that are about to be imported from prospects.txt (that is to say, records with the same primary key), then a decision should be made as to what MariaDB is to do about the duplicates. The SQL statement, as it stands above, will cause MariaDB to try to import the duplicate records and to create duplicate rows in prospect_contact for them. If the table's properties are set not to allow duplicates, then MariaDB will kick out errors. To get MariaDB to replace the duplicate existing rows with the ones being imported in, one would add the REPLACE
just before the INTO TABLE
clause like this:
To import only records for prospects that are not already in prospect_contact, one would substitute REPLACE with the IGNORE
flag. This instructs MariaDB to ignore records read from the text file that already exist in the table.
For importing data into a table while it's in use, table access needs to be addressed. If access to the table by other users may not be interrupted, then a LOW_PRIORITY
flag can be added to the statement. This tells MariaDB that the loading of this data is a low priority. One would only need to change the first line of the SQL statement above to set its priority to low:
If the LOW_PRIORITY
flag isn't included, the table are locked temporarily during the import and other users are prevented from accessing it.
I mentioned earlier that uploading of the text file should not be done in binary mode so as to avoid the difficulties associated with Windows line endings. If this is unavoidable, however, there is an easy way to import binary row-endings with MariaDB. One would just specify the appropriate hexadecimals for a carriage-return combined with a line-feed (i.e., CRLF
) as the value of TERMINATED BY
:
Notice that there are intentionally no quotes around the binary value. If there were, MariaDB would take the value for text and not a binary code. The semi-colon is not part of the value; it's the SQL statement terminator.
Earlier we also stated that the first row in the spreadsheet containing the column headings should be deleted before exporting to avoid the difficulty of importing the headings as a record. It's actually pretty easy to tell MariaDB to just skip the top line. One would add the following line to the very end of the statement:
The number of lines for MariaDB to ignore can, of course, be more than one.
Another difficulty arises when some Windows application wizards export data with each field surrounded by double-quotes, as well as around the start and end of records. This can be a problem when a field contains a double-quote. To deal with this, some applications use back-slash () to escape embedded double-quotes, to indicate that a particular double-quote is not a field ending but part of the field's content. However, some applications will use a different character (like a pound-sign) to escape embedded quotes. This can cause problems if MariaDB isn't prepared for the odd escape-character. MariaDB will think the escape character is actually text and the embedded quote-mark, although it's escaped, is a field ending. The unenclosed text that follows are imported into the next column and the remaining columns are one column off, leaving the last column not imported. As maddening as this can be, it's quite manageable in MariaDB by adding an ENCLOSED BY
and an ESCAPED BY
clause:
In the Foreign Data Basics section above, we said that the columns in the spreadsheet should be put in the same order and quantity as the receiving table. This really isn't necessary if MariaDB is cued in as to what it should expect. To illustrate, let's assume that prospect_contact has four columns in the following order: row_id
, name_first
, name_last
, telephone
. Whereas, the spreadsheet has only three columns, differently named, in this order: Last Name
, First Name
, Telephone
. If the spreadsheet isn't adjusted, then the SQL statement will need to be changed to tell MariaDB the field order:
This SQL statement tells MariaDB the name of each table column associated with each spreadsheet column in the order that they appear in the text file. From there it will naturally insert the data into the appropriate columns in the table. As for columns that are missing like row_id, MariaDB will fill in those fields with the default value if one has been supplied in the table's properties. If not, it will leave the field as NULL. Incidentally, we slipped in the binary [TAB] (0x09)
as a field delimiter.
For some clients and for certain situations it may be of value to be able to import data into MariaDB without using the client. This could be necessary when constructing a shell script to import text files on an automated, regular schedule. To accomplish this, the (mysqlimport
before ) utility may be used as it encompasses the statement and can easily be run from a script. So if one wants to enter the involved SQL statement at the end of the last section above, the following could be entered from the command-line (i.e., not in the client):
Although this statement is written over several lines here, it either has to be on the same line when entered or a space followed by a back-slash has to be entered at the end of each line (as seen here) to indicate that more follows. Since the above is entered at the command-line prompt, the user isn't logged into MariaDB. Therefore the first line contains the user name and password for to give to MariaDB. The password itself is optional, but the directive --password
(without the equal sign) isn't. If the password value is not given in the statement, then the user are prompted for it. Notice that the order of directives doesn't matter after the initial command, except that the database and file name go last. Regarding the file name, its prefix must be the same as the table—the dot and the extension are ignored. This requires that prospects.txt
be renamed to prospect_contact.txt
. If the file isn't renamed, then MariaDB would create a new table called prospects and the --replace
option would be pointless. After the file name, incidentally, one could list more text files, separated by a space, to process using . We've added the --verbose
directive so as to be able to see what's going on. One probably would leave this out in an automated script. By the way, --low-priority
and --ignore-lines
are available.
Some web hosting companies do not allow the use of or statements due to security vulnerabilities in these statements for them. To get around this, some extra steps are necessary to avoid having to manually enter the data one row at a time. First, one needs to have MariaDB installed on one's local workstation. For simplicity, we'll assume this is done and is running Linux on the main partition and MS Windows® on an extra partition. Recapping the on-going example of this article based on these new circumstances, one would boot up into Windows and start MS Excel®, load the client's spreadsheet into it and then run the export wizard as before—saving the file prospects.txt to the 'My Documents' directory. Then one would reboot into Linux and mount the Windows partition and copy the data text file to /tmp
in Linux, locally. Next one would log into the local (not the client's) MariaDB server and import the text file using a as we've extensively outline above. From there one would exit MariaDB and export the data out of MariaDB using the utility locally, from the command-line like this:
This creates an interesting text file complete with all of the SQL commands necessary to insert the data back into MariaDB one record, one at a time. When you run , it's very educational to open up it in a text editor to see what it generates.
After creating this table dump, one would upload the resulting file (in ASCII mode) to the /tmp
directory on the client's web server. From the command prompt on the client's server one would enter the following:
This line along with the line show above are simple approaches. Like the Windows application wizard, with mariadb-dump one can specify the format of the output file and several other factors. One important factor related to the scenario used in this article is the statement that are embedded in the output file. This will fail and kick out an error because of the existing table prospect_contact in the client's database. To limit the output to only statements and no statements, the line would look like this:
Notice that we've used acceptable abbreviations for the user name and the password directives. Since the password was given here, the user are prompted for it.
The utility usually works pretty well. However, one feature it's lacking at this time is a REPLACE
flag as is found in the statement and with the tool. So if a record already exists in the prospect_contact
, it won't be imported. Instead it will kick out an error message and stop at that record, which can be a mess if one has imported several hundred rows and have several hundred more to go. One easy fix for this is to open up prospects.sql
in a text editor and do a search on the word INSERT
and replace it with REPLACE
. The syntax of both of these statements are the same, fortunately. So one would only need to replace the keyword for new records to be inserted and for existing records to be replaced.
It's always amazing to me how much can be involved in the simplest of statements in MariaDB. MariaDB is deceptively powerful and feature rich. One can keep the statements pretty minimal or one can develop a fairly detailed, single statement to allow for accuracy of action. There are many other aspects of importing data into MariaDB that we did not address—in particular dealing with utilities. We also didn't talk about the Perl modules that could be used to convert data files. These can be useful in scripting imports. There are many ways in which one can handle importing data. Hopefully, this article has presented most of the basics and pertinent advanced details that may be of use to most MariaDB developers.
This page is licensed: CC BY-SA / Gnu FDL
For a basic overview, see .
There are four main kinds of indexes; primary keys (unique and not null), unique indexes (unique and can be null), plain indexes (not necessarily unique) and full-text indexes (for full-text searching).
The terms 'KEY' and 'INDEX' are generally used interchangeably, and statements should work with either keyword.
A primary key is unique and can never be null. It will always identify only one record, and each record must be represented. Each table can only have one primary key.
In tables, all indexes contain the primary key as a suffix. Thus, when using this storage engine, keeping the primary key as small as possible is particularly important. If a primary key does not exist and there are no UNIQUE indexes, InnoDB creates a 6-bytes clustered index which is invisible to the user.
Many tables use a numeric ID field as a primary key. The attribute can be used to generate a unique identity for new rows, and is commonly-used with primary keys.
Primary keys are usually added when the table is created with the statement. For example, the following creates a primary key on the ID field. Note that the ID field had to be defined as NOT NULL, otherwise the index could not have been created.
You cannot create a primary key with the command. If you do want to add one after the table has already been created, use , for example:
Tables in the INFORMATION_SCHEMA
database can be queried to find tables that do not have primary keys. For example, here is a query using the and tables that can be used:
A Unique Index must be unique, but it can have columns that may be NULL. So each key value identifies only one record, but not each record needs to be represented.
MariaDB starting with
Unique, if index type is not specified, is normally a BTREE index that can also be used by the optimizer to find rows. If the key is longer than the max key length for the used storage engine and the storage engine supports long unique index, a HASH key are created. This enables MariaDB to enforce uniqueness for any type or number of columns.
For example, to create a unique key on the Employee_Code field, as well as a primary key, use:
Unique keys can also be added after the table is created with the command, or with the command, for example:
and
Indexes can contain more than one column. MariaDB is able to use one or more columns on the leftmost part of the index, if it cannot use the whole index. (except for the HASH index type).
Take another example:
Since the index is defined as unique over both columns a and b, the following row is valid, as while neither a nor b are unique on their own, the combination is unique:
The fact that a UNIQUE
constraint can be NULL
is often overlooked. In SQL any NULL
is never equal to anything, not even to another NULL
. Consequently, a UNIQUE
constraint will not prevent one from storing duplicate rows if they contain null values:
Indeed, in SQL two last rows, even if identical, are not equal to each other:
In MariaDB you can combine this with to enforce uniqueness over a subset of rows in a table:
This table structure ensures that all active or on-hold users have distinct names, but as soon as a user is deleted, his name is no longer part of the uniqueness constraint, and another user may get the same name.
If a unique index consists of a column where trailing pad characters are stripped or ignored, inserts into that column where values differ only by the number of trailing pad characters will result in a duplicate-key error.
MariaDB starting with
For some engines, like InnoDB, UNIQUE
can be used with any type of columns or any number of columns.
If the key length is longer than the max key length supported by the engine, a HASH key are created.
This can be seen with SHOW CREATE TABLE table_name
or SHOW INDEX FROM table_name
:
Indexes do not necessarily need to be unique. For example:
Full-text indexes support full-text indexing and searching. See the section.
In general you should only add indexes to match the queries your application uses. Any extra will waste resources. In an application with very small tables, indexes will not make much difference but as soon as your tables are larger than your buffer sizes the indexes will start to speed things up dramatically.
Using the statement on your queries can help you decide which columns need indexing.
If you query contains something like LIKE '%word%'
, without a fulltext index you are using a full table scan every time, which is very slow.
If your table has a large number of reads and writes, consider using delayed writes. This uses the db engine in a "batch" write mode, which cuts down on disk io, therefore increasing performance.
Use the command to create an index.
If you are building a large table then for best performance add the index after the table is populated with data. This is to increase the insert performance and remove the index overhead during inserts.
You can view which indexes are present on a table, as well as details about them, with the statement.
If you want to know how to re-create an index, run SHOW CREATE TABLE
.
If an index is rarely used (or not used at all) then remove it to increase INSERT, and UPDATE performance.
If are enabled, the table stores the index usage.
If the is enabled and the log_queries_not_using_indexes
server system variable is ON
, the queries which do not use indexes are logged.
The initial version of this article was copied, with permission, from on 2012-10-30.
CC BY-SA / Gnu FDL
CREATE TABLE `Employees` (
`ID` TINYINT(3) UNSIGNED NOT NULL AUTO_INCREMENT,
`First_Name` VARCHAR(25) NOT NULL,
`Last_Name` VARCHAR(25) NOT NULL,
`Position` VARCHAR(25) NOT NULL,
PRIMARY KEY (`ID`)
);
ALTER TABLE Employees ADD PRIMARY KEY(ID);
SELECT t.TABLE_SCHEMA, t.TABLE_NAME
FROM information_schema.TABLES AS t
LEFT JOIN information_schema.KEY_COLUMN_USAGE AS c
ON t.TABLE_SCHEMA = c.CONSTRAINT_SCHEMA
AND t.TABLE_NAME = c.TABLE_NAME
AND c.CONSTRAINT_NAME = 'PRIMARY'
WHERE t.TABLE_SCHEMA NOT IN ('information_schema', 'performance_schema', 'mysql', 'sys')
AND c.CONSTRAINT_NAME IS NULL;
### INSERT INTO `securedb`.`t_long_keys`
### SET
### @1=1 /* INT meta=0 nullable=0 is_null=0 */
### @2='a' /* VARSTRING(4073) meta=4073 nullable=1 is_null=0 */
### @3=580 /* LONGINT meta=0 nullable=1 is_null=0 */
CREATE TABLE `Employees` (
`ID` TINYINT(3) UNSIGNED NOT NULL,
`Employee_Code` VARCHAR(25) NOT NULL,
`First_Name` VARCHAR(25) NOT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY `UK_EmpCode` (`Employee_Code`) -- Naming the unique key is good practice
);
ALTER TABLE Employees ADD UNIQUE `UK_HomePhone` (`Home_Phone`);
CREATE UNIQUE INDEX `IX_Position` ON Employees(Position);
CREATE TABLE t1 (a INT NOT NULL, b INT, UNIQUE (a,b));
INSERT INTO t1 VALUES (1,1), (2,2);
INSERT INTO t1 VALUES (2,1); -- Valid: (2,1) is unique, though '2' in 'a' and '1' in 'b' are not individually unique here.
SELECT * FROM t1;
+---+------+
| a | b |
+---+------+
| 1 | 1 |
| 2 | 1 |
| 2 | 2 |
+---+------+
INSERT INTO t1 VALUES (3,NULL), (3, NULL); -- Both rows are inserted
SELECT * FROM t1;
+---+------+
| a | b |
+---+------+
| 1 | 1 |
| 2 | 1 |
| 2 | 2 |
| 3 | NULL |
| 3 | NULL |
+---+------+
SELECT (3, NULL) = (3, NULL);
+-----------------------+
| (3, NULL) = (3, NULL) |
+-----------------------+
| 0 | -- 0 means false
+-----------------------+
CREATE TABLE Table_1 (
user_name VARCHAR(10),
status ENUM('Active', 'On-Hold', 'Deleted'),
del CHAR(0) AS (IF(status IN ('Active', 'On-Hold'), '', NULL)) PERSISTENT,
UNIQUE(user_name, del)
);
-- Example table definition (simplified for brevity)
CREATE TABLE t_long_keys (
a INT PRIMARY KEY,
b BLOB,
c1 VARCHAR(1000),
UNIQUE KEY `uk_b` (b),
UNIQUE KEY `uk_c1` (c1)
) ENGINE=InnoDB;
-- SHOW CREATE TABLE might reveal 'USING HASH' for uk_b or uk_c1 if they exceed length limits
SHOW CREATE TABLE t_long_keys\G
...
UNIQUE KEY `uk_b` (`b`) USING HASH,
...
CREATE TABLE t2 (a INT NOT NULL, b INT, INDEX `idx_a_b` (a,b));
INSERT INTO t2 VALUES (1,1), (2,2), (2,2); -- Duplicate (2,2) is allowed
SELECT * FROM t2;
+---+------+
| a | b |
+---+------+
| 1 | 1 |
| 2 | 2 |
| 2 | 2 |
+---+------+
SHOW INDEX FROM Employees;
SHOW CREATE TABLE Employees\G
MariaDB has many built-in functions that can be used to manipulate strings of data. With these functions, one can format data, extract certain characters, or use search expressions. Good developers should be aware of the string functions that are available. Therefore, in this article we will go through several string functions, grouping them by similar features, and provide examples of how they might be used.
There are several string functions that are used to format text and numbers for nicer display. A popular and very useful function for pasting together the contents of data fields with text is the CONCAT() function. As an example, suppose that a table called contacts has a column for each sales contact's first name and another for the last name. The following SQL statement would put them together:
SELECT CONCAT(name_first, ' ', name_last)
AS Name
FROM contacts;
This statement will display the first name, a space, and then the last name together in one column. The AS clause will change the column heading of the results to Name.
A less used concatenating function is CONCAT_WS(). It will put together columns with a separator between each. This can be useful when making data available for other programs. For instance, suppose we have a program that will import data, but it requires the fields to be separated by vertical bars. We could just export the data, or we could use a SELECT statement like the one that follows in conjunction with an interface written with an API language like Perl:
SELECT CONCAT_WS('|', col1, col2, col3)
FROM table1;
The first element above is the separator. The remaining elements are the columns to be strung together.
If we want to format a long number with commas every three digits and a period for the decimal point (e.g., 100,000.00), we can use the function FORMAT() like so:
SELECT CONCAT('$', FORMAT(col5, 2))
FROM table3;
In this statement, the CONCAT() will place a dollar sign in front of the numbers found in the col5
column, which are formatted with commas by FORMAT(). The 2
within the FORMAT() stipulates two decimal places.
Occasionally, one will want to convert the text from a column to either all upper-case letters or all lower-case letters. In the example that follows, the output of the first column is converted to upper-case and the second to lower-case:
SELECT UCASE(col1),
LCASE(col2)
FROM table4;
When displaying data in forms, it's sometimes useful to pad the data displayed with zeros or dots or some other filler. This can be necessary when dealing with VARCHAR columns where the width varies to help the user to see the column limits. There are two functions that may be used for padding: LPAD() and RPAD().
SELECT RPAD(part_nbr, 8, '.') AS 'Part Nbr.',
LPAD(description, 15, '_') AS Description
FROM catalog;
In this SQL statement, dots are added to the right end of each part number. So a part number of "H200" will display as "H200....", but without the quotes. Each part's description will have under-scores preceding it. A part with a description of "brass hinge" will display as "brass hinge".
If a column is a CHAR data-type, a fixed width column, then it may be necessary to trim any leading or trailing spaces from displays. There are a few functions to accomplish this task. The LTRIM() function will eliminate any leading spaces to the left. So "H200
" becomes "H200
". For columns with trailing spaces, spaces on the right, RTRIM() will work: "H500
" becomes "H500
". A more versatile trimming function, though, is TRIM(). With it one can trim left, right or both. Below are a few examples:
SELECT TRIM(LEADING '.' FROM col1),
TRIM(TRAILING FROM col2),
TRIM(BOTH '_' FROM col3),
TRIM(col4)
FROM table5;
In the first TRIM() clause, the padding component is specified; the leading dots are to be trimmed from the output of col1
. The trailing spaces are trimmed off of col2
—space is the default. Both leading and trailing under-scores are trimmed from col3
above. Unless specified, BOTH is the default. So leading and trailing spaces are trimmed from col4
in the statement here.
When there is a need to extract specific elements from a column, MariaDB has a few functions that can help. Suppose a column in the table contacts contains the telephone numbers of sales contacts, including the area-codes, but without any dashes or parentheses. The area-code of each could be extracted for sorting with the LEFT() and the telephone number with the RIGHT() function.
SELECT LEFT(telephone, 3) AS area_code,
RIGHT(telephone, 7) AS tel_nbr
FROM contacts
ORDER BY area_code;
In the LEFT() function above, the column telephone is given along with the number of characters to extract, starting from the first character on the left in the column. The RIGHT() function is similar, but it starts from the last character on the right, counting left to capture, in this statement, the last seven characters. In the SQL statement above, area_code is reused to order the results set. To reformat the telephone number, it are necessary to use the SUBSTRING() function.
SELECT CONCAT('(', LEFT(telephone, 3), ') ',
SUBSTRING(telephone, 4, 3), '-',
MID(telephone, 7)) AS 'Telephone Number'
FROM contacts
ORDER BY LEFT(telephone, 3);
In this SQL statement, the CONCAT() function is employed to assemble some characters and extracted data to produce a common display for telephone numbers (e.g., (504) 555-1234). The first element of the CONCAT() is an opening parenthesis. Next, a LEFT() is used to get the first three characters of telephone, the area-code. After that a closing parenthesis, along with a space is added to the output. The next element uses the SUBSTRING() function to extract the telephone number's prefix, starting at the fourth position, for a total of three characters. Then a dash is inserted into the display. Finally, the function MID() extracts the remainder of the telephone number, starting at the seventh position. The functions MID() and SUBSTRING() are interchangeable and their syntax are the same. By default, for both functions, if the number of characters to capture isn't specified, then it's assumed that the remaining ones are to be extracted.
There are a few functions in MariaDB that can help in manipulating text. One such function is REPLACE(). With it every occurrence of a search parameter in a string can be replaced. For example, suppose we wanted to replace the title Mrs. with Ms. in a column containing the person's title, but only in the output. The following SQL would do the trick:
SELECT CONCAT(REPLACE(title, 'Mrs.', 'Ms.'),
' ', name_first, ' ', name_last) AS Name
FROM contacts;
We're using the ever handy CONCAT() function to put together the contact's name with spaces. The REPLACE() function extracts each title and replaces Mrs. with Ms., where applicable. Otherwise, for all other titles, it displays them unchanged.
If we want to insert or replace certain text from a column (but not all of its contents), we could use the INSERT() function in conjunction with the LOCATE() function. For example, suppose another contacts table has the contact's title and full name in one column. To change the occurrences of Mrs. to Ms., we could not use REPLACE() since the title is embedded in this example. Instead, we would do the following:
SELECT INSERT(name, LOCATE(name, 'Mrs.'), 4, 'Ms.')
FROM contacts;
The first element of the INSERT() function is the column. The second element which contains the LOCATE() is the position in the string that text is to be inserted. The third element is optional; it states the number of characters to overwrite. In this case, Mrs. which is four characters is overwritten with Ms. (the final element), which is only three. Incidentally, if 0 is specified, then nothing is overwritten, text is inserted only. As for the LOCATE() function, the first element is the column and the second the search text. It returns the position within the column where the text is found. If it's not found, then 0 is returned. A value of 0 for the position in the INSERT() function negates it and returns the value of name unchanged.
On the odd chance that there is a need to reverse the content of a column, there's the REVERSE() function. You would just place the column name within the function. Another minor function is the REPEAT() function. With it a string may be repeated in the display:
SELECT REPEAT(col1, 2)
FROM table1;
The first component of the function above is the string or column to be repeated. The second component states the number of times it's to be repeated.
The function CHAR_LENGTH() is used to determine the number of characters in a string. This could be useful in a situation where a column contains different types of information of specific lengths. For instance, suppose a column in a table for a college contains identification numbers for students, faculty, and staff. If student identification numbers have eight characters while others have less, the following will count the number of student records:
SELECT COUNT(school_id)
AS 'Number of Students'
FROM table8
WHERE CHAR_LENGTH(school_id)=8;
The COUNT() function above counts the number of rows that meet the condition of the WHERE
clause.
In a SELECT statement, an ORDER BY clause can be used to sort a results set by a specific column. However, if the column contains IP addresses, a simple sort may not produce the desired results:
SELECT ip_address
FROM computers WHERE server='Y'
ORDER BY ip_address LIMIT 3;
+-------------+
| ip_address |
+-------------+
| 10.0.1.1 |
| 10.0.11.1 |
| 10.0.2.1 |
+-------------+
In the limited results above, the IP address 10.0.2.1 should be second. This happens because the column is being sorted lexically and not numerically. The function INET_ATON() will solve this sorting problem.
SELECT ip_address
FROM computers WHERE server='Y'
ORDER BY INET_ATON(ip_address) LIMIT 3;
Basically, the INET_ATON() function will convert IP addresses to regular numbers for numeric sorting. For instance, if we were to use the function in the list of columns in a SELECT statement, instead of the WHERE
clause, the address 10.0.1.1 would return 167772417, 10.0.11.1 will return 167774977, and 10.0.2.1 the number 167772673. As a complement to INET_ATON(), the function INET_NTOA() will translate these numbers back to their original IP addresses.
MariaDB is fairly case insensitive, which usually is fine. However, to be able to check by case, the STRCMP() function can be used. It converts the column examined to a string and makes a comparison to the search parameter.
SELECT col1, col2
FROM table6
WHERE STRCMP(col3, 'text')=0;
If there is an exact match, the function STRCMP() returns 0. So if col3
here contains "Text", it won't match. Incidentally, if col3
alphabetically is before the string to which it's compared, a -1
are returned. If it's after it, a 1
is returned.
When you have list of items in one string, the SUBSTRING_INDEX() can be used to pull out a sub-string of data. As an example, suppose we have a column which has five elements, but we want to retrieve just the first two elements. This SQL statement will return them:
SELECT SUBSTRING_INDEX(col4, '|', 2)
FROM table7;
The first component in the function above is the column or string to be picked apart. The second component is the delimiter. The third is the number of elements to return, counting from the left. If we want to grab the last two elements, we would use a negative two to instruct MariaDB to count from the right end.
There are more string functions available in MariaDB. A few of the functions mentioned here have aliases or close alternatives. There are also functions for converting between ASCII, binary, hexi-decimal, and octal strings. And there are also string functions related to text encryption and decryption that were not mentioned. However, this article has given you a good collection of common string functions that will assist you in building more powerful and accurate SQL statements.
This page is licensed: CC BY-SA / Gnu FDL
Despite a MariaDB developer's best planning, occasionally one needs to change the structure or other aspects of tables. This is not very difficult, but some developers are unfamiliar with the syntax for the functions used in MariaDB to accomplish this. And some changes can be very frustrating. In this article we'll explore the ways to alter tables in MariaDB and we'll give some precautions about related potential data problems.
For the examples in this article, we will refer to a database called db1
containing a table called clients
. The clients
table is for keeping track of client names and addresses. To start off, we'll enter a DESCRIBE statement to see what the table looks like:
DESCRIBE clients;
+-------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+-------------+------+-----+---------+-------+
| cust_id | int(11) | | PRI | 0 | |
| name | varchar(25) | YES | | NULL | |
| address | varchar(25) | YES | | NULL | |
| city | varchar(25) | YES | | NULL | |
| state | char(2) | YES | | NULL | |
| zip | varchar(10) | YES | | NULL | |
| client_type | varchar(4) | YES | | NULL | |
+-------------+-------------+------+-----+---------+-------+
This is a very simple table that will hold very little information. However, it's sufficient for the examples here in which we will change several of its columns. Before doing any structural changes to a table in MariaDB, especially if it contains data, one should make a backup of the table to be changed. There are a few ways to do this, but some choices may not be permitted by your web hosting company. Even if your database is on your own server, though, the mariadb-dump utility is typically the best tool for making and restoring backups in MariaDB, and it's generally permitted by web hosting companies. To backup the clients table with mariadb-dump, we will enter the following from the command-line:
mariadb-dump --user='username' --password='password' --add-locks db1 clients > clients.sql
As you can see, the username and password are given on the first line. On the next line, the --add-locks
option is used to lock the table before backing up and to unlock automatically it when the backup is finished. There are many other options in mariadb-dump that could be used, but for our purposes this one is all that's necessary. Incidentally, this statement can be entered in one line from the shell (i.e., not from the mariadb
client), or it can be entered on multiple lines as shown here by using the back-slash (i.e., /
) to let the shell know that more is to follow. On the third line above, the database name is given, followed by the table name. The redirect (i.e., >
) tells the shell to send the results of the dump to a text file called clients.sql
in the current directory. A directory path could be put in front of the file name to create the file elsewhere. If the table should need to be restored, the following can be run from the shell:
mariadb --user='username' --password='password' db1 < clients.sql
Notice that this line does not use the mariadb-dump
utility. It uses the mariadb
client from the outside, so to speak. When the dump file (clients.sql
) is read into the database, it will delete the clients
table and it's data in MariaDB before restoring the backup copy with its data. So be sure that users haven't added data in the interim. In the examples in this article, we are assuming that there isn't any data in the tables yet.
In order to add a column to an existing MariaDB table, one would use the ALTER TABLE statement. To demonstrate, suppose that it has been decided that there should be a column for the client's account status (i.e., active or inactive). To make this change, the following is entered:
ALTER TABLE clients
ADD COLUMN status CHAR(2);
This will add the column status
to the end with a fixed width of two characters (i.e., AC for active and IA for inactive). In looking over the table again, it's decided that another field for client apartment numbers or the like needs to be added. That data could be stored in the address column, but it would better for it to be in a separate column. An ALTER TABLE statement could be entered like above, but it will look tidier if the new column is located right after the address column. To do this, we'll use the AFTER
option:
ALTER TABLE clients
ADD COLUMN address2 VARCHAR(25)
AFTER address;
By the way, to add a column to the first position, you would replace the last line of the SQL statement above to read like this:
...
FIRST;
Before moving on, let's take a look at the table's structure so far:
DESCRIBE clients;
+-------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+-------------+------+-----+---------+-------+
| cust_id | int(11) | | PRI | 0 | |
| name | varchar(25) | YES | | NULL | |
| address | varchar(25) | YES | | NULL | |
| address2 | varchar(25) | YES | | NULL | |
| city | varchar(25) | YES | | NULL | |
| state | char(2) | YES | | NULL | |
| zip | varchar(10) | YES | | NULL | |
| client_type | varchar(4) | YES | | NULL | |
| status | char(2) | YES | | NULL | |
+-------------+-------------+------+-----+---------+-------+
After looking over the above table display, it's decided that it might be better if the status column has the choices of 'AC' and 'IA' enumerated. To make this change, we'll enter the following SQL statement:
ALTER TABLE clients
CHANGE status status ENUM('AC','IA');
Notice that the column name status is specified twice. Although the column name isn't being changed, it still must be respecified. To change the column name (from status
to active
), while leaving the enumerated list the same, we specify the new column name in the second position:
ALTER TABLE clients
CHANGE status active ENUM('AC','IA');
Here we have the current column name and then the new column name, along with the data type specifications (i.e., ENUM
), even though the result is only a name change. With the CHANGE
clause everything must be stated, even items that are not to be changed.
In checking the table structure again, more changes are decided on: The column address is to be renamed to address1 and changed to forty characters wide. Also, the enumeration of active is to have 'yes' and 'no' choices. The problem with changing enumerations is that data can be clobbered in the change if one isn't careful. We've glossed over this possibility before because we are assuming that clients is empty. Let's take a look at how the modifications suggested could be made with the table containing data:
ALTER TABLE clients
CHANGE address address1 VARCHAR(40),
MODIFY active ENUM('yes','NO','AC','IA');
UPDATE clients
SET active = 'yes'
WHERE active = 'AC';
UPDATE clients
SET active = 'NO'
WHERE active = 'IA';
ALTER TABLE clients
MODIFY active ENUM('yes','NO');
The first SQL statement above changes address and modifies active in preparation for the transition. Notice the use of a MODIFY
clause. It works the same as CHANGE
, but it is only used for changing data types and not column names. Therefore, the column name isn't respecified. Notice also that there is a comma after the CHANGE clause. You can string several CHANGE
and MODIFY
clauses together with comma separators. We've enumerated both the new choices and the old ones to be able to migrate the data. The two UPDATE statements are designed to adjust the data accordingly and the last ALTER TABLE statement is to remove the old enumerated choices for the status column.
In talking to the boss, we find out that the client_type
column isn't going to be used. So we enter the following in MariaDB:
ALTER TABLE clients
DROP client_type;
This deletes client_type
and its data, but not the whole table, obviously. Nevertheless, it is a permanent and non-reversible action; there won't be a confirmation request when using the mariadb client. This is how it is with all MariaDB DROP statements and clauses. So be sure that you want to delete an element and its data before using a DROP.
As mentioned earlier, be sure that you have a backup of your tables before doing any structured changes.
You may have noticed that the results of the DESCRIBE statements shown before have a heading called 'Default' and just about all of the fields have a default value of NULL. This means that there are no default values and a null value is allowed and are used if a value isn't specified when a row is created. To be able to specify a default value other than NULL, an ALTER TABLE statement can be entered with a SET
clause. Suppose we're located in Louisiana and we want a default value of 'LA' for state since that's where our clients are usually located. We would enter the following to set the default:
ALTER TABLE clients
ALTER state SET DEFAULT 'LA';
Notice that the second line starts with ALTER
and not CHANGE
. If we change our mind about having a default value for state, we would enter the following to reset it back to NULL (or whatever the initial default value would be based on the data type):
ALTER TABLE clients
ALTER state DROP DEFAULT;
This particular DROP
doesn't delete data, by the way.
One of the most irritating tasks in making changes to a table for newcomers is dealing with indexes. If they try to rename a column that is indexed by only using an ALTER TABLE statement like we used earlier, they will get a frustrating and confusing error message:
ALTER TABLE clients
CHANGE cust_id client_id INT
PRIMARY KEY;
ERROR 1068: Multiple primary key defined
If they're typing this column change from memory, they will wear themselves out trying different deviations thinking that they remembered the syntax wrong. What most newcomers to MariaDB don't seem to realize is that the index is separate from the indexed column. To illustrate, let's take a look at the index for clients by using the SHOW INDEX statement:
SHOW INDEX FROM clients\G
*************************** 1. row ***************************
TABLE: clients
Non_unique: 0
Key_name: PRIMARY
Seq_in_index: 1
Column_name: cust_id
Collation: A
Cardinality: 0
Sub_part: NULL
Packed: NULL
Comment:
1 row in set (0.00 sec)
The text above shows that behind the scenes there is an index associated with cust_id
. The column cust_id
is not the index. Incidentally, the G at the end of the SHOW INDEX statement is to display the results in portrait instead of landscape format. Before the name of an indexed column can be changed, the index related to it must be eliminated. The index is not automatically changed or deleted. Therefore, in the example above, MariaDB thinks that the developer is trying to create another primary key index. So, a DROP
clause for the index must be entered first and then a CHANGE
for the column name can be made along with the establishing of a new index:
ALTER TABLE clients
DROP PRIMARY KEY,
CHANGE cust_id
client_id INT PRIMARY KEY;
The order of these clauses is necessary. The index must be dropped before the column can be renamed. The syntax here is for a PRIMARY KEY
. There are other types of indexes, of course. To change a column that has an index type other than a PRIMARY KEY
. Assuming for a moment that cust_id
has a UNIQUE
index, this is what we would enter to change its name:
ALTER TABLE clients
DROP UNIQUE cust_id
CHANGE cust_id
client_id INT UNIQUE;
Although the index type can be changed easily, MariaDB won't permit you to do so when there are duplicate rows of data and when going from an index that allows duplicates (e.g., INDEX
) to one that doesn't (e.g., UNIQUE
). If you actually do want to eliminate the duplicates, though, you can add the IGNORE
flag to force the duplicates to be deleted:
ALTER IGNORE TABLE clients
DROP INDEX cust_id
CHANGE cust_id
client_id INT UNIQUE;
In this example, we're not only changing the indexed column's name, but we're also changing the index type from INDEX
to UNIQUE
. And, again, the IGNORE
flag tells MariaDB to ignore any records with duplicate values for cust_id
.
The previous sections covered how to make changes to columns in a table. Sometimes you may want to rename a table. To change the name of the clients
table to client_addresses we enter this:
RENAME TABLE clients
TO client_addresses;
The RENAME TABLE statement will also allows a table to be moved to another database just by adding the receiving database's name in front of the new table name, separated by a dot. Of course, you can move a table without renaming it. To move the newly named client_addresses
table to the database db2, we enter this:
RENAME TABLE client_addresses
TO db2.client_addresses;
Finally, with tables that contain data (excluding InnoDB tables), occasionally it's desirable to resort the data within the table. Although the ORDER BY clause in a SELECT statement can do this on the fly as needed, sometimes developers want to do this somewhat permanently to the data within the table based on a particular column or columns. It can be done by entering the following:
ALTER TABLE client_addresses
ORDER BY city, name;
Notice that we're sorting by the city first and then by the client's name. Now when the developer enters a SELECT statement without an ORDER BY clause, the results are already ordered by the default of city and then name, at least until more data is added to the table.
This is not applicable to InnoDB tables, the default, which are ordered according to the clustered index, unless the primary key is defined on the specific columns.
Good planning is certainly important in developing a MariaDB database. However, as you can see, MariaDB is malleable enough that it can be reshaped without much trouble. Just be sure to make a backup before restructuring a table and be sure to check your work and the data when you're finished. With all of this in mind, you should feel comfortable in creating tables since they don't have to be perfect from the beginning.
This page is licensed: CC BY-SA / Gnu FDL
CREATE TABLE t1 ( a INT );
CREATE TABLE t2 ( b INT );
CREATE TABLE student_tests (
name CHAR(10), test CHAR(10),
score TINYINT, test_date DATE
);
INSERT INTO t1 VALUES (1), (2), (3);
INSERT INTO t2 VALUES (2), (4);
INSERT INTO student_tests
(name, test, score, test_date) VALUES
('Chun', 'SQL', 75, '2012-11-05'),
('Chun', 'Tuning', 73, '2013-06-14'),
('Esben', 'SQL', 43, '2014-02-11'),
('Esben', 'Tuning', 31, '2014-02-09'),
('Kaolin', 'SQL', 56, '2014-01-01'),
('Kaolin', 'Tuning', 88, '2013-12-29'),
('Tatiana', 'SQL', 87, '2012-04-28'),
('Tatiana', 'Tuning', 83, '2013-09-30');
CREATE TABLE student_details (
id INT NOT NULL AUTO_INCREMENT, name CHAR(10),
date_of_birth DATE, PRIMARY KEY (id)
);
INSERT INTO student_details (name,date_of_birth) VALUES
('Chun', '1993-12-31'),
('Esben','1946-01-01'),
('Kaolin','1996-07-16'),
('Tatiana', '1988-04-13');
SELECT * FROM student_details;
+----+---------+---------------+
| id | name | date_of_birth |
+----+---------+---------------+
| 1 | Chun | 1993-12-31 |
| 2 | Esben | 1946-01-01 |
| 3 | Kaolin | 1996-07-16 |
| 4 | Tatiana | 1988-04-13 |
+----+---------+---------------+
SELECT * FROM t1 INNER JOIN t2 ON t1.a = t2.b;
SELECT MAX(a) FROM t1;
+--------+
| MAX(a) |
+--------+
| 3 |
+--------+
SELECT MIN(a) FROM t1;
+--------+
| MIN(a) |
+--------+
| 1 |
+--------+
SELECT AVG(a) FROM t1;
+--------+
| AVG(a) |
+--------+
| 2.0000 |
+--------+
SELECT name, MAX(score) FROM student_tests GROUP BY name;
+---------+------------+
| name | MAX(score) |
+---------+------------+
| Chun | 75 |
| Esben | 43 |
| Kaolin | 88 |
| Tatiana | 87 |
+---------+------------+
SELECT name, test, score FROM student_tests ORDER BY score DESC;
+---------+--------+-------+
| name | test | score |
+---------+--------+-------+
| Kaolin | Tuning | 88 |
| Tatiana | SQL | 87 |
| Tatiana | Tuning | 83 |
| Chun | SQL | 75 |
| Chun | Tuning | 73 |
| Kaolin | SQL | 56 |
| Esben | SQL | 43 |
| Esben | Tuning | 31 |
+---------+--------+-------+
SELECT name,test, score FROM student_tests WHERE score=(SELECT MIN(score) FROM student);
+-------+--------+-------+
| name | test | score |
+-------+--------+-------+
| Esben | Tuning | 31 |
+-------+--------+-------+
SELECT name, test, score FROM student_tests st1 WHERE score = (
SELECT MAX(score) FROM student st2 WHERE st1.name = st2.name
);
+---------+--------+-------+
| name | test | score |
+---------+--------+-------+
| Chun | SQL | 75 |
| Esben | SQL | 43 |
| Kaolin | Tuning | 88 |
| Tatiana | SQL | 87 |
+---------+--------+-------+
SELECT CURDATE() AS today;
+------------+
| today |
+------------+
| 2014-02-17 |
+------------+
SELECT name, date_of_birth, TIMESTAMPDIFF(YEAR,date_of_birth,'2014-08-02') AS age
FROM student_details;
+---------+---------------+------+
| name | date_of_birth | age |
+---------+---------------+------+
| Chun | 1993-12-31 | 20 |
| Esben | 1946-01-01 | 68 |
| Kaolin | 1996-07-16 | 18 |
| Tatiana | 1988-04-13 | 26 |
+---------+---------------+------+
SELECT @avg_score:= AVG(score) FROM student_tests;
+-------------------------+
| @avg_score:= AVG(score) |
+-------------------------+
| 67.000000000 |
+-------------------------+
SELECT * FROM student_tests WHERE score > @avg_score;
+---------+--------+-------+------------+
| name | test | score | test_date |
+---------+--------+-------+------------+
| Chun | SQL | 75 | 2012-11-05 |
| Chun | Tuning | 73 | 2013-06-14 |
| Kaolin | Tuning | 88 | 2013-12-29 |
| Tatiana | SQL | 87 | 2012-04-28 |
| Tatiana | Tuning | 83 | 2013-09-30 |
+---------+--------+-------+------------+
SET @count = 0;
SELECT @count := @count + 1 AS counter, name, date_of_birth FROM student_details;
+---------+---------+---------------+
| counter | name | date_of_birth |
+---------+---------+---------------+
| 1 | Chun | 1993-12-31 |
| 2 | Esben | 1946-01-01 |
| 3 | Kaolin | 1996-07-16 |
| 4 | Tatiana | 1988-04-13 |
+---------+---------+---------------+
SELECT table_schema AS `DB`, table_name AS `TABLE`,
ROUND(((data_length + index_length) / 1024 / 1024), 2) `Size (MB)`
FROM information_schema.TABLES
ORDER BY (data_length + index_length) DESC;
+--------------------+---------------------------------------+-----------+
| DB | Table | Size (MB) |
+--------------------+---------------------------------------+-----------+
| wordpress | wp_simple_history_contexts | 7.05 |
| wordpress | wp_posts | 6.59 |
| wordpress | wp_simple_history | 3.05 |
| wordpress | wp_comments | 2.73 |
| wordpress | wp_commentmeta | 2.47 |
| wordpress | wp_simple_login_log | 2.03 |
...
CREATE TABLE t (id INT, f1 VARCHAR(2));
INSERT INTO t VALUES (1,'a'), (2,'a'), (3,'b'), (4,'a');
SELECT * FROM t t1, t t2 WHERE t1.f1=t2.f1 AND t1.id<>t2.id AND t1.id=(
SELECT MAX(id) FROM t tab WHERE tab.f1=t1.f1
);
+------+------+------+------+
| id | f1 | id | f1 |
+------+------+------+------+
| 4 | a | 1 | a |
| 4 | a | 2 | a |
+------+------+------+------+
DELETE FROM t WHERE id IN (
SELECT t2.id FROM t t1, t t2 WHERE t1.f1=t2.f1 AND t1.id<>t2.id AND t1.id=(
SELECT MAX(id) FROM t tab WHERE tab.f1=t1.f1
)
);
Query OK, 2 rows affected (0.120 sec)
SELECT * FROM t;
+------+------+
| id | f1 |
+------+------+
| 3 | b |
| 4 | a |
+------+------
LOAD DATA INFILE '/tmp/prospects.txt'
INTO TABLE prospect_contact
FIELDS TERMINATED BY '|';
LOAD DATA INFILE '/tmp/prospects.txt'
INTO TABLE prospect_contact
FIELDS TERMINATED BY '|'
LINES STARTING BY '"'
TERMINATED BY '"\r\n';
...
LINES STARTING BY '\''
...
LOAD DATA INFILE '/tmp/prospects.txt'
REPLACE INTO TABLE prospect_contact
FIELDS TERMINATED BY '|'
LINES STARTING BY '"'
TERMINATED BY '"\n';
LOAD DATA LOW_PRIORITY INFILE '/tmp/prospects.txt'
...
...
TERMINATED BY 0x0d0a;
...
IGNORE 1 LINES;
LOAD DATA LOW_PRIORITY INFILE '/tmp/prospects.txt'
REPLACE INTO TABLE prospect_contact
FIELDS TERMINATED BY '"'
ENCLOSED BY '"' ESCAPED BY '#'
LINES STARTING BY '"'
TERMINATED BY '"\n'
IGNORE 1 LINES;
LOAD DATA LOW_PRIORITY INFILE '/tmp/prospects.txt'
REPLACE INTO TABLE sales_dept.prospect_contact
FIELDS TERMINATED BY 0x09
ENCLOSED BY '"' ESCAPED BY '#'
TERMINATED BY 0x0d0a
IGNORE 1 LINES
(name_last, name_first, telephone);
mariadb-import --user='marie_dyer' --password='angelle1207' \
--fields-terminated-by=0x09 --lines-terminated-by=0x0d0a \
--replace --low-priority --fields-enclosed-by='"' \
--fields-escaped-by='#' --ignore-lines='1' --verbose \
--columns='name_last, name_first, telephone' \
sales_dept '/tmp/prospect_contact.txt'
mariadb-dump --user='root' --password='geronimo' sales_dept prospect_contact > /tmp/prospects.sql
mariadb --user='marie_dyer' --password='angelle12107' sales_dept < '/tmp/prospects.sql'
mariadb-dump -u marie_dyer -p --no-create-info sales_dept prospect_contact > /tmp/prospects.sql
CREATE TABLE `Employees` (
`ID` TINYINT(3) UNSIGNED NOT NULL AUTO_INCREMENT,
`First_Name` VARCHAR(25) NOT NULL,
`Last_Name` VARCHAR(25) NOT NULL,
`Position` VARCHAR(25) NOT NULL,
`Home_Address` VARCHAR(50) NOT NULL,
`Home_Phone` VARCHAR(12) NOT NULL,
PRIMARY KEY (`ID`)
);
ALTER TABLE Employees ADD PRIMARY KEY(ID);
SELECT t.TABLE_SCHEMA, t.TABLE_NAME
FROM information_schema.TABLES AS t
LEFT JOIN information_schema.KEY_COLUMN_USAGE AS c
ON t.TABLE_SCHEMA = c.CONSTRAINT_SCHEMA
AND t.TABLE_NAME = c.TABLE_NAME
AND c.CONSTRAINT_NAME = 'PRIMARY'
WHERE t.TABLE_SCHEMA != 'information_schema'
AND t.TABLE_SCHEMA != 'performance_schema'
AND t.TABLE_SCHEMA != 'mysql'
AND c.CONSTRAINT_NAME IS NULL;
CREATE TABLE `Employees` (
`ID` TINYINT(3) UNSIGNED NOT NULL,
`First_Name` VARCHAR(25) NOT NULL,
`Last_Name` VARCHAR(25) NOT NULL,
`Position` VARCHAR(25) NOT NULL,
`Home_Address` VARCHAR(50) NOT NULL,
`Home_Phone` VARCHAR(12) NOT NULL,
`Employee_Code` VARCHAR(25) NOT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY (`Employee_Code`)
);
ALTER TABLE Employees ADD UNIQUE `EmpCode`(`Employee_Code`);
CREATE UNIQUE INDEX HomePhone ON Employees(Home_Phone);
CREATE TABLE t1 (a INT NOT NULL, b INT, UNIQUE (a,b));
INSERT INTO t1 VALUES (1,1), (2,2);
SELECT * FROM t1;
+---+------+
| a | b |
+---+------+
| 1 | 1 |
| 2 | 2 |
+---+------+
INSERT INTO t1 VALUES (2,1);
SELECT * FROM t1;
+---+------+
| a | b |
+---+------+
| 1 | 1 |
| 2 | 1 |
| 2 | 2 |
+---+------+
INSERT INTO t1 VALUES (3,NULL), (3, NULL);
SELECT * FROM t1;
+---+------+
| a | b |
+---+------+
| 1 | 1 |
| 2 | 1 |
| 2 | 2 |
| 3 | NULL |
| 3 | NULL |
+---+------+
SELECT (3, NULL) = (3, NULL);
+---------------------- +
| (3, NULL) = (3, NULL) |
+---------------------- +
| 0 |
+---------------------- +
CREATE TABLE Table_1 (
user_name VARCHAR(10),
status ENUM('Active', 'ON-Hold', 'Deleted'),
del CHAR(0) AS (IF(status IN ('Active', 'ON-Hold'),'', NULL)) persistent,
UNIQUE(user_name,del)
)
CREATE TABLE t1 (a INT PRIMARY KEY,
b BLOB,
c1 VARCHAR(1000),
c2 VARCHAR(1000),
c3 VARCHAR(1000),
c4 VARCHAR(1000),
c5 VARCHAR(1000),
c6 VARCHAR(1000),
c7 VARCHAR(1000),
c8 VARCHAR(1000),
c9 VARCHAR(1000),
UNIQUE KEY `b` (b),
UNIQUE KEY `all_c` (c1,c2,c3,c4,c6,c7,c8,c9)) ENGINE=myisam;
SHOW CREATE TABLE t1\G
*************************** 1. row ***************************
TABLE: t1
CREATE TABLE: CREATE TABLE `t1` (
`a` INT(11) NOT NULL,
`b` BLOB DEFAULT NULL,
`c1` VARCHAR(1000) DEFAULT NULL,
`c2` VARCHAR(1000) DEFAULT NULL,
`c3` VARCHAR(1000) DEFAULT NULL,
`c4` VARCHAR(1000) DEFAULT NULL,
`c5` VARCHAR(1000) DEFAULT NULL,
`c6` VARCHAR(1000) DEFAULT NULL,
`c7` VARCHAR(1000) DEFAULT NULL,
`c8` VARCHAR(1000) DEFAULT NULL,
`c9` VARCHAR(1000) DEFAULT NULL,
PRIMARY KEY (`a`),
UNIQUE KEY `b` (`b`) USING HASH,
UNIQUE KEY `all_c` (`c1`,`c2`,`c3`,`c4`,`c6`,`c7`,`c8`,`c9`) USING HASH
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci
CREATE TABLE t2 (a INT NOT NULL, b INT, INDEX (a,b));
INSERT INTO t2 VALUES (1,1), (2,2), (2,2);
SELECT * FROM t2;
+---+------+
| a | b |
+---+------+
| 1 | 1 |
| 2 | 2 |
| 2 | 2 |
+---+------+
This page is licensed: CC BY-SA / Gnu FDL
This article covers
PARTITIONing uses and non-uses
How to Maintain a time-series PARTITIONed table
AUTO_INCREMENT secrets
First, my Opinions on PARTITIONing
Taken from
#1: Don't use until you know how and why it will help.
Don't use PARTITION unless you will have >1M rows
No more than 50 PARTITIONs on a table (open, show table status, etc, are impacted) (fixed in MySQL 5.6.6?; a better fix coming eventually in 5.7)
PARTITION BY RANGE is the only useful method.
SUBPARTITIONs are not useful.
The partition field should not be the field first in any key.
It is OK to have an as the first part of a compound key, or in a non-UNIQUE index.
It is so tempting to believe that PARTITIONing will solve performance problems. But it is so often wrong.
PARTITIONing splits up one table into several smaller tables. But table size is rarely a performance issue. Instead, I/O time and indexes are the issues.
A common fallacy: "Partitioning will make my queries run faster". It won't. Ponder what it takes for a 'point query'. Without partitioning, but with an appropriate index, there is a BTree (the index) to drill down to find the desired row. For a billion rows, this might be 5 levels deep. With partitioning, first the partition is chosen and "opened", then a smaller BTree (of say 4 levels) is drilled down. Well, the savings of the shallower BTree is consumed by having to open the partition. Similarly, if you look at the disk blocks that need to be touched, and which of those are likely to be cached, you come to the conclusion that about the same number of disk hits is likely. Since disk hits are the main cost in a query, Partitioning does not gain any performance (at least for this typical case). The 2D case (below) gives the main contradiction to this discussion.
Use case #1 -- time series. Perhaps the most common use case where PARTITIONing shines is in a dataset where "old" data is periodically deleted from the table. RANGE PARTITIONing by day (or other unit of time) lets you do a nearly instantaneous DROP PARTITION plus REORGANIZE PARTITION instead of a much slower DELETE. Much of this blog is focused on this use case. This use case is also discussed in
The big win for Case #1: DROP PARTITION is a lot faster than DELETEing a lot of rows.
Use case #2 -- 2-D index. INDEXes are inherently one-dimensional. If you need two "ranges" in the WHERE clause, try to migrate one of them to PARTITIONing.
Finding the nearest 10 pizza parlors on a map needs a 2D index. Partition pruning sort of gives a second dimension. See Latitude/Longitude Indexing That uses PARTITION BY RANGE(latitude) together with PRIMARY KEY(longitude, ...)
The big win for Case #2: Scanning fewer rows.
Use case #3 -- hot spot. This is a bit complicated to explain. Given this combination:
A table's index is too big to be cached, but the index for one partition is cacheable, and
The index is randomly accessed, and
Data ingestion would normally be I/O bound due to updating the index Partitioning can keep all the index "hot" in RAM, thereby avoiding a lot of I/O.
The big win for Case #3: Improving caching to decrease I/O to speed up operations.
For to work (in any table), it must be the first field in some index. Period. There are no other requirements on indexing it.
Being the first field in some index lets the engine find the 'next' value when opening the table.
AUTO_INCREMENT need not be UNIQUE. What you lose: prevention of explicitly inserting a duplicate id. (This is rarely needed, anyway.)
Examples (where id is AUTO_INCREMENT):
PRIMARY KEY (...), INDEX(id)
PRIMARY KEY (...), UNIQUE(id, partition_key) -- not useful
INDEX(id), INDEX(...) (but no UNIQUE keys)
PRIMARY KEY(id), ... -- works only if id is the partition key (not very useful)
Let's focus on the maintenance task involved in Case #1, as described above.
You have a large table that is growing on one end and being pruned on the other. Examples include news, logs, and other transient information. PARTITION BY RANGE is an excellent vehicle for such a table.
DROP PARTITION is much faster than DELETE. (This is the big reason for doing this flavor of partitioning.)
Queries often limit themselves to 'recent' data, thereby taking advantage of "partition pruning".
Depending on the type of data, and how long before it expires, you might have daily or weekly or hourly (etc) partitions.
There is no simple SQL statement to "drop partitions older than 30 days" or "add a new partition for tomorrow". It would be tedious to do this by hand every day.
After which you have...
Perhaps you noticed some odd things in the example. Let me explain them.
Partition naming: Make them useful.
from20120415 ... 04-16: Note that the LESS THAN is the next day's date
The "start" partition: See paragraph below.
The "future" partition: This is normally empty, but it can catch overflows; more later.
The range key (dt) must be included in any PRIMARY or UNIQUE key.
The range key (dt) should be last in any keys it is in -- You have already "pruned" with it; it is almost useless in the index, especially at the beginning.
DATETIME, etc -- I picked this datatype because it is typical for a time series. Newer MySQL versions allow TIMESTAMP. INT could be used; etc.
There is an extra day (03-16 thru 04-16): The latest day is only partially full.
Why the bogus "start" partition? If an invalid datetime (Feb 31) were to be used, the datetime would turn into NULL. NULLs are put into the first partition. Since any SELECT could have an invalid date (yeah, this stretching things), the partition pruner always includes the first partition in the resulting set of partitions to search. So, if the SELECT must scan the first partition, it would be slightly more efficient if that partition were empty. Hence the bogus "start" partition. Longer discussion, by The Data Charmer 5.5 eliminates the bogus check, but only if you switch to a new syntax:
More on the "future" partition. Sooner or later the cron/EVENT to add tomorrow's partition will fail to run. The worst that could happen is for tomorrow's data to be lost. The easiest way to prevent that is to have a partition ready to catch it, even if this partition is normally always empty.
Having the "future" partition makes the ADD PARTITION script a little more complex. Instead, it needs to take tomorrow's data from "future" and put it into a new partition. This is done with the REORGANIZE command shown. Normally nothing need be moved, and the ALTER takes virtually zero time.
DROP if the oldest partition is "too old".
Add 'tomorrow' near the end of today, but don't try to add it twice.
Do not count partitions -- there are two extra ones. Use the partition names or information_schema.PARTITIONS.PARTITION_DESCRIPTION.
DROP/Add only once in the script. Rerun the script if you need more.
Run the script more often than necessary. For daily partitions, run the script twice a day, or even hourly. Why? Automatic repair.
As I have said many times, in many places, BY RANGE is perhaps the only useful variant. And a time series is the most common use for PARTITIONing.
(as discussed here) DATETIME/DATE with TO_DAYS()
DATETIME/DATE with TO_DAYS(), but with 7-day intervals
TIMESTAMP with TO_DAYS(). (version 5.1.43 or later)
PARTITION BY RANGE COLUMNS(DATETIME) (5.5.0)
PARTITION BY RANGE(TIMESTAMP) (version 5.5.15 / 5.6.3)
PARTITION BY RANGE(TO_SECONDS()) (5.6.0)
INT UNSIGNED with constants computed as unix timestamps.
INT UNSIGNED with constants for some non-time-based series.
MEDIUMINT UNSIGNED containing an "hour id": FLOOR(FROM_UNIXTIME(timestamp) / 3600)
Months, Quarters, etc: Concoct a notation that works.
How many partitions?
Under, say, 5 partitions -- you get very little of the benefits.
Over, say, 50 partitions, and you hit inefficiencies elsewhere.
Certain operations (SHOW TABLE STATUS, opening the table, etc) open every partition.
, before version 5.6.6, would lock all partitions before pruning!
Partition pruning does not happen on INSERTs (until Version 5.6.7), so INSERT needs to open all the partitions.
A possible 2-partition use case:
8192 partitions is a hard limit (1024 before ).
Before "native partitions" (5.7.6), each partition consumed a chunk of memory.
The complexity of the code is in the discovery of the PARTITION names, especially of the oldest and the 'next'.
To run the demo,
Install Perl and DBIx::DWIW (from CPAN).
copy the txt file (link above) to demo_part_maint.pl
execute perl demo_part_maint.pl to get the rest of the instructions
The program will generate and execute (when needed) either of these:
Original writing -- Oct, 2012; Use cases added: Oct, 2014; Refreshed: June, 2015; 8.0: Sep, 2016
PARTITIONing requires at least MySQL 5.1
The tips in this document apply to MySQL, MariaDB, and Percona.
Future (as envisioned in 2016):
MySQL 5.7.6 has "native partitioning for InnoDB".
FOREIGN KEY support, perhaps in a later 8.0.xx.
"GLOBAL INDEX" -- this would avoid the need for putting the partition key in every unique index, but make DROP PARTITION costly. This is farther into the future.
MySQL 8.0, released Sep, 2016, not yet GA)
Only InnoDB tables can be partitioned -- MariaDB is likely to continue maintaining Partitioning on non-InnoDB tables, but Oracle is clearly not.
Some of the problems having lots of partitions are lessened by the Data-Dictionary-in-a-table.
Native partitioning will give:
This will improve performance slightly by combining two "handlers" into one.
Decreased memory usage, especially when using a large number of partitions.
Rick James graciously allowed us to use this article in the documentation.
has other useful tips, how-tos, optimizations, and debugging tips.
Original source:
This page is licensed: CC BY-SA / Gnu FDL
ALTER TABLE tbl
DROP PARTITION from20120314;
ALTER TABLE tbl
REORGANIZE PARTITION future INTO (
PARTITION from20120415 VALUES LESS THAN (TO_DAYS('2012-04-16')),
PARTITION future VALUES LESS THAN MAXVALUE);
CREATE TABLE tbl (
dt DATETIME NOT NULL, -- or DATE
...
PRIMARY KEY (..., dt),
UNIQUE KEY (..., dt),
...
)
PARTITION BY RANGE (TO_DAYS(dt)) (
PARTITION START VALUES LESS THAN (0),
PARTITION from20120315 VALUES LESS THAN (TO_DAYS('2012-03-16')),
PARTITION from20120316 VALUES LESS THAN (TO_DAYS('2012-03-17')),
...
PARTITION from20120414 VALUES LESS THAN (TO_DAYS('2012-04-15')),
PARTITION from20120415 VALUES LESS THAN (TO_DAYS('2012-04-16')),
PARTITION future VALUES LESS THAN MAXVALUE
);
PARTITION BY RANGE COLUMNS(dt) (
PARTITION day_20100226 VALUES LESS THAN ('2010-02-27'), ...
ALTER TABLE tbl REORGANIZE PARTITION
future
INTO (
PARTITION from20150606 VALUES LESS THAN (736121),
PARTITION future VALUES LESS THAN MAXVALUE
)
ALTER TABLE tbl
DROP PARTITION from20150603
The simplest way to retrieve data from MariaDB is to use the SELECT statement. Since the SELECT statement is an essential SQL statement, it has many options available with it. It's not necessary to know or use them all—you could execute very basic SELECT statements if that satisfies your needs. However, as you use MariaDB more, you may need more powerful SELECT statements. In this article we will go through the basics of SELECT and will progress to more involved SELECT statements;we will move from the beginner level to the more intermediate and hopefully you will find some benefit from this article regardless of your skill level. For absolute beginners who are just starting with MariaDB, you may want to read the MariaDB Basics article.
In order to follow the examples that follow, you can create and pre-populate the tables, as follows:
CREATE OR REPLACE TABLE books (
isbn CHAR(20) PRIMARY KEY,
title VARCHAR(50),
author_id INT,
publisher_id INT,
year_pub CHAR(4),
description TEXT );
CREATE OR REPLACE TABLE authors
(author_id INT AUTO_INCREMENT PRIMARY KEY,
name_last VARCHAR(50),
name_first VARCHAR(50),
country VARCHAR(50) );
INSERT INTO authors (name_last, name_first, country) VALUES
('Kafka', 'Franz', 'Czech Republic'),
('Dostoevsky', 'Fyodor', 'Russia');
INSERT INTO books (title, author_id, isbn, year_pub) VALUES
('The Trial', 1, '0805210407', '1995'),
('The Metamorphosis', 1, '0553213695', '1995'),
('America', 2, '0805210644', '1995'),
('Brothers Karamozov', 2, '0553212168', ''),
('Crime & Punishment', 2, '0679420290', ''),
('Crime & Punishment', 2, '0553211757', ''),
('Idiot', 2, '0192834118', ''),
('Notes from Underground', 2, '067973452X', '');
If you are unclear what these statements do, review the MariaDB Primer and MariaDB Basics tutorials.
The basic, minimal elements of the SELECT statement call for the keyword SELECT
, of course, the columns to select or to retrieve, and the table from which to retrieve rows of data. Actually, for the columns to select, we can use the asterisk as a wildcard to select all columns in a particular table. Using a database from a fictitious bookstore, we might enter the following SQL statement to get a list of all columns and rows in a table containing information on books:
SELECT * FROM books;
+------------+------------------------+-----------+--------------+----------+-------------+
| isbn | title | author_id | publisher_id | year_pub | description |
+------------+------------------------+-----------+--------------+----------+-------------+
| 0192834118 | Idiot | 2 | NULL | | NULL |
| 0553211757 | Crime & Punishment | 2 | NULL | | NULL |
| 0553212168 | Brothers Karamozov | 2 | NULL | | NULL |
| 0553213695 | The Metamorphosis | 1 | NULL | 1995 | NULL |
| 0679420290 | Crime & Punishment | 2 | NULL | | NULL |
| 067973452X | Notes from Underground | 2 | NULL | | NULL |
| 0805210407 | The Trial | 1 | NULL | 1995 | NULL |
| 0805210644 | America | 2 | NULL | 1995 | NULL |
+------------+------------------------+-----------+--------------+----------+-------------+
8 rows in set (0.001 sec)
This will retrieve all of the data contained in the books
table. As you can see, not all values have been populated. If we want to retrieve only certain columns, we would list them in place of the asterisk in a comma-separated list like so:
SELECT isbn, title, author_id
FROM books;
+------------+------------------------+-----------+
| isbn | title | author_id |
+------------+------------------------+-----------+
| 0192834118 | Idiot | 2 |
| 0553211757 | Crime & Punishment | 2 |
| 0553212168 | Brothers Karamozov | 2 |
| 0553213695 | The Metamorphosis | 1 |
| 0679420290 | Crime & Punishment | 2 |
| 067973452X | Notes from Underground | 2 |
| 0805210407 | The Trial | 1 |
| 0805210644 | America | 2 |
+------------+------------------------+-----------+
8 rows in set (0.001 sec)
This narrows the width of the results set by retrieving only three columns, but it still retrieves all of the rows in the table. If the table contains thousands of rows of data, this may be more data than we want. If we want to limit the results to just a few books, say five, we would include what is known as a LIMIT clause:
SELECT isbn, title, author_id
FROM books
LIMIT 5;
+------------+--------------------+-----------+
| isbn | title | author_id |
+------------+--------------------+-----------+
| 0192834118 | Idiot | 2 |
| 0553211757 | Crime & Punishment | 2 |
| 0553212168 | Brothers Karamozov | 2 |
| 0553213695 | The Metamorphosis | 1 |
| 0679420290 | Crime & Punishment | 2 |
+------------+--------------------+-----------+
5 rows in set (0.001 sec)
This will give us the first five rows found in the table. If we want to get the next ten found, we would add a starting point parameter just before the number of rows to display, separated by a comma:
SELECT isbn, title, author_id
FROM books
LIMIT 5, 10;
+------------+------------------------+-----------+
| isbn | title | author_id |
+------------+------------------------+-----------+
| 067973452X | Notes from Underground | 2 |
| 0805210407 | The Trial | 1 |
| 0805210644 | America | 2 |
+------------+------------------------+-----------+
3 rows in set (0.001 sec)
In this case, there are only three further records.
The previous statements have narrowed the number of columns and rows retrieved, but they haven't been very selective. Suppose that we want only books written by a certain author, say Dostoevsky. Looking in the authors table we find that his author identification number is 2. Using a WHERE
clause, we can retrieve a list of books from the database for this particular author like so:
SELECT isbn, title
FROM books
WHERE author_id = 2
LIMIT 5;
+------------+------------------------+
| isbn | title |
+------------+------------------------+
| 0192834118 | Idiot |
| 0553211757 | Crime & Punishment |
| 0553212168 | Brothers Karamozov |
| 0679420290 | Crime & Punishment |
| 067973452X | Notes from Underground |
+------------+------------------------+
5 rows in set (0.000 sec)
I removed the author_id from the list of columns to select, but left the basic LIMIT clause in because we want to point out that the syntax is fairly strict on ordering of clauses and flags. You can't enter them in any order. You'll get an error in return.
The SQL statements we've looked at thus far will display the titles of books in the order in which they're found in the database. If we want to put the results in alphanumeric order based on the values of the title column, for instance, we would add an ORDER BY clause like this:
SELECT isbn, title
FROM books
WHERE author_id = 2
ORDER BY title ASC
LIMIT 5;
+------------+--------------------+
| isbn | title |
+------------+--------------------+
| 0805210644 | America |
| 0553212168 | Brothers Karamozov |
| 0553211757 | Crime & Punishment |
| 0679420290 | Crime & Punishment |
| 0192834118 | Idiot |
+------------+--------------------+
5 rows in set (0.001 sec)
Notice that the ORDER BY clause goes after the WHERE
clause and before the LIMIT clause. Not only will this statement display the rows in order by book title, but it will retrieve only the first five based on the ordering. That is to say, MariaDB will first retrieve all of the rows based on the WHERE
clause, order the data based on the ORDER BY
clause, and then display a limited number of rows based on the LIMIT clause. Hence the reason for the order of clauses. You may have noticed that we slipped in the ASC flag. It tells MariaDB to order the rows in ascending order for the column name it follows. It's not necessary, though, since ascending order is the default. However, if we want to display data in descending order, we would replace the flag with DESC
. To order by more than one column, additional columns may be given in the ORDER BY
clause in a comma separated list, each with the ASC
or DESC
flags if preferred.
So far we've been working with one table of data containing information on books for a fictitious bookstore. A database will usually have more than one table, of course. In this particular database, there's also one called authors in which the name and other information on authors is contained. To be able to select data from two tables in one SELECT statement, we will have to tell MariaDB that we want to join the tables and will need to provide a join point. This can be done with a JOIN clause as shown in the following SQL statement, with the results following it:
SELECT isbn, title,
CONCAT(name_first, ' ', name_last) AS author
FROM books
JOIN authors USING (author_id)
WHERE name_last = 'Dostoevsky'
ORDER BY title ASC
LIMIT 5;
+------------+--------------------+-------------------+
| isbn | title | author |
+------------+--------------------+-------------------+
| 0805210644 | America | Fyodor Dostoevsky |
| 0553212168 | Brothers Karamozov | Fyodor Dostoevsky |
| 0553211757 | Crime & Punishment | Fyodor Dostoevsky |
| 0679420290 | Crime & Punishment | Fyodor Dostoevsky |
| 0192834118 | Idiot | Fyodor Dostoevsky |
+------------+--------------------+-------------------+
5 rows in set (0.00 sec)
Our SELECT statement is getting hefty, but it's the same one to which we've been adding. Don't let the clutter fluster you. Looking for the new elements, let's focus on the JOIN clause first. There are a few possible ways to construct a join. This method works if both tables contain a column of the same name and value. Otherwise you'll have to redo the JOIN
clause to look something like this:
...
JOIN authors ON author_id = row_id
...
This excerpt is based on the assumption that the key field in the authors table is not called author_id, but row_id instead. There's much more that can be said about joins, but that would make for a much longer article. If you want to learn more on joins, look at MariaDB's documentation page on JOIN
syntax.
Looking again at the last full SQL statement above, you must have spotted the CONCAT() function that we added to the on-going example statement. This string function takes the values of the columns and strings given and pastes them together, to give one neat field in the results. We also employed the AS parameter to change the heading of the results set for the field to author. This is much tidier. Since we joined the books and the authors tables together, we were able to search for books based on the author's last name rather than having to look up the author ID first. This is a much friendlier method, albeit more complicated. Incidentally, we can have MariaDB check columns from both tables to narrow our search. We would just add column = value pairs, separated by commas in the WHERE clause. Notice that the string containing the author's name is wrapped in quotes—otherwise, the string would be considered a column name and we'd get an error.
The name Dostoevsky is sometimes spelled Dostoevskii, as well as a few other ways. If we're not sure how it's spelled in the authors table, we could use the LIKE operator instead of the equal sign, along with a wildcard. If we think the author's name is probably spelled either of the two ways mentioned, we could enter something like this:
SELECT isbn, title,
CONCAT(name_first, ' ', name_last) AS author
FROM books
JOIN authors USING (author_id)
WHERE name_last LIKE 'Dostoevsk%'
ORDER BY title ASC
LIMIT 5;
+------------+--------------------+-------------------+
| isbn | title | author |
+------------+--------------------+-------------------+
| 0805210644 | America | Fyodor Dostoevsky |
| 0553212168 | Brothers Karamozov | Fyodor Dostoevsky |
| 0553211757 | Crime & Punishment | Fyodor Dostoevsky |
| 0679420290 | Crime & Punishment | Fyodor Dostoevsky |
| 0192834118 | Idiot | Fyodor Dostoevsky |
+------------+--------------------+-------------------+
5 rows in set (0.001 sec)
This will match any author last name starting with Dostoevsk. Notice that the wildcard here is not an asterisk, but a percent-sign.
There are many flags or parameters that can be used in a SELECT statement. To list and explain all of them with examples would make this a very lengthy article. The reality is that most people never use some of them anyway. So, let's take a look at a few that you may find useful as you get more involved with MariaDB or if you work with large tables on very active servers.
The first flag that may be given, it goes immediately after the SELECT
keyword, is ALL
. By default, all rows that meet the requirements of the various clauses given are selected, so this isn't necessary. If instead we would only want the first occurrence of a particular criteria to be displayed, we could add the DISTINCT option. For instance, for authors like Dostoevsky there are several printings of a particular title. In the results shown earlier you may have noticed that there were two copies of Crime & Punishment listed, however they have different ISBN numbers and different publishers. Suppose that for our search we only want one row displayed for each title. We could do that like so:
SELECT DISTINCT title
FROM books
JOIN authors USING (author_id)
WHERE name_last = 'Dostoevsky'
ORDER BY title;
+------------------------+
| title |
+------------------------+
| America |
| Brothers Karamozov |
| Crime & Punishment |
| Idiot |
| Notes from Underground |
+------------------------+
We've thinned out the ongoing SQL statement a bit for clarity. This statement will result in only one row displayed for Crime & Punishment and it are the first one found.
If we're retrieving data from an extremely busy database, by default any other SQL statements entered simultaneously which are changing or updating data are executed before a SELECT statement. SELECT
statements are considered to be of lower priority. However, if we would like a particular SELECT
statement to be given a higher priority, we can add the keyword HIGH_PRIORITY. Modifying the previous SQL statement for this factor, we would enter it like this:
SELECT DISTINCT HIGH_PRIORITY title
FROM books
JOIN authors USING (author_id)
WHERE name_last = 'Dostoevsky'
ORDER BY title;
+------------------------+
| title |
+------------------------+
| America |
| Brothers Karamozov |
| Crime & Punishment |
| Idiot |
| Notes from Underground |
+------------------------+
You may have noticed in the one example earlier in which the results are shown, that there's a status line displayed that specifies the number of rows in the results set. This is less than the number of rows that were found in the database that met the statement's criteria. It's less because we used a LIMIT clause. If we add the SQL_CALC_FOUND_ROWS flag just before the column list, MariaDB will calculate the number of columns found even if there is a LIMIT
clause.
SELECT SQL_CALC_FOUND_ROWS isbn, title
FROM books
JOIN authors USING (author_id)
WHERE name_last = 'Dostoevsky'
LIMIT 5;
+------------+------------------------+
| isbn | title |
+------------+------------------------+
| 0192834118 | Idiot |
| 0553211757 | Crime & Punishment |
| 0553212168 | Brothers Karamozov |
| 0679420290 | Crime & Punishment |
| 067973452X | Notes from Underground |
+------------+------------------------+
5 rows in set (0.001 sec)
To retrieve this information, though, we will have to use the FOUND_ROWS() function like so:
SELECT FOUND_ROWS();
+--------------+
| FOUND_ROWS() |
+--------------+
| 6 |
+--------------+
1 row in set (0.000 sec
This value is temporary and are lost if the connection is terminated. It cannot be retrieved by any other client session. It relates only to the current session and the value for the variable when it was last calculated.
There are several more parameters and possibilities for the SELECT statement that we had to skip to keep this article a reasonable length. A popular one that we left out is the GROUP BY clause for calculating aggregate data for columns (e.g., an average). There are several flags for caching results and a clause for exporting a results set to a text file. If you would like to learn more about SELECT and all of the options available, look at the on-line documentation for SELECT statements.
This page is licensed: CC BY-SA / Gnu FDL
The recording of date and time in a MariaDB database is a very common requirement. For gathering temporal data, one needs to know which type of columns to use in a table. More importantly is knowing how to record chronological data and how to retrieve it in various formats. Although this is a seemingly basic topic, there are many built-in time functions that can be used for more accurate SQL statements and better formatting of data. In this article we will explore these various aspects of how to do time with MariaDB.
Since date and time are only numeric strings, they can be stored in a regular character column. However, by using temporal data type columns, you can make use of several built-in functions offered by MariaDB. Currently, there are five temporal data types available: DATE
, TIME
, DATETIME
, TIMESTAMP
, and YEAR
. The DATE
column type is for recording the date only and is basically in this format: yyyy-mm-dd
. The TIME
column type is for recording time in this format: hhh:mm:ss
. To record a combination of date and time, there is the DATETIME
column type: yyyy-mm-dd hh:mm:ss
.
The TIMESTAMP
column is similar to DATETIME
, but it's a little limited in its range of allowable time. It starts at the Unix epoch time (1970-01-01) and ends on 2106-02-07. Finally, the YEAR
data type is for recording only the year in a column: yy
or yyyy
. For the examples in this article, DATE
, TIME
, and DATETIME
columns are used. The database that are referenced is for a fictitious psychiatry practice that keeps track of its patients and billable hours in MariaDB.
The TIMESTAMP
column is similar to DATETIME
, but it's a little limited in its range of allowable time. It starts at the Unix epoch time (i.e., 1970-01-01) and ends on 2038-01-19. Finally, the YEAR
data type is for recording only the year in a column: yy
or yyyy
. For the examples in this article, DATE
, TIME
, and DATETIME
columns are used. The database that are referenced is for a fictitious psychiatry practice that keeps track of its patients and billable hours in MariaDB.
To record the current date and time in a MariaDB table, there are a few built-in functions that may be used. First, to record the date there are the functions CURRENT_DATE and CURDATE( ) (depending on your style), which both produce the same results (e.g., 2017-08-01). Notice that CURDATE( ) requires parentheses and the other does not. With many functions a column name or other variables are placed inside of the parentheses to get a result. With functions like CURDATE( ), there is nothing that may go inside the parenthesis. Since these two functions retrieve the current date in the format of the DATE
column type, they can be used to fill in a DATE
column when inserting a row:
INSERT INTO billable_work
(doctor_id, patient_id, session_date)
VALUES('1021', '1256', CURRENT_DATE);
The column session_date is a DATE
column. Notice that there are no quotes around the date function. If there were it would be taken as a literal value rather than a function. Incidentally, I've skipped discussing how the table was set up. If you're not familiar with how to set up a table, you may want to read the MariaDB Basics article. To see what was just recorded by the INSERT statement above, the following may be entered (results follow):
SELECT rec_id, doctor_id,
patient_id, session_date
FROM billable_work
WHERE rec_id=LAST_INSERT_ID();
+--------+-----------+------------+--------------+
| rec_id | doctor_id | patient_id | session_date |
+--------+-----------+------------+--------------+
| 2462 | 1021 | 1256 | 2017-08-23 |
+--------+-----------+------------+--------------+
Notice in the billable_work table that the primary key column (i.e., rec_id
) is an automatically generated and incremental number column (i.e., AUTO_INCREMENT
). As long as another record is not created or the user does not exit from the mariadb client or otherwise end the session, the LAST_INSERT_ID( ) function will retrieve the value of the rec_id
for the last record entered by the user.
To record the time of an appointment for a patient in a time data type column, CURRENT_TIME or CURTIME( ) are used in the same way to insert the time. The following is entered to update the row created above to mark the starting time of the appointment—another SELECT statement follows with the results:
UPDATE billable_work
SET session_time=CURTIME()
WHERE rec_id='2462';
SELECT patient_id, session_date, session_time
FROM billable_work
WHERE rec_id='2462';
+------------+--------------+--------------+
| patient_id | session_date | session_time |
+------------+--------------+--------------+
| 1256 | 2017-08-23 | 10:30:23 |
+------------+--------------+--------------+
The column session_time is a time column. To record the date and time together in the same column, CURRENT_TIMESTAMP or SYSDATE( ) or NOW( ) can be used. All three functions produce the same time format: yyyy-mm-dd hh:mm:ss
. Therefore, the column's data type would have to be DATETIME
to use them.
Although MariaDB records the date in a fairly agreeable format, you may want to present the date when it's retrieved in a different format. Or, you may want to extract part of the date, such as only the day of the month. There are many functions for reformatting and selectively retrieving date and time information. To start off with, let's select a column with a data type of DATE
and look at the functions available for retrieving each component. To extract the year, there's the YEAR( ) function. For extracting just the month, the MONTH( ) function could be called upon. And to grab the day of the month, DAYOFMONTH( ) will work. Using the record entered above, here's what an SQL statement and its results would look like in which the session date is broken up into separate parts, but in a different order:
SELECT MONTH(session_date) AS Month,
DAYOFMONTH(session_date) AS Day,
YEAR(session_date) AS Year
FROM billable_work
WHERE rec_id='2462';
+-------+------+------+
| Month | Day | Year |
+-------+------+------+
| 8 | 23 | 2017 |
+-------+------+------+
For those who aren't familiar with the keyword AS
, it's used to label a column's output and may be referenced within an SQL statement. Splitting up the elements of a date can be useful in analyzing a particular element. If the bookkeeper of the fictitious psychiatry office needed to determine if the day of the week of each session was on a Saturday because the billing rate would be higher (time and a half), the DAYOFWEEK( ) function could be used. To spice up the examples, let's wrap the date function up in an IF( ) function that tests for the day of the week and sets the billing rate accordingly.
SELECT patient_id AS 'Patient ID',
session_date AS 'Date of Session',
IF(DAYOFWEEK(session_date)=6, 1.5, 1.0)
AS 'Billing Rate'
FROM billable_work
WHERE rec_id='2462';
+-------------+-----------------+--------------+
| Patient ID | Date of Session | Billing Rate |
+-------------+-----------------+--------------+
| 1256 | 2017-08-23 | 1.5 |
+-------------+-----------------+--------------+
Since we've slipped in the IF( ) function, we should explain it's format. The test condition is listed first within the parentheses. In this case, the test is checking if the session date is the sixth day of the week. Then, what MariaDB should display is given if the test passes, followed by the result if it fails.
Similar to the DAYOFWEEK( ) function, there's also WEEKDAY( ). The only difference is that for DAYOFWEEK( ) the first day of the week is Sunday—with WEEKDAY( ) the first day is Monday. Both functions represent the first day with 0 and the last with 6
. Having Saturday and Sunday symbolized by 5
and 6
can be handy in constructing an IF statement that has a test component like "WEEKDAY(session_date) > 4
" to determine if a date is a weekend day. This is cleaner than testing for values of 0
and 6
.
There is a function for determining the day of the year: DAYOFYEAR( ). It's not used often, but it is available if you ever need it. Occasionally, though, knowing the quarter of a year for a date can be useful for financial accounting. Rather than set up a formula in a script to determine the quarter, the QUARTER( ) function can do this easily. For instance, suppose an accountant wants a list of a doctor's sessions for each patient for the previous quarter. These three SQL statements could be entered in sequence to achieve the results that follow:
SET @LASTQTR:=IF((QUARTER(CURDATE())-1)=0,
4, QUARTER(CURDATE())-1);
SET @YR:=IF(@LASTQTR=4,
YEAR(NOW())-1, YEAR(NOW()));
SELECT patient_id AS 'Patient ID',
COUNT(session_time)
AS 'Number of Sessions'
FROM billable_work
WHERE QUARTER(session_date) = @LASTQTR
AND YEAR(session_date) = @YR
AND doctor_id='1021'
GROUP BY patient_id
ORDER BY patient_id LIMIT 5;
+------------+--------------------+
| Patient ID | Number of Sessions |
+------------+--------------------+
| 1104 | 10 |
| 1142 | 7 |
| 1203 | 18 |
| 1244 | 6 |
| 1256 | 12 |
+------------+--------------------+
This example is the most complicated so far. But it's not too difficult to understand if we pull it apart. The first SQL statement sets up a user variable containing the previous quarter (i.e., 1, 2, 3, or 4). This variable are needed in the other two statements. The IF( ) clause in the first statement checks if the quarter of the current date minus one is zero. It will equal zero when it's run during the first quarter of a year. During a first quarter, of course, the previous quarter is the fourth quarter of the previous year. So, if the equation equals zero, then the variable @LASTQTR
is set to 4
. Otherwise, @LASTQTR
is set to the value of the current quarter minus one. The second statement is necessary to ensure that the records for the correct year are selected. So, if @LASTQTR
equals four, then @YR
needs to equal last year. If not, @YR
is set to the current year. With the user variables set to the correct quarter and year, the SELECT statement can be entered. The COUNT( ) function counts the number of appointments that match the WHERE
clause for each patient based on the GROUP BY clause. The WHERE
clause looks for sessions with a quarter that equals @LASTQTR
and a year that equals @YR
, as well as the doctor's identification number. In summary, what we end up with is a set of SQL statements that retrieve the desired information regardless of which quarter or year it's entered.
The last section covered how to retrieve pieces of a date column. Now let's look at how to do the same with a time column. To extract just the hour of a time saved in MariaDB, the HOUR( ) function could be used. For the minute and second, there's MINUTE( ) and SECOND( ). Let's put them all together in one straightforward SELECT statement:
SELECT HOUR(session_time) AS Hour,
MINUTE(session_time) AS Minute,
SECOND(session_time) AS Second
FROM billable_work
WHERE rec_id='2462';
+------+--------+--------+
| Hour | Minute | Second |
+------+--------+--------+
| 10 | 30 | 00 |
+------+--------+--------+
All of the examples given so far have involved separate columns for date and time. The EXTRACT( ) function, however, will allow a particular component to be extracted from a combined column type (i.e., DATETIME
or TIMESTAMP
). The format is EXTRACT(date_type FROM date_column)
where date_type is the component to retrieve and date_column is the name of the column from which to extract data. To extract the year, the date_type would be YEAR
; for month, MONTH
is used; and for day, there's DAY
. To extract time elements, HOUR
is used for hour, MINUTE
for minute, and SECOND
for second. Although that's all pretty simple, let's look at an example. Suppose the table billable_work has a column called appointment
(a datetime
column) that contains the date and time for which the appointment was scheduled (as opposed to the time it actually started in session_time
). To get the hour and minute for a particular date, the following SQL statement will suffice:
SELECT patient_name AS Patient,
EXTRACT(HOUR FROM appointment) AS Hour,
EXTRACT(MINUTE FROM appointment) AS Minute
FROM billable_work, patients
WHERE doctor_id='1021'
AND EXTRACT(MONTH FROM appointment)='8'
AND EXTRACT(DAY FROM appointment)='30'
AND billable_work.patient_id =
patients.patient_id;
This statement calls upon another table (patients
) which holds patient information such as their names. It requires a connecting point between the tables (i.e., the patient_id
from each table). If you're confused on how to form relationships between tables in a SELECT statement, you may want to go back and read the Getting Data from MariaDB article. The SQL statement above would be used to retrieve the appointments for one doctor for one day, giving results like this:
+-------------------+------+--------+
| Patient | Hour | Minute |
+-------------------+------+--------+
| Michael Zabalaoui | 10 | 00 |
| Jerry Neumeyer | 11 | 00 |
| Richard Stringer | 13 | 30 |
| Janice Sogard | 14 | 30 |
+-------------------+------+--------+
In this example, the time elements are separated and they don't include the date. With the EXTRACT( ) function, however, you can also return combined date and time elements. There is DAY_HOUR
for the day and hour; there's DAY_MINUTE
for the day, hour, and minute; DAY_SECOND
for day, hour, minute, and second; and YEAR_MONTH
for year and month. There are also some time only combinations: HOUR_MINUTE
for hour and minute; HOUR_SECOND
for hour, minute, and second; and MINUTE_SECOND
for minute and second. However, there's not a MONTH_DAY
to allow the combining of the two extracts in the WHERE
clause of the last SELECT statement above. Nevertheless, we'll modify the example above and use the HOUR_MINUTE
date_type to retrieve the hour and minute in one resulting column. It would only require the second and third lines to be deleted and replaced with this:
...
EXTRACT(HOUR_MINUTE FROM appointment)
AS Appointment
...
+-------------------+-------------+
| Patient | Appointment |
+-------------------+-------------+
| Michael Zabalaoui | 1000 |
| Jerry Neumeyer | 1100 |
| Richard Stringer | 1330 |
| Janice Sogard | 1430 |
+-------------------+-------------+
The problem with this output, though, is that the times aren't very pleasing looking. For more natural date and time displays, there are a few simple date formatting functions available and there are the DATE_FORMAT( ) and TIME_FORMAT( ) functions.
The simple functions that we mentioned are used for reformatting the output of days and months. To get the date of patient sessions for August, but in a more wordier format, MONTHNAME( ) and DAYNAME( ) could be used:
SELECT patient_name AS Patient,
CONCAT(DAYNAME(appointment), ' - ',
MONTHNAME(appointment), ' ',
DAYOFMONTH(appointment), ', ',
YEAR(appointment)) AS Appointment
FROM billable_work, patients
WHERE doctor_id='1021'
AND billable_work.patient_id =
patients.patient_id
AND appointment>'2017-08-01'
AND appointment<'2017-08-31'
LIMIT 1;
+-------------------+-----------------------------+
| Patient | Appointment |
+-------------------+-----------------------------+
| Michael Zabalaoui | Wednesday - August 30, 2017 |
+-------------------+-----------------------------+
In this statement the CONCAT( ) splices together the results of several date functions along with spaces and other characters. The EXTRACT( ) function was eliminated from the WHERE
clause and instead a simple numeric test for sessions in August was given. Although EXTRACT( ) is fairly straightforward, this all can be accomplished with less typing by using the DATE_FORMAT( ) function.
The DATE_FORMAT( ) function has over thirty options for formatting the date to your liking. Plus, you can combine the options and add your own separators and other text. The syntax is DATE_FORMAT(date_column, 'options & characters')
. As an example, let's reproduce the last SQL statement by using the DATE_FORMAT( ) function for formatting the date of the appointment and for scanning for appointments in July only:
SELECT patient_name AS Patient,
DATE_FORMAT(appointment, '%W - %M %e, %Y')
AS Appointment
FROM billable_work, patients
WHERE doctor_id='1021'
AND billable_work.patient_id =
patients.patient_id
AND DATE_FORMAT(appointment, '%c') = 8
LIMIT 1;
This produces the exact same output as above, but with a more succinct statement. The option %W
gives the name of the day of the week. The option %M
provides the month's name. The option %e
displays the day of the month (%d
would work, but it left-pads single-digit dates with zeros). Finally, %Y
is for the four character year. All other elements within the quotes (i.e., the spaces, the dash, and the comma) are literal characters for a nicer display.
With DATE_FORMAT( ), time elements of a field also can be formatted. For instance, suppose we also wanted the hour and minute of the appointment. We would only need to change the second line of the SQL statement above (to save space, patient_name was eliminated):
SELECT
DATE_FORMAT(appointment, '%W - %M %e, %Y at %r')
AS Appointment
...
+--------------------------------------------+
| Appointment |
+--------------------------------------------+
| Wednesday - August 30, 2017 at 02:11:19 AM |
+--------------------------------------------+
The word at was added along with the formatting option %r
which gives the time with AM or PM at the end.
Although it may be a little confusing at first, once you've learned some of the common formatting options, DATE_FORMAT( ) is much easier to use than EXTRACT( ). There are many more options to DATE_FORMAT( ) that we haven't mentioned. For a complete list of the options available, see the DATE_FORMAT( ) documentation page.
In addition to DATE_FORMAT( ), MariaDB has a comparable built-in function for formating only time: TIME_FORMAT( ). The syntax is the same and uses the same options as DATE_FORMAT( ), except only the time related formatting options apply. As an example, here's an SQL statement that a doctor might use at the beginning of each day to get a list of her appointments for the day:
SELECT patient_name AS Patient,
TIME_FORMAT(appointment, '%l:%i %p')
AS Appointment
FROM billable_work, patients
WHERE doctor_id='1021'
AND billable_work.patient_id =
patients.patient_id
AND DATE_FORMAT(appointment, '%Y-%m-%d') =
CURDATE();
+-------------------+-------------+
| Patient | Appointment |
+-------------------+-------------+
| Michael Zabalaoui | 10:00 AM |
| Jerry Neumeyer | 11:00 AM |
| Richard Stringer | 01:30 PM |
| Janice Sogard | 02:30 PM |
+-------------------+-------------+
The option %l
provides the hours 01 through 12. The %p
at the end indicates (with the AM or PM) whether the time is before or after noon. The %i
option gives the minute. The colon and the space are for additional display appeal. Of course, all of this can be done exactly the same way with the DATE_FORMAT( ) function. As for the DATE_FORMAT( ) component in the WHERE clause here, the date is formatted exactly as it are with CURDATE( ) (i.e., 2017-08-30) so that they may be compared properly.
Many developers use PHP, Perl, or some other scripting language with MariaDB. Sometimes developers will solve retrieval problems with longer scripts rather than learn precisely how to extract temporal data with MariaDB. As you can see in several of the examples here (particularly the one using the QUARTER( ) function), you can accomplish a great deal within MariaDB. When faced with a potentially complicated SQL statement, try creating it in the mariadb client first. Once you get what you need (under various conditions) and in the format desired, then copy the statement into your script. This practice can greatly help you improve your MariaDB statements and scripting code.
CC BY-SA / Gnu FDL
WHITE PAPER
The Ultimate Guide to High Availability with MariaDB
WEBINAR
MariaDB 101: Learning the Basics of MariaDB
WEBINAR
MariaDB 101: Learning the Basics of MariaDB
Regular and reliable backups are essential to successful recovery of mission critical applications. backup and restore operations are performed using MariaDB Enterprise Backup, an enterprise-build of .
MariaDB Enterprise Backup is compatible with MariaDB Enterprise Server.
MariaDB Backup creates a file-level backup of data from the MariaDB Community Server data directory. This backup includes , and the encrypted and unencrypted tablespaces of supported storage engines (e.g., , , ).
MariaDB Enterprise Server implements:
Full backups, which contain all data in the database.
Incremental backups, which contain modifications since the last backup.
Partial backups, which contain a subset of the tables in the database.
Backup support is specific to storage engines. All supported storage engines enable full backup. The InnoDB storage engine additionally supports incremental backup.
Note: MariaDB Enterprise Backup does not support backups of MariaDB ColumnStore. Backup of MariaDB ColumnStore can be performed using . Backup of data ingested to MariaDB ColumnStore can also occur pre-ingestion, such as in the case of HTAP where backup could occur of transactional data in MariaDB Enterprise Server, and restore of data to MariaDB ColumnStore would then occur through reprocessing..
A feature of MariaDB Enterprise Backup and MariaDB Enterprise Server, non-blocking backups minimize workload impact during backups. When MariaDB Enterprise Backup connects to MariaDB Enterprise Server, staging operations are initiated to protect data during read.
Non-blocking backup functionality differs from historical backup functionality in the following ways:
MariaDB Enterprise Backup in MariaDB Enterprise Server includes enterprise-only optimizations to backup staging, including DDL statement tracking, which reduces lock-time during backups.
MariaDB Backup in MariaDB Community Server 10.4 and later will block writes, log tables, and statistics.
Older MariaDB Community Server releases used FLUSH TABLES WITH READ LOCK, which closed open tables and only allowed tables to be reopened with a read lock during the duration of backups.
MariaDB Enterprise Backup creates complete or incremental backups of MariaDB Enterprise Server data, and is also used to restore data from backups produced using MariaDB Enterprise Backup.
Full backups produced using MariaDB Enterprise Server are not initially point-in-time consistent, and an attempt to restore from a raw full backup will cause InnoDB to crash to protect the data.
Incremental backups produced using MariaDB Enterprise Backup contain only the changes since the last backup and cannot be used standalone to perform a restore.
To restore from a backup, you first need to prepare the backup for point-in-time consistency using the --prepare
command:
Running the --prepare
command on a full backup synchronizes the tablespaces, ensuring that they are point-in-time consistent and ready for use in recovery.
Running the --prepare
command on an incremental backup synchronizes the tablespaces and also applies the updated data into the previous full backup, making it a complete backup ready for use in recovery.
Running the --prepare
command on data that is to be used for a partial restore (when restoring only one or more selected tables) requires that you also use the --export
option to create the necessary .cfg
files to use in recovery.
When MariaDB Enterprise Backup restores from a backup, it copies or moves the backup files into the MariaDB Enterprise Server data directory, as defined by the datadir system variable.
For MariaDB Backup to safely restore data from full and incremental backups, the data directory must be empty. One way to achieve this is to move the data directory aside to a unique directory name:
Make sure that the Server is stopped.
Move the data directory to a unique name (e.g., /var/lib/mysql-2020-01-01
) OR remove the old data directory (depending on how much space you have available).
Create a new (empty) data directory (e.g., mkdir /var/lib/mysql)
.
Run MariaDB Backup to restore the databases into that directory.
Change the ownership of all the restored files to the correct system user (e.g., chown -R mysql:mysql /var/lib/mysql
).
Start MariaDB Enterprise Server, which now uses the restored data directory.
When ready, and if you have not already done so, delete the old data directory to free disk space.
When MariaDB Backup performs a backup operation, it not only copies files from the data directory but also connects to the running MariaDB Enterprise Server.
This connection to MariaDB Enterprise Server is used to manage locks that prevent the Server from writing to a file while being read for a backup.
MariaDB Backup establishes this connection based on the user credentials specified with the --user
and --password
options when performing a backup.
It is recommended that a dedicated user be created and authorized to perform backups.
MariaDB Backup requires this user to have the RELOAD, PROCESS, LOCK TABLES,
and REPLICATION CLIENT
privileges.
In the above example, MariaDB Backup would run on the local system that runs MariaDB Enterprise Server. Where backups may be run against a remote server, the user authentication and authorization should be adjusted.
While MariaDB Backup requires a user for backup operations, no user is required for restore operations since restores occur while MariaDB Enterprise Server is not running.
MariaDB Backup requires this user to have the RELOAD, PROCESS, LOCK TABLES,
and REPLICATION CLIENT
privileges.
In the above example, MariaDB Backup would run on the local system that runs MariaDB Enterprise Server. Where backups may be run against a remote server, the user authentication and authorization should be adjusted.
While MariaDB Backup requires a user for backup operations, no user is required for restore operations since restores occur while MariaDB Enterprise Server is not running.
Full backups performed with MariaDB Backup contain all table data present in the database.
When performing a full backup, MariaDB Backup makes a file-level copy of the MariaDB Enterprise Server data directory. This backup omits log data such as the binary logs (binlog), error logs, general query logs, and slow query logs.
When you perform a full backup, MariaDB Backup writes the backup to the --target-dir
path. The directory must be empty or non-existent and the operating system user account must have permission to write to that directory. A database user account is required to perform the backup.
The version of mariadb-backup
or mariadb-backup
should be the same version as the MariaDB Enterprise Server version. When the version does not match the server version, errors can sometimes occur, or the backup can sometimes be unusable.
To create a backup, execute mariadb-backup or mariadb-backup with the --backup
option, and provide the database user account credentials using the --user
and --password
options:
Subsequent to the above example, the backup is now available in the designated --target-dir
path.
A raw full backup is not and must be prepared before it can be used for a restore. The backup can be prepared any time after the backup is created and before the backup is restored. However, MariaDB recommends preparing a backup immediately after taking the backup to ensure that the backup is consistent.
The backup should be prepared with the same version of MariaDB Backup that was used to create the backup.
To prepare the backup, execute mariadb-backup or mariadb-backup with the --prepare
option:
For best performance, the --use-memory
option should be set to the server's innodb_buffer_pool_size
value.
Once a full backup has been prepared to be point-in-time consistent, MariaDB Backup is used to copy backup data to the MariaDB Enterprise Server data directory.
To restore from a full backup:
Stop the MariaDB Enterprise Server
Empty the data directory
Restore from the "full" directory using the --copy-back
option:
MariaDB Backup writes to the data directory as the current user, which can be changed using sudo
. To confirm that restored files are properly owned by the user that runs MariaDB Enterprise Server, run a command like this (adapted for the correct user/group):
Once this is done, start MariaDB Enterprise Server:
When the Server starts, it works from the restored data directory.
Full backups of large data-sets can be time-consuming and resource-intensive. MariaDB Backup supports the use of incremental backups to minimize this impact.
While full backups are resource-intensive at time of backup, the resource burden around incremental backups occurs when preparing for restore. First, the full backup is prepared for restore, then each incremental backup is applied.
When you perform an incremental backup, MariaDB Backup compares a previous full or incremental backup to what it finds on MariaDB Community Server. It then creates a new backup containing the incremental changes.
Incremental backup is supported for InnoDB tables. Tables using other storage engines receive full backups even during incremental backup operations.
To increment a full backup, use the --incremental-basedir
option to indicate the path to the full backup and the --target-dir
option to indicate where you want to write the incremental backup:
In this example, MariaDB Backup reads the /data/backups/full directory
, and MariaDB Enterprise Server then creates an incremental backup in the /data/backups/inc1
directory.
An incremental backup must be applied to a prepared full backup before it can be used in a restore operation. If you have multiple full backups to choose from, pick the nearest full backup prior to the incremental backup that you want to restore. You may also want to back up your full-backup directory, as it are modified by the updates in the incremental data.
If your full backup directory is not yet prepared, run this to make it consistent:
Then, using the prepared full backup, apply the first incremental backup's data to the full backup in an incremental preparation step:
Once the incremental backup has been applied to the full backup, the full backup directory contains the changes from the incremental backup (that is, the inc1/ directory). Feel free to remove inc1/ to save disk space.
Once you have prepared the full backup directory with all the incremental changes you need (as described above), stop the MariaDB Community Server, its data directory, and restore from the original full backup directory using the --copy-back option:
MariaDB Backup writes files into the data directory using either the current user or root (in the case of a sudo operation), which may be different from the system user that runs the database. Run the following to recursively update the ownership of the restored files and directories:
Then, start MariaDB Enterprise Server. When the Server starts, it works from the restored data directory.
In a partial backup, MariaDB Backup copies a specified subset of tablespaces from the MariaDB Enterprise Server data directory. Partial backups are useful in establishing a higher frequency of backups on specific data, at the expense of increased recovery complexity. In selecting tablespaces for a partial backup, please consider referential integrity.
Command-line options can be used to narrow the set of databases or tables to be included within a backup:
For example, you may wish to produce a partial backup, which excludes a specific database:
Partial backups can also be incremental:
As with full and incremental backups, partial backups are not point-in-time consistent. A partial backup must be prepared before it can be used for recovery.
A partial restore can be performed from a full backup or partial backup.
The preparation step for either partial or full backup restoration requires the use of transportable tablespaces for InnoDB. As such, each prepare operation requires the --export option:
When using a partial incremental backup for restore, the incremental data must be applied to its prior partial backup data before its data is complete. If performing partial incremental backups, run the prepare statement again to apply the incremental changes onto the partial backup that served as the base.
Unlike full and incremental backups, you cannot restore partial backups directly using MariaDB Backup. Further, as a partial backup does not contain a complete data directory, you cannot restore MariaDB Community Server to a startable state solely with a partial backup.
To restore from a partial backup, you need to prepare a table on the MariaDB Community Server, then manually copy the files into the data directory.
The details of the restore procedure depend on the characteristics of the table:
As partial restores are performed while the server is running, not stopped, care should be taken to prevent production workloads during restore activity.
Note: You can also use data from a full backup in a partial restore operation if you have prepared the data using the --export
option as described above.
To restore a non-partitioned table from a backup, first create a new table on MariaDB Community Server to receive the restored data. It should match the specifications of the table you're restoring.
Be extra careful if the backup data is from a server with a different version than the restore server, as some differences (such as a differing ROW_FORMAT
) can cause an unexpected result.
Create an empty table for the data being restored:
Modify the table to discard the tablespace:
You can copy (or move) the files for the table from the backup to the data directory:
Use a wildcard to include both the .ibd and .cfg files. Then, change the owner to the system user running MariaDB Community Server:
Lastly, import the new tablespace:
MariaDB Community Server looks in the data directory for the tablespace you copied in, then imports it for use. If the table is encrypted, it also looks for the encryption key with the relevant key ID that the table data specifies.
Repeat this step for every table you wish to restore.
Restoring a partitioned table from a backup requires a few extra steps compared to restoring a non-partitioned table.
To restore a partitioned table from a backup, first create a new table on MariaDB Community Server to receive the restored data. It should match the specifications of the table you're restoring, including the partition specification.
Be extra careful if the backup data is from a server with a different version than the restore server, as some differences (such as a differing ROW_FORMAT) can cause an unexpected result.
Create an empty table for the data being restored:
Then create a second empty table matching the column specification, but without partitions. This is your working table:
For each partition you want to restore, discard the working table's tablespace:
Then, copy the table files from the backup, using the new name:
Change the owner to that of the user running MariaDB Community Server:
Import the copied tablespace:
Lastly, exchange the partition, copying the tablespace from the working table into the partition file for the target table:
Repeat the above process for each partition until you have them all exchanged into the target table. Then delete the working table, as it's no longer necessary:
This restores a partitioned table.
When restoring a table with a full-text search (FTS) index, InnoDB may throw a schema mismatch error.
In this case, to restore the table, it is recommended to:
Remove the corresponding .cfg file.
Restore data to a table without any secondary indexes including FTS.
Add the necessary secondary indexes to the restored table.
For example, to restore table t1 with FTS index from database db1:
In the MariaDB shell, drop the table you are going to restore:
Create an empty table for the data being restored:
Modify the table to discard the tablespace:
In the operating system shell, copy the table files from the backup to the data directory of the corresponding database:
Remove the .cfg file from the data directory:
Change the owner of the newly copied files to the system user running MariaDB Community Server:
In the MariaDB shell, import the copied tablespace:
Verify that the data has been successfully restored:
Add the necessary secondary indexes:
The table is now fully restored:
Recovering from a backup restores the data directory at a specific point-in-time, but it does not restore the binary log. In a point-in-time recovery, you begin by restoring the data directory from a full or incremental backup, then use the mysqlbinlog utility to recover the binary log data to a specific point in time.
First, prepare the backup as you normally would for a or backup:
When MariaDB Backup runs on a MariaDB Community Server where binary logs is enabled, it stores binary log information in the xtrabackup_binlog_info
file. Consult this file to find the name of the binary log position to use. In the following example, the log position is 321.
Update the configuration file to use a new data directory.
Using MariaDB Backup, restore from the backup to the new data directory:
Then change the owner to the MariaDB Community Server system user:
Start MariaDB Community Server:
Using the binary log file in the old data directory, the start position in the xtrabackup_binlog_info
file, the date and time you want to restore to, and the mysqlbinlog
utility to create an SQL file with the binary log changes:
Lastly, run the binary log SQL to restore the databases:
This page is: Copyright © 2025 MariaDB. All rights reserved.
CREATE USER 'mariadb-backup'@'localhost'
IDENTIFIED BY 'mbu_passwd';
GRANT RELOAD, PROCESS, LOCK TABLES, BINLOG MONITOR
ON *.*
TO 'mariadb-backup'@'localhost';
CREATE USER 'mariadb-backup'@'localhost'
IDENTIFIED BY 'mbu_passwd';
GRANT RELOAD, PROCESS, LOCK TABLES, REPLICATION CLIENT
ON *.*
TO 'mariadb-backup'@'localhost';
sudo mariadb-backup --backup \
--target-dir=/data/backups/full \
--user=mariadb-backup \
--password=mbu_passwd
sudo mariadb-backup --prepare \
--use-memory=34359738368 \
--target-dir=/data/backups/full
mariadb-backup --copy-back --target-dir=/data/backups/full
chown -R mysql:mysql /var/lib/mysql
sudo systemctl start mariadb
mariadb-backup --backup \
--incremental-basedir=/data/backups/full \
--target-dir=/data/backups/inc1 \
--user=mariadb-backup \
--password=mbu_passwd
mariadb-backup --prepare --target-dir=/data/backups/full
mariadb-backup --prepare \
--target-dir=/data/backups/full \
--incremental-dir=/data/backups/inc1
mariadb-backup --copy-back --target-dir=/data/backups/full
chown -R mysql:mysql /var/lib/mysql
--databases
List of databases to include
--databases-exclude
List of databases to omit from the backup
--databases-file
Path to file listing the databases to include
--tables
List of tables to include
--tables-exclude
List of tables to exclude
--tables-file
Path to file listing the tables to include
mariadb-backup --backup \
--target-dir=/data/backups/part \
--user=mariadb-backup \
--password=mbu_passwd \
--database-exclude=test
mariadb-backup --backup \
--incremental-basedir=/data/backups/part \
--target-dir=/data/backups/part_inc1 \
--user=mariadb-backup \
--password=mbu_passwd \
--database-exclude=test
mariadb-backup --prepare --export --target-dir=/data/backups/part
mariadb-backup --prepare --export \
--target-dir=/data/backups/part \
--incremental-dir=/data/backups/part_inc1
CREATE TABLE test.address_book (
id INT PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(255),
email VARCHAR(255));
ALTER TABLE test.address_book DISCARD TABLESPACE;
# cp /data/backups/part_inc1/test/address_book.* /var/lib/mysql/test
# chown mysql:mysql /var/lib/mysql/test/address_book.*
ALTER TABLE test.address_book IMPORT TABLESPACE;
CREATE TABLE test.students (
id INT PRIMARY KEY AUTO_INCREMENT
name VARCHAR(255),
email VARCHAR(255),
graduating_year YEAR)
PARTITION BY RANGE (graduating_year) (
PARTITION p9 VALUES LESS THAN 2019
PARTITION p1 VALUES LESS THAN MAXVALUE
);
CREATE TABLE test.students_work AS
SELECT * FROM test.students WHERE NULL;
ALTER TABLE test.students_work DISCARD TABLESPACE;
# cp /data/backups/part_inc1/test/students.ibd /var/lib/mysql/test/students_work.ibd
# cp /data/backups/part_inc1/test/students.cfg /var/lib/mysql/test/students_work.cfg
# chown mysql:mysql /var/lib/mysql/test/students_work.*
ALTER TABLE test.students_work IMPORT TABLESPACE;
ALTER TABLE test.students EXCHANGE PARTITION p0 WITH TABLE test.students_work;
DROP TABLE test.students_work;
DROP TABLE IF EXISTS db1.t1;
CREATE TABLE db1.t1(f1 CHAR(10)) ENGINE=INNODB;
ALTER TABLE db1.t1 DISCARD TABLESPACE;
$ sudo cp /data/backups/part/db1/t1.* /var/lib/mysql/db1
$ sudo rm /var/lib/mysql/db1/t1.cfg
$ sudo chown mysql:mysql /var/lib/mysql/db1/t1.*
ALTER TABLE db1.t1 IMPORT TABLESPACE;
SELECT * FROM db1.t1;
+--------+
| f1 |
+--------+
| ABC123 |
+--------+
ALTER TABLE db1.t1 FORCE, ADD FULLTEXT INDEX f_idx(f1);
SHOW CREATE TABLE db1.t1\G
*************************** 1. row ***************************
Table: t1
Create Table: CREATE TABLE `t1` (
`f1` char(10) DEFAULT NULL,
FULLTEXT KEY `f_idx` (`f1`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_general_ci
mariadb-backup --prepare --target-dir=/data/backups/full
cat /data/backups/full/xtraback_binlog_info
mariadb-node4.00001 321
[mysqld]
datadir=/var/lib/mysql_new
mariadb-backup --copy-back --target-dir=/data/backups/full
chown -R mysql:mysql /var/lib/mysql_new
sudo systemctl start mariadb
mysqlbinlog --start-position=321 \
--stop-datetime="2019-06-28 12:00:00" \
/var/lib/mysql/mariadb-node4.00001 \
> mariadb-binlog.sql
mysql -u root -p < mariadb-binlog.sql
WEBINAR
MariaDB 101: Learning the Basics of MariaDB
WEBINAR
MariaDB 101: Learning the Basics of MariaDB
WEBINAR
MariaDB 101: Learning the Basics of MariaDB
WEBINAR
MariaDB 101: Learning the Basics of MariaDB
WEBINAR
MariaDB 101: Learning the Basics of MariaDB
WEBINAR
MariaDB 101: Learning the Basics of MariaDB
WEBINAR
MariaDB 101: Learning the Basics of MariaDB
In MariaDB, a table can be split in smaller subsets. Both data and indexes are partitioned.
There can be several reasons to use this feature:
If you often need to delete a large set of rows, such as all rows for a given year, using partitions can help, as dropping a partition with many rows is very fast, while deleting a lot of rows can be very slow.
Very large tables and indexes can be slow even with optimized queries. But if the target table is partitioned, queries that read a small number of partitions can be much faster. However, this means that the queries have to be written carefully in order to only access a given set of partitions.
Partitioning allows one to distribute files over multiple storage devices. For example, we can have historical data on slower, larger disks (historical data are not supposed to be frequently read); and current data can be on faster disks, or SSD devices.
In case we separate historical data from recent data, we will probably need to take regular backups of one partition, not the whole table.
When partitioning a table, the use should decide:
a partitioning type;
a partitioning expression.
A partitioning type is the method used by MariaDB to decide how rows are distributed over existing partitions. Choosing the proper partitioning type is important to distribute rows over partitions in an efficient way.
With some partitioning types, a partitioning expression is also required. A partitioning function is an SQL expression returning an integer or temporal value, used to determine which partition will contain a given row. The partitioning expression is used for all reads and writes on involving the partitioned table, thus it should be fast.
MariaDB supports the following partitioning types:
By default, MariaDB permits partitioning. You can determine this by using the statement, for example:
If partition is listed as DISABLED:
MariaDB has either been built without partitioning support, or has been started with the option, or one of its variants:
and you will not be able to create partitions.
It is possible to create a new partitioned table using .
allows one to:
Partition an existing table;
Remove partitions from a partitioned table (with all data in the partition);
Add/remove partitions, or reorganize them, as long as the partitioning function allows these operations (see below);
Exchange a partition with a table;
Perform administrative operations on some or all partitions (analyze, optimize, check, repair).
[ALTER TABLE](../../reference/sql-statements-and-structure/sql-statements/data-definition/alter/alter-table.md) ... ADD PARTITION
can be used to add partitions to an existing table:
With partitions, it is only possible to add a partition to the high end of the range, not the low end. For example, the following results in an error:
You can work around this by using REORGANIZE PARTITION to split the partition instead. See .
[ALTER TABLE](../../reference/sql-statements-and-structure/sql-statements/data-definition/alter/alter-table.md) ... COALESCE PARTITION
is used to reduce the number of HASH or KEY partitions by the specified number. For example, given the following table with 5 partitions:
The following statement will reduce the number of partitions by 2, leaving the table with 3 partitions:
MariaDB starting with
[ALTER TABLE](../../reference/sql-statements-and-structure/sql-statements/data-definition/alter/alter-table.md) ... CONVERT PARTITION
can, from , be used to convert partitions in an existing table to a standalone table:
CONVERT TABLE
does the reverse, converting a table into a partition:
MariaDB starting with
When converting tables to a partition, validation is performed on each row to ensure it meets the partition requirements. This can be very slow in the case of larger tables.
From , it is possible to disable this validation by specifying the WITHOUT VALIDATION
option.
WITH VALIDATION
will result in the validation being performed, and is the default behaviour.
An alternative (and the only method prior to ) to convert partitions to tables is to use [ALTER TABLE](../../reference/sql-statements-and-structure/sql-statements/data-definition/alter/alter-table.md) ... EXCHANGE PARTITION
. This requires having to manually do the following steps:
create an empty table with the same structure as the partition
exchange the table with the partition
drop the empty partition
For example:
Similarly, to do the reverse and convert a table into a partition [ALTER TABLE](../../reference/sql-statements-and-structure/sql-statements/data-definition/alter/alter-table.md) ... EXCHANGE PARTITION
can also be used, with the following manual steps required:
create the partition
exchange the partition with the table
drop the old table:
For example:
[ALTER TABLE](../../reference/sql-statements-and-structure/sql-statements/data-definition/alter/alter-table.md) ... DROP PARTITION
can be used to drop specific partitions (and discard all data within the specified partitions) for and partitions. It cannot be used on or partitions. To rather remove all partitioning, while leaving the data unaffected, see .
<=
=
[ALTER TABLE](../../reference/sql-statements-and-structure/sql-statements/data-definition/alter/alter-table.md) t1 EXCHANGE PARTITION p1 WITH TABLE t2
permits one to exchange a partition or subpartition with another table.
The following requirements must be met:
Table t1 must be partitioned, and table t2 cannot be partitioned
Table t2 cannot be a temporary table
Table t1 and t2 must otherwise be identica
Any existing row in t2 must match the conditions for storage in the exchanged partition p1 unless, from , the WITHOUT VALIDATION
option is specified.
MariaDB will by default perform the validation to see that each row meets the partition requirements, and the statement will fail if a row does not fit.
This attempted exchange fails, as the value already in t2, 2015-05-05
is outside of the partition conditions:
MariaDB starting with
This validation is performed for each row, and can be very slow in the case of larger tables.
From , it is possible to disable this validation by specifying the WITHOUT VALIDATION
option.
WITH VALIDATION
will result in the validation being performed, and is the default behaviour.
[ALTER TABLE](../../reference/sql-statements-and-structure/sql-statements/data-definition/alter/alter-table.md) ... REMOVE PARTITIONING
will remove all partitioning from the table, while leaving the data unaffected. To rather drop a particular partition (and discard all of its data), see .
Reorganizing partitions allows one to adjust existing partitions, without losing data. Specifically, the statement can be used for:
Splitting an existing partition into multiple partitions
Merging a number of existing partitions into a new, single, partition
Changing the ranges for a subset of existing partitions defined using VALUES LESS THAN
Changing the value lists for a subset of partitions defined using VALUES I.
Renaming partitions
An existing partition can be split into multiple partitions. This can also be used to add a new partition at the low end of a partition (which is not possible by ).
Similarly, if MAXVALUE binds the high end:
A number of existing partitions can be merged into a new partition, for example:
The REORGANIZE PARTITION statement can also be used for renaming partitions. Note that this creates a copy of the partition:
[ALTER TABLE](../../reference/sql-statements-and-structure/sql-statements/data-definition/alter/alter-table.md) ... TRUNCATE PARTITION
will remove all data from the specified partition/s, leaving the table and partition structure unchanged. Partitions don't need to be contiguous.
Similar to , key distributions for specific partitions can also be analyzed and stored, for example:
Similar to , specific partitions can be checked for errors, for example:
The ALL
keyword can be used in place of the list of partition names, and the check operation are performed on all partitions.
Similar to , specific partitions can be repaired, for example:
As with , the QUICK
and EXTENDED
options are available. However, the USE_FRM
option cannot be used with this statement on a partitioned table.
REPAIR PARTITION
will fail if there are duplicate key errors. ALTER IGNORE TABLE ... REPAIR PARTITION
can be used in this case.
The ALL
keyword can be used in place of the list of partition names, and the repair operation are performed on all partitions.
Similar to , specific partitions can be checked for errors, for example:
OPTIMIZE PARTITION
does not support per-partition optimization on InnoDB tables, and will issue a warning and cause the entire table to rebuilt and analyzed. ALTER TABLE ... REBUILD PARTITION
and ALTER TABLE ... ANALYZE PARTITION
can be used instead.
The ALL
keyword can be used in place of the list of partition names, and the optimize operation are performed on all partitions.
Some MariaDB allow more interesting uses for partitioning.
The storage engine allows one to:
Treat a set of identical defined tables as one.
A MyISAM table can be in many different MERGE sets and also used separately.
allows one to:
Move partitions of the same table on different servers. In this way, the workload can be distributed on more physical or virtual machines (data sharding).
All partitions of a SPIDER table can also live on the same machine. In this case there are a small overhead (SPIDER will use connections to localhost), but queries that read multiple partitions will use parallel threads.
allows one to:
Build a table whose partitions are tables using different storage engines (like InnoDB, MyISAM, or even engines that do not support partitioning).
Build an indexable, writeable table on several data files. These files can be in different formats.
See also:
contains information about existing partitions.
for suggestions on using partitions
This page is licensed: CC BY-SA / Gnu FDL
SHOW PLUGINS;
...
| Aria | ACTIVE | STORAGE ENGINE | NULL | GPL |
| FEEDBACK | DISABLED | INFORMATION SCHEMA | NULL | GPL |
| partition | ACTIVE | STORAGE ENGINE | NULL | GPL |
+-------------------------------+----------+--------------------+---------+---------+
| partition | DISABLED | STORAGE ENGINE | NULL | GPL |
+-------------------------------+----------+--------------------+---------+---------+
--skip-partition
--disable-partition
--partition=OFF
ADD PARTITION [IF NOT EXISTS] (partition_definition)
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016)
);
ALTER TABLE t1 ADD PARTITION (
PARTITION p4 VALUES LESS THAN (2017),
PARTITION p5 VALUES LESS THAN (2018)
);
ALTER TABLE t1 ADD PARTITION (
PARTITION p0a VALUES LESS THAN (2012)
);
ERROR 1493 (HY000): VALUES LESS THAN value must be strictly increasing for each partition
COALESCE PARTITION number
CREATE OR REPLACE TABLE t1 (v1 INT)
PARTITION BY KEY (v1)
PARTITIONS 5;
ALTER TABLE t1 COALESCE PARTITION 2;
CONVERT PARTITION partition_name TO TABLE tbl_name
CONVERT TABLE normal_table TO partition_definition
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016)
);
INSERT INTO t1 VALUES ('2013-11-11'),('2014-11-11'),('2015-11-11');
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2013-11-11 00:00:00 |
| 2014-11-11 00:00:00 |
| 2015-11-11 00:00:00 |
+---------------------+
ALTER TABLE t1 CONVERT PARTITION p3 TO TABLE t2;
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2013-11-11 00:00:00 |
| 2014-11-11 00:00:00 |
+---------------------+
SELECT * FROM t2;
+--------------+
| dt |
+--------------+
| 2015-11-11 00:00:00 |
+---------------------+
SHOW CREATE TABLE t1\G
*************************** 1. row ***************************
TABLE: t1
CREATE TABLE: CREATE TABLE `t1` (
`dt` datetime NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci
PARTITION BY RANGE (year(`dt`))
(PARTITION `p0` VALUES LESS THAN (2013) ENGINE = InnoDB,
PARTITION `p1` VALUES LESS THAN (2014) ENGINE = InnoDB,
PARTITION `p2` VALUES LESS THAN (2015) ENGINE = InnoDB)
SHOW CREATE TABLE t2\G
*************************** 1. row ***************************
TABLE: t2
CREATE TABLE: CREATE TABLE `t2` (
`dt` datetime NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci
ALTER TABLE t1 CONVERT TABLE t2 TO PARTITION p3 VALUES LESS THAN (2016);
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2013-11-11 00:00:00 |
| 2014-11-11 00:00:00 |
| 2015-11-11 00:00:00 |
+---------------------+
3 rows in set (0.001 sec)
SELECT * FROM t2;
ERROR 1146 (42S02): Table 'test.t2' doesn't exist
SHOW CREATE TABLE t1\G
*************************** 1. row ***************************
TABLE: t1
CREATE TABLE: CREATE TABLE `t1` (
`dt` datetime NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci
PARTITION BY RANGE (year(`dt`))
(PARTITION `p0` VALUES LESS THAN (2013) ENGINE = InnoDB,
PARTITION `p1` VALUES LESS THAN (2014) ENGINE = InnoDB,
PARTITION `p2` VALUES LESS THAN (2015) ENGINE = InnoDB,
PARTITION `p3` VALUES LESS THAN (2016) ENGINE = InnoDB)
CONVERT TABLE normal_table TO partition_definition [{WITH | WITHOUT} VALIDATION]
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016)
);
INSERT INTO t1 VALUES ('2013-11-11'),('2014-11-11'),('2015-11-11');
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2013-11-11 00:00:00 |
| 2014-11-11 00:00:00 |
| 2015-11-11 00:00:00 |
+---------------------+
CREATE OR REPLACE TABLE t2 LIKE t1;
ALTER TABLE t2 REMOVE PARTITIONING;
ALTER TABLE t1 EXCHANGE PARTITION p3 WITH TABLE t2;
ALTER TABLE t1 DROP PARTITION p3;
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2013-11-11 00:00:00 |
| 2014-11-11 00:00:00 |
+---------------------+
SELECT * FROM t2;
+--------------+
| dt |
+--------------+
| 2015-11-11 00:00:00 |
+---------------------+
SHOW CREATE TABLE t1\G
*************************** 1. row ***************************
TABLE: t1
CREATE TABLE: CREATE TABLE `t1` (
`dt` datetime NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci
PARTITION BY RANGE (year(`dt`))
(PARTITION `p0` VALUES LESS THAN (2013) ENGINE = InnoDB,
PARTITION `p1` VALUES LESS THAN (2014) ENGINE = InnoDB,
PARTITION `p2` VALUES LESS THAN (2015) ENGINE = InnoDB)
SHOW CREATE TABLE t2\G
*************************** 1. row ***************************
TABLE: t2
CREATE TABLE: CREATE TABLE `t2` (
`dt` datetime NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci
ALTER TABLE t1 ADD PARTITION (PARTITION p3 VALUES LESS THAN (2016));
ALTER TABLE t1 EXCHANGE PARTITION p3 WITH TABLE t2;
DROP TABLE t2;
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2013-11-11 00:00:00 |
| 2014-11-11 00:00:00 |
| 2015-11-11 00:00:00 |
+---------------------+
SHOW CREATE TABLE t1\G
*************************** 1. row ***************************
TABLE: t1
CREATE TABLE: CREATE TABLE `t1` (
`dt` datetime NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_swedish_ci
PARTITION BY RANGE (year(`dt`))
(PARTITION `p0` VALUES LESS THAN (2013) ENGINE = InnoDB,
PARTITION `p1` VALUES LESS THAN (2014) ENGINE = InnoDB,
PARTITION `p2` VALUES LESS THAN (2015) ENGINE = InnoDB,
PARTITION `p3` VALUES LESS THAN (2016) ENGINE = InnoDB)
DROP PARTITION [IF EXISTS] partition_names
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016)
);
INSERT INTO t1 VALUES ('2012-11-15');
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2012-11-15 00:00:00 |
+---------------------+
ALTER TABLE t1 DROP PARTITION p0;
SELECT * FROM t1;
Empty set (0.002 sec)
EXCHANGE PARTITION partition_name WITH TABLE tbl_name
EXCHANGE PARTITION partition_name WITH TABLE tbl_name [{WITH | WITHOUT} VALIDATION]
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014)
);
CREATE OR REPLACE TABLE t2 (
dt DATETIME NOT NULL
) ENGINE = InnoDB;
INSERT INTO t1 VALUES ('2012-01-01'),('2013-01-01');
INSERT INTO t2 VALUES ('2013-02-02');
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2012-01-01 00:00:00 |
| 2013-01-01 00:00:00 |
+---------------------+
SELECT * FROM t2;
+--------------+
| dt |
+--------------+
| 2013-02-02 00:00:00 |
+---------------------+
ALTER TABLE t1 EXCHANGE PARTITION p1 WITH TABLE t2;
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2012-01-01 00:00:00 |
| 2013-02-02 00:00:00 |
+---------------------+
SELECT * FROM t2;
+--------------+
| dt |
+--------------+
| 2013-01-01 00:00:00 |
+---------------------+
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014)
);
CREATE OR REPLACE TABLE t2 (
dt DATETIME NOT NULL
) ENGINE = InnoDB;
INSERT INTO t1 VALUES ('2012-02-02'),('2013-03-03');
INSERT INTO t2 VALUES ('2015-05-05');
ALTER TABLE t1 EXCHANGE PARTITION p1 WITH TABLE t2;
ERROR 1526 (HY000): Table has no partition for value 0
ALTER TABLE t1 EXCHANGE PARTITION p1 WITH TABLE t2 WITHOUT VALIDATION;
Query OK, 0 rows affected (0.048 sec)
REMOVE PARTITIONING
ALTER TABLE t1 REMOVE PARTITIONING;
REORGANIZE PARTITION [partition_names INTO (partition_definitions)]
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016)
);
ALTER TABLE t1 REORGANIZE PARTITION p0 INTO (
PARTITION p0a VALUES LESS THAN (2012),
PARTITION p0b VALUES LESS THAN (2013)
);
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016),
PARTITION p4 VALUES LESS THAN MAXVALUE
);
ALTER TABLE t1 REORGANIZE PARTITION p4 INTO (
PARTITION p4 VALUES LESS THAN (2017),
PARTITION p5 VALUES LESS THAN MAXVALUE
);
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016)
);
ALTER TABLE t1 REORGANIZE PARTITION p2,p3 INTO (
PARTITION p2 VALUES LESS THAN (2016)
);
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016)
);
ALTER TABLE t1 REORGANIZE PARTITION p3 INTO (
PARTITION p3 VALUES LESS THAN (2017)
);
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016)
);
ALTER TABLE t1 REORGANIZE PARTITION p3 INTO (
PARTITION p3_new VALUES LESS THAN (2016)
);
TRUNCATE PARTITION partition_names
CREATE OR REPLACE TABLE t1 (
dt DATETIME NOT NULL
)
ENGINE = InnoDB
PARTITION BY RANGE (YEAR(dt))
(
PARTITION p0 VALUES LESS THAN (2013),
PARTITION p1 VALUES LESS THAN (2014),
PARTITION p2 VALUES LESS THAN (2015),
PARTITION p3 VALUES LESS THAN (2016)
);
INSERT INTO t1 VALUES ('2012-11-01'),('2013-11-02'),('2014-11-03'),('2015-11-04');
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2012-11-01 00:00:00 |
| 2013-11-02 00:00:00 |
| 2014-11-03 00:00:00 |
| 2015-11-04 00:00:00 |
+---------------------+
ALTER TABLE t1 TRUNCATE PARTITION p0,p2;
SELECT * FROM t1;
+--------------+
| dt |
+--------------+
| 2013-11-02 00:00:00 |
| 2015-11-04 00:00:00 |
+---------------------+
ALTER TABLE t1 ANALYZE PARTITION p0,p1,p3;
+---------+---------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+---------+---------+----------+----------+
| test.t1 | analyze | status | OK |
+---------+---------+----------+----------+
CHECK PARTITION {ALL | PARTITION [,partition2 ...]}
ALTER TABLE t1 CHECK PARTITION p1,p3;
+---------+-------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+---------+-------+----------+----------+
| test.t1 | check | status | OK |
+---------+-------+----------+----------+
REPAIR PARTITION {ALL | partition [,partition2 ...]} [QUICK] [EXTENDED]
ALTER TABLE t1 REPAIR PARTITION p0,p3;
+---------+--------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+---------+--------+----------+----------+
| test.t1 | repair | status | OK |
+---------+--------+----------+----------+
OPTIMIZE PARTITION {ALL | PARTITION [,partition2 ...]}
ALTER TABLE t1 OPTIMIZE PARTITION p0,p3;
+---------+----------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+---------+----------+----------+----------+
| test.t1 | optimize | status | OK |
+---------+----------+----------+----------+
mariadb-backup is an open source tool provided by MariaDB for performing physical online backups of InnoDB, Aria and MyISAM tables. For InnoDB, “hot online” backups are possible. It was originally forked from Percona XtraBackup 2.3.8. It is available on Linux and Windows.
This tool provides a production-quality, nearly non-blocking method for performing full backups on running systems. While partial backups with mariadb-backup are technically possible, they require many steps and cannot be restored directly onto existing servers containing other data.
mariadb-backup
supports all of the main features of Percona XtraBackup 2.3.8, plus:
Backup/Restore of tables using Data-at-Rest Encryption.
Backup/Restore of tables using InnoDB Page Compression.
mariadb-backup SST method with Galera Cluster.
Microsoft Windows support.
Backup/Restore of tables using the MyRocks storage engine. See Files Backed up by mariadb-backup: MyRocks Data Files for more information.
Supported Features in MariaDB Enterprise Backup
MariaDB Backup supports some additional features, such as:
Minimizes locks during the backup to permit more concurrency and to enable faster backups.
This relies on the usage of BACKUP STAGE
commands and DDL logging.
This includes no locking during the copy phase of ALTER TABLE
statements, which tends to be the longest phase of these statements.
Provides optimal backup support for all storage engines that store things on local disk.
MariaDB Backup does not support some additional features.
Percona XtraBackup requires more locks to run than MariaDB. In addition, any running ALTER TABLE will block Percona XtraBackup until it completes.
Percona XtraBackup copies its InnoDB redo log files to the file xtrabackup_logfile
, while mariadb-backup uses the file ib_logfile0.
Percona XtraBackup's libgcrypt-based encryption of backups is not supported by mariadb-backup.
There is no symbolic link from mariadb-backup
to innobackupex, as there is for xtrabackup. Instead, mariadb-backup
has the --innobackupex command-line option to enable innobackupex-compatible options.
The --compact and --rebuild_indexes options are not supported.
Support for --stream=tar was removed from mariadb-backup
in .
mariadb-backup
does not support lockless binlog.
Difference in Versioning Schemes
Each Percona XtraBackup release has two version numbers--the Percona XtraBackup version number and the version number of the MySQL Server release that it is based on. For example:
xtrabackup version 2.2.8 based on MySQL server 5.6.22
Each mariadb-backup release only has one version number, and it is the same as the version number of the MariaDB Server release that it is based on. For example:
mariadb-backup based on MariaDB server 10.2.15-MariaDB Linux (x86_64)
mariadb-backup
The mariadb-backup
executable is included in binary tarballs on Linux.
mariadb-backup can also be installed via a package manager on Linux. In order to do so, your system needs to be configured to install from one of the MariaDB repositories.
You can configure your package manager to install it from MariaDB Corporation's MariaDB Package Repository by using the MariaDB Package Repository setup script.
You can also configure your package manager to install it from MariaDB Foundation's MariaDB Repository by using the MariaDB Repository Configuration Tool.
Installing with yum/dnf
On RHEL, CentOS, Fedora, and other similar Linux distributions, it is highly recommended to install the relevant RPM package from MariaDB's repository using yum or dnf. Starting with RHEL 8 and Fedora 22, yum
has been replaced by dnf
, which is the next major version of yum
. However, yum
commands still work on many systems that use dnf
. For example:
sudo yum install MariaDB-backup
Installing with apt-get
On Debian, Ubuntu, and other similar Linux distributions, it is highly recommended to install the relevant DEB package from MariaDB's repository using apt-get. For example:
sudo apt-get install mariadb-backup
Installing with zypper
On SLES, OpenSUSE, and other similar Linux distributions, it is highly recommended to install the relevant RPM package from MariaDB's repository using zypper. For example:
sudo zypper install MariaDB-backup
The mariadb-backup
executable is included in MSI and ZIP packages on Windows.
When using the Windows MSI installer, mariadb-backup
can be installed by selecting Backup utilities:
The command to use mariadb-backup
and the general syntax is:
mariadb-backup <options>
For in-depth explanations on how to use mariadb-backup
, see:
Options supported by mariadb-backup can be found on the mariadb-backup Options page.
mariadb-backup
will currently silently ignore unknown command-line options, so be extra careful about accidentally including typos in options or accidentally using options from later mariadb-backup
versions. The reason for this is that mariadb-backup
currently treats command-line options and options from option files equivalently. When it reads from these option files, it has to read a lot of options from the server option groups read by mariadbd. However, mariadb-backup
does not know about many of the options that it normally reads in these option groups. If mariadb-backup
raised an error or warning when it encountered an unknown option, then this process would generate a large amount of log messages under normal use. Therefore, mariadb-backup
is designed to silently ignore the unknown options instead. See MDEV-18215 about that.
In addition to reading options from the command-line, mariadb-backup can also read options from option files.
The following options relate to how MariaDB command-line tools handles option files. They must be given as the first argument on the command-line:
--print-defaults
Print the program argument list and exit.
--no-defaults
Don't read default options from any option file.
--defaults-file=#
Only read default options from the given option file.
--defaults-extra-file=#
Read this file after the global files are read.
--defaults-group-suffix=#
In addition to the default option groups, also read option groups with this suffix.
mariadb-backup reads server options from the following option groups from option files:
[mariadb-backup]
Options read by mariadb-backup
. Available starting with and .
[mariadb-backup]
Options read by mariadb-backup
. Available starting with and .
[xtrabackup]
Options read by mariadb-backup
and Percona XtraBackup.
[server]
Options read by MariaDB Server. Available starting with , , and .
[mysqld]
Options read by mariadbd
, which includes both MariaDB Server and MySQL Server (where it is called mysqld
).
[mysqld-X.Y]
Options read by a specific version of mysqld, which includes both MariaDB Server and MySQL Server. For example: [mysqld-10.4]
. Available starting with , , and .
[mariadb]
Options read by MariaDB Server. Available starting with , , and .
[mariadb-X.Y]
Options read by a specific version of MariaDB Server. For example: [mariadb-10.4]
. Available starting with , , and .
[mariadbd]
Options read by MariaDB Server. Available starting with and .
[mariadbd-X.Y]
Options read by a specific version of MariaDB Server. For example: [mariadbd-10.4]
. Available starting with and .
[client-server]
Options read by all MariaDB client programs and the MariaDB Server. This is useful for options like socket and port, which is common between the server and the clients. Available starting with , , and .
[galera]
Options read by MariaDB Server, but only if it is compiled with Galera Cluster support. In MariaDB 10.1 and later, all builds on Linux are compiled with Galera Cluster support. When using one of these builds, options from this option group are read even if the Galera Cluster functionality is not enabled. Available starting with , , and on systems compiled with Galera Cluster support.
mariadb-backup reads client options from the following option groups from option files:
[mariadb-backup]
Options read by mariadb-backup. Available starting with and .
[mariadb-backup]
Options read by mariadb-backup. Available starting with and
[xtrabackup]
Options read by mariadb-backup and Percona XtraBackup.
[client]
Options read by all MariaDB and MySQL client programs, which includes both MariaDB and MySQL clients. For example, mysqldump.
[client-server]
Options read by all MariaDB client programs and the MariaDB Server. This is useful for options like socket and port, which is common between the server and the clients. Available starting with , , and .
[client-mariadb]
Options read by all MariaDB client programs. Available starting with , , and .
mariadb-backup
needs to authenticate with the database server when it performs a backup operation (i.e. when the --backup option is specified). For most use cases, the user account that performs the backup needs to have the following global privileges on the database server.
In 10.5 and later the required privileges are:
CREATE USER 'mariadb-backup'@'localhost' IDENTIFIED BY 'mypassword';
GRANT RELOAD, PROCESS, LOCK TABLES, BINLOG MONITOR ON *.* TO 'mariadb-backup'@'localhost';
Prior to 10.5, the required privileges are:
CREATE USER 'mariadb-backup'@'localhost' IDENTIFIED BY 'mypassword';
GRANT RELOAD, PROCESS, LOCK TABLES, REPLICATION CLIENT ON *.* TO 'mariadb-backup'@'localhost';
If your database server is also using the MyRocks storage engine, then the user account that performs the backup will also need the SUPER
global privilege. This is because mariadb-backup
creates a checkpoint of this data by setting the rocksdb_create_checkpoint system variable, which requires this privilege. See MDEV-20577 for more information.
CONNECTION ADMIN
is also required where --kill-long-queries-timeout is greater than 0
, and --no-lock isn't applied in order to KILL queries. Prior to 10.5 a SUPER
privilege is required instead of CONNECTION ADMIN
.
REPLICA MONITOR
(or alias SLAVE MONITOR
) is also required where --galera-info or --slave-info is specified.
To use the --history option, the backup user also needs to have the following privileges granted:
GRANT CREATE, ALTER, INSERT ON mysql.mariadb_backup_history TO 'mariadb-backup'@'localhost';
Prior to MariaDB 10.11, the necessary permissions to use --history were:
GRANT CREATE, INSERT ON PERCONA_SCHEMA.* TO 'mariadb-backup'@'localhost';
If you're upgrading from an older version and you want to use the new default table without losing your backup history, you can move and rename the current table in this way:
RENAME TABLE PERCONA_SCHEMA.xtrabackup_history TO mysql.mariadb_backup_history;
ALTER TABLE mysql.mariadb_backup_history MODIFY format ENUM('file', 'tar', 'mbstream') DEFAULT NULL;
The user account information can be specified with the --user and --password command-line options. For example:
mariadb-backup --backup \
--target-dir=/var/mariadb/backup/ \
--user=mariadb-backup --password=mypassword
The user account information can also be specified in a supported client option group in an option file. For example:
[mariadb-backup]
user=mariadb-backup
password=mypassword
mariadb-backup
does not need to authenticate with the database server when preparing or restoring a backup.
mariadb-backup
has to read MariaDB's files from the file system. Therefore, when you run mariadb-backup
as a specific operating system user, you should ensure that user account has sufficient permissions to read those files.
If you are using Linux and if you installed MariaDB with a package manager, then MariaDB's files will probably be owned by the mysql
user and the mysql
group.
mariadb-backup
with Data-at-Rest Encryptionmariadb-backup
supports Data-at-Rest Encryption.
mariadb-backup will query the server to determine which key management and encryption plugin is being used, and then it will load that plugin itself, which means that mariadb-backup
needs to be able to load the key management and encryption plugin's shared library.
mariadb-backup will also query the server to determine which encryption keys it needs to use.
In other words, mariadb-backup
is able to figure out a lot of encryption-related information on its own, so normally one doesn't need to provide any extra options to backup or restore encrypted tables.
mariadb-backup
backs up encrypted and unencrypted tables as they are on the original server. If a table is encrypted, then the table will remain encrypted in the backup. Similarly, if a table is unencrypted, then the table will remain unencrypted in the backup.
The primary reason that mariadb-backup needs to be able to encrypt and decrypt data is that it needs to apply InnoDB redo log records to make the data consistent when the backup is prepared. As a consequence, mariadb-backup does not perform many encryption or decryption operations when the backup is initially taken. MariaDB performs more encryption and decryption operations when the backup is prepared. This means that some encryption-related problems (such as using the wrong encryption keys) may not become apparent until the backup is prepared.
mariadb-backup
for Galera SSTsThe mariadb-backup
SST method uses the mariadb-backup
utility for performing SSTs. See mariadb-backup SST method for more information.
mariadb-backup
mariadb-backup
backs up many different files in order to perform its backup operation. See Files Backed up by mariadb-backup for a list of these files.
mariadb-backup
mariadb-backup
creates several different types of files during the backup and prepare phases. See Files Created by mariadb-backup for a list of these files.
mariadb-backup
can store the binary log position in the backup. See --binlog-info. This can be used for point-in-time recovery and to use the backup to setup a slave with the correct binlog position.
Prior to , , and , mariadb-backup
doesn't read server options from all option groups supported by the server. In those versions, it only looks for server options in the following server option groups:
[xtrabackup]
Options read by Percona XtraBackup and mariadb-backup
.
[mariadb-backup]
Options read by Percona XtraBackup and mariadb-backup
. Available starting with and .
[mysqld]
Options read by , which includes both MariaDB Server and MySQL Server (where it is called mysqld
).
Those versions do not read server options from the following option groups supported by the server:
[server]
Options read by MariaDB Server. Available starting with , , and .
[mysqld-X.Y]
Options read by a specific version of mysqld, which includes both MariaDB Server and MySQL Server. For example: [mysqld-5.5]
. Available starting with , , and .
[mariadb]
Options read by MariaDB Server. Available starting with , , and .
[mariadb-X.Y]
Options read by a specific version of MariaDB Server. For example: [mariadb-10.3]
. Available starting with , , and .
[client-server]
Options read by all MariaDB client programs and the MariaDB Server. This is useful for options like socket and port, which is common between the server and the clients. Available starting with , , and .
[galera]
Options read by MariaDB Server, but only if it is compiled with Galera Cluster support. In MariaDB 10.1 and later, all builds on Linux are compiled with Galera Cluster support. When using one of these builds, options from this option group are read even if the Galera Cluster functionality is not enabled. Available starting with , , and on systems compiled with Galera Cluster support.
See MDEV-18347 for more information.
Prior to , , and , if you were performing a --copy-back operation, and if you did not explicitly specify a value for the --datadir option either on the command line or one of the supported server option groups in an option file, then mariadb-backup
would not default to the server's default datadir
. Instead, mariadb-backup
would fail with an error. For example:
Error: datadir must be specified.
The solution is to explicitly specify a value for the --datadir option either on the command line or in one of the supported server option groups in an option file. For example:
[mysqld]
datadir=/var/lib/mysql
In , , and and later, mariadb-backup will default to the server's default datadir
value.
See MDEV-12956 for more information.
Prior to and , if concurrent DDL was executed while the backup was taken, then that could cause various kinds of problems to occur.
One example is that if DDL caused any tablespace IDs to change (such as TRUNCATE TABLE or RENAME TABLE), then that could cause the effected tables to be inconsistent in the backup. In this scenario, you might see errors about mismatched tablespace IDs when the backup is prepared.
For example, the errors might look like this:
2018-12-07 07:49:32 7f51b3184820 InnoDB: Error: table 'DB1/TAB_TEMP'
InnoDB: in InnoDB data dictionary has tablespace id 1355633,
InnoDB: but a tablespace with that id does not exist. There is
InnoDB: a tablespace of name DB1/TAB_TEMP and id 1354713, though. Have
InnoDB: you deleted or moved .ibd files?
InnoDB: Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/innodb-troubleshooting-datadict.html
InnoDB: for how to resolve the issue.
Or they might look like this:
2018-07-12 21:24:14 139666981324672 [Note] InnoDB: Ignoring data file 'db1/tab1.ibd' with space ID 200485, since the redo log references db1/tab1.ibd with space ID 200484.
Some of the problems related to concurrent DDL are described below.
Problems solved by setting --lock-ddl-per-table (mariadb-backup
command-line option added in ):
If a table is dropped during the backup, then it might still exists after the backup is prepared.
If a table exists when the backup starts, but it is dropped before the backup copies it, then the tablespace file can't be copied, so the backup would fail.
Problems solved by setting --innodb_log_optimize_ddl=OFF (MariaDB Server system variable added in and removed in 10.6.0):
If the backup noticed concurrent DDL, then it might fail with
ALTER TABLE or OPTIMIZE TABLE was executed during backup
Problems solved by --innodb_safe_truncate=ON (MariaDB Server system variable in and removed in 10.3.0):
If a table is created during the backup, then it might not exist in the backup after prepare.
If a table is renamed during the backup after the tablespace file was copied, then the table may not exist after the backup is prepared.
If a table is dropped and created under the same name during the backup after the tablespace file was copied, then the table will have the wrong tablespace ID when the backup is prepared.
Note that, with the removal of innodb_log_optimize_ddl
and innodb_safe_truncate
, the above problems were definitely solved.
Problems solved by other bug fixes:
If --lock-ddl-per-table is used and if a table is concurrently being dropped or renamed, then mariadb-backup can fail to acquire the MDL lock.
These problems are only fixed in MariaDB 10.2 and later, so it is not recommended to execute concurrent DDL when using mariadb-backup
with MariaDB 10.1.
See MDEV-13563, MDEV-13564, MDEV-16809, and MDEV-16791 for more information.
Prior to , mariadb-backup
users could run into issues if they restored a backup by manually copying the files from the backup into the datadir while the directory still contained pre-existing InnoDB redo log files. The backup itself did not contain InnoDB redo log files with the traditional ib_logfileN
file names, so the pre-existing log files would remain in the datadir. If the server were started with these pre-existing log files, then it could perform crash recovery with them, which could cause the database to become inconsistent or corrupt.
In these MariaDB versions, this problem could be avoided by not restoring the backup by manually copying the files and instead restoring the backup by using mariadb-backup
and providing the --copy-back option, since mariadb-backup
deletes pre-existing InnoDB redo log files from the datadir during the restore process.
In and later, mariadb-backup
prevents this issue by creating an empty InnoDB redo log file called ib_logfile0
as part of the --prepare stage. That way, if the backup is manually restored, any pre-existing InnoDB redo log files would get overwritten by the empty one.
See MDEV-13311 for more information.
If mariadb-backup
uses more file descriptors than the system is configured to allow, then users can see errors like the following:
2019-02-12 09:48:38 7ffff7fdb820 InnoDB: Operating system error number 23 in a file operation.
InnoDB: Error number 23 means 'Too many open files in system'.
InnoDB: Some operating system error numbers are described at
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/operating-system-error-codes.html
InnoDB: Error: could not open single-table tablespace file ./db1/tab1.ibd
InnoDB: We do not continue the crash recovery, because the table may become
InnoDB: corrupt if we cannot apply the log records in the InnoDB log to it.
InnoDB: To fix the problem and start mysqld:
InnoDB: 1) If there is a permission problem in the file and mysqld cannot
InnoDB: open the file, you should modify the permissions.
InnoDB: 2) If the table is not needed, or you can restore it from a backup,
InnoDB: then you can remove the .ibd file, and InnoDB will do a normal
InnoDB: crash recovery and ignore that table.
InnoDB: 3) If the file system or the disk is broken, and you cannot remove
InnoDB: the .ibd file, you can set innodb_force_recovery > 0 in my.cnf
InnoDB: and force InnoDB to continue crash recovery here.
Prior to , , and , mariadb-backup
would actually ignore the error and continue the backup. In some of those cases, mariadb-backup
would even report a successful completion of the backup to the user. In later versions, mariadb-backup
will properly throw an error and abort when this error is encountered. See MDEV-19060 for more information.
When this error is encountered, one solution is to explicitly specify a value for the --open-files-limit option either on the command line or in one of the supported server option group s in an option file. For example:
[mariadb-backup]
open_files_limit=65535
An alternative solution is to set the soft and hard limits for the user account that runs mariadb-backup
by adding new limits to /etc/security/limits.conf. For example, if mariadb-backup
is run by the mysql
user, then you could add lines like the following:
mysql soft nofile 65535
mysql hard nofile 65535
After the system is rebooted, the above configuration should set new open file limits for the mysql
user, and the user's ulimit
output should look like the following:
ulimit -Sn
65535
ulimit -Hn
65535
+, +
Stable
+,
Beta
Alpha
How to backup with MariaDB (video)
MariaDB point-in-time recovery (video)
mariadb-backup and Restic (video)
This page is licensed: CC BY-SA / Gnu FDL
mariadb-backup
Options--apply-log
Prepares an existing backup to restore to the MariaDB Server. This is only valid in innobackupex
mode, which can be enabled with the --innobackupex option.
Files that mariadb-backup
generates during --backup operations in the target directory are not ready for use on the Server. Before you can restore the data to MariaDB, you first need to prepare the backup.
In the case of full backups, the files are not point in time consistent, since they were taken at different times. If you try to restore the database without first preparing the data, InnoDB rejects the new data as corrupt. Running mariadb-backup
with the --prepare command readies the data so you can restore it to MariaDB Server. When working with incremental backups, you need to use the --prepare
command and the --incremental-dir option to update the base backup with the deltas from an incremental backup.
mariadb-backup --innobackupex --apply-log
Once the backup is ready, you can use the --copy-back or the --move-back commands to restore the backup to the server.
--apply-log-only
If this option is used when preparing a backup, then only the redo log apply stage are performed, and other stages of crash recovery are ignored. This option is used with incremental backups.
Note: This option is not needed or supported anymore.
--backup
Backs up your databases.
Using this command option, mariadb-backup
performs a backup operation on your database or databases. The backups are written to the target directory, as set by the --target-dir option.
mariadb-backup --backup
--target-dir /path/to/backup \
--user user_name --password user_passwd
mariadb-backup
can perform full and incremental backups. A full backup creates a snapshot of the database in the target directory. An incremental backup checks the database against a previously taken full backup, (defined by the --incremental-basedir option) and creates delta files for these changes.
In order to restore from a backup, you first need to run mariadb-backup
with the --prepare command option, to make a full backup point-in-time consistent or to apply incremental backup deltas to base. Then you can run mariadb-backup
again with either the --copy-back or --move-back commands to restore the database.
For more information, see Full Backup and Restore and Incremental Backup and Restore.
--binlog-info
Defines how mariadb-backup
retrieves the binary log coordinates from the server.
--binlog-info[=OFF | ON | LOCKLESS | AUTO]
The --binlog-info
option supports the following retrieval methods. When no retrieval method is provided, it defaults to AUTO
.
OFF
Disables the retrieval of binary log information
ON
Enables the retrieval of binary log information, performs locking where available to ensure consistency
LOCKLESS
Unsupported option
AUTO
Enables the retrieval of binary log information using ON
or LOCKLESS
where supported
Using this option, you can control how mariadb-backup
retrieves the server's binary log coordinates corresponding to the backup.
When enabled, whether using ON
or AUTO
, mariadb-backup
retrieves information from the binlog during the backup process. When disabled with OFF
, mariadb-backup
runs without attempting to retrieve binary log information. You may find this useful when you need to copy data without metadata like the binlog or replication coordinates.
mariadb-backup --binlog-info --backup
Currently, the LOCKLESS
option depends on features unsupported by MariaDB Server. See the description of the xtrabackup_binlog_pos_innodb file for more information. If you attempt to run mariadb-backup
with this option, then it causes the utility to exit with an error.
--close-files
Defines whether you want to close file handles.
Using this option, you can tell mariadb-backup
that you want to close file handles. Without this option, mariadb-backup
keeps files open in order to manage DDL operations. When working with particularly large tablespaces, closing the file can make the backup more manageable. However, it can also lead to inconsistent backups. Use at your own risk.
mariadb-backup --close-files --prepare
--compress
This option was deprecated as it relies on the no longer maintained QuickLZ library. It are removed in a future release - versions supporting this function will not be affected. It is recommended to instead backup to a stream (stdout), and use a 3rd party compression library to compress the stream, as described in Using Encryption and Compression Tools With mariadb-backup.
Defines the compression algorithm for backup files.
--compress[=compression_algorithm]
The --compress
option only supports the now deprecated quicklz
algorithm.
quicklz
Uses the QuickLZ compression algorithm
mariadb-backup --compress --backup
If a backup is compressed using this option, then mariadb-backup
will record that detail in the xtrabackup_info file.
--compress-chunk-size
Deprecated, for details see the --compress option.
Defines the working buffer size for compression threads.
--compress-chunk-size=#
mariadb-backup
can perform compression operations on the backup files before writing them to disk. It can also use multiple threads for parallel data compression during this process. Using this option, you can set the chunk size each thread uses during compression. It defaults to 64K
.
mariadb-backup --backup --compress \
--compress-threads=12 --compress-chunk-size=5M
To further configure backup compression, see the --compress and --compress-threads options.
--compress-threads
Deprecated, for details see the --compress option.
Defines the number of threads to use in compression.
--compress-threads=#
mariadb-backup
can perform compression operations on the backup files before writing them to disk. Using this option, you can define the number of threads you want to use for this operation. You may find this useful in speeding up the compression of particularly large databases. It defaults to single-threaded.
mariadb-backup --compress --compress-threads=12 --backup
To further configure backup compression, see the --compress and --compress-chunk-size options.
--copy-back
Restores the backup to the data directory.
Using this command, mariadb-backup
copies the backup from the target directory to the data directory, as defined by the --datadir option. You must stop the MariaDB Server before running this command. The data directory must be empty. If you want to overwrite the data directory with the backup, use the --force-non-empty-directories option.
Bear in mind, before you can restore a backup, you first need to run mariadb-backup with the --prepare option. In the case of full backups, this makes the files point-in-time consistent. With incremental backups, this applies the deltas to the base backup. Once the backup is prepared, you can run --copy-back
to apply it to MariaDB Server.
mariadb-backup --copy-back --force-non-empty-directories
Running the --copy-back
command copies the backup files to the data directory. Use this command if you want to save the backup for later. If you don't want to save the backup for later, use the --move-back command.
--core-file
Defines whether to write a core file.
Using this option, you can configure mariadb-backup to dump its core to file in the event that it encounters fatal signals. You may find this useful for review and debugging purposes.
mariadb-backup --core-file --backup
--databases
Defines the databases and tables you want to back up.
--databases="database[.table][ database[.table] ...]"
Using this option, you can define the specific database or databases you want to back up. In cases where you have a particularly large database or otherwise only want to back up a portion of it, you can optionally also define the tables on the database.
mariadb-backup --backup \
--databases="example.table1 example.table2"
In cases where you want to back up most databases on a server or tables on a database, but not all, you can set the specific databases or tables you don't want to back up using the --databases-exclude option.
If a backup is a partial backup, then mariadb-backup will record that detail in the xtrabackup_info file.
In innobackupex
mode, which can be enabled with the --innobackupex option, the --databases
option can be used as described above, or it can be used to refer to a file, just as the --databases-file option can in the normal mode.
--databases-exclude
Defines the databases you don't want to back up.
--databases-exclude="database[.table][ database[.table] ...]"
Using this option, you can define the specific database or databases you want to exclude from the backup process. You may find it useful when you want to back up most databases on the server or tables on a database, but would like to exclude a few from the process.
mariadb-backup --backup \
--databases="example" \
--databases-exclude="example.table1 example.table2"
To include databases in the backup, see the --databases option option
If a backup is a partial backup, then mariadb-backup will record that detail in the xtrabackup_info file.
--databases-file
Defines the path to a file listing databases and/or tables you want to back up.
--databases-file="/path/to/database-file"
Format the databases file to list one element per line, with the following syntax:
database[.table]
In cases where you need to back up a number of databases or specific tables in a database, you may find the syntax for the --databases and --databases-exclude options a little cumbersome. Using this option you can set the path to a file listing the databases or databases and tables you want to back up.
For instance, imagine you list the databases and tables for a backup in a file called main-backup
.
cat main-backup
example1
example2.table1
example2.table2
mariadb-backup --backup --databases-file=main-backup
If a backup is a partial backup, then mariadb-backup will record that detail in the xtrabackup_info file.
-h, --datadir
Defines the path to the database root.
--datadir=PATH
Using this option, you can define the path to the source directory. This is the directory that mariadb-backup reads for the data it backs up. It should be the same as the MariaDB Server datadir system variable.
mariadb-backup --backup -h /var/lib64/mysql
--debug-sleep-before-unlock
This is a debug-only option used by the Xtrabackup test suite.
--decompress
Deprecated, for details see the --compress option.
This option requires that you have the qpress
utility installed on your system.
Defines whether you want to decompress previously compressed backup files.
When you run mariadb-backup with the --compress option, it compresses the subsequent backup files, using the QuickLZ algorithm. Using this option, mariadb-backup decompresses the compressed files from a previous backup.
For instance, run a backup with compression,
mariadb-backup --compress --backup
Then decompress the backup,
mariadb-backup --decompress
You can enable the decryption of multiple files at a time using the --parallel option. By default, mariadb-backup does not remove the compressed files from the target directory. If you want to delete these files, use the --remove-original option.
--debug-sync
Defines the debug sync point. This option is only used by the mariadb-backup test suite.
--defaults-extra-file
Defines the path to an extra default option file.
--defaults-extra-file=/path/to/config
Using this option, you can define an extra default option file for mariadb-backup. Unlike --defaults-file, this file is read after the default option files are read, allowing you to only overwrite the existing defaults.
mariadb-backup --backup \
--defaults-file-extra=addition-config.cnf \
--defaults-file=config.cnf
--defaults-file
Defines the path to the default option file.
--defaults-file=/path/to/config
Using this option, you can define a default option file for mariadb-backup. Unlike the --defaults-extra-file option, when this option is provided, it completely replaces all default option files.
mariadb-backup --backup \
--defaults-file="config.cnf
--defaults-group
Defines the option group to read in the option file.
--defaults-group="name"
In situations where you find yourself using certain mariadb-backup options consistently every time you call it, you can set the options in an option file. The --defaults-group
option defines what option group mariadb-backup reads for its options.
Options you define from the command-line can be set in the configuration file using minor formatting changes. For instance, if you find yourself perform compression operations frequently, you might set --compress-threads and --compress-chunk-size options in this way:
[mariadb-backup]
compress_threads = 12
compress_chunk_size = 64K
Now whenever you run a backup with the --compress option, it always performs the compression using 12 threads and 64K chunks.
mariadb-backup --compress --backup
See mariadb-backup Overview: Server Option Groups and mariadb-backup Overview: Client Option Groups for a list of the option groups read by mariadb-backup by default.
--encrypted-backup
When this option is used with --backup
, if mariadb-backup encounters a page that has a non-zero key_version
value, then mariadb-backup assumes that the page is encrypted.
Use --skip-encrypted-backup
instead to allow mariadb-backup to copy unencrypted tables that were originally created before MySQL 5.1.48.
--export
If this option is provided during the --prepare
stage, then it tells mariadb-backup to create .cfg
files for each InnoDB file-per-table tablespace. These .cfg
files are used to import transportable tablespaces in the process of restoring partial backups and restoring individual tables and partitions.
The --export
option could require rolling back incomplete transactions that had modified the table. This will likely create a "new branch of history" that does not correspond to the server that had been backed up, which makes it impossible to apply another incremental backup on top of such additional changes. The option should only be applied when doing a --prepare
of the last incremental.
mariadb-backup --prepare --export
mariadb-backup did not support the --export option. See MDEV-13466 about that. In earlier versions of MariaDB, this means that mariadb-backup could not create .cfg
files for InnoDB file-per-table tablespaces during the --prepare
stage. You can still import file-per-table tablespaces without the .cfg
files in many cases, so it may still be possible in those versions to restore partial backups or to restore individual tables and partitions with just the .ibd
files. If you have a full backup and you need to create .cfg
files for InnoDB file-per-table tablespaces, then you can do so by preparing the backup as usual without the --export
option, and then restoring the backup, and then starting the server. At that point, you can use the server's built-in features to copy the transportable tablespaces.
--extra-lsndir
Saves an extra copy of the xtrabackup_checkpoints and xtrabackup_info files into the given directory.
--extra-lsndir=PATH
When using the --backup command option, mariadb-backup produces a number of backup files in the target directory. Using this option, you can have mariadb-backup produce additional copies of the xtrabackup_checkpoints and xtrabackup_info files in the given directory.
mariadb-backup --extra-lsndir=extras/ --backup
This is especially usefull when using --stream for streaming output, e.g. for compression and/or encryption using external tools in combination with incremental backups, as the xtrabackup_checkpoints file necessary to determine the LSN to continue the incremental backup from is still accessible without uncompressing / decrypting the backup file first. Simply pass in the --extra-lsndir
of the previous backup as--incremental-basedir
--force-non-empty-directories
Allows --copy-back or --move-back command options to use non-empty target directories.
When using mariadb-backup with the --copy-back or --move-back command options, they normally require a non-empty target directory to avoid conflicts. Using this option with either of command allows mariadb-backup to use a non-empty directory.
mariadb-backup --force-non-empty-directories --copy-back
Bear in mind that this option does not enable overwrites. When copying or moving files into the target directory, if mariadb-backup finds that the target file already exists, it fails with an error.
--ftwrl-wait-query-type
Defines the type of query allowed to complete before mariadb-backup issues the global lock.
--ftwrl-wait-query-type=[ALL | UPDATE | SELECT]
The --ftwrl-wait-query-type
option supports the following query types. The default value is ALL
.
ALL
Waits until all queries complete before issuing the global lock
SELECT
Waits until SELECT statements complete before issuing the global lock
UPDATE
Waits until UPDATE statements complete before issuing the global lock
When mariadb-backup runs, it issues a global lock to prevent data from changing during the backup process. When it encounters a statement in the process of executing, it waits until the statement is finished before issuing the global lock. Using this option, you can modify this default behavior to ensure that it waits only for certain query types, such as for SELECT and UPDATE statements.
mariadb-backup --backup \
--ftwrl-wait-query-type=UPDATE
--ftwrl-wait-threshold
Defines the minimum threshold for identifying long-running queries for FTWRL.
--ftwrl-wait-threshold=#
When mariadb-backup runs, it issues a global lock to prevent data from changing during the backup process and ensure a consistent record. If it encounters statements still in the process of executing, it waits until they complete before setting the lock. Using this option, you can set the threshold at which mariadb-backup engages FTWRL. When it --ftwrl-wait-timeout is not 0 and a statement has run for at least the amount of time given this argument, mariadb-backup waits until the statement completes or until the --ftwrl-wait-timeout expires before setting the global lock and starting the backup.
mariadb-backup --backup \
--ftwrl-wait-timeout=90 \
--ftwrl-wait-threshold=30
--ftwrl-wait-timeout
Defines the timeout to wait for queries before trying to acquire the global lock. The global lock refers to BACKUP STAGE BLOCK_COMMIT. The global lock refers to FLUSH TABLES WITH READ LOCK (FTWRL).
--ftwrl-wait-timeout=#
When mariadb-backup runs, it acquires a global lock to prevent data from changing during the backup process and ensure a consistent record. If it encounters statements still in the process of executing, it can be configured to wait until the statements complete before trying to acquire the global lock.
If the --ftwrl-wait-timeout
is set to 0, then mariadb-backup tries to acquire the global lock immediately without waiting. This is the default value.
If the --ftwrl-wait-timeout
is set to a non-zero value, then mariadb-backup waits for the configured number of seconds until trying to acquire the global lock.
Starting in MariaDB 10.5.3, mariadb-backup will exit if it can't acquire the global lock after waiting for the configured number of seconds. In earlier versions, it could wait for the global lock indefinitely, even if --ftwrl-wait-timeout
was set to a non-zero value.
mariadb-backup --backup \
--ftwrl-wait-query-type=UPDATE \
--ftwrl-wait-timeout=5
--galera-info
Defines whether you want to back up information about a Galera Cluster node's state.
When this option is used, mariadb-backup creates an additional file called xtrabackup_galera_info, which records information about a Galera Cluster node's state. It records the values of the wsrep_local_state_uuid and wsrep_last_committed status variables.
You should only use this option when backing up a Galera Cluster node. If the server is not a Galera Cluster node, then this option has no effect.
This option, when enabled and used with GTID replication, will rotate the binary logs at backup time.
mariadb-backup --backup --galera-info
--history
Defines whether you want to track backup history in the PERCONA_SCHEMA.xtrabackup_history
table.
--history[=name]
When using this option, mariadb-backup records its operation in a table on the MariaDB Server. Passing a name to this option allows you group backups under arbitrary terms for later processing and analysis.
mariadb-backup --backup --history=backup_all
Currently, the table it uses by default is named mysql.mariadb_backup_history
. Prior to , the default table was PERCONA_SCHEMA.xtrabackup_history
.
mariadb-backup will also record this in the xtrabackup_info file.
-H, --host
Defines the host for the MariaDB Server you want to backup.
--host=name
Using this option, you can define the host to use when connecting to a MariaDB Server over TCP/IP. By default, mariadb-backup attempts to connect to the local host.
mariadb-backup --backup \
--host="example.com"
--include
This option is a regular expression to be matched against table names in databasename.tablename format. It is
equivalent to the --tables option. This is only valid in innobackupex
mode, which can be enabled with the --innobackupex option.
--incremental
Defines whether you want to take an increment backup, based on another backup. This is only valid in innobackupex
mode, which can be enabled with the --innobackupex option.
mariadb-backup --innobackupex --incremental
Using this option with the --backup command option makes the operation incremental rather than a complete overwrite. When this option is specified, either the --incremental-lsn or[--incremental-basedir](mariadb-backup-options.md#-incremental-basedir) options can also be given. If neither option is given, option [--incremental-basedir](mariadb-backup-options.md#-incremental-basedir) is used by default, set to the first timestamped backup directory in the backup base directory.
mariadb-backup --innobackupex --backup --incremental \
--incremental-basedir=/data/backups \
--target-dir=/data/backups
If a backup is a incremental backup, then mariadb-backup will record that detail in the xtrabackup_info file.
--incremental-basedir
Defines whether you want to take an incremental backup, based on another backup.
--incremental-basedir=PATH
Using this option with the --backup command option makes the operation incremental rather than a complete overwrite. mariadb-backup will only copy pages from .ibd
files if they are newer than the backup in the specified directory.
mariadb-backup --backup \
--incremental-basedir=/data/backups \
--target-dir=/data/backups
If a backup is a incremental backup, then mariadb-backup will record that detail in the xtrabackup_info file.
--incremental-dir
Defines whether you want to take an incremental backup, based on another backup.
--increment-dir=PATH
Using this option with --prepare command option makes the operation incremental rather than a complete overwrite. mariadb-backup will apply .delta
files and log files into the target directory.
mariadb-backup --prepare \
--increment-dir=backups/
If a backup is a incremental backup, then mariadb-backup will record that detail in the xtrabackup_info file.
--incremental-force-scan
Defines whether you want to force a full scan for incremental backups.
When using mariadb-backup to perform an incremental backup, this option forces it to also perform a full scan of the data pages being backed up, even when there's bitmap data on the changes. MariaDB does not support changed page bitmaps, so this option is useless in those versions. See MDEV-18985 for more information.
mariadb-backup --backup \
--incremental-basedir=/path/to/target \
--incremental-force-scan
--incremental-history-name
Defines a logical name for the backup.
--incremental-history-name=name
mariadb-backup can store data about its operations on the MariaDB Server. Using this option, you can define the logical name it uses in identifying the backup.
mariadb-backup --backup \
--incremental-history-name=morning_backup
Currently, the table it uses by default is named mysql.mariadb_backup_history
. Prior to , the default table was PERCONA_SCHEMA.xtrabackup_history
.
mariadb-backup will also record this in the xtrabackup_info file.
--incremental-history-uuid
Defines a UUID for the backup.
--incremental-history-uuid=name
mariadb-backup can store data about its operations on the MariaDB Server. Using this option, you can define the UUID it uses in identifying a previous backup to increment from. It checks --incremental-history-name, --incremental-basedir, and --incremental-lsn. If mariadb-backup fails to find a valid lsn, it generates an error.
mariadb-backup --backup \
--incremental-history-uuid=main-backup012345678
Currently, the table it uses is named PERCONA_SCHEMA.xtrabackup_history
, but expect that name to change in future releases. See MDEV-19246 for more information.
mariadb-backup will also record this in the xtrabackup_info file.
--incremental-lsn
Defines the sequence number for incremental backups.
--incremental-lsn=name
Using this option, you can define the sequence number (LSN) value for --backup operations. During backups, mariadb-backup only copies .ibd
pages newer than the specified values.
WARNING: Incorrect LSN values can make the backup unusable. It is impossible to diagnose this issue.
--innobackupex
Deprecated
Enables innobackupex
mode, which is a compatibility mode.
mariadb-backup --innobackupex
In innobackupex
mode, mariadb-backup has the following differences:
To prepare a backup, the --apply-log option is used instead of the --prepare option.
To create an incremental backup, the --incremental option is supported.
The --no-timestamp option is supported.
To create a partial backup, the --include option is used instead of the --tables option.
To create a partial backup, the --databases option can still be used, but it's behavior changes slightly.
The --target-dir option is not used to specify the backup directory. The backup directory should instead be specified as a standalone argument.
The primary purpose of innobackupex
mode is to allow scripts and tools to more easily migrate to mariadb-backup if they were originally designed to use the innobackupex
utility that is included with Percona XtraBackup. It is not recommended to use this mode in new scripts, since it is not guaranteed to be supported forever. See MDEV-20552 for more information.
--innodb
This option has no effect. Set only for MySQL option compatibility.
--innodb-adaptive-hash-index
Enables InnoDB Adaptive Hash Index.
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option you can explicitly enable the InnoDB Adaptive Hash Index. This feature is enabled by default for mariadb-backup. If you want to disable it, use --skip-innodb-adaptive-hash-index.
mariadb-backup --backup \
--innodb-adaptive-hash-index
--innodb-autoextend-increment
Defines the increment in megabytes for auto-extending the size of tablespace file.
--innodb-autoextend-increment=36
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can set the increment in megabytes for automatically extending the size of tablespace data file in InnoDB.
mariadb-backup --backup \
--innodb-autoextend-increment=35
--innodb-buffer-pool-filename
Using this option has no effect. It is available to provide compatibility with the MariaDB Server.
--innodb-buffer-pool-size
Defines the memory buffer size InnoDB uses the cache data and indexes of the table.
--innodb-buffer-pool-size=124M
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can configure the buffer pool for InnoDB operations.
mariadb-backup --backup \
--innodb-buffer-pool-size=124M
--innodb-checksum-algorithm
innodb_checksum_algorithm was deprecated in MariaDB 10.5.10 and removed in .
In earlier versions, it is used to define the checksum algorithm.
--innodb-checksum-algorithm=crc32
| strict_crc32
| innodb
| strict_innodb
| none
| strict_none
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can specify the algorithm mariadb-backup uses when checksumming on InnoDB tables. Currently, MariaDB supports the following algorithms CRC32
, STRICT_CRC32
, INNODB
, STRICT_INNODB
, NONE
, STRICT_NONE
.
mariadb-backup --backup \
---innodb-checksum-algorithm=strict_innodb
--innodb-data-file-path
Defines the path to individual data files.
--innodb-data-file-path=/path/to/file
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option you can define the path to InnoDB data files. Each path is appended to the --innodb-data-home-dir option.
mariadb-backup --backup \
--innodb-data-file-path=ibdata1:13M:autoextend \
--innodb-data-home-dir=/var/dbs/mysql/data
--innodb-data-home-dir
Defines the home directory for InnoDB data files.
--innodb-data-home-dir=PATH
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option you can define the path to the directory containing InnoDB data files. You can specific the files using the --innodb-data-file-path option.
mariadb-backup --backup \
--innodb-data-file-path=ibdata1:13M:autoextend \
--innodb-data-home-dir=/var/dbs/mysql/data
--innodb-doublewrite
Enables doublewrites for InnoDB tables.
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. When using this option, mariadb-backup improves fault tolerance on InnoDB tables with a doublewrite buffer. By default, this feature is enabled. Use this option to explicitly enable it. To disable doublewrites, use the --skip-innodb-doublewrite option.
mariadb-backup --backup \
--innodb-doublewrite
--innodb-encrypt-log
Defines whether you want to encrypt InnoDB logs.
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can tell mariadb-backup that you want to encrypt logs from its InnoDB activity.
--innodb-file-io-threads
Defines the number of file I/O threads in InnoDB.
--innodb-file-io-threads=#
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can define the number of file I/O threads mariadb-backup uses on InnoDB tables.
mariadb-backup --backup \
--innodb-file-io-threads=5
--innodb-file-per-table
Defines whether you want to store each InnoDB table as an .ibd
file.
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option causes mariadb-backup to store each InnoDB table as an .ibd
file in the target directory.
--innodb-flush-method
Defines the data flush method. Ignored from .
--innodb-flush-method=fdatasync
| O_DSYNC
| O_DIRECT
| O_DIRECT_NO_FSYNC
| ALL_O_DIRECT
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can define the data flush method mariadb-backup uses with InnoDB tables.
mariadb-backup --backup \
--innodb-flush-method==_DIRECT_NO_FSYNC
--innodb-io-capacity
Defines the number of IOP's the utility can perform.
--innodb-io-capacity=#
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can limit the I/O activity for InnoDB background tasks. It should be set around the number of I/O operations per second that the system can handle, based on drive or drives being used.
mariadb-backup --backup \
--innodb-io-capacity=200
--innodb-log-checksums
Defines whether to include checksums in the InnoDB logs.
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can explicitly set mariadb-backup to include checksums in the InnoDB logs. The feature is enabled by default. To disable it, use the --skip-innodb-log-checksums option.
mariadb-backup --backup \
--innodb-log-checksums
--innodb-log-buffer-size
This option has no functionality in mariadb-backup. It exists for MariaDB Server compatibility.
--innodb-log-files-in-group
This option has no functionality in mariadb-backup. It exists for MariaDB Server compatibility.
--innodb-log-group-home-dir
Defines the path to InnoDB log files.
--innodb-log-group-home-dir=PATH
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can define the path to InnoDB log files.
mariadb-backup --backup \
--innodb-log-group-home-dir=/path/to/logs
--innodb-max-dirty-pages-pct
Defines the percentage of dirty pages allowed in the InnoDB buffer pool.
--innodb-max-dirty-pages-pct=#
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can define the maximum percentage of dirty, (that is, unwritten) pages that mariadb-backup allows in the InnoDB buffer pool.
mariadb-backup --backup \
--innodb-max-dirty-pages-pct=80
--innodb-open-files
Defines the number of files kept open at a time.
--innodb-open-files=#
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can set the maximum number of files InnoDB keeps open at a given time during backups.
mariadb-backup --backup \
--innodb-open-files=10
--innodb-page-size
Defines the universal page size.
--innodb-page-size=#
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can define the universal page size in bytes for mariadb-backup.
mariadb-backup --backup \
--innodb-page-size=16k
--innodb-read-io-threads
Defines the number of background read I/O threads in InnoDB.
--innodb-read-io-threads=#
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can set the number of I/O threads MariaDB uses when reading from InnoDB.
mariadb-backup --backup \
--innodb-read-io-threads=4
--innodb-undo-directory
Defines the directory for the undo tablespace files.
--innodb-undo-directory=PATH
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can define the path to the directory where you want MariaDB to store the undo tablespace on InnoDB tables. The path can be absolute.
mariadb-backup --backup \
--innodb-undo-directory=/path/to/innodb_undo
--innodb-undo-tablespaces
Defines the number of undo tablespaces to use.
--innodb-undo-tablespaces=#
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can define the number of undo tablespaces you want to use during the backup.
mariadb-backup --backup \
--innodb-undo-tablespaces=10
--innodb-use-native-aio
Defines whether you want to use native AI/O.
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can enable the use of the native asynchronous I/O subsystem. It is only available on Linux operating systems.
mariadb-backup --backup \
--innodb-use-native-aio
--innodb-write-io-threads
Defines the number of background write I/O threads in InnoDB.
--innodb-write-io-threads=#
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can set the number of background write I/O threads mariadb-backup uses.
mariadb-backup --backup \
--innodb-write-io-threads=4
--kill-long-queries-timeout
Defines the timeout for blocking queries.
--kill-long-queries-timeout=#
When mariadb-backup runs, it issues a FLUSH TABLES WITH READ LOCK
statement. It then identifies blocking queries. Using this option you can set a timeout in seconds for these blocking queries. When the time runs out, mariadb-backup kills the queries.
The default value is 0, which causes mariadb-backup to not attempt killing any queries.
mariadb-backup --backup \
--kill-long-queries-timeout=10
--kill-long-query-type
Defines the query type the utility can kill to unblock the global lock.
--kill-long-query-type=ALL | UPDATE | SELECT
When mariadb-backup encounters a query that sets a global lock, it can kill the query in order to free up MariaDB Server for the backup. Using this option, you can choose the types of query it kills: SELECT, UPDATE, or both set with ALL
. The default is ALL
.
mariadb-backup --backup \
--kill-long-query-type=UPDATE
--lock-ddl-per-table
Prevents DDL for each table to be backed up by acquiring MDL lock on that. NOTE: Unless --no-lock option was also specified, conflicting DDL queries , are killed at the end of backup This is done avoid deadlock between "FLUSH TABLE WITH READ LOCK", user's DDL query (ALTER, RENAME), and MDL lock on table.
--log
This option has no functionality. It is set to ensure compatibility with MySQL.
--log-bin
Defines the base name for the log sequence.
--log-bin[=name]
Using this option you, you can set the base name for mariadb-backup to use in log sequences.
--log-copy-interval
Defines the copy interval between checks done by the log copying thread.
--log-copy-interval=#
Using this option, you can define the copy interval mariadb-backup uses between checks done by the log copying thread. The given value is in milliseconds.
mariadb-backup --backup \
--log-copy-interval=50
--log-innodb-page-corruption
Continue backup if InnoDB corrupted pages are found. The pages are logged in innodb_corrupted_pages
and backup is finished with error. --prepare will try to fix corrupted pages. If innodb_corrupted_pages
exists after --prepare in base backup directory, backup still contains corrupted pages and can not be considered as consistent.
Added in MariaDB 10.5.9
--move-back
Restores the backup to the data directory.
Using this command, mariadb-backup moves the backup from the target directory to the data directory, as defined by the --datadir option. You must stop the MariaDB Server before running this command. The data directory must be empty. If you want to overwrite the data directory with the backup, use the --force-non-empty-directories option.
Bear in mind, before you can restore a backup, you first need to run mariadb-backup with the --prepare option. In the case of full backups, this makes the files point-in-time consistent. With incremental backups, this applies the deltas to the base backup. Once the backup is prepared, you can run --move-back
to apply it to MariaDB Server.
mariadb-backup --move-back \
--datadir=/var/mysql
Running the --move-back
command moves the backup files to the data directory. Use this command if you don't want to save the backup for later. If you do want to save the backup for later, use the --copy-back command.
--mysqld
Used internally to prepare a backup.
--no-backup-locks
mariadb-backup locks the database by default when it runs. This option disables support for Percona Server's backup locks.
When backing up Percona Server, mariadb-backup would use backup locks by default. To be specific, backup locks refers to the LOCK TABLES FOR BACKUP
and LOCK BINLOG FOR BACKUP
statements. This option can be used to disable support for Percona Server's backup locks. This option has no effect when the server does not support Percona's backup locks.
Deprecated and has no effect from , , and as MariaDB will now always use backup locks for better performance. See MDEV-32932.
mariadb-backup --backup --no-backup-locks
--no-lock
Disables table locks with the FLUSH TABLE WITH READ LOCK
statement.
Using this option causes mariadb-backup to disable table locks with the FLUSH TABLE WITH READ LOCK
statement. Only use this option if:
You are not executing DML statements on non-InnoDB tables during the backup. This includes the mysql
database system tables (which are MyISAM).
You are not executing any DDL statements during the backup.
You are not using the file "xtrabackup_binlog_info", which is not consistent with the data when --no-lock is used. Use the file "xtrabackup_binlog_pos_innodb" [link] instead.
All tables you're backing up use the InnoDB storage engine.
mariadb-backup --backup --no-lock
If you're considering --no-lock
due to backups failing to acquire locks, this may be due to incoming replication events preventing the lock. Consider using the --safe-slave-backup option to momentarily stop the replica thread. This alternative may help the backup to succeed without resorting to --no-lock
.
The --no-lock option only provides a consistent backup if the user ensures that no DDL or non-transactional table updates occur during the backup. The --no-lock option is not supported by MariaDB plc.
--no-timestamp
This option prevents creation of a time-stamped subdirectory of the BACKUP-ROOT-DIR given on the command line. When it is specified, the backup is done in BACKUP-ROOT-DIR instead. This is only valid in innobackupex
mode, which can be enabled with the --innobackupex option.
--no-version-check
Disables version check.
Using this option, you can disable mariadb-backup version check.
mariadb-backup --backup --no-version-check
--open-files-limit
Defines the maximum number of file descriptors.
--open-files-limit=#
Using this option, you can define the maximum number of file descriptors mariadb-backup reserves with setrlimit()
.
mariadb-backup --backup \
--open-files-limit=
--parallel
Defines the number of threads to use for parallel data file transfer.
--parallel=#
Using this option, you can set the number of threads mariadb-backup uses for parallel data file transfers. By default, it is set to 1.
-p, --password
Defines the password to use to connect to MariaDB Server.
--password=passwd
When you run mariadb-backup, it connects to MariaDB Server in order to access and back up the databases and tables. Using this option, you can set the password mariadb-backup uses to access the server. To set the user, use the --user option.
mariadb-backup --backup \
--user=root \
--password=root_password
--plugin-dir
Defines the directory for server plugins.
--plugin-dir=PATH
Using this option, you can define the path mariadb-backup reads for MariaDB Server plugins. It only uses it during the --prepare phase to load the encryption plugin. It defaults to the plugin_dir server system variable.
mariadb-backup --backup \
--plugin-dir=/var/mysql/lib/plugin
--plugin-load
Defines the encryption plugins to load.
--plugin-load=name
Using this option, you can define the encryption plugin you want to load. It is only used during the --prepare phase to load the encryption plugin. It defaults to the server --plugin-load
option.
The option was removed.
-P, --port
Defines the server port to connect to.
--port=#
When you run mariadb-backup, it connects to MariaDB Server in order to access and back up your databases and tables. Using this option, you can set the port the utility uses to access the server over TCP/IP. To set the host, see the --host option. Use mysql --help
for more details.
mariadb-backup --backup \
--host=192.168.11.1 \
--port=3306
--prepare
Prepares an existing backup to restore to the MariaDB Server.
Files that mariadb-backup generates during --backup operations in the target directory are not ready for use on the Server. Before you can restore the data to MariaDB, you first need to prepare the backup.
In the case of full backups, the files are not point in time consistent, since they were taken at different times. If you try to restore the database without first preparing the data, InnoDB rejects the new data as corrupt. Running mariadb-backup with the --prepare
command readies the data so you can restore it to MariaDB Server. When working with incremental backups, you need to use the --prepare
command and the --incremental-dir option to update the base backup with the deltas from an incremental backup.
mariadb-backup --prepare
Once the backup is ready, you can use the --copy-back or the --move-back commands to restore the backup to the server.
--print-defaults
Prints the utility argument list, then exits.
Using this argument, MariaDB prints the argument list to stdout and then exits. You may find this useful in debugging to see how the options are set for the utility.
mariadb-backup --print-defaults
--print-param
Prints the MariaDB Server options needed for copyback.
Using this option, mariadb-backup prints to stdout the MariaDB Server options that the utility requires to run the --copy-back command option.
mariadb-backup --print-param
--rollback-xa
By default, mariadb-backup will not commit or rollback uncommitted XA transactions, and when the backup is restored, any uncommitted XA transactions must be manually committed using XA COMMIT
or manually rolled back using XA ROLLBACK
.
MariaDB starting with 10.5
mariadb-backup's --rollback-xa
option is not present because the server has more robust ways of handling uncommitted XA transactions.
This is an experimental option. Do not use this option in older versions. Older implementation can cause corruption of InnoDB data.
--rsync
Defines whether to use rsync.
During normal operation, mariadb-backup transfers local non-InnoDB files using a separate call to cp
for each file. Using this option, you can optimize this process by performing this transfer with rsync, instead.
mariadb-backup --backup --rsync
This option is not compatible with the --stream option.
Deprecated and has no effect from , , and as rsync will not work on tables that are in use. See MDEV-32932.
--safe-slave-backup
Stops replica SQL threads for backups.
When running mariadb-backup on a server that uses replication, you may occasionally encounter locks that block backups. Using this option, it stops replica SQL threads and waits until the Slave_open_temp_tables
in the SHOW STATUS
statement is zero. If there are no open temporary tables, the backup runs, otherwise the SQL thread starts and stops until there are no open temporary tables.
mariadb-backup --backup \
--safe-slave-backup \
--safe-slave-backup-timeout=500
The backup fails if the Slave_open_temp_tables
doesn't reach zero after the timeout period set by the --safe-slave-backup-timeout option.
--safe-slave-backup-timeout
Defines the timeout for replica backups.
--safe-slave-backup-timeout=#
When running mariadb-backup on a server that uses replication, you may occasionally encounter locks that block backups. With the --safe-slave-backup option, it waits until the Slave_open_temp_tables
in the SHOW STATUS
statement reaches zero. Using this option, you set how long it waits. It defaults to 300.
mariadb-backup --backup \
--safe-slave-backup \
--safe-slave-backup-timeout=500
--secure-auth
Refuses client connections to servers using the older protocol.
Using this option, you can set it explicitly to refuse client connections to the server when using the older protocol, from before 4.1.1. This feature is enabled by default. Use the --skip-secure-auth option to disable it.
mariadb-backup --backup --secure-auth
--skip-innodb-adaptive-hash-index
Disables InnoDB Adaptive Hash Index.
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option you can explicitly disable the InnoDB Adaptive Hash Index. This feature is enabled by default for mariadb-backup. If you want to explicitly enable it, use --innodb-adaptive-hash-index.
mariadb-backup --backup \
--skip-innodb-adaptive-hash-index
--skip-innodb-doublewrite
Disables doublewrites for InnoDB tables.
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. When doublewrites are enabled, InnoDB improves fault tolerance with a doublewrite buffer. By default this feature is turned on. Using this option you can disable it for mariadb-backup. To explicitly enable doublewrites, use the --innodb-doublewrite option.
mariadb-backup --backup \
--skip-innodb-doublewrite
--skip-innodb-log-checksums
Defines whether to exclude checksums in the InnoDB logs.
mariadb-backup initializes its own embedded instance of InnoDB using the same configuration as defined in the configuration file. Using this option, you can set mariadb-backup to exclude checksums in the InnoDB logs. The feature is enabled by default. To explicitly enable it, use the --innodb-log-checksums option.
--skip-secure-auth
Refuses client connections to servers using the older protocol.
Using this option, you can set it accept client connections to the server when using the older protocol, from before 4.1.1. By default, it refuses these connections. Use the --secure-auth option to explicitly enable it.
mariadb-backup --backup --skip-secure-auth
--slave-info
Prints the binary log position and the name of the primary server.
If the server is a replica, then this option causes mariadb-backup to print the hostname of the replica's replication primary and the binary log file and position of the replica's SQL thread to stdout
.
This option also causes mariadb-backup to record this information as a CHANGE MASTER command that can be used to set up a new server as a replica of the original server's primary after the backup has been restored. This information are written to the xtrabackup_slave_info file.
mariadb-backup does not check if GTIDs are being used in replication. It takes a shortcut and assumes that if the gtid_slave_pos system variable is non-empty, then it writes the CHANGE MASTER command with the MASTER_USE_GTID option set to slave_pos
. Otherwise, it writes the CHANGE MASTER command with the MASTER_LOG_FILE and MASTER_LOG_POS options using the primary's binary log file and position. See MDEV-19264 for more information.
mariadb-backup --slave-info
-S, --socket
Defines the socket for connecting to local database.
--socket=name
Using this option, you can define the UNIX domain socket you want to use when connecting to a local database server. The option accepts a string argument. For more information, see the mysql --help
command.
mariadb-backup --backup \
--socket=/var/mysql/mysql.sock
--ssl
Enables TLS. By using this option, you can explicitly configure mariadb-backup to encrypt its connection with TLS when communicating with the server. You may find this useful when performing backups in environments where security is extra important or when operating over an insecure network.
TLS is also enabled even without setting this option when certain other TLS options are set. For example, see the descriptions of the following options:
--ssl-ca
--ssl-capath
--ssl-cert
--ssl-cipher
--ssl-key
--ssl-ca
Defines a path to a PEM file that should contain one or more X509 certificates for trusted Certificate Authorities (CAs) to use for TLS. This option requires that you use the absolute path, not a relative path. For example:
--ssl-ca=/etc/my.cnf.d/certificates/ca.pem
This option is usually used with other TLS options. For example:
mariadb-backup --backup \
--ssl-cert=/etc/my.cnf.d/certificates/client-cert.pem \
--ssl-key=/etc/my.cnf.d/certificates/client-key.pem \
--ssl-ca=/etc/my.cnf.d/certificates/ca.pem
See Secure Connections Overview: Certificate Authorities (CAs) for more information.
This option implies the --ssl option.
--ssl-capath
Defines a path to a directory that contains one or more PEM files that should each contain one X509 certificate for a trusted Certificate Authority (CA) to use for TLS. This option requires that you use the absolute path, not a relative path. For example:
--ssl-capath=/etc/my.cnf.d/certificates/ca/
This option is usually used with other TLS options. For example:
mariadb-backup --backup \
--ssl-cert=/etc/my.cnf.d/certificates/client-cert.pem \
--ssl-key=/etc/my.cnf.d/certificates/client-key.pem \
--ssl-ca=/etc/my.cnf.d/certificates/ca.pem \
--ssl-capath=/etc/my.cnf.d/certificates/ca/
The directory specified by this option needs to be run through the openssl rehash command.
See Secure Connections Overview: Certificate Authorities (CAs) for more information
This option implies the --ssl option.
--ssl-cert
Defines a path to the X509 certificate file to use for TLS. This option requires that you use the absolute path, not a relative path. For example:
--ssl-cert=/etc/my.cnf.d/certificates/client-cert.pem
This option is usually used with other TLS options. For example:
mariadb-backup --backup \
--ssl-cert=/etc/my.cnf.d/certificates/client-cert.pem \
--ssl-key=/etc/my.cnf.d/certificates/client-key.pem \
--ssl-ca=/etc/my.cnf.d/certificates/ca.pem
This option implies the --ssl option.
--ssl-cipher
Defines the list of permitted ciphers or cipher suites to use for TLS. For example:
--ssl-cipher=name
This option is usually used with other TLS options. For example:
mariadb-backup --backup \
--ssl-cert=/etc/my.cnf.d/certificates/client-cert.pem \
--ssl-key=/etc/my.cnf.d/certificates/client-key.pem \
--ssl-ca=/etc/my.cnf.d/certificates/ca.pem
--ssl-cipher=TLSv1.2
To determine if the server restricts clients to specific ciphers, check the ssl_cipher system variable.
This option implies the --ssl option.
--ssl-crl
Defines a path to a PEM file that should contain one or more revoked X509 certificates to use for TLS. This option requires that you use the absolute path, not a relative path. For example:
--ssl-crl=/etc/my.cnf.d/certificates/crl.pem
This option is usually used with other TLS options. For example:
mariadb-backup --backup \
--ssl-cert=/etc/my.cnf.d/certificates/client-cert.pem \
--ssl-key=/etc/my.cnf.d/certificates/client-key.pem \
--ssl-ca=/etc/my.cnf.d/certificates/ca.pem \
--ssl-crl=/etc/my.cnf.d/certificates/crl.pem
See Secure Connections Overview: Certificate Revocation Lists (CRLs) for more information.
This option is only supported if mariadb-backup was built with OpenSSL. If mariadb-backup was built with yaSSL, then this option is not supported. See TLS and Cryptography Libraries Used by MariaDB for more information about which libraries are used on which platforms.
--ssl-crlpath
Defines a path to a directory that contains one or more PEM files that should each contain one revoked X509 certificate to use for TLS. This option requires that you use the absolute path, not a relative path. For example:
--ssl-crlpath=/etc/my.cnf.d/certificates/crl/
This option is usually used with other TLS options. For example:
mariadb-backup --backup \
--ssl-cert=/etc/my.cnf.d/certificates/client-cert.pem \
--ssl-key=/etc/my.cnf.d/certificates/client-key.pem \
--ssl-ca=/etc/my.cnf.d/certificates/ca.pem \
--ssl-crlpath=/etc/my.cnf.d/certificates/crl/
The directory specified by this option needs to be run through the openssl rehash command.
See Secure Connections Overview: Certificate Revocation Lists (CRLs) for more information.
This option is only supported if mariadb-backup was built with OpenSSL. If mariadb-backup was built with yaSSL, then this option is not supported. See TLS and Cryptography Libraries Used by MariaDB for more information about which libraries are used on which platforms.
--ssl-key
Defines a path to a private key file to use for TLS. This option requires that you use the absolute path, not a relative path. For example:
--ssl-key=/etc/my.cnf.d/certificates/client-key.pem
This option is usually used with other TLS options. For example:
mariadb-backup --backup \
--ssl-cert=/etc/my.cnf.d/certificates/client-cert.pem \
--ssl-key=/etc/my.cnf.d/certificates/client-key.pem \
--ssl-ca=/etc/my.cnf.d/certificates/ca.pem
This option implies the --ssl option.
--ssl-verify-server-cert
Enables server certificate verification. This option is disabled by default.
This option is usually used with other TLS options. For example:
mariadb-backup --backup \
--ssl-cert=/etc/my.cnf.d/certificates/client-cert.pem \
--ssl-key=/etc/my.cnf.d/certificates/client-key.pem \
--ssl-ca=/etc/my.cnf.d/certificates/ca.pem \
--ssl-verify-server-cert
--stream
Streams backup files to stdout.
--stream=xbstream
Using this command option, you can set mariadb-backup to stream the backup files to stdout in the given format. Currently, the supported format is xbstream
.
mariadb-backup --stream=xbstream > backup.xb
To extract all files from the xbstream archive into a directory use the mbstream
utility
mbstream -x < backup.xb
If a backup is streamed, then mariadb-backup will record the format in the xtrabackup_info file.
--tables
Defines the tables you want to include in the backup.
--tables=REGEX
Using this option, you can define what tables you want mariadb-backup to back up from the database. The table values are defined using Regular Expressions. To define the tables you want to exclude from the backup, see the --tables-exclude option.
mariadb-backup --backup \
--databases=example
--tables=nodes_* \
--tables-exclude=nodes_tmp
If a backup is a partial backup, then mariadb-backup will record that detail in the xtrabackup_info file.
--tables-exclude
Defines the tables you want to exclude from the backup.
--tables-exclude=REGEX
Using this option, you can define what tables you want mariadb-backup to exclude from the backup. The table values are defined using Regular Expressions. To define the tables you want to include from the backup, see the --tables option.
mariadb-backup --backup \
--databases=example
--tables=nodes_* \
--tables-exclude=nodes_tmp
If a backup is a partial backup, then mariadb-backup will record that detail in the xtrabackup_info file.
--tables-file
Defines path to file with tables for backups.
--tables-file=/path/to/file
Using this option, you can set a path to a file listing the tables you want to back up. mariadb-backup iterates over each line in the file. The format is database.table
.
mariadb-backup --backup \
--databases=example \
--tables-file=/etc/mysql/backup-file
If a backup is a partial backup, then mariadb-backup will record that detail in the xtrabackup_info file.
--target-dir
Defines the destination directory.
--target-dir=/path/to/target
Using this option you can define the destination directory for the backup. mariadb-backup writes all backup files to this directory. mariadb-backup will create the directory, if it does not exist (but it will not create the full path recursively, i.e. at least parent directory if the --target-dir must exist=
mariadb-backup --backup \
--target-dir=/data/backups
--throttle
Defines the limit for I/O operations per second in IOS values.
--throttle=#
Using this option, you can set a limit on the I/O operations mariadb-backup performs per second in IOS values. It is only used during the --backup command option.
--tls-version
This option accepts a comma-separated list of TLS protocol versions. A TLS protocol version will only be enabled if it is present in this list. All other TLS protocol versions will not be permitted. For example:
--tls-version="TLSv1.2,TLSv1.3"
This option is usually used with other TLS options. For example:
mariadb-backup --backup \
--ssl-cert=/etc/my.cnf.d/certificates/client-cert.pem \
--ssl-key=/etc/my.cnf.d/certificates/client-key.pem \
--ssl-ca=/etc/my.cnf.d/certificates/ca.pem \
--tls-version="TLSv1.2,TLSv1.3"
See Secure Connections Overview: TLS Protocol Versions for more information.
-t, --tmpdir
Defines path for temporary files.
--tmpdir=/path/tmp[;/path/tmp...]
Using this option, you can define the path to a directory mariadb-backup uses in writing temporary files. If you want to use more than one, separate the values by a semicolon (that is, ;
). When passing multiple temporary directories, it cycles through them using round-robin.
mariadb-backup --backup \
--tmpdir=/data/tmp;/tmp
--use-memory
Defines the buffer pool size that is used during the prepare stage.
--use-memory=124M
Using this option, you can define the buffer pool size for mariadb-backup. Use it instead of buffer_pool_size
.
mariadb-backup --prepare \
--use-memory=124M
--user
Defines the username for connecting to the MariaDB Server.
--user=name
-u name
When mariadb-backup runs, it connects to the specified MariaDB Server to get its backups. Using this option, you can define the database user used for authentication. Starting from MariaDB 10.5.24, , , , , , , , if the --user
option is ommited, the user name is detected from the OS.
mariadb-backup --backup \
--user=root \
--password=root_passwd
--verbose
Displays verbose output
mariadb-backup --verbose
--version
Prints version information.
Using this option, you can print the mariadb-backup version information to stdout.
mariadb-backup --version
This page is licensed: CC BY-SA / Gnu FDL