Versions that aren't stable will be suffixed with -rc, or -alpha to clearly show their release status, and enables Renovatebot and other that follow semantic versioning to follow updates.

For a consistent application between testing an production environment using the SHA hash of the image is recommended like docker.io/library/mariadb@sha256:29fe5062baf36bae8ec68f21a3dce4f0372dadc185e687624f1252fc49d91c67. There is a list of mapping and history of tags to SHA hash on the Docker Library repository.

Puppet Supported Modules page in Puppet Forge.

Some drawbacks of using events are the following:

• Events can only perform tasks that can be developed in SQL. So, for example, it is not possible to send alerts. Access to files or remote databases is limited.

• The event scheduler runs as a single thread. This means that events that are scheduled to run while another event is running will wait until the other event has stopped. This means that there is no guarantee that an event will run on exactly it's scheduled time. This should not be a problem as long as one ensures that events are short lived.

• For more events limitations, see Event Limitations.

In many cases you may prefer to develop scripts in an external programming language. However, you should know that simple tasks consisting of a few queries can easily be implemented as events.

Good Practices

When using events to automate tasks, there are good practices one may want to follow.

Move your SQL code in a stored procedure. All the event will do is to call a stored procedures. Several events may call the same stored procedure, maybe with different parameters. The procedure may also be called manually, if necessary. This will avoid code duplication. This will separate the logic from the schedule, making it possible to change an event without a risk of making changes to the logic, and the other way around.

Just like cron jobs, events should log whether if they succeed or not. Logging debug messages may also be useful for non-trivial events. This information can be logged into a dedicated table. The contents of the table can be monitored by a monitoring tool like Grafana. This allows to visualize in a dashboard the status of events, and send alerts in case of a failure.

Examples

Some examples of tasks that could easily be automated with events:

• Copying data from a remote table to a local table by night, using the CONNECT storage engine. This can be a good idea if many rows need be copied, because data won't be sent to an external client.

• Periodically delete historical data. For example, rows that are older than 5 years. Nothing prevents us from doing this with an external script, but probably this wouldn't add any value.

• Periodically delete invalid rows. In an e-commerce, they could be abandoned carts. In a messaging system, they could be messages to users that don't exist anymore.

• Add a new partition to a table and drop the oldest one (partition rotation).

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

To communicate with Docker API, Ansible needs a proper Python module installed on the Ansible node (docker or docker-py).

Several roles exist to deploy Docker and configure it. They can be found in Ansible Galaxy.

References

Further information can be found in Ansible documentation.

• Docker Guide.

• docker_container module.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

See Ansible Overview - concepts for more details and an example.

Architectural Differences

The architecture of the various systems is different. Their architectures determine how a deploy physically works, and what is needed to be able to deploy.

Storing Secrets

Often our automation repositories need to contain secrets, like MariaDB user passwords or private keys for SSH authentication.

Both Ansible and Puppet support integration with secret stores, like Hashicorp Vault.

Ecosystems and Communities

Automation software communities are very important, because they make available a wide variety of modules to handle specific software.

See Also

For more information about the systems mentioned in this page, from MariaDB users perspective:

• Ansible and MariaDB.

• Puppet and MariaDB.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

We also specify a timezone-update tag, so we can apply the role to only update the timezone tables.

Running the Script

The next task runs mariadb-tzinfo-to-sql.

We use the shell module to run the command. Running a command in this way is not idempotent, so we specify when: timezone_info.changed to only run it when necessary. Some warnings may be generated, so we pipe the output of mysql_tzinfo_to_sql to grep to filter warnings out.

Using Galera

If we're using we'll want to only update the timezone tables in one node, because the other nodes will replicate the changes. For our convenience, we can run this operation on the first node. If the nodes hostnames are defined in a list called cluster_hosts, we can check if the current node is the first in this way:

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

Each hierarchy allows to one choose the proper configuration file for a resource, based on certain criteria. For example criteria may include node names, node groups, operating systems, or datacenters. Hierarchies are defined in a hiera.yaml file, which also defines a path for the files in each hierarchy.

Puppet facts are commonly used to select the proper files to use. For example, a path may be defined as "os/%{facts.os.name}.yaml". In this case, each resource will use a file named after the operating system it uses, in the os directory. You may need to use custom facts, for example to check which microservices will use a MariaDB server, or in which datacenter it runs.

We do not have to create a file for each possible value of a certain fact. We can define a default configuration file with settings that are reasonable for most resources. Other files, when included, will override some of the default settings.

A hiera configuration file will look like this:

This file would include the global files, the OS-specific files and the node-specific files. Each hierarchy will override settings from previous hierarchies.

We can actually have several hiera configuration files. hiera.yaml is the global file. But we will typically have additional hiera configuration files for each environment. So we can include the configuration files that apply to production, staging, etc, plus global configuration files that should be included for every environment.

Importantly, we can also have hiera configuration files for each module. So, for example, a separate mariadb/hiera.yaml file may defined the hierarchies for MariaDB servers. This allow us to define, for example, different configuration files for MariaDB and for MaxScale, as most of the needed settings are typically different.

Configuration Files

You probably noticed that, in the previous example, we defined data_hash: yaml_data, which indicates that configuration files are written in YAML. Other allowed formats are JSON and HOCON. The data_hash setting is defined in defaults, but it can be overridden by hierarchies.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

The administrators of a system should be particularly careful to upgrade the kernel whenever security bugs to these features are fixed.

It is important to note that when we upgrade the kernel, runC or Docker itself we cause downtime for all the containers running on the system.

Images Security

Containers are built from images. If security is a major concern, you should make sure that the images you use are secure.

If you want to be sure that you are pulling authentic images, you should only pull images signed with Docker Content Trust. Signing only ensure authenticity or origin, it doesn't dictate that entity is trustworthy.

Updated images should be used. An image usually downloads packages information at build time. If the image is not recently built, a newly created container will have old packages. Updating the packages on container creation and regularly re-updating them will ensure that the container uses packages with the most recent versions. Rebuilding an image often will reduce the time necessary to update the packages the first time.

Security bugs are usually important for a database server, so you don't want your version of MariaDB to contain known security bugs. But suppose you also have a bug in Docker, in runC, or in the kernel. A bug in a user-facing application may allow an attacker to exploit a bug in those lower level technologies. So, after gaining access to the container, an attacker may gain access to the host system. This is why system administrators should keep both the host system and the software running in the containers updated.

References

For more information, see the following links:

• Container Security from Red Hat.

• Docker security on Docker documentation.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

To run ping on a specific group or host, we can just replace "all" with a group name or host name from the inventory:

Running Commands on Remote Servers

The previous examples show how to run an Ansible module on remote servers. But it's also possible to run custom commands over SSH. Here's how:

This command shows the value of $PATH on all servers in the inventory "production-mariadb".

We can also run commands as root by adding the -b (or --become) option:

Applying Roles to Remote Servers

We saw how to run commands on remote hosts. Applying roles to remote hosts is not much harder, we just need to add some information. An example:

Let's see what changed:

• ansible-playbook is the executable file that we need to call to apply playbooks and roles.

• production-mariadb.yml is the play that associates the servers listed in the inventory to their roles.

If we call ansible-playbook with no additional arguments, we will apply all applicable roles to all the servers mentioned in the play.

To only apply roles to certain servers, we can use the -l parameter to specify a group, an individual host, or a pattern:

We can also apply tasks from roles selectively. Tasks may optionally have tags, and each tag corresponds to an operation that we may want to run on our remote hosts. For example, a "mariadb" role could have the "timezone-update" tag, to update the contents of the timezone tables. To only apply the tasks with the "timezone-update" tag, we can use this command:

Using tags is especially useful for database servers. While most of the technologies typically managed by Ansible are stateless (web servers, load balancers, etc.) database servers are not. We must pay special attention not to run tasks that could cause a database server outage, for example destroying its data directory or restarting the service when it is not necessary.

Check mode

We should always test our playbooks and roles on test servers before applying them to production. However, if test servers and production servers are not exactly in the same state (which means, some facts may differ) it is still possible that applying roles will fail. If it fails in the initial stage, Ansible will not touch the remote hosts at all. But there are cases where Ansible could successfully apply some tasks, and fail to apply another task. After the first failure, ansible-playbook will show errors and exit. But this could leave a host in an inconsistent state.

Ansible has a check mode that is meant to greatly reduce the chances of a failure. When run in check mode, ansible-playbook will read the inventory, the play and roles; it will figure out which tasks need to be applied; then it will connect to target hosts, read facts, and value all the relevant variables. If all these steps succeed, it is unlikely that running ansible-playbook without check mode will fail.

To run ansible-playbook in check mode, just add the --check (or -C) parameter.

References

Further documentation can be found in the Ansible website:

• ansible tool.

• ansible-playbook tool.

• Validating tasks: check mode and diff mode.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

ANSIBLE_ASK_PASS can be specified. If this is the case, Ansible will prompt the user asking to type an SSH password.

Avoiding Sharing Secrets

As a general rule, any configuration that implies communicating sensible information to the persons who will connects to a system implies some degree of risk. Therefore, the most common choice is to allow users to login into remote systems with their local usernames, using SSH keys.

Once Ansible is able to connect remote hosts, it can also be used to install the public keys of some users to grant them access. Sharing these keys implies no risk. Sharing private keys is never necessary, and must be avoided.

MariaDB has a UNIX_SOCKET plugin that can be used to let some users avoid entering a password, as far as they're logged in the operating system. This authentication method is used by default for the root user. This is a good way to avoid having one more password and possibly writing to a .my.cnf file so that the user doesn't have to type it.

Even for users who connect remotely, it is normally not necessary to insert passwords in an Ansible file. When we create a user with a password, a hash of the original password is stored in MariaDB. That hash can be found in the mysql.user table. To know the hash of a password without even creating a user, we can use the PASSWORD() function:

When we create a user, we can actually specify a hash instead of the password the user will have to type:

ansible-vault

Even if you try to avoid sharing secrets, it's likely you'll have to keep some in Ansible. For example, MariaDB users that connect remotely have passwords, and if we want Ansible to create and manage those users, the hashes must be placed somewhere in our Ansible repository. While a hash cannot be converted back to a password, treating hashes as secrets is usually a good idea. Ansible provides a native way to handle secrets: ansible-vault.

In the simplest case, we can manage all our passwords with a single ansible-vault password. When we add or change a new password in some file (typically a file in host_vars or group_vars) we'll use ansible-vault to crypt this password. While doing so, we'll be asked to insert our ansible-vault password. When we apply a role and Ansible needs to decrypt this password, it will ask us to enter again our ansible-vault password.

ansible-vault can use more than one password. Each password can manage a different set of secrets. So, for example, some users may have the password to manage regular MariaDB users passwords, and only one may have the password that is needed to manage the root user.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

• curl, to call HTTP URLs and get their output.

• js, to extract or transform information from a JSON document.

Example: Check When a New Version Becomes GA

A trivial use case is to write a script that checks the list of MariaDB GA major versions and, when something changes, send us an email. So we can test the newest GA version and eventually install it.

The script in this example will be extremely simple. We'll do it this way:

• Retrieve the JSON object describing all MariaDB versions.

• For each element of the array, only show the release_id and release_status properties, and concatenate them.

• Apply a filter, so we only select the rows containing 'stable' but not 'old' (so we exclude 'Old Stable').

• From the remaining rows, only show the first column (the version number).

• If the results we obtained are different from the previously written file (see last point) send an email.

• Save the results into a file.

This is something that we can easily do with a Unix shell:

The only non-standard command here is jq. It is a great way to manipulate JSON documents, so if you don't know it you may want to take a look at jq documentation.

How to Use the API with a Python Script

To use the API with Python, we need a module that is able to send HTTP requests and parse a JSON output. The requests module has both these features. It can be installed as follows:

The following script prints stable versions to the standard output:

requests.get() makes an HTTP call of type GET, and requests.json() returns a dictionary representing the previously obtained JSON document.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

With an inventory of this type, it will be possible to run Bolt actions against all the targets that are members of a group:

In the examples in the rest of the page, the --targets parameter will be indicated in this way, for simplicity: --targets <targets>.

Running Commands on Targets

The simplest way to run a command remotely is the following:

Copying Files

To copy a file or a whole directory to targets:

To copy a file or a whole directory from the targets to the local host:

Running Scripts on Targets

We can use Bolt to run a local script on remote targets. Bolt will temporarily copy the script to the targets, run it, and delete it from the targets. This is convenient for scripts that are meant to only run once.

Running Tasks on Targets

Puppet tasks are not always as powerful as custom scripts, but they are simpler and many of them are idempotent. The following task stops MariaDB replication:

Applying Puppet Code on Targets

It is also possible to apply whole manifests or portions of Puppet code (resources) on the targets.

To apply a manifest:

To apply a resource description:

Bolt Resources and References

• Bolt documentation.

• Bolt on GitHub.

Further information about the concepts explained in this page can be found in Bolt documentation:

• Inventory Files in Bolt documentation.

• Applying Puppet code in Bolt documentation.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

Note that, if we don't want to use Orchestrator to automate operations, we can still use it as a dynamic inventory. Other tools can use it to obtain a list of existing MariaDB servers via its REST API or CLI commands.

Orchestrator has several big users, listed in the documentation Users page. It is also included in the PMM monitoring solution.

To install Orchestrator, see:

• The install.md for a manual installation;

• The links in README.md, to install Orchestrator using automation tools.

Supported Topologies

Currently, Orchestrator fully supports MariaDB GTID, replication, and semi-synchronous replication. While Orchestrator does not support Galera specific logic, it works with Galera clusters. For details, see Supported Topologies and Versions in Orchestrator documentation.

Architecture

Orchestrator consists of a single executable called orchestrator. This is a process that periodically connects to the target servers. It will run SQL queries against target servers, so it needs a user with proper permissions. When the process is running, a GUI is available via a web browser, at the URL 'localhost:3000'. It also exposes a REST API (see Using the web API in the Orchestrator documentation).

Orchestrator expects to find a JSON configuration file called orchestrator.conf.json, in /etc.

A database is used to store the configuration and the state of the target servers. By default, this is done using built-in SQLite. However, it is possible to use an external MariaDB or MySQL server instance.

If a cluster of Orchestrator instances is running, only one central database is used. One Orchestrator node is active, while the others are passive and are only used for failover. If the active node crashes or becomes unreachable, one of the other nodes becomes the active instance. The active_node table shows which node is active. Nodes communicate between them using the Raft protocol.

CLI Examples

As mentioned, Orchestrator can be used from the command-line. Here you can find some examples.

List clusters:

Discover a specified instance and add it to the known topology:

Forget about an instance:

Move a replica to a different master:

Move a replica up, so that it becomes a "sibling" of its master:

Move a replica down, so that it becomes a replica of its"sibling":

Make a node read-only:

Make a node writeable:

The --debug and --stack options can be added to the above commands to make them more verbose.

Orchestrator Resources and References

• Orchestrator on GitHub.

• Documentation.

• Raft consensus protocol website.

The README.md file lists some related community projects, including modules to automate Orchestrator with Puppet and other technologies.

On GitHub you can also find links to projects that allow the use of automation software to deploy and manage Orchestrator.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

1. Boot Disk > Change

Switch the operating system to a modern Ubuntu release x86/64 CPU architecture, or similar free tier offering.

1. Create a firewall rule in the Firewall Policies section of the console. After naming it, change the targets, add 0.0.0.0/0 as a source IP range, and open TCP port 3306. Then Click create.
2. Connect using Google Cloud’s built in browser SSH. Accept all prompts for authorization.

Install Docker on the GCE VM

For more detailed instructions, refer to Installing and Using MariaDB via Docker

1. Escalate to root Escalate to root

1. Install Docker

1. Pull Docker image

1. Start MDRB docker process at your terminal / command line.

Mounting a Directory for Persistent MySQL Storage

When using the -v flag to mount a directory as /var/lib/mysql, you ensure that your MySQL data is stored persistently.

File Path Instructions:

• Windows: Use a file path format similar to C:\Users\YourUsername\Documents\YourDirectory.

• Linux: Always use absolute paths instead of relative ones.

1. Shell into container

2. Login to MRDB inside container using the root password specified in step 12.

1. Setup admin account with permission for remote connection, configure access control Execute these SQL commands in sequence:

Obviously replace these passwords with something that is a bit more secure than you see in this example for anything other than development purposes.

1. Setup service account for your app with permission for remote connection, configure access control Execute these SQL commands in sequence:

Obviously replace these passwords with something that is a bit more secure than you see in this example for anything other than development purposes.

1. Load up your database from your preexisting SQL script that contains CREATE DATABASE; USE DATABASE; and CREATE TABLE statements.

Copy the external IP address of your VM instance from the Console in the VM instances list.

To run your database creation script using MariaDB, open a new terminal window and navigate to the directory containing the script, init.sql. Then, execute the following command, replacing ww.xx.yyy.zzz with your IP address:

_{This page is licensed: CC BY-SA / Gnu FDL}

We will discuss a bit later how this applies to MariaDB, and more generally to database servers.

When something should change, for example some software version or configuration, normally Dockerfiles are updated and containers are recreated from the latest image versions. For this reason, containers shouldn't contain anything that shouldn't be lost, and recreating them should be an extremely cheap operation. Docker Compose or the Swarm mode are used to declare which containers form a certain environment, and how they communicate with each other.

On the contrary, Ansible and Puppet are mainly built to manage the configuration of existing servers. It doesn't recreate servers, it changes their configuration. So Docker and Ansible have very different approaches. For this reason, Ansible and Puppet are not frequently used to deploy containers to production. However, using them together can bring some benefits, especially for development environments.

More on this later in the page. First, we need to understand how these concepts apply to database servers.

Stateful Technologies

Using ephemeral containers works very well for stateless technologies, like web servers and proxies. These technologies virtually only need binaries, configuration and small amounts of data (web pages). If some data need to be restored after a container creation, it will be a fast operation.

In the case of a database, the problem is that data can be large and need to be written somewhere. We don't want all databases to disappear when we destroy a container. Even if we had an up-to-date backup, restoring it would take time.

However, OCI Containers has features called volumes. A volume is a directory in the host system mapped to a directory in one or more containers. Volumes are not destroyed when containers are destroyed. They can be used to share data between any number of containers and the host system. Therefore, they are also a good way to persist data.

Suppose a MariaDB container called mariadb-main-01 uses a volume that is mapped to /var/docker/volumes/mariadb-main. At some point we want to use a more recent MariaDB version. As explained earlier, the container way to do this is to destroy the container and create a new one that uses a more recent version of the MariaDB image.

So, we will destroy mariadb-main-01. The volume is still there. Then we create a new container with the same name, but based on a newer image. We make sure to link the volume to the new container too, so it will be able to use /var/docker/volumes/mariadb-main again. At this point we may want to run mariadb-upgrade, but apart from that, everything should just work.

The container runtime implementations also provide the opportunity to create a volume with an explicit name and this is also persistent. The actual location on the filesystem is managed by the runtime.

The above described steps are simple, but running them manually is time consuming and error-prone. Automating them with some automation software like Ansible or Puppet is often desirable.

Ways to Deploy Containers

Containers can be deployed in the following ways:

• Manually. See Installing and Using MariaDB via Docker. This is not recommended for production, or for complex environments. However, it can easily be done for the simplest cases. If we want to make changes to our custom images, we'll need to modify the Dockerfiles, destroy the containers and recreate them.

• With Docker Compose. See Setting Up a LAMP Stack with Docker Compose for a simple example. When we modify a Dockerfile, we'll need to destroy the containers and recreate them, which is usually as simple as running docker compose down followed by docker compose up. After changing docker-cmpose.yml (maybe to add a container or a network) we'll simply need to run docker compose up again, because it is idempotent.

• Using Ansible, Puppet or other automation software, as mentioned before. We can use Ansible or Puppet to create the containers, and run them again every time we want to apply some change to the containers. This means that the containers are potentially created once and modified any number of times.

In all these cases, it is entirely possible to add Vagrant to the picture. Vagrant is a way to deploy or provision several hosts, including virtual machines (the most common case), and containers. It is agnostic in regarding the underlying technology, so it can deploy to a virtual machine, a container, or even a remote server in the same way. Containers can work with Vagrant in two ways:

• As a provisioner. In this case Vagrant will most commonly deploy a virtual machine, and will use Docker to setup the applications that need to run in it, as containers. This guarantees a higher level of isolation, compared to running the containers in the local host. Especially if you have different environments to deploy locally, because you can have them on different virtual machines.

• As a provider. Vagrant will deploy one or more containers locally. Once each container is up, Vagrant can optionally use a provisioner on it, to make sure that the container runs the proper software with proper configuration. In this case, Ansible, Puppet or other automation software can be used as a provisioner. But again, this is optional: it is possible to make changes to the Dockerfiles and recreate the containers every time.

Benefits of Managing Containers with Automation Software

Containers can be entirely managed with Docker Compose or the Swarm mode. This is often a good idea.

However, choosing to use automation software like Ansible or Puppet has some benefits too. Benefits include:

• Containers allow working without modifying the host system, and their creation is very fast. Much faster than virtual machines. This makes containers desirable for development environments.

• As explained, making all containers ephemeral and using volumes to store important data is possible. But this means adding some complexity to adapt an ephemeral philosophy to technologies that are not ephemeral by nature (databases). Also, many database professionals don't like this approach. Using automation software allows easily triggering upgrades and configuration changes in the containers, treating them as non-ephemeral systems.

• Sometimes containers are only used in development environments. If production databases are managed via Ansible, Puppet, or other automation software, this could lead to some code duplication. Dealing with configuration changes using the same procedures will reduce the cost of maintenance.

• While recreating containers is fast, being able to apply small changes with Ansible or Puppet can be more convenient in some cases: particularly if we write files into the container itself, or if recreating a container bootstrap involves some lengthy procedure.

• Trying to do something non-standard with Dockerfiles can be tricky. For example, running two processes in a container is possible but can be problematic, as containers are designed to run single main process per container. However there are situations when this is desirable. For example PMM containers run several different processes. Launching additional processes with Ansible or Puppet may be easier than doing it with a Dockerfile.

With all this in mind, let's see some examples of cases when managing containers with Ansible, Puppet or other automation software is preferable, rather than destroying containers every time we want to make a change:

• We use Ansible or Puppet in production, and we try to keep development environments as similar as possible to production. By using Ansible/Puppet in development too, we can reuse part of the code.

• We make changes to the containers often, and recreating containers is not as fast as it should be (for example because a MariaDB dump needs to be restored).

• Creating a container implies some complex logic that does not easily fit a Dockerfile or Docker Compose (including, but not limited to, running multiple processes per container).

That said, every case is different. There are environments where these advantages do not apply, or bring a very small benefit. In those cases, the cost of adding some automation with Ansible, Puppet or similar software is probably not justified.

How to Deploy to Container from Orchestration Software

Suppose you want to manage containers configuration with Ansible.

At a first glance, the simplest way is to run Ansible in the host system. It will need to connect to the containers via SSH, so they need to expose the 22 port. But we have multiple containers, so we'll need to map the 22 port of each container to a different port in the host. This is hard to maintain and potentially insecure: in production you want to avoid exposing any container port to the host.

A better solution is to run Ansible itself in a container. The playbooks will be in a container volume, so we can access them from the host system to manage them more easily. The Ansible container will communicate with other containers using a container network, using the standard 22 port (or another port of your choice) for all containers.

See Also

See these pages on how to manage containers with different automation technologies:

• Deploying Containers with Ansible.

• Deploying Containers with Puppet.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

If you want to deploy MariaDB Enterprise Server without Docker, alternative deployment methods are available.

Use Cases

MariaDB Enterprise Server can be deployed with Docker to support use cases that require software to be rapidly deployed on existing infrastructure, such as:

• Continuously create and destroy automated testing environments as part of a continuous integration (CI) pipeline

• Create a small test environment on a local workstation

• Create multiple isolated test environments on the same host

• Deployment alongside related containers using Docker Compose

Compatibility

The following products and versions can be deployed using the MariaDB Enterprise Docker Registry:

• MariaDB Enterprise Server 11.8

• MariaDB Enterprise Server 11.4

• MariaDB Enterprise Server 10.6

For details about which storage engines and plugins are supported in the images for each version, see "MariaDB Enterprise Docker Registry".

Deploy Enterprise Server in a Docker Container

To deploy MariaDB Enterprise Server in a Docker container, follow the instructions below.

Step 1: Retrieve Customer Download Token

MariaDB Corporation requires customers to authenticate when logging in to the MariaDB Enterprise Docker Registry. A customer-specific Customer Download Token must be provided as the password.

Customer Download Tokens are available through the MariaDB Customer Portal.

To retrieve the customer download token for your account:

1. Navigate to the Customer Download Token at the MariaDB Customer Portal.

2. Log in using your MariaDB ID.

3. Copy the Customer Download Token to use as the password when logging in to the MariaDB Enterprise Docker Registry.

Step 2: Log In to Docker Registry

Log in to the MariaDB Enterprise Docker Registry by executing docker login:

When prompted, enter the login details:

• As the user name, enter the email address associated with your MariaDB ID.

• As the password, enter your Customer Download Token.

The login details will be saved.

Confirm the login details were saved by checking the /.docker/config.json file for a JSON object named "docker.mariadb.com" inside an "auths" parent JSON object:

Step 3: Choose an Image Tag

The enterprise-server repository in the MariaDB Enterprise Docker Registry contains images for different MariaDB Enterprise Server releases using specific tags. Before continuing, you will need to decide which tag to use.

To deploy a container using the most recent image for the latest MariaDB Enterprise Server release series (currently 11.8), use the latest tag.

For additional information, see "MariaDB Enterprise Docker Registry: Tags".

Step 4: Pull Docker Image

Pull the Docker image with the chosen tag by executing docker pull:

Confirm the Docker image has been pulled by executing docker images:

Step 5: Create a Container

Create a container using the pulled Docker image by executing docker run:

• Configure the container and set the root password using environment variables by setting the --env command-line option.

• Configure TCP port bindings for the container by setting the --publish or --publish-all command-line options.

• Configure MariaDB Enterprise Server by setting mariadbd command-line options.

Confirm the container is running by executing docker ps:

By default, Docker uses Docker bridge networking for new containers. For details on how to use host networking for new containers, see "Create a Container with Host Networking".

Step 6: Connect to Container

Connect to the container by executing MariaDB Client on the container using docker exec:

Confirm the container is using the correct version of MariaDB Enterprise Server by querying the version system variable with the SHOW GLOBAL VARIABLES statement:

Exit the container using exit:

Step 7: Stop Container

Stop a Docker container using docker stop:

Confirm the container is stopped by executing docker ps:

Step 8: Remove Container

Remove a Docker container using docker rm:

Confirm the container is removed by executing docker ps:

_{This page is: Copyright © 2025 MariaDB. All rights reserved.}

In the first line we declare that we are using version 3 of the Docker compose language.

Then we have the list of services, namely the web and the mariadb services.

Let's see the properties of the services:

About Volumes

It is good practice to create volumes for:

• The data directory, so we don't lose data when a container is created or replaced, perhaps to upgrade MariaDB.

• The directory where we put all the logs, if it is not the datadir.

• The directory containing all configuration files (for development environments), so we can edit those files with the editor installed in the host system. Normally no editor is installed in containers. In production we don't need to do this, because we can copy files from a repository located in the host system to the containers.

Note that Docker Compose variables are just placeholders for values. Compose does not support assignment, conditionals or loops.

Using Variables

In the above example you can see several variables, like ${MARIADB_VERSION}. Before executing the file, Docker Compose will replace this syntax with the MARIADB_VERSION variable.

Variables allow making Docker Compose files more re-usable: in this case, we can use any MariaDB image version without modifying the Docker Compose file.

The most common way to pass variables is to write them into a file. This has the benefit of allowing us to version the variable file along with the Docker Compose file. It uses the same syntax you would use in BASH:

For bigger setups, it could make sense to use different environment files for different services. To do so, we need to specify the file to use in the Compose file:

Docker Compose Commands

Docker Compose is operated using docker-compose. Here we'll see the most common commands. For more commands and for more information about the commands mentioned here, see the documentation.

Docker Compose assumes that the Composer file is located in the current directory and it's called docker-compose.yml. To use a different file, the -f <filename> parameter must be specified.

Docker Compose Resources and References

• Overview of Docker Compose in the Docker documentation.

• Compose file in the Docker documentation.

• Docker Compose on GitHub.

Further information about the concepts explained in this page can be found in Docker documentation:

• Environment variables in Compose.

• Overview of docker-compose CLI.

• Compose command-line reference.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

The MARIADB_AUTO_UPGRADE=1 will regenerate the .my-healthcheck.cnf file if missing and recreate the healthcheck users of the database with a new random password. The current port configuration of the MariaDB container is written into this file.

The MARIADB_MYSQL_LOCALHOST_USER=1, MARIADB_MYSQL_LOCALHOST_GRANTS environment variables can also be used, but with the creation of the healthcheck user, these are backwards compatible.

Compose File Example

An example of a compose file that uses the healthcheck.sh to determine a healthy service as a dependency before starting a WordPress service:

Tests

The various health check flags used for monitoring the status and readiness of a MariaDB server.

Parameters

To customize the health checks described previously, you can use the following parameters:

Examples

Switch to mysql user, and check if can connect and the innodb is initialized.

Switch to mysql user, check if connections can be made. For the replication channel archive, ensure IO and SQL threads are running, the seconds behind master is less than 600 seconds, and the SQL remaining delay is less than 30 seconds. For the channel1, the seconds behind master is limited to 10 seconds maximum.

_{This page is licensed: CC BY-SA / Gnu FDL}

Start The Container With Server Options

To start the container in the background with the MariaDB server image, run:

Additionally, environment variables are also provided.

Get the list of running containers

Note: specify the flag -a in case you want to see all containers

Start the client from the container

To start the mariadb client inside the created container and run specific commands, run the following:

Inspect logs of a container

In the logs you can find status information about the server, plugins, generated passwords, errors and so on.

Restart the container

Run commands within the container

Use a volume to specify configuration options

One can specify custom configuration files through the /etc/mysql/conf.d volume during container startup.

Use a volume to specify grants during container start

User created with the environment variables has full grants only to the MARIADB_DATABASE. In order to override those grants, one can specify grants to a user, or execute any SQL statements from host file to docker-entrypoint-initdb.d. In the local_init_dir directory we can find the file, created like this:

See Also

• Installing and using MariaDB via Docker

_{This page is licensed: CC BY-SA / Gnu FDL}

Most images are based on another image. The base image is specified at the beginning of the Dockerfile, with the FROM directive. If the base image is not present in the local system, it is downloaded from the repository specified, or if not specified, from the default repository of the build program. This is often Docker Hub. For example, we can build a mariadb-rocksdb:10.5 image starting from the debian:13 image. In this way, we'll have all the software included in a standard Debian image, and we'll add MariaDB and its configuration upon that image.

All the following Dockerfile directives are compiled into a new Docker image, identified by an SHA256 string. Each of these images is based on the image compiled from the previous directive. A physical compiled image can serve as a base for any number of images. This mechanism saves a lot of disk space, download time and build time.

The following diagram shows the relationship between Dockerfiles, images and containers:

Dockerfile Syntax

Here's a simple Dockerfile example:

This example is not very good for practical purposes, but it shows what a Dockerfile looks like.

First, we declare that the base image to use is ubuntu:20.04.

Then we run some commands to install MariaDB from the Ubuntu default repositories and stop the MariaDB service.

We define some metadata about the image with LABEL. Any label is valid.

We declare that the port 3306 (MariaDB default port) should be exposed. However, this has no effect if the port is not exposed at container creation.

We also define a healthcheck. This is a command that is run to check if the container is healthy. If the return code is 0 the healthcheck succeeds, if it's 1 it fails. In the MariaDB specific case, we want to check that it's running and able to answer a simple query. This is better than just checking that MariaDB process is running, because MariaDB could be running but unable to respond, for example because max_connections was reached or data si corrupted. We read a system variable, because we should not assume that any user-created table exists. We also specify --start-period to allow some time for MariaDB to start, keeping in mind that restarting it may take some time if some data is corrupted. Note that there can be only one healthcheck: if the command is specified multiple times, only the last occurrence will take effect.

Finally, we start the container command: mariadbd. This command is run when a container based on this image starts. When the process stops or crashes, the container will immediately stop.

Note that, in a container, we normally run mariadbd directly or in an entrypoint script exec mariadbd, rather than running mysqld_safe or running MariaDB as a service. Containers restart can be handled by the container service. See automatic restart.

See the documentation links below to learn the syntax allowed in a Dockerfile.

Using Variables

It is possible to use variables in a Dockerfile. This allows us, for example, to install different packages, install different versions of a package, or configure software differently depending on how variables are set, without modifying the Dockerfile itself.

To use a variable, we can do something like this:

Here ARG is used after the FROM directive, thus the variable cannot be used in FROM. It is also possible to declare a variable before FROM, so we can use a variable to select the base image to use or its tag, but in this case the variable cannot be used after the FROM directive, unless ARG is re-declared after the FROM. Here is an example:

We'll have to assign variables a value when we build the Dockerfile, in this way:

Note that Dockerfile variables are just placeholders for values. Dockerfiles do not support assignment, conditionals or loops.

Versioning and Deploying Images

Dockerfiles are normally versioned, as well as the files that are copied to the images.

Once an image is built, it can be pushed to a container registry. Whenever an image is needed on a host to start containers from it, it is pulled from the registry.

Container registries

A default container registry for OCI images is Docker Hub. It contains Docker Official Images maintained by the Docker Library team and the community. Any individual or organization can open an account and push images to Docker Hub. Most Docker images are open source: the Dockerfiles and the needed files to build the images are usually on GitHub.

It is also possible to setup a self-hosted registry. Images can be pushed to that registry and pulled from it, instead of using Docker Hub. If the registry is not publicly accessible, it can be used to store images used by the organization without making them publicly available.

But a self-hosted registry can also be useful for open source images: if an image is available on Docker Hub and also on a self-hosted registry, in case Docker Hub is down or not reachable, it will still be possible to pull images.

Choosing Image Names and Tags

The names of images developed by the community follow this schema:

It doesn't matter if the maintainer is an individual or an organization. For images available on Docker Hub, the maintainer is the name of a Docker Hub account.

Official images maintained by the Docker Library maintainers have the implicit name of library filled in by the container fetching tool. For example, the official MariaDB image is called mariadb which is an alias for docker.io/library/mariadb.

All images have a tag, which identifies the version or the variant of an image. For example, all MariaDB versions available on Docker are used as image tags. MariaDB 10.11 is called mariadb:10.11.

By conversion, tags form a hierarchy. So for example, there is a 10.1.1 tag whose meaning will not change over time. 10.5 will always identify the latest stable version in the 10.5 branch. For some time it was 10.5.1, then it became 10.5.2, and so on.

When we pull an image without specifying a tag (ie, docker pull mariadb), we are implicitly requiring the image with the latest tag. This is even more mutable: at different periods of time, it pointed to the latest 10.0 version, to the latest 10.1 version, and so on.

In production, it is always better to know for sure which version we are installing. Therefore it is better to specify a tag whose meaning won't change over time, like 10.5.21. To keep to a latest LTS version, the lts can be used.

Pushing and Pulling Images

To pull an image from Docker Hub or a self-hosted registry, we use the docker pull command. For example:

This command downloads the specified image if it is not already present in the system, or if the local version is not up to date.

After modifying a Dockerfile, we can build an image in this way:

This step can be automated by services like Docker Hub and GitHub. Check those service's documentation to find out how this feature works.

Once an image is created, it can be pushed to a registry. We can do it in this way:

Docker Content Trust

Docker has a feature called Docker Content Trust (DCT). It is a system used to digitally sign images, based on PEM keys. For environments where security is a major concern, it is important to sign images before pushing them. This can be done with both Docker Hub and self-hosted registries.

Good Practices and Caveats

As mentioned, a Dockerfile is built by creating a new image for each directive that follows FROM. This leads to some considerations.

• Sometimes it can be a good idea to run several shell commands in a single RUN directive to avoid creating images that are not useful.

• Modifying a directive means that all subsequent directives also need to be rebuilt. When possible, directives that are expected to change often should follow directives that will change seldom.

• Directives like LABEL or EXPOSE should be placed close to the end of Dockerfiles. In this way they will be rebuilt often, but this operation is cheap. On the other side, changing a label should not trigger a long rebuild process.

• Variables should be used to avoid Dockerfiles proliferation. But if a variable is used, changing its value should be tested. So, be sure not to use variables without a good reason.

• Writing logic into a Dockerfile is impossible or very hard. Call shell scripts instead, and write your logic into them. For example, in a shell script it is easy to perform a certain operation only if a variable is set to a certain value.

• If you need MariaDB containers with different configurations or different sets of plugins, use the method explained above. Do not create several Dockerfiles, with different tags, for each desired configuration or plugin set. This may lead to undesired code duplication and increased maintenance costs.

References

More details can be found in the Docker documentation:

• Dockerfile reference.

• docker build.

• Repositories.

• Deploy a registry server.

• .

See also:

• Privacy-Enhanced Mail on Wikipedia.

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

Therefore manifests are declarative. You don't write the steps to achieve the desired result. Instead, you describe the desired result. When Puppet detects differences between your description and the current state of a server, it decides what to do to fix those differences.

Manifests are also idempotent. You don't need to worry about the effects of applying a manifest twice. This may happen (see Architecture below) but it won't have any side effects.

Defining Resources

Here's an example of how to describe a resource in a manifest:

This block describes a resource. The resource type is file, while the resource itself is /etc/motd. The description consists of a set of attributes. The most important is ensure, which in this case states that the file must exist. It is also common to use this resource to indicate that a file (probably created by a previous version of the manifest) doesn't exist.

These classes of resource types exist:

• Built-in resources, or Puppet core resources: Resources that are part of Puppet, maintained by the Puppet team.

• Defined resources: Resources that are defined as a combination of other resources. They are written in the Puppet domain-specific language.

• Custom resources: Resources that are written by users, in the Ruby language.

To obtain information about resources:

To group several resources in a reusable class:

There are several ways to include a class. For example:

Defining Nodes

Puppet has a main manifest that could be a site.pp file or a directory containing .pp files. For simple infrastructures, we can define the nodes here. For more complex infrastructures, we may prefer to import other files that define the nodes.

Nodes are defined in this way:

The resource type is node. Then we specify a hostname that is used to match this node to an existing host. This can also be a list of hostnames, a regular expression that matches multiple nodes, or the default keyword that matches all hosts. To use a regular expression:

Concepts

The most important Puppet concepts are the following:

Architecture

Depending on how the user decides to deploy changes, Puppet can use two different architectures:

• An Agent-master architecture. This is the preferred way to use Puppet.

• A standalone architecture, that is similar to Ansible architecture.

Agent-master Architecture

A Puppet master stores a catalog for each target. There may be more than one Puppet master, for redundancy.

Each target runs a Puppet agent in background. Each Puppet agent periodically connects to the Puppet master, sending its facts. The Puppet master compiles the relevant manifest using the facts it receives, and send back a catalog. Note that it is also possible to store the catalogs in PuppetDB instead.

Once the Puppet agent receives the up-to-date catalog, it checks all resources and compares them with its current state. It applies the necessary changes to make sure that its state reflects the resources present in the catalog.

Standalone Architecture

With this architecture, the targets run Puppet apply. This application usually runs as a Linux cron job or a Windows scheduled task, but it can also be manually invoked by the user.

When Puppet apply runs, it compiles the latest versions of manifests using the local facts. Then it checks every resource from the resulting catalogs and compares it to the state of the local system, applying changes where needed.

Newly created or modified manifests are normally deployed to the targets, so Puppet apply can read them from the local host. However it is possible to use PuppetDB instead.

PuppetDB

PuppetDB is a Puppet node that runs a PostgreSQL database to store information that can be used by other nodes. PuppetDB can be used with both the Agent-master and the standalone architectures, but it is always optional. However it is necessary to use some advanced Puppet features.

PuppetDB stored the following information:

• The latest facts from each target.

• The latest catalogs, compiled by Puppet apply or a Puppet master.

• Optionally, the recent history of each node activities.

External Node Classifiers

With both architectures, it is possible to have a component called an External Node Classifier (ENC). This is a script or an executable written in any language that Puppet can call to determine the list of classes that should be applied to a certain target.

An ENC received a node name in input, and should return a list of classes, parameters, etc, as a YAML hash.

Bolt

Bolt can be used in both architectures to run operations against a target or a set of targets. These operations can be commands passed manually to Bolt, scripts, Puppet tasks or plans. Bolt directly connects to targets via ssh and runs system commands.

See Bolt Examples to get an idea of what you can do with Bolt.

hiera

hiera is a hierarchical configuration system that allows us to:

• Store configuration in separate files;

• Include the relevant configuration files for every server we automate with Puppet.

See Puppet hiera Configuration System for more information.

Puppet Resources

• Puppet documentation.

• forge.puppet.com.

• Puppet on GitHub.

• Puppet on Wikipedia.

More information about the topics discussed in this page can be found in the Ansible documentation:

• Puppet Glossary in Puppet documentation.

• Overview of Puppet's architecture in Puppet documentation.

• PuppetDB documentation.

• Classifying nodes in Puppet documentation.

• in Puppet documentation.

• .

Content initially contributed by Vettabase Ltd.

_{This page is licensed: CC BY-SA / Gnu FDL}

A custom resource adds an API endpoint, so the resource can be managed via the API server. It includes functionality to get information about the resource, like a list of the existing servers.

A custom controller implements the checks that must be performed against the resource to check if its state should be corrected using the API. In the case of MariaDB, some reasonable checks would be verifying that it accepts connections, replication is running, and a server is (or is not) read only.

MariaDB Enterprise Operator

MariaDB Enterprise Operator provides a seamless way to run and operate containerized versions of MariaDB Enterprise Server and MaxScale on Kubernetes, allowing you to leverage Kubernetes orchestration and automation capabilities. This document outlines the features and advantages of using Kubernetes and the MariaDB Enterprise Operator to streamline the deployment and management of MariaDB and MaxScale instances.

MariaDB Community Operator

mariadb-operator is a Kubernetes operator that allows you to run and operate MariaDB in a cloud native way. It aims to declaratively manage MariaDB instances using Kubernetes CRDs instead of imperative commands.

It's available in both Artifact Hub and Operator Hub and supports the following features:

• Easily provision standalone MariaDB servers in Kubernetes.

• Multiple highly available topologies supported:

  • Asynchronous replication

  • Synchronous multi-master via Galera

  • as database proxy to load balance requests and perform failover/switchover operations

• Flexible configuration. .

• based on and .

• based on .

• Multiple : S3 compatible, PVCs, Kubernetes volumes and VolumeSnapshots.

• Flexible backup configuration: , , , , , , ...

• : restore the closest available backup to the specified time.

• from: Physical backups, logical backups, S3, PVCs, VolumeSnapshots...

• : roll out replica Pods one by one, wait for each of them to become ready, and then proceed with the primary Pod, using ReplicasFirstPrimaryLast.

• Manual : OnDelete and Never.

• Automated .

• . Automatically trigger when my.cnf changes.

• operator reconciliation for maintenance operations.

• Issue, configure and rotate and CAs.

• Native integration with . Automatically create Certificate resources.

• via and maxscale-exporter.

• Native integration with . Automatically create ServiceMonitor resources.

• Declaratively manage : , and logical .

• Declaratively manage resources in .

• Configure for your applications.

• Orchestrate and schedule .

• Validation webhooks to provide CRD immutability.

• Additional printer columns to report the current CRD status.

• CRDs designed according to the Kubernetes .

• Install it using , or .

• Multiple : cluster-wide and single namespace.

• Helm chart to deploy and its associated CRs.

• Multi-arch distroless .

• GitOps friendly.