One of the new things in MariaDB MaxScale 2.2 is the addition of a fully integrated administrative interface implemented as a JSON-API conforming REST API. The MariaDB MaxScale REST API is a great way to monitor and interact with MariaDB MaxScale from a wide variety of programming languages but it is not the most convenient thing for command line use. For this very reason, we added a new command line client, MaxCtrl.

The MaxCtrl client is intended to be the successor/complement/alternative of MaxAdmin, the classic admin client for MariaDB MaxScale. It leverages the MaxScale REST API to provide human-readable information about MariaDB MaxScale and its modules to the user. While providing the same set of functionality that MaxAdmin does, it comes with the added benefit of being ready for multi-MaxScale setups (more on that later on).

One of the biggest benefits of the MaxCtrl client is the fact that you don’t need to be on the same server as MariaDB MaxScale to use it in a secure fashion. Since the MaxScale REST API supports HTTPS, you can now safely manage MariaDB MaxScale via encrypted connections. The client, written in Node.js, is a self-contained executable that works on a wide variety of platforms.

Getting Started with MaxCtrl

As most of the time MariaDB MaxScale is initially managed locally, the MaxCtrl client is included as a part of the maxscale package and the name of the executable is maxctrl. This means that when you install MariaDB MaxScale, you already have MaxCtrl installed as well.

The REST API listens to localhost on port 8989 by default. This is to make it more secure so that, upon installation, you don’t expose the API until you configure it and make it fully secure. In addition to this, the API requires authentication before any information can be accessed. The same users that are used in MaxAdmin also work for the REST API. This means that the default admin:mariadb credentials are also the default credentials for the REST API.

The end-user experience should be relatively similar to how MaxAdmin works but with a slightly modernized look and feel. Take, for example, the list servers command.

We can see a familiar layout of the data but with a more readable table-like formatting.

Since processing HTTP requests consisting of JSON in shell scripts is only slightly less painful compared to parsing MaxAdmin output, we’ve added the –tsv option to make it much easier.

As can be seen, the output is directly consumable by the majority of UNIX tools. For instance, if we want to only see the name and state of each server, we simply pipe the output of maxctrl –tsv list servers to cut -f 1,5.

Another nice feature of MaxCtrl is that it integrates the help output into the client program. This means that if you wonder what the exact syntax was to alter server parameters, you can simply look it up from the help output.

By starting from maxctrl help, you can explore all the commands that are available in MaxCtrl. In addition to the client itself, the same documentation can be found in the MariaDB Knowledgebase.

Connecting to Remote MaxScales

After the initial installation and configuration of a MariaDB MaxScale instance, management will often be done remotely. This means that a secure network connection is required and we achieve that by enabling the encrypted HTTPS protocol of the MaxScale REST API. To enable it, configure the admin_ssl_key, admin_ssl_cert and admin_ssl_ca_cert parameters to point to PEM format private key, public certificate and CA certificate file.

[maxscale]
threads=auto
admin_ssl_key=/var/lib/my-key.pem
admin_ssl_cert=/var/lib/my-cert.pem
admin_ssl_ca_cert=/var/lib/my-ca-cert.pem

Then just use the –secure option of maxctrl to enable encrypted connections and give the hostname and port as the parameters. If you are using a self-signed certificate, pass the path to the CA certificate with –tls-ca-cert and disable server certificate verification with –tls-verify-server-cert=false.

As our example uses localhost:8989 as the hostname and port and we use self-signed certificates, we add –hosts=localhost:8989 –secure –tls-verify-server-cert=false as the extra options to maxctrl. We also explicitly define the path to our self-signed CA certificate to tell MaxCtrl to use a custom CA certificate instead of using the bundled OS certificates.

By using HTTPS, MariaDB MaxScale can be securely managed over open networks. This makes management of multiple remote MaxScale servers significantly more convenient.

Managing Multiple MaxScales

With the addition of the REST API, multiple MaxScale instances can now be controlled from a remote location. As we saw in the previous section, the description of the –hosts parameter mentions a list of MaxScale hosts. This means that you can execute the same command on multiple MaxScale servers and see the output grouped by each host. Here’s an example of the output when a server is set into maintenance mode in two different MaxScale instances (localhost:8989 and localhost:8990).

In addition to being able to execute these simpler commands on multiple MaxScale instances, you can execute cluster-wide commands with the cluster group of commands.

The cluster diff command shows all differences in the logical configuration of each MaxScale server. This is done by comparing the APIs of the hosts given with the –hosts option to the host given as the value to the cluster diff command. If we compare the two local setups, we see that the only difference these two servers have is the listener port they listen on.

If we swap the hosts around, we’ll the how the other host differs.

This command is intended to be a tool for detecting configuration divergence between multiple MaxScale instances. The output is not in the traditional diff format mainly due to the fact that, based on our testing, a diff of two JSON objects doesn’t produce actionable output. You always need to see the full object, or at least the path of the object that changed, to know what the real difference is. The diff calculation may still receive small changes in the beta to make the output more concise and readable.

Compared to the cluster diff command the cluster sync command is used to synchronize the configurations of multiple MaxScales.

Similar to other cluster commands, the list of MaxScale instances to synchronize are given with the –hosts option. The target MaxScale, used as the baseline, is given as the argument to the command.

The command builds on top of the cluster diff command by first detecting what parts are different, what parts are new and what parts were removed. This is all done via the REST API which allows MaxCtrl to leverage the HTTP methods for first detecting the changes and then creating, deleting and updating all the REST API resources.

This command can be used to converge the runtime configurations of MaxScales that have diverged due to downtime. The best way to understand how it works in practice is to see it in action.

Example: Converging Diverged Configurations

We have two MaxScale servers, maxscale-a and maxscale-b, in front of a two node master-slave cluster. Both MaxScales are configured with the following configuration.

[maxscale]
threads=auto

[server1]
type=server
address=192.168.0.100
port=3306
protocol=MariaDBBackend

[server2]
type=server
address=192.168.0.101
port=3306
protocol=MariaDBBackend

[Cluster-Monitor]
type=monitor
module=mariadbmon
servers=server1,server2
user=maxuser
passwd=maxpwd
monitor_interval=10000

[Read-Write-Service]
type=service
router=readwritesplit
servers=server1,server2
user=maxuser
passwd=maxpwd

[Read-Write-Listener]
type=listener
service=Read-Write-Service
protocol=MariaDBClient
port=4006

Let’s say that some maintenance needs to be done on maxscale-b and we need to stop it. While we’re doing the maintenance, we realize that we want to change the monitoring interval from 10 seconds down to 2 seconds on the running MaxScale. Since maxscale-b is down, we can only do this on maxscale-a. We do it with the following command.

maxctrl --hosts maxscale-a alter monitor Cluster-Monitor monitor_interval 2000

Once we finish the maintenance on maxscale-b, we bring it up. The problem is that even though the base configurations are the same, the runtime configurations are different. We could just run the same commands again on the second MaxScale and they would end up in the same state. A more convenient way to synchronize is to simply run the following command.

maxctrl --hosts maxscale-b cluster sync maxscale-a

Now both of the MaxScale servers have the same logical configuration. This can be verified by executing the cluster diff command to see that there are no differences between the two installations.

You can also use MaxCtrl to manage multiple MaxScale instances that are managed by a Keepalived setup. For a detailed look into how MariaDB MaxScale is used with Keepalived, read the great article by Esa Korhonen on MariaDB MaxScale and Keepalived.

Summary

As can be seen in MaxCtrl, providing an easy-to-use API enriches the way you can interact with software and opens up new opportunities for further improvements.

We’d love to hear your input on how to improve MaxCtrl and the REST API. You can install it from the MariaDB download page. The MaxCtrl client can be found in the maxscale-client package which is a part of the MariaDB MaxScale repository.