A Google Group exists for MariaDB MaxScale. The Group is used to discuss ideas, issues and communicate with the MariaDB MaxScale community. Send email to maxscale@googlegroups.com or use the forum interface.

Bugs can be reported in the MariaDB Jira jira.mariadb.org

Installing MariaDB MaxScale

Information about installing MariaDB MaxScale, either from a repository or by building from source code, is included in the MariaDB MaxScale Installation Guide.

The same guide also provides basic information on running MariaDB MaxScale. More detailed information about configuring MariaDB MaxScale can be found in the Configuration Guide.

_{This page is licensed: CC BY-SA / Gnu FDL}

We want two services and ports to which the client application can connect. One service routes client connections to the primary server, the other load balances between replica servers. To achieve this, we need to define two services in the configuration file.

Create the following two sections in your configuration file. The section names are the names of the services and should be meaningful. For this tutorial, we use the names_Write-Service_ and Read-Service.

router defines the routing module used. Here we use readconnroute for connection-level routing.

A service needs a list of servers to route queries to. The server names must match the names of server sections in the configuration file and not the hostnames or addresses of the servers.

The router_options-parameter tells the readconnroute-module which servers it should route a client connection to. For the write service we use the master-type and for the read service the slave-type.

The user and password parameters define the credentials the service uses to populate user authentication data. These users were created at the start of the MaxScale Tutorial.

For increased security, see password encryption.

Configuring the Listener

To allow network connections to a service, a network ports must be associated with it. This is done by creating a separate listener section in the configuration file. A service may have multiple listeners but for this tutorial one per service is enough.

The service parameter tells which service the listener connects to. For the_Write-Listener_ we set it to Write-Service and for the Read-Listener we set it to Read-Service.

A listener must define the network port to listen on.

The optional address-parameter defines the local address the listener should bind to. This may be required when the host machine has multiple network interfaces. The default behavior is to listen on all network interfaces (the IPv6 address ::).

Starting MariaDB MaxScale

For the last steps, please return to MaxScale Tutorial.

_{This page is licensed: CC BY-SA / Gnu FDL}

The actual Xpand monitor configuration looks as follows:

The mandatory parameters are the object type, the monitor module to use, the list of servers to use for bootstrapping and the username and password to use when connecting to the servers.

The monitor_interval parameter specifies how frequently the monitor should ping the health check port of each server and the cluster_monitor_interval specifies how frequently the monitor should do a complete cluster check, that is, access the system tables of the Cluster for checking the Cluster configuration. The default values are 2000 and 60000, that is, 2 seconds and 1 minute, respectively.

For each detected Xpand node a corresponding MaxScale server object will be created, whose name is @@<Monitor-Name>:node-<id>, where _Monitor-Name_ is the name of the monitor, in this exampleXpand` and id is the node id of the Xpand node. So, with a cluster of three nodes, the created servers might be named like.

Note that as these are created at runtime and may disappear at any moment, depending on changes happening in and made to the Xpand cluster, they should never be referred to directly from service configurations. Instead, services should refer to the monitor, as shown in the following:

Instead of listing the servers of the service explicitly using the servers parameter as usually is the case, the service refers to the Xpand monitor using the cluster parameter. This will cause the service to use the Xpand nodes that the Xpand monitor discovers at runtime.

For additional details, please consult the monitor documentation.

_{This page is licensed: CC BY-SA / Gnu FDL}

If the cdcusers file cannot be found, the service user (maxuser:maxpwd in the example) can be used to connect through the CDC protocol.

For more details, refer to the CDC users documentation.

Protocol Phases

Connection and Authentication

• Client connects to MaxScale CDC protocol listener.

• Send the authentication message which includes the user and the SHA1 of the password

In the future, optional flags could be implemented.

Registration

• Sending UUID

• Specify the output format (AVRO or JSON) for data retrieval.

Data Request

• Send CDC commands to retrieve router statistics or to query for data events

Protocol Details

Authentication

The authentication starts when the client sends the hexadecimal representation of the username concatenated with a colon (:) and the SHA1 of the password.

bin2hex(username + ':' + SHA1(password))

For example the user foobar with a password of foopasswd should send the following hexadecimal string

Server returns OK on success and ERR on failure.

Registration

REGISTER

REGISTER UUID=UUID, TYPE={JSON | AVRO}

Register as a client to the service.

Example:

Server returns OK on success and ERR on failure.

Change Data Capture Commands

REQUEST-DATA

REQUEST-DATA DATABASE.TABLE[.VERSION] [GTID]

This command fetches data from specified table in a database and returns the output in the requested format (AVRO or JSON). Data records are sent to clients and if new AVRO versions are found (e.g. mydb.mytable.0000002.avro) the new schema and data will be sent as well.

The data will be streamed until the client closes the connection.

Clients should continue reading from network in order to automatically gets new events.

Example:

Example Client

MaxScale includes an example CDC client application written in Python 3. You can find the source code for it in the MaxScale repository.

_{This page is licensed: CC BY-SA / Gnu FDL}

Behavior

The order the servers are defined in is the order in which the servers are queried. This means that the results are ordered based on the servers parameter of the service. The result will only be completed once all servers have executed this.

All commands executed via this router will be executed on all servers. This means that an INSERT through the cat router will send it to all servers. In the case of commands that do not return resultsets, the response of the last server is sent to the client. This means that if one of the earlier servers returns a different result, the client will not see it.

As the intended use-case of the router is to mainly reduce multiple result sets into one, it has no mechanisms to prevent writes from being executed on slave servers (which would cause data corruption or replication failure). Take great care when performing administrative operations though this router.

If a connection to one of the servers is lost, the client connection will also be closed.

Example

Here is a simple example service definition that uses the servers from the Configuring Servers tutorial and the credentials from the MaxScale Tutorial.

_{This page is licensed: CC BY-SA / Gnu FDL}

Monitor User

The monitor user does not require any special grants to monitor a Galera cluster. To create a user for the monitor, execute the following SQL.

_{This page is licensed: CC BY-SA / Gnu FDL}

If a custom location was used for the key file, give it as the first argument tomaxpasswd and pass the password to be encrypted as the second argument. For more information, see maxkeys --help.

Here is an example configuration that uses an encrypted password.

If the key file is not in the default location, the datadir parameter must be set to the directory that contains it.

_{This page is licensed: CC BY-SA / Gnu FDL}

After configuring the servers and the monitor, we create a read-write-splitter service configuration. Create the following section in your configuration file. The section name is also the name of the service and should be meaningful. For this tutorial, we use the name Splitter-Service.

router defines the routing module used. Here we use readwritesplit for query-level read-write-splitting.

A service needs a list of servers where queries will be routed to. The server names must match the names of server sections in the configuration file and not the hostnames or addresses of the servers.

The user and password parameters define the credentials the service uses to populate user authentication data. These users were created at the start of the MaxScale Tutorial.

For increased security, see password encryption.

Configuring the Listener

To allow network connections to a service, a network ports must be associated with it. This is done by creating a separate listener section in the configuration file. A service may have multiple listeners but for this tutorial one is enough.

The service parameter tells which service the listener connects to. For the_Splitter-Listener_ we set it to Splitter-Service.

A listener must define the network port to listen on.

The optional address-parameter defines the local address the listener should bind to. This may be required when the host machine has multiple network interfaces. The default behavior is to listen on all network interfaces (the IPv6 address ::).

Starting MariaDB MaxScale

For the last steps, please return to MaxScale Tutorial.

_{This page is licensed: CC BY-SA / Gnu FDL}

The default way the communication between MaxAdmin and MariaDB MaxScale is handled has been changed from an internet socket to a Unix domain socket. The former alternative is still available but has been deprecated.

If no arguments are given to MaxAdmin, it will attempt to connect to MariaDB MaxScale using a Unix domain socket. After the upgrade you will need to provide at least one internet socket related flag - -h, -P,-u or -p - to force MaxAdmin to use the internet socket approach.

E.g.

MySQL Monitor

The MySQL Monitor now assigns the stale state to the master server by default. In addition to this, the slave servers receive the stale slave state when they lose the connection to the master. This should not cause changes in behavior but the output of MaxAdmin will show new states when replication is broken.

_{This page is licensed: CC BY-SA / Gnu FDL}

The database users should be configured according to the configuration guide. The MaxScale Tutorial contains easy to follow instructions on how to set up MaxScale.

This tutorial will assume the user is using of the binary distributions available and has installed this in the default location. The process of configuring MariaDB MaxScale will be covered within this document. The installation and configuration of the MariaDB servers will not be covered in-depth.

Preparing MaxScale

Follow the MaxScale Tutorial to install and prepare the required database users for MaxScale. You don't need to create the configuration file for MaxScale as it will be covered in the next section.

Creating Your MariaDB MaxScale Configuration

The first step in the creation of your maxscale.cnf file is to define the global maxscale section. This section configures the number of threads MariaDB MaxScale uses. A good rule of thumb is to use at most as may threads as you have CPUs. MariaDB MaxScale uses few threads for internal operations so one or two threads less than the maximum should be enough.

After this we configure two servers we will use to shard our database. The accounts_east server will hold one schema and the accounts_west will hold another schema. We will use these two servers to create our sharded database.

The next step is to configure the service which the users connect to. This section defines which router to use, which servers to connect to and the credentials to use. The router we use in this tutorial is the schemarouter.

After this we configure a listener for the service. The listener is the actual port the user connects to. We will use the port 4000.

The final step is to configure a monitor which will monitor the state of the servers. The monitor will notify MariaDB MaxScale if the servers are down. We add the two servers to the monitor, define the credentials to use and we set the monitoring cycle interval.

After this we have a fully working configuration and we can move on to starting MariaDB MaxScale.

Starting MariaDB MaxScale

Upon completion of the configuration process MariaDB MaxScale is ready to be started . This may either be done manually by running the maxscale command or via the service interface. The service scripts are located in the /etc/init.d/ folder and are accessible through both the service and systemctl commands.

MariaDB MaxScale is now ready to start accepting client connections and routing them. Queries are routed to the right servers based on the database they target and switching between the shards is seamless since MariaDB MaxScale keeps the session state intact between servers.

If MariaDB MaxScale fails to start, check the error log in /var/log/maxscale to see what sort of errors were detected.

Note: As the sharding solution in MaxScale is relatively simple, cross-database queries between two or more shards are not supported.

_{This page is licensed: CC BY-SA / Gnu FDL}

This is the minimum set of requirements that must be met to build the MaxScale core package. Some modules in MaxScale require optional extra dependencies.

• libuuid (binlogrouter)

• boost (binlogrouter)

• Bison 2.7 or later (dbfwfilter)

• Flex 2.5.35 or later (dbfwfilter)

• librdkafka (kafkacdc, kafkaimporter and mirror)

• memcached (storage_memcached for the cache filter)

• hiredis (storage_redis for the cache filter)

Some of these dependencies are not available on all operating systems and are downloaded automatically during the build step. To skip the building of modules that need automatic downloading of the dependencies, use -DBUNDLE=N when configuring CMake.

Quickstart

This installs MaxScale as if it was installed from a package. Install git before running the following commands.

Required Packages

For a definitive list of packages, consult the install_build_deps.sh script.

Configuring the Build

The tests and other parts of the build can be controlled via CMake arguments.

Here is a small table with the names of the most common parameters and what they control. These should all be given as parameters to the -D switch inNAME=VALUE format (e.g. -DBUILD_TESTS=Y).

Note: You can look into defaults.cmake for a list of the CMake variables.

Running unit tests

To run the MaxScale unit test suite, configure the build with -DBUILD_TESTS=Y, compile and then run the make test command.

Building MariaDB MaxScale packages

If you wish to build packages, just add -DPACKAGE=Y to the CMake invocation and build the package with make package instead of installing MaxScale withmake install. This process will create a RPM/DEB package depending on your system.

To build a tarball, add -DTARBALL=Y to the cmake invokation. This will create a maxscale-x.y.z.tar.gz file where x.y.z is the version number.

Some Debian and Ubuntu systems suffer from a bug where make package fails with errors from dpkg-shlibdeps. This can be fixed by running make beforemake package and adding the path to the libmaxscale-common.so library to the LD_LIBRARY_PATH environment variable.

Installing optional components

The MaxScale build system is split into multiple components. The main component is the core MaxScale package which contains MaxScale and all the modules. This is the default component that is build, installed and packaged. There is also the experimental component that contains all experimental modules which are not considered as part of the core MaxScale package and are either alpha or beta quality modules.

To build the experimental modules along with the MaxScale core components, invoke CMake with -DTARGET_COMPONENT=core,experimental.

_{This page is licensed: CC BY-SA / Gnu FDL}

Introduction

MaxGUI is a browser-based interface for MaxScale REST-API and query execution.

Enabling MaxGUI

To enable MaxGUI in a testing mode, add admin_host=0.0.0.0 andadmin_secure_gui=false under the [maxscale] section of the MaxScale configuration file. Once enabled, MaxGUI will be available on port 8989:http://127.0.0.1:8989/

Securing the GUI

To make MaxGUI secure, set admin_secure_gui=true and configure both theadmin_ssl_key and admin_ssl_cert parameters.

See Configuration Guide and Configuration and Hardening for instructions on how to harden your MaxScale installation for production use.

Authentication

MaxGUI uses the same credentials as maxctrl. The default username is admin with mariadb as the password.

Internally, MaxGUI uses JSON Web Tokens as the authentication method for persisting the user's session. If the Remember me checkbox is ticked, the session will persist for 24 hours. Otherwise, the session will expire as soon as MaxGUI is closed.

To log out, simply click the username section in the top right corner of the page header to access the logout menu.

Pages

Dashboard

This page provides an overview of MaxScale configuration which includes Monitors, Servers, Services, Sessions, Listeners, and Filters.

By default, the refresh interval is 10 seconds.

Detail

This page shows information on each MaxScale object and allow to edit its parameter, relationships and perform other manipulation operations.

Access this page by clicking on the MaxScale object name on the dashboard page

Visualization

This page visualizes MaxScale configuration and clusters.

• Configuration: Visualizing MaxScale configuration.

• Cluster: Visualizing a replication cluster into a tree graph and provides manual cluster manipulation operations such asswitchover, reset-replication, release-locks, failover, rejoin . At the moment, it supports only servers monitored by Monitor using mariadbmon module.

Access this page by clicking the graph icon on the sidebar navigation.

Settings

This page shows and allows editing of MaxScale parameters.

Access this page by clicking the gear icon on the sidebar navigation.

Logs Archive

Realtime MaxScale logs can be accessed by clicking the logs icon on the sidebar navigation.

Workspace

The "Workspace" page offers a versatile set of tools for effectively managing data and database interactions. It includes the following key tasks:

1. Run Queries

Execute queries on various servers, services, or listeners to retrieve data and perform database operations. Visualize query results using different graph types such as line, bar, or scatter graphs. Export query results in formats like CSV or JSON for further analysis and sharing.

2. Data Migration

The "Data Migration" feature facilitates seamless transitions from PostgreSQL to MariaDB. Transfer data and database structures between the two systems while ensuring data integrity and consistency throughout the process.

_{This page is licensed: CC BY-SA / Gnu FDL}

• global_script

• session_script

Filter Parameters

The luafilter has two parameters. They control which scripts will be called by the filter. Both parameters are optional but at least one should be defined. If both global_script and session_script are defined, the entry points in both scripts will be called.

global_script

The global Lua script. The parameter value is a path to a readable Lua script which will be executed.

This script will always be called with the same global Lua state and it can be used to build a global view of the whole service.

session_script

The session level Lua script. The parameter value is a path to a readable Lua script which will be executed once for each session.

Each session will have its own Lua state meaning that each session can have a unique Lua environment. Use this script to do session specific tasks.

Lua Script Calling Convention

The entry points for the Lua script expect the following signatures:

• nil createInstance(name) - global script only, called when the script is first loaded

  • When the global script is loaded, it first executes on a global level before the luafilter calls the createInstance function in the Lua script with the filter's name as its argument.

• nil newSession(string, string) - new session is created

  • After the session script is loaded, the newSession function in the Lua scripts is called. The first parameter is the username of the client and the second parameter is the client's network address.

• nil closeSession() - session is closed

  • The closeSession function in the Lua scripts will be called.

• (nil | bool | string) routeQuery(string) - query is being routed

  • The Luafilter calls the routeQuery functions of both the session and the global script. The query is passed as a string parameter to the routeQuery Lua function and the return values of the session specific function, if any were returned, are interpreted. If the first value is bool, it is interpreted as a decision whether to route the query or to send an error packet to the client. If it is a string, the current query is replaced with the return value and the query will be routed. If nil is returned, the query is routed normally.

• nil clientReply(string) - reply to a query is being routed

  • This function is called with the name of the server that returned the response.

• string diagnostic() - global script only, print diagnostic information

  • If the Lua function returns a string that is valid JSON, it will be decoded as JSON and displayed as such in the REST API. If the object does not decode into JSON, it will be stored as a JSON string.

These functions, if found in the script, will be called whenever a call to the matching entry point is made.

Script Template

Here is a script template that can be used to try out the luafilter. Copy it into a file and add global_script=<path to script> into the filter configuration. Make sure the file is readable by the maxscale user.

Functions Exposed by the Luafilter

The luafilter exposes the following functions that can be called inside the Lua script API endpoints.

• string mxs_get_type_mask()

• Returns the type of the current query being executed as a string. The values are the string versions of the query types defined in query_classifier.h are separated by vertical bars (|). This function can only be called from the routeQuery entry point.

• string mxs_get_operation()

• Returns the current operation type as a string. The values are defined in query_classifier.h. This function can only be called from the routeQuery entry point.

• string mxs_get_canonical()

• Returns the canonical version of a query by replacing all user-defined constant values with question marks. This function can only be called from the routeQuery entry point.

• number mxs_get_session_id()

• This function returns the session ID of the current session. Inside thecreateInstance and diagnostic endpoints this function will always return the value 0.

• string mxs_get_db()

• Returns the current default database used by the connection.

• string mxs_get_user()

• Returns the username of the client connection.

• string mxs_get_host()

• Returns the address of the client connection.

Example Configuration and Script

Here is a minimal configuration entry for a luafilter definition.

And here is a script that opens a file in /tmp/ and logs output to it.

_{This page is licensed: CC BY-SA / Gnu FDL}

Filters can be divided into a number of categories

Logging filters

Logging filters do not in any way alter the statement or results of the statements that are passed through MariaDB MaxScale. They merely log some information about some or all of the statements and/or result sets.

Two examples of logging filters are contained within the MariaDB MaxScale, a filter that will log all statements and another that will log only a number of statements, based on the duration of the execution of the query.

Statement rewriting filters

Statement rewriting filters modify the statements that are passed through the filter. This allows a filter to be used as a mechanism to alter the statements that are seen by the database, an example of the use of this might be to allow an application to remain unchanged when the underlying database changes or to compensate for the migration from one database schema to another.

Result set manipulation filters

A result set manipulation filter is very similar to a statement rewriting but applies to the result set returned rather than the statement executed. An example of this may be obfuscating the values in a column.

Routing hint filters

Routing hint filters are filters that embed hints in the request that can be used by the router onto which the query is passed. These hints include suggested destinations as well as metric that may be used by the routing process.

Firewall filters

A firewall filter is a mechanism that allows queries to be blocked within MariaDB MaxScale before they are sent on to the database server for execution. They allow constructs or individual queries to be intercepted and give a level of access control that is more flexible than the traditional database grant mechanism.

Pipeline control filters

A pipeline filter is one that has an affect on how the requests are routed within the internal MariaDB MaxScale components. The most obvious version of this is the ability to add a "tee" connector in the pipeline, duplicating the request and sending it to a second MariaDB MaxScale service for processing.

Filter Definition

Filters are defined in the configuration file, typically maxscale.cnf, using a section for each filter instance. The content of the filter sections in the configuration file various from filter to filter, however there are always to entries present for every filter, the type and module.

The type is used by the configuration manager within MariaDB MaxScale to determine what this section is defining and the module is the name of the plugin that implements the filter.

When a filter is used within a service in MariaDB MaxScale the entry filters= is added to the service definition in the ini file section for the service. Multiple filters can be defined using a syntax akin to the Linux shell pipe syntax.

The names used in the filters= parameter are the names of the filter definition sections in the ini file. The same filter definition can be used in multiple services and the same filter module can have multiple instances, each with its own section in the ini file.

Filter Examples

The filters that are bundled with the MariaDB MaxScale are documented separately, in this section a short overview of how these might be used for some simple tasks will be discussed. These are just examples of how these filters might be used, other filters may also be easily added that will enhance the MariaDB MaxScale functionality still further.

Log The 30 Longest Running Queries

The top filter can be used to measure the execution time of every statement within a connection and log the details of the longest running statements.

The first thing to do is to define a filter entry in the ini file for the top filter. In this case we will call it "top30". The type is filter and the module that implements the filter is called topfilter.

In the definition above we have defined two filter specific parameters, the count of the number of statement to be logged and a filebase that is used to define where to log the information. This filename is a stem to which a session id is added for each new connection that uses the filter.

The filter keeps track of every statement that is executed, monitors the time it takes for a response to come back and uses this as the measure of execution time for the statement. If the time is longer than the other statements that have been recorded, then this is added to the ordered list within the filter. Once 30 statements have been recorded those statements that have been recorded with the least time are discarded from the list. The result is that at any time the filter has a list of the 30 longest running statements in each session.

When the session ends a report will be written for the session into the logfile defined. That report will include the top 30 longest running statements, plus summary data for the session;

• The time the connection was opened.

• The host the connection was from.

• The username used in the connection.

• The duration of the connection.

• The total number of statements executed in the connection.

• The average execution time for a statement in this connection.

Duplicate Data From Your Application Into Cassandra

The scenario we are using in this example is one in which you have an online gaming application that is designed to work with a MariaDB database. The database schema includes a high score table which you would like to have access to in a Cassandra cluster. The application is already using MariaDB MaxScale to connect to a MariaDB Galera cluster, using a service names BubbleGame. The definition of that service is as follows

The table you wish to store in Cassandra in called HighScore and will contain the same columns in both the MariaDB table and the Cassandra table. The first step is to install a MariaDB instance with the Cassandra storage engine to act as a bridge server between the relational database and Cassandra. In this bridge server add a table definition for the HighScore table with the engine type set to Cassandra. See Cassandra Storage Engine Overview for details. Add this server into the MariaDB MaxScale configuration and create a service that will connect to this server.

Next add a filter definition for the tee filter that will duplication insert statements that are destined for the HighScore table to this new service.

The above filter definition will cause all statements that match the regular expression inset.*HighScore.*values to be duplication and sent not just to the original destination, via the router but also to the service named Cassandra.

The final step is to add the filter to the BubbleGame service to enable the use of the filter.

_{This page is licensed: CC BY-SA / Gnu FDL}

The Xpand monitor is capable of autonomously figuring out the cluster configuration, but in order to get going there must be at least one_server_-section referring to a node in the Xpand cluster.

That server defintion will be used by the monitor in order to connect to the Xpand cluster. There can be more than one such "bootstrap" definition to cater for the case that the node used as a bootstrap server is down when MaxScale starts.

NOTE These bootstrap servers should only be referred to from the Xpand monitor configuration, but never from a service.

Monitor

In the Xpand monitor section, the bootstrap servers are referred to in the same way as "ordinary" servers are referred to in other monitors.

The user defined by the user parameter needs the following grants:

In case the same user is used both for the monitor and the service (see below), then the user must be given the grants required by the service as well.

The bootstrap servers are only used for connecting to the Xpand cluster; thereafter the Xpand monitor will dynamically find out the cluster configuration.

The discovered cluster configuration will be stored (the ips and ports of the Xpand nodes) and upon subsequent restarts the Xpand monitor will use that information if the bootstrap servers happen to be unavailable.

With the configuration above maxctrl list servers might output the following:

All servers whose name start with @@ have been detected dynamically.

Note that the address 10.2.224.101 appears twice; once forBootstrap-1 and another time for @@Xpand:node-6. The Xpand monitor will create a dynamic server instance for all nodes in the Xpand cluster; also for the ones used in bootstrap server sections.

Service

The service is specified as follows:

The user defined by the user parameter needs the following grants:

In case the same user is used both for the monitor (see above) and the service, then the user must be given the grants required by the monitor as well.

Note that the service does not list any specific servers, but instead refers, using the argument cluster, to the Xpand monitor.

In practice this means that the service will use the servers of the monitor named Xpand and in the case of a Xpand monitor those servers will be the ones that the monitor has detected dynamically. That is, when setup like this, the service will automatically adjust to any changes taking place in the Xpand cluster.

NOTE There is no need to specify any router_options, but the default router_options=running provides the desired behaviour. In particular do not specify router_options=master as that will cause only a single node to be used.

Listener

To complete the configuration, a listener must be specified.

Xpand and Readwritesplit

The primary purpose of the router readwritesplit is to split statements between one primary and multiple replicas. In the case of Xpand, all servers will be primaries, but readwritesplit may still be the right choise.

Namely, as readwritesplit is transaction aware and capable of replaying transactions, it can be used for hiding certain events taking place in Xpand from the clients that use it.

For instance, whenever a node is removed from or added to a Xpand cluster there will be a group change, which is visible to a client as a transaction rollback. However, if readwritesplit is used and transaction replay is enabled, then MaxScale may be able to hide the group change so that the client only detects a slight delay.

Apart from the service section, the configuration when using_readwritesplit_ is identical to the readconnroute configuration described above.

Service

The service is specified as follows:

With this configuration, subject to the boundary conditions of transaction replaying, a client will neither notice group change events nor the disappearance of the very node the client is connected to. In that latter case, MaxScale will simply connect to another node and replay the current transaction (if one is active). For detailed information about the transaction replay functionality, please refer to the readwritesplitdocumentation.

NOTE It is vital to haveslave_selection_criteria=LEAST_GLOBAL_CONNECTIONS, as otherwise connections will not be distributed evenly across all Xpand nodes.

As a rule of thumb, use readwritesplit if it is important that changes taking place in the cluster configuration are hidden from the applications, otherwise use readconnroute.

_{This page is licensed: CC BY-SA / Gnu FDL}

The tarball has been built with the assumption that it will be installed in /usr/local. However, it is possible to install it in any directory, but in that case MariaDB MaxScale must be invoked with a flag.

Installing as root in /usr/local

If you have root access to the system you probably want to install MariaDB MaxScale under the user and group maxscale.

The required steps are as follows:

Creating the symbolic link is necessary, since MariaDB MaxScale has been built with the assumption that the plugin directory is /usr/local/maxscale/lib/maxscale.

The symbolic link also makes it easy to switch between different versions of MariaDB MaxScale that have been installed side by side in /usr/local; just make the symbolic link point to another installation.

In addition, the first time you install MariaDB MaxScale from a tarball you need to create the following directories:

and make maxscale the owner of them:

The following step is to create the MariaDB MaxScale configuration file /etc/maxscale.cnf. The file etc/maxscale.cnf.template can be used as a base. Please refer to Configuration Guide for details.

When the configuration file has been created, MariaDB MaxScale can be started.

The -d flag causes maxscale not to turn itself into a daemon, which is adviseable the first time MariaDB MaxScale is started, as it makes it easier to spot problems.

If you want to place the configuration file somewhere else but in /etc you can invoke MariaDB MaxScale with the --config flag, for instance, --config=/usr/local/maxscale/etc/maxscale.cnf.

Note also that if you want to keep everything under /usr/local/maxscale you can invoke MariaDB MaxScale using the flag --basedir.

That will cause MariaDB MaxScale to look for its configuration file in/usr/local/maxscale/etc and to store all runtime files under /usr/local/maxscale/var.

Installing in any Directory

Enter a directory where you have the right to create a subdirectory. Then do as follows.

The next step is to create the MaxScale configuration file maxscale-x.y.z/etc/maxscale.cnf. The file maxscale-x.y.z/etc/maxscale.cnf.template can be used as a base. Please refer to Configuration Guide for details.

When the configuration file has been created, MariaDB MaxScale can be started.

With the flag --basedir, MariaDB MaxScale is told where the lib, etc and var directories are found. Unless it is specified, MariaDB MaxScale assumes the lib directory is found in /usr/local/maxscale, and the var and etc directories in /.

It is also possible to specify the directories and the location of the configuration file individually. Invoke MaxScale like

to find out the appropriate flags.

_{This page is licensed: CC BY-SA / Gnu FDL}

Every MaxScale service with a MariaDB protocol listener requires knowledge of the user accounts defined on the backend databases. The service maintains this information in an internal component called the user account manager (UAM). The UAM queries relevant data from the mysql-database of the backends and stores it. Typically, only the current primary server is queried, as all servers are assumed to have the same users. The service settings user and password define the credentials used when fetching user accounts.

The service uses the stored data when authenticating clients, checking their passwords and database access rights. This results in an authentication process very similar to the MariaDB Server itself. Unauthorized users are generally detected already at the MaxScale level instead of the backend servers. This may not apply in some cases, for example if MaxScale is using old user account data.

If authentication fails, the UAM updates its data from a backend. MaxScale may attempt authenticating the client again with the refreshed data without communicating the first failure to the client. This transparent user data update does not always work, in which case the client should try to log in again.

As the UAM is shared between all listeners of a service, its settings are defined in the service configuration. For more information, search the configuration guide for users_refresh_time, users_refresh_interval and_auth_all_servers_. Other settings which affect how the UAM connects to backends are the global settings auth_connect_timeout and local_address, and the various server-level ssl-settings.

Required grants

To properly fetch user account information, the MaxScale service user must be able to read from various tables in the mysql-database: user, db,tables_priv, columns_priv, procs_priv, proxies_priv and roles_mapping. The user should also have the SHOW DATABASES-grant.

If using MariaDB ColumnStore, the following grant is required:

Limitations and troubleshooting

When a client logs in to MaxScale, MaxScale sees the client's IP address. When MaxScale then connects the client to backends (using the client's username and password), the backends see the connection coming from the IP address of MaxScale. If the client user account is to a wildcard host ('alice'@'%'), this is not an issue. If the host is restricted ('alice'@'123.123.123.123'), authentication to backends will fail.

There are two primary ways to deal with this:

1. Duplicate user accounts. For every user account with a restricted hostname an equivalent user account for MaxScale is added ('alice'@'maxscale-ip').

2. Use proxy protocol.

Option 1 limits the passwords for user accounts with shared usernames. Such accounts must use the same password since they will effectively share the MaxScale-to-backend user account. Option 2 requires server support.

See MaxScale Troubleshooting for additional information on how to solve authentication issues.

Wildcard database grants

MaxScale supports wildcards _ and % for database-level grants. As with MariaDB Server, grant select on test_.* to 'alice'@'%'; gives access totest_ as well as test1, test2 and so on. If the GRANT command escapes the wildcard (grant select on test_.* to 'alice'@'%';) both MaxScale and the MariaDB Server interpret it as only allowing access to test_. _ and % are only interpreted as wildcards when the grant is to a database:grant select on test_.t1 to 'alice'@'%'; only grants access to the_test_.t1_-table, not to test1.t1.

Authenticator options

The listener configuration defines authentication options which only affect the listener. authenticator defines the authentication plugins to use.authenticator_options sets various options. These options may affect an individual authentication plugin or the authentication as a whole. The latter are explained below. Multiple options can be given as a comma-separated list.

skip_authentication

• Type: boolean

• Mandatory: No

• Dynamic: No

• Default: false

If enabled, MaxScale will not check the passwords of incoming clients and just assumes that they are correct. Wrong passwords are instead detected when MaxScale tries to authenticate to the backend servers.

This setting is mainly meant for failure tolerance in situations where the password check is performed outside of MaxScale. If, for example, MaxScale cannot use an LDAP-server but the backend databases can, enabling this setting allows clients to log in. Even with this setting enabled, a user account matching the incoming client username and IP must exist on the backends for MaxScale to accept the client.

This setting is incompatible with standard MariaDB/MySQL authentication plugin (MariaDBAuth in MaxScale). If enabled, MaxScale cannot authenticate clients to backend servers using standard authentication.

match_host

• Type: boolean

• Mandatory: No

• Dynamic: No

• Default: true

If disabled, MaxScale does not require that a valid user account entry for incoming clients exists on the backends. Specifically, only the client username needs to match a user account, hostname/IP is ignored.

This setting may be used to force clients to connect through MaxScale. Normally, creating the user jdoe@% will allow the user jdoe to connect from any IP-address. By disabling match_host and replacing the user with_jdoe@maxscale-IP_, the user can still connect from any client IP but will be forced to go through MaxScale.

lower_case_table_names

• Type: number

• Mandatory: No

• Dynamic: No

• Default: 0

Controls database name matching for authentication when an incoming client logs in to a non-empty database. The setting functions similar to the MariaDB Server setting and should be set to the value used by the backends.

The setting accepts the values 0, 1 or 2:

• 0: case-sensitive matching (default)

• 1: convert the requested database name to lower case before using case-insensitive matching. Assumes that database names on the server are stored in lower case.

• 2: use case-insensitive matching.

true and false are also accepted for backwards compatibility. These map to 1 and 0, respectively.

The identifier names are converted using an ASCII-only function. This means that non-ASCII characters will retain their case-sensitivity.

Starting with MaxScale versions 2.5.25, 6.4.6, 22.08.5 and 23.02.2, the behavior of lower_case_table_names=1 is identical with how the MariaDB server behaves. In older releases the comparisons were done in a case-sensitive manner after the requested database name was converted into lowercase. Usinglower_case_table_names=2 will behave identically in all versions which makes it a safe alternative to use when a mix of older and newer MaxScale versions is being used.

_{This page is licensed: CC BY-SA / Gnu FDL}

Overview

The namedserverfilter is a MariaDB MaxScale filter module able to route queries to servers based on regular expression (regex) matches. Since it is a filter instead of a router, the NamedServerFilter only sets routing suggestions. It requires a compatible router to be effective. Currently, bothreadwritesplit and hintrouter take advantage of routing hints in the data packets. This filter uses the PCRE2 library for regular expression matching.

Configuration

The filter accepts settings in two modes: legacy and indexed. Only one of the modes may be used for a given filter instance. The legacy mode is meant for backwards compatibility and allows only one regular expression and one server name in the configuration. In indexed mode, up to 25 regex-server pairs are allowed in the form match01 - target01, match02 - target02 and so on. Also, in indexed mode, the server names (targets) may contain a list of names or special tags ->master or ->slave.

All parameters except the deprecated match and target parameters can be modified at runtime. Any modifications to the filter configuration will only affect sessions created after the change has completed.

Below is a configuration example for the filter in indexed-mode. The legacy mode is not recommended and may be removed in a future release. In the example, a SELECT on TableOne (match01) results in routing hints to two named servers, while a SELECT on TableTwo is suggested to be routed to the primary server of the service. Whether a list of server names is interpreted as a route-to-any or route-to-all is up to the attached router. The HintRouter sees a list as a suggestion to route-to-any. For additional information on hints and how they can also be embedded into SQL-queries, see Hint-Syntax.

Filter Parameters

NamedServerFilter requires at least one matchXY - targetXY pair.

matchXY

• Type: regex

• Mandatory: No

• Dynamic: Yes

• Default: None

matchXY defines a PCRE2 regular expression against which the incoming SQL query is matched. XY must be a number in the range 01 - 25. Each match-setting pairs with a similarly indexed target-setting. If one is defined, the other must be defined as well. If a query matches the pattern, the filter attaches a routing hint defined by the target-setting to the query. The_options_-parameter affects how the patterns are compiled.

options

• Type: enum

• Mandatory: No

• Dynamic: Yes

• Values: ignorecase, case, extended

• Default: ignorecase

Regular expression options for matchXY.

targetXY

• Type: string

• Mandatory: No

• Dynamic: Yes

• Default: None

The hint which is attached to the queries matching the regular expression defined by_matchXY_. If a compatible router is used in the service the query will be routed accordingly. The target can be one of the following:

• a server or service name (adds a HINT_ROUTE_TO_NAMED_SERVER hint)

• a list of server names, comma-separated (adds severalHINT_ROUTE_TO_NAMED_SERVER hints)

• ->master (adds a HINT_ROUTE_TO_MASTER hint)

• ->slave (adds a HINT_ROUTE_TO_SLAVE hint)

• ->all (adds a HINT_ROUTE_TO_ALL hint)

The support for service names was added in MaxScale 6.3.2. Older versions of MaxScale did not accept service names in the target parameters.

source

• Type: string

• Mandatory: No

• Dynamic: Yes

• Default: None

This optional parameter defines an IP address or mask which a connecting client's IP address is matched against. Only sessions whose address matches this setting will have this filter active and performing the regex matching. Traffic from other client IPs is simply left as is and routed straight through.

Since MaxScale 2.1 it's also possible to use % wildcards:

Note that using source=% to match any IP is not allowed.

Since MaxScale 2.3 it's also possible to specify multiple addresses separated by comma. Incoming client connections are subsequently checked against each.

user

• Type: string

• Mandatory: No

• Dynamic: Yes

• Default: None

This optional parameter defines a username the connecting client username is matched against. Only sessions that are connected using this username will have the match and routing hints applied to them. Traffic from other users is simply left as is and routed straight through.

Additional remarks

The maximum number of accepted match - target pairs is 25.

In the configuration file, the indexed match and target settings may be in any order and may skip numbers. During SQL-query matching, however, the regexes are tested in ascending order: match01, match02, match03 and so on. As soon as a match is found for a given query, the routing hints are written and the packet is forwarded to the next filter or router. Any remaining match regexes are ignored. This means the match - target pairs should be indexed in priority order, or, if priority is not a factor, in order of decreasing match probability.

Binary-mode prepared statements (COM_STMT_PREPARE) are handled by matching the prepared sql against the match-parameters. If a match is found, the routing hints are attached to any execution of that prepared statement. Text- mode prepared statements are not supported in this way. To divert them, use regular expressions which match the specific "EXECUTE"-query.

Examples

Example 1 - Route queries targeting a specific table to a server

This will route all queries matching the regular expression *from *users to the server named server2. The filter will ignore character case in queries.

A query like SELECT * FROM users would be routed to server2 where as a query like SELECT * FROM accounts would be routed according to the normal rules of the router.

_{This page is licensed: CC BY-SA / Gnu FDL}

The precise installation process varies from one distribution to another. Details on package installation can be found in the Installation Guide.

Creating a user account for MaxScale

MaxScale checks that incoming clients are valid. To do this, MaxScale needs to retrieve user authentication information from the backend databases. Create a special user account for this purpose by executing the following SQL commands on the primary server of your database cluster. The following tutorials will use these credentials.

MariaDB versions 10.2.2 to 10.2.10 also require GRANT SELECT ON mysql.* TO 'maxscale'@'%';

Creating client user accounts

Because MariaDB MaxScale sits between the clients and the backend databases, the backend databases will see all clients as if they were connecting from MaxScale's address. This usually means that two sets of grants for each user are required.

For example, assume that the user 'jdoe'@'client-host' exists and MaxScale is located at_maxscale-host_. If 'jdoe'@'client-host' needs to be able to connect through MaxScale, another user, 'jdoe'@'maxscale-host', must be created. The second user must have the same password and similar grants as 'jdoe'@'client-host'.

The quickest way to do this is to first create the new user:

Then do a SHOW GRANTS query:

Then copy the same grants to the 'jdoe'@'maxscale-host' user.

An alternative to generating two separate accounts is to use one account with a wildcard host ('jdoe'@'%') which covers both hosts. This is more convenient but less secure than having specific user accounts as it allows access from all hosts.

Creating the configuration file

MaxScale reads its configuration from /etc/maxscale.cnf. A template configuration is provided with the MaxScale installation.

A global maxscale section is included in every MaxScale configuration file. This section sets the values of various global parameters, such as the number of threads MaxScale uses to handle client requests. To set thread count to the number of available cpu cores, set the following.

Configuring the servers

Read the Configuring Servers mini-tutorial for server configuration instructions.

Configuring the monitor

The type of monitor used depends on the type of cluster used. For a primary-replica cluster read Configuring MariaDB Monitor. For a Galera cluster read Configuring Galera Monitor.

Configuring the services and listeners

This part is covered in two different tutorials. For a fully automated read-write-splitting setup, read the Read Write Splitting Tutorial. For a simple connection based setup, read the Connection Routing Tutorial.

Starting MaxScale

After configuration is complete, MariaDB MaxScale is ready to start. For systems that use systemd, use the systemctl command.

For older SysV systems, use the service command.

If MaxScale fails to start, check the error log in /var/log/maxscale/maxscale.log to see if any errors are detected in the configuration file.

Checking MaxScale status with MaxCtrl

The maxctrl-command can be used to confirm that MaxScale is running and the services, listeners and servers have been correctly configured. The following shows expected output when using a read-write-splitting configuration.

MariaDB MaxScale is now ready to start accepting client connections and route queries to the backend cluster.

More options can be found in the Configuration Guide,readwritesplit module documentation and readconnroute module documentation.

For more information about MaxCtrl and how to secure it, see the REST-API Tutorial.

_{This page is licensed: CC BY-SA / Gnu FDL}

The primary server where we will be replicating from needs to have binary logging enabled, binlog_format set to row and binlog_row_image set tofull. These can be enabled by adding the two following lines to the my.cnf file of the primary.

You can find out more about replication formats from the

Configuring MaxScale

We start by adding two new services into the configuration file. The first service is the binlogrouter service which will read the binary logs from the primary server. The second service will read the binlogs as they are streamed from the primary and convert them into Avro format files.

The source parameter in the avro-service points to the replication-service we defined before. This service will be the data source for the avrorouter. The_filestem_ is the prefix in the binlog files and start_index is the binlog number to start from. With these parameters, the avrorouter will start reading events from binlog binlog.000015.

Note that the filestem and start_index must point to the file that is the first binlog that the binlogrouter will replicate. For example, if the first file you are replicating is my-binlog-file.001234, set the parameters tofilestem=my-binlog-file and start_index=1234.

For more information on the avrorouter options, read the Avrorouter Documentation.

Preparing the data in the primary server

Before starting the MaxScale process, we need to make sure that the binary logs of the primary server contain the DDL statements that define the table layouts. What this means is that the CREATE TABLE statements need to be in the binary logs before the conversion process is started.

If the binary logs contain data modification events for tables that aren't created in the binary logs, the Avro schema of the table needs to be manually created. There are multiple ways to do this:

• Dump the database to a replica, configure it to replicate from the primary and point MaxScale to this replica (this is the recommended method as it requires no extra steps)

• Use the cdc_schema Go utility and copy the generated .avsc files to the avrodir

• Use the Python version of the schema generator and copy the generated .avsc files to the avrodir

If you used the schema generator scripts, all Avro schema files for tables that are not created in the binary logs need to be in the location pointed to by the_avrodir_ parameter. The files use the following naming:<database>.<table>.<schema_version>.avsc. For example, the schema file name of the test.t1 table would be test.t1.0000001.avsc.

Starting MariaDB MaxScale

The next step is to start MariaDB MaxScale and set up the binlogrouter. We do that by connecting to the MySQL listener of the replication_router service and executing a few commands.

NOTE: GTID replication is not currently supported and file-and-position replication must be used.

This will start the replication of binary logs from the primary server at 172.18.0.1 listening on port 3000. The first file that the binlogrouter replicates is binlog.000015. This is the same file that was configured as the starting file in the avrorouter.

For more details about the SQL commands, refer to the Binlogrouter documentation.

After the binary log streaming has started, the avrorouter will automatically start processing the binlogs.

Creating and Processing Data

Next, create a simple test table and populated it with some data by executing the following statements.

To use the cdc.py command line client to connect to the CDC service, we must first create a user. This can be done via maxctrl by executing the following command.

This will create the maxuser:maxpwd credentials which can then be used to request a JSON data stream of the test.t1 table that was created earlier.

The output is a stream of JSON events describing the changes done to the database.

The first record is always the JSON format schema for the table describing the types and names of the fields. All records that follow it represent the changes that have happened on the database.

_{This page is licensed: CC BY-SA / Gnu FDL}

Ed25519 is a highly secure authentication method based on public key cryptography. It is used with the auth_ed25519-plugin of MariaDB Server.

When a client authenticates via ed25519, MaxScale first sends them a random message. The client signs the message using their password as private key and sends the signature back. MaxScale then checks the signature using the public key fetched from the mysql.user-table. The client password or an equivalent token is never exposed. For more information, see .

The security of this authentication scheme presents a problem for a proxy such as MaxScale since MaxScale needs to log in to backend servers on behalf of the client. Since each server will generate their own random messages, MaxScale cannot simply forward the original signature. Either the real password is required, or a different authentication scheme must be used between MaxScale and backends. The MaxScale ed25519auth-plugin supports both alternatives.

Configuration

To begin, add "ed25519auth" to the list of authenticators for a listener.

MaxScale will now authenticate incoming clients with ed25519 if their user account has plugin set to "ed25519" in the mysql.user-table. However, routing queries will fail since MaxScale cannot authenticate to backends. To continue, either use a mapping file or enable sha256-mode. Sha256-mode is enabled with the following settings.

ed_mode

This setting defines the authentication mode used. Two values are supported:

• ed25519 (default) Digital signature based authentication. Requires mapping for backend support.

• sha256 Authenticate client with caching_sha2_password-plugin instead. Requires either SSL or configured RSA-keys.

ed_rsa_privkey_path and ed_rsa_pubkey_path

Defines the RSA-keys used for encrypting the client password if SSL is not in use. Should point to files with the private and public keys.

Using a mapping file

To enable MaxScale to authenticate to backends,user mapping can be used. The mapping and backend passwords are given in a json-file. The client can map to an identical username or to another user, and the backend authentication scheme can be something else than ed25519.

The following example maps user "alpha" to "beta" and MaxScale then uses standard authentication to log into backends as "beta". User "alpha" authenticates to MaxScale using whatever method configured in the server. User "gamma" does not map to another user, just the password is given.

MaxScale configuration:

/home/joe/mapping.json:

Using sha256-authentication

The mapping-based solution requires the DBA to maintain a file with user passwords, which has security and upkeep implications. To avoid this, MaxScale can instead use the caching_sha2_password-plugin to authenticate the client. This authentication scheme transmits the client password to MaxScale in full, allowing MaxScale to log into backends using ed25519. MaxScale effectively lies to the client about its authentication plugin and then uses the correct plugin with the backends. Enable sha256-authentication by setting authentication option ed_mode to "sha256".

sha256-authentication is best used with encrypted connections. The example below shows a listener configured for sha256-mode and SSL.

If SSL is not in use, caching_sha2_password transmits the password using RSA-encryption. In this case, MaxScale needs the public and private RSA-keys. MaxScale sends the public key to the client if they don't already have it and the client uses it to encrypt the password. MaxScale then uses the private key to decrypt the password. The example below shows a listener configured for sha256-mode without SSL.

The keyfiles can be generated with OpenSSL using the following commands.

_{This page is licensed: CC BY-SA / Gnu FDL}

Overview

The throttle filter is used to limit the maximum query frequency (QPS - queries per second) of a database session to a configurable value. The main use cases are to prevent a rogue session (client side error) and a DoS attack from overloading the system.

The throttling is dynamic. The query frequency is not limited to an absolute value. Depending on the configuration the throttle will allow some amount of high frequency queries, or especially short bursts with no frequency limitation.

Configuration

Basic Configuration

This configuration states that the query frequency will be throttled to around 500 qps, and that the time limit a query is allowed to stay at the maximum frequency is 60 seconds. All values involving time are configured in milliseconds. With the basic configuration the throttling will be nearly immediate, i.e. a session will only be allowed very short bursts of high frequency querying.

When a session has been continuously throttled for throttling_duration milliseconds, or 60 seconds in this example, MaxScale will disconnect the session.

Allowing high frequency bursts

The two parameters max_qps and sampling_duration together define how a session is throttled.

Suppose max qps is 400 qps and sampling duration is 10 seconds. Since QPS is not an instantaneous measure, but one could say it has a granularity of 10 seconds, we see that over the 10 seconds 10*400 = 4000 queries are allowed before throttling kicks in.

With these values, a fresh session can start off with a speed of 2000 qps, and maintain that speed for 2 seconds before throttling starts.

If the client continues to query at high speed and throttling duration is set to 10 seconds, Maxscale will disconnect the session 12 seconds after it started.

Filter Parameters

max_qps

• Type: number

• Mandatory: Yes

• Dynamic: Yes

Maximum queries per second.

This is the frequency to which a session will be limited over a given time period. QPS is not measured as an instantaneous value but over a configurable sampling duration (see sampling_duration).

throttling_duration

• Type: duration

• Mandatory: Yes

• Dynamic: Yes

This defines how long a session is allowed to be throttled before MaxScale disconnects the session.

sampling_duration

• Type: duration

• Mandatory: No

• Dynamic: Yes

• Default: 250ms

Sampling duration defines the window of time over which QPS is measured. This parameter directly affects the amount of time that high frequency queries are allowed before throttling kicks in.

The lower this value is, the more strict throttling becomes. Conversely, the longer this time is, the longer bursts of high frequency querying is allowed.

continuous_duration

• Type: duration

• Mandatory: No

• Dynamic: Yes

• Default: 2s

This value defines what continuous throttling means. Continuous throttling starts as soon as the filter throttles the frequency. Continuous throttling ends when no throttling has been performed in the past continuous_duration time.

_{This page is licensed: CC BY-SA / Gnu FDL}