Start and Configure MariaDB Agent Manager on MariaDB MaxScale Nodes

Overview

The MariaDB Agent Manager coordinates deployment of the agents needed to consolidate and push data to the SkySQL Observability service.

This page shows how to start and configure MariaDB Agent Manager on MaxScale nodes, so that they can be monitored by the SkySQL Observability service.

With MariaDB Agent Manager, MaxScale nodes can be configured as Monitored Hosts. In development environments, users might not want to dedicate a standalone host to be a Dedicated Forwarding Host. In those environments, one Monitored Host (such as any MaxScale node) can also serve as a Forwarding Host. However, in production environments, MariaDB recommends using a Dedicated Forwarding Host.

Compatibility

MariaDB MaxScale

  • MariaDB MaxScale 2.5

  • MariaDB MaxScale 6

  • MariaDB MaxScale 22.08

  • MariaDB MaxScale 23.02

Operating Systems

  • CentOS 7

  • Red Hat Enterprise Linux 7

  • Red Hat Enterprise Linux 8

  • Rocky Linux 8

  • Ubuntu 18.04 LTS

  • Ubuntu 20.04 LTS

  • Ubuntu 22.04 LTS

Prerequisites

Before performing this procedure, install MariaDB Agent Manager on each Forwarding Host and Monitored Host.

Step 1: Login as Appropriate OS User

Login as the appropriate user, which depends on which systemd mode MariaDB Agent Manager uses for your operating system:

  • On operating systems that use MariaDB Agent Manager in user systemd mode, perform the procedure as the non-root OS user that runs the services. When logging in, do not use su to switch to the user, because a full login session is required to start the systemd user manager instance used by MariaDB Agent Manager's systemd services.

  • On operating systems that use MariaDB Agent Manager in system systemd mode, perform the procedure as the root OS user.

Step 2: Configure Firewalls

MariaDB Agent Manager communicates over TCP between the Forwarding Host and each Monitored Host. MariaDB Agent Manager also communicates over TCP between the Forwarding Host and the SkySQL Observability service.

Firewall configuration must support this communication.

Forwarding Server ports

If the MaxScale node is also acting as a Forwarding Host, allow incoming traffic on the following ports:

Port

Source

Purpose

TCP/9303

localhost

API port (overridden by specifying the --api-port option when executing the mariadb_agent_manager start sub-command)

MaxScale ports

On each Monitored Host running MariaDB MaxScale, allow incoming traffic on the following ports:

Port

Source

Purpose

MaxScale listener database port (TCP port varies)

Each Forwarding Host

See "MaxScale Listener Database port parameter

TCP/8989

Each Forwarding Host

MaxScale REST API port

TCP/9100

Each Forwarding Host

node_exporter listening port

TCP/9105

Each Forwarding Host

maxscale_exporter listening port

In environments where non-default ports are used, the firewall configuration must be adapted for the specific ports in use.

Step 3: Configure MaxScale

The MaxScale node might require some additional configuration.

Configure REST API

MariaDB Agent Manager's maxscale_exporter communicates with MaxScale using the REST API.

If a MaxScale node is a Monitored Host, the node's MaxScale configuration might need to be adapted to allow REST API access from the Forwarding Hosts. The default configuration file is located at /etc/maxscale.cnf.

The following MaxScale parameters are relevant for MariaDB Agent Manager:

Parameter

Description

admin_enabled

The default value of true is required for maxscale_exporter. If this parameter has been set to false, maxscale_exporter will not function.

admin_host

The default value of 127.0.0.1 would allow connections from the maxscale_exporter process running on the host. This parameter can be changed to 0.0.0.0 to allow REST API and MaxCtrl connections from other hosts.

admin_port

The default value of 8989 works for maxscale_exporter, but the port might need to be changed if the port is already being used.

admin_ssl_key

If TLS is required, then the path to the private key file must be configured.

admin_ssl_cert

If TLS is required, then the path to the certificate file must be configured.

admin_ssl_ca_cert

If TLS is required, then the path to the Certificate Authority (CA) chain file must be configured.

For example, if TLS is not required, a minimalist MaxScale configuration file would only need to set admin_host:

[maxscale]
threads          = auto
admin_host       = 0.0.0.0

Restart MaxScale

On the MaxScale host, if the MaxScale configuration was changed in the previous step, MaxScale should be restarted:

$ sudo systemctl restart maxscale

Step 4: Create and Alter MaxScale Accounts

On each MaxScale node, some MaxScale accounts are required.

Secure Default MaxScale Account

MaxScale's default administrative user is named admin, and the default password is mariadb. For security reasons, MariaDB recommends changing the default password.

On each MaxScale node, change the default password using the maxctrl alter user command:

$ maxctrl --user=admin --password='mariadb' \
   alter user admin 'ADMIN_PASSWORD'

Create MaxScale Account

If you previously created a MaxScale user when you ran the mariadb_agent_manager install sub-command, you can skip this step.

MariaDB Agent Manager requires a MaxScale user account for maxscale_exporter. MariaDB recommends creating a dedicated MaxScale user account for MariaDB Agent Manager instead of using the default account.

If your topology has multiple MaxScale nodes, the user account should be created on each individual MaxScale node.

On each MaxScale node, create a dedicated MaxScale user account using one of the following methods:

Step 5: Prepare maxscale-credentials.json

On each Monitored Host, exporters authenticate using credentials that are stored locally. These credentials must initially be provided in a credentials file.

If you created a MaxScale user account on this server using the mariadb_agent_manager create-user or mariadb_agent_manager create-user install sub-commands, the file should have been created automatically by MariaDB Agent Manager. If not, you need to create the file manually.

The MariaDB Agent Manager credentials file is in JSON format. These instructions assume that the credentials file is named maxscale-credentials.json

On each Monitored Host running MariaDB MaxScale, create a credentials file.

Create a maxscale section with the user account and password details from "Step 4: Create and Alter MaxScale Accounts":

{
  "cred_store": {
    "maxscale": {
      "user": "mariadbagent",
      "pwd": "USER_PASSWORD"
    }
  }
}

Step 6: Encrypt Passwords

On each Monitored Host running MariaDB MaxScale, the passwords in the credentials file must be encrypted by executing the mariadb_agent_manager encrypt-creds sub-command:

$ mariadb_agent_manager encrypt-creds -p ~/maxscale-credentials.json
The following files have been successfully created:
Encrypted credentials:    /home/USER/.config/mariadb-agent-manager/cred.json
Secret key:               /home/USER/.config/mariadb-agent-manager/.key.txt
  • The -p option specifies the path to the credentials file. Alternatively, the REMOTE_AGENT_CREDENTIALS environment variable can be used.

MariaDB Agent Manager encrypts the credentials and saves the encrypted credentials to its configuration directory.

After the credentials have been encrypted, MariaDB Agent Manager deletes the credentials file. If you need to preserve the credentials file, ensure that you have a backup.

Step 7: Prepare configuration.json

MariaDB Agent Manager reads a configuration file to understand the database infrastructure. These instructions assume that the configuration file is named configuration.json. The MariaDB Agent Manager configuration file is in JSON format.

Create a configuration file:

  • Select the example configuration file for your topology and edit to reflect your database infrastructure and parameters.

  • Alternatively, a configuration file can be created using the configuration wizard by executing the mariadb_agent_manager init sub-command.

The configuration file must be copied to each Forwarding Host and each Monitored Host.

Replicated Transactions Topology

{
  "datacenter": {
    "name": "Example Data Center",
    "location": "Oregon, US",
    "database_services": [
      {
        "name": "example-replicated",
        "type": "replicated",
        "servers": [
          {
            "name": "mxs1",
            "type": "maxscale",
            "hostname": "mxs1.example.com"
          },
          {
            "name": "mdb1",
            "type": "server",
            "hostname": "mdb1.example.com"
          },
          {
            "name": "mdb2",
            "type": "server",
            "hostname": "mdb2.example.com"
          },
          {
            "name": "mdb3",
            "type": "server",
            "hostname": "mdb3.example.com"
          }
        ]
      }
    ]
  }
}

Distributed Transactions Topology

{
  "datacenter": {
    "name": "Example Data Center",
    "location": "Oregon, US",
    "database_services": [
      {
        "name": "example-distributed",
        "type": "distributed",
        "servers": [
          {
            "name": "mxs1",
            "type": "maxscale",
            "hostname": "mxs1.example.com"
          },
          {
            "name": "xpand1",
            "type": "xpand-backend",
            "hostname": "xpand1.example.com"
          },
          {
            "name": "xpand2",
            "type": "xpand-backend",
            "hostname": "xpand2.example.com"
          },
          {
            "name": "xpand3",
            "type": "xpand-backend",
            "hostname": "xpand3.example.com"
          }
        ]
      }
    ]
  }
}

Multi-Node Analytics Topology

{
  "datacenter": {
    "name": "Example Data Center",
    "location": "Oregon, US",
    "database_services": [
      {
        "name": "example-columnstore",
        "type": "columnstore",
        "servers": [
          {
            "name": "mxs1",
            "type": "maxscale",
            "hostname": "mxs1.example.com"
          },
          {
            "name": "cs_node1",
            "type": "server",
            "hostname": "mdb1.example.com"
          },
          {
            "name": "cs_node2",
            "type": "server",
            "hostname": "mdb2.example.com"
          },
          {
            "name": "cs_node3",
            "type": "server",
            "hostname": "mdb3.example.com"
          }
        ]
      }
    ]
  }
}

Galera Cluster Topology

{
  "datacenter": {
    "name": "Example Data Center",
    "location": "Oregon, US",
    "database_services": [
      {
        "name": "example-galera",
        "type": "galera",
        "servers": [
          {
            "name": "mxs1",
            "type": "maxscale",
            "hostname": "mxs1.example.com"
          },
          {
            "name": "mdb1",
            "type": "server",
            "hostname": "mdb1.example.com"
          },
          {
            "name": "mdb2",
            "type": "server",
            "hostname": "mdb2.example.com"
          },
          {
            "name": "mdb3",
            "type": "server",
            "hostname": "mdb3.example.com"
          }
        ]
      }
    ]
  }
}

Step 8: Start MariaDB Agent Manager

On each Forwarding Host and Monitored Host running MariaDB MaxScale, MariaDB Agent Manager must be started by executing the mariadb_agent_manager start sub-command.

  • If the MaxScale node is also acting as a Forwarding Host, MariaDB Agent Manager can be started for the first time with the following options:

    $ mariadb_agent_manager start -c ~/configuration.json -k ~/api_token.txt -f
    
    • The -c option specifies the path to the configuration file. Alternatively, the REMOTE_AGENT_CONFIG environment variable can be used.

    • The -k option specifies the path to a file that contains your SkySQL API Key. Alternatively, the REMOTE_AGENT_API_KEY environment variable can be used. Setting the API key is only required the first time that the services are started on the Forwarding Host, because MariaDB Agent Manager saves the API key to a file. If the option or environment variable are not set, MariaDB Agent Manager prompts the user to enter the API key.

    • The -f option indicates that the node is a Forwarding Host. Alternatively, the FORWARDING_SERVER environment variable can be used. If the host is not defined in the configuration file, MariaDB Agent Manager assumes that it is a dedicated Forwarding Host. If the host is defined in the configuration file, MariaDB Agent Manager handles it as a combination Forwarding and Monitored Host.

  • On each Monitored Host, MariaDB Agent Manager can be started with the following options:

    $ mariadb_agent_manager start -c ~/configuration.json -k ~/api_token.txt
    
    • The -c option specifies the path to the configuration file. Alternatively, the REMOTE_AGENT_CONFIG environment variable can be used.

    • If you plan to use Fluent Bit to collect and process logs, the -k option is required to specify the path to a file that contains your SkySQL API Key. Alternatively, the REMOTE_AGENT_API_KEY environment variable can be used. Setting the API key is only required the first time that the services are started on the Monitored Host, because MariaDB Agent Manager saves the API key to a file. If the option or environment variable are not set, MariaDB Agent Manager prompts the user to enter the API key.

After the service has been successfully started, MariaDB Agent Manager prints the status of the components running on the host:

STATUS of the SERVICES:
Service: Maxscale Exporter                        active
Service: Node Exporter                            active

Next Steps

If you still need to start and configure MariaDB Agent Manager on additional node types:

If you have already started and configured MariaDB Agent Manager on all nodes: