> For the complete documentation index, see [llms.txt](https://mariadb.com/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://mariadb.com/docs/mariadb-cloud/cloud-management/observability.md). # Observability MariaDB Cloud Observability exposes runtime logs and metrics through APIs and integrations like Datadog and Splunk, requiring an API key and Database ID for instrumentation and dashboard configuration This page provides a high-level overview of the Observability functionality in MariaDB Cloud. In order to interact with our Observability APIs, an [API KEY](/docs/mariadb-cloud/security/managing-api-keys.md) must be generated. Throughout this document, we will refer to it as `{{SKYSQL_API_KEY}}`. Additionally, you will need the MariaDB Cloud Database ID, available by clicking on any of your existing services from the [MariaDB Cloud Console](https://app.skysql.com/) and navigating to the Details page. We will refer to the Database ID as `{{SKYSQL_DATABASE_ID}}` throughout this document. For the impatient reader, we jump right to the [Integrations section](#integrations), then for ones who are building custom instrumentation, we provide a detailed list of [APIs](#apis) and their relevant documentation. ## Integrations ### Datadog Using the [Datadog](https://www.datadoghq.com/) integration, you can instrument Observability metrics from MariaDB Cloud into your Datadog account. This integration allows you to monitor and visualize MariaDB Cloud metrics alongside other services in your Datadog dashboard. #### Requirements You will need your Datadog API key to set up the integration. We will refer to it as `{{DD_API_KEY}}` throughout this document. #### Agent Setup You will need to configure the Datadog Agent to pull metrics from us. Here is an example of how you can set up the [DataDog Agent](https://docs.datadoghq.com/agent/): 1. Create a local directory for configuration to be mapped to the Docker Container: ```shell mkdir -p /home/datadog-agent/openmetrics ``` 2. Create a `conf.yaml` file in your `openmetrics` directory with: ```yaml init_config: instances: - openmetrics_endpoint: https://api.skysql.com/observability/v2/metrics namespace: {{SKYSQL_DATABASE_ID}} extra_headers: X-API-KEY: {{SKYSQL_API_KEY}} metrics: - '.*' ``` 3. Send the metrics to the correct DataDog Site. You should refer to the [DataDog Site documentation](https://docs.datadoghq.com/getting_started/site/) to determine the correct `SITE PARAMETER` for your account. This resource provides a comprehensive list of Datadog sites and their corresponding `SITE PARAMETER` values, ensuring that your data is sent to the correct regional Datadog instance. We will refer to it as `{{DD_SITE_PARAMETER}}` throughout this document. 4. Run the Datadog Agent Docker Container with the following command: ```shell docker run -v /home/datadog-agent/openmetrics:/etc/datadog-agent/conf.d/openmetrics.d:ro \ -e DD_API_KEY={{DD_API_KEY}} -e DD_HOSTNAME="my-agent" -e DD_SITE="{{DD_SITE_PARAMETER}}" \ -e DD_LOG_LEVEL="info" gcr.io/datadoghq/agent:7 ``` 5. You should see the metrics soon to be available in [DataDog Metrics Explorer](https://app.datadoghq.com/metric/explorer). #### Testing [MariaDB Cloud APIs](#apis) If you can always check if the Observability API is working successfully by calling it directly: ```shell curl --location 'https://api.skysql.com/observability/v2/metrics' \ --header 'X-API-KEY: {{SKYSQL_API_KEY}}' ``` ### Splunk Using the [Splunk](https://www.splunk.com/) integration, you can send MariaDB Cloud logs and metrics to Splunk Cloud Platform, where you can search them with SPL and chart them on dashboards next to the rest of your monitored infrastructure. The integration is distributed as a package in the public [mariadb-corporation/splunk-integration](https://github.com/mariadb-corporation/splunk-integration) repository, which holds the collector scripts, deployment examples, dashboard examples, and the full setup instructions. #### Requirements * A MariaDB Cloud API key (`{{SKYSQL_API_KEY}}`) with read access to the Observability API. * A Splunk Cloud Platform instance with the HTTP Event Collector (HEC) enabled. * A HEC token with write access to the target index. Logs go to an events index (`mariadb_logs` by default); metrics go to a **Metrics**-type index (`mariadb_metrics` by default). * Python 3.7 or later with the `requests` library. A Splunk universal forwarder is not required. Both collectors push data directly to Splunk Cloud through HEC. #### What the package provides The package contains two collectors. They are independent, so you can deploy either one on its own or both together. | Collector | Endpoints used | What it sends to Splunk | | --------- | -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Logs | `observability/v2/logs/query`, `observability/v2/logs/archive` | Error log, audit log, and MaxScale log lines, extracted from the downloaded log archives and converted to HEC events. A per-archive checkpoint keeps already-ingested lines from being sent again. | | Metrics | `observability/v2/metrics` | The Prometheus-format metric series from the [metrics](#metrics) endpoint, parsed and converted to HEC metric events. | #### Deployment options Each collector runs in any of these modes: * **Daemon** (recommended) — a persistent process that polls on a fixed interval (`--daemon --interval `), managed by systemd, launchd, or a Kubernetes deployment. The repository includes an example unit file for each. * **Standalone run** — a single collection cycle, useful for verifying end-to-end delivery before you set up a service. * **AWS Lambda** — scheduled functions triggered by EventBridge Scheduler, with credentials read from AWS Secrets Manager and the logs checkpoint stored in Amazon S3. Both a Terraform stack and a CloudFormation template are included. Configuration is entirely through environment variables — `MARIADB_API_KEY`, `SPLUNK_HEC_URL`, and `SPLUNK_HEC_TOKEN` are required, with optional overrides for the index, source, sourcetype, batch size, and retry behavior. There is no configuration file. See the repository for the complete list. {% hint style="info" %} The repository also ships example Splunk dashboards and SPL searches for both signals, in `logs/SPLUNK_DASHBOARDS.md` and `metrics/SPLUNK_DASHBOARDS.md`. {% endhint %} Detailed documentation on how to interact with our [APIs](#apis) follows: ## APIs To build instrumentation around our [Observability APIs](https://apidocs.skysql.com/#/Observability), we expose the following endpoints: #### Logs MariaDB Cloud exposes a set of log-related endpoints under `observability/v2/logs`, allowing you to: * Retrieve logs within a specified date range * Download log archives * Query log types and servers * Manage log retention settings Refer to the [Observability section of the MariaDB Cloud API docs](https://apidocs.skysql.com/#/Observability) for the full list of parameters and responses. #### Metrics You can retrieve metrics (in Prometheus-compatible format) from MariaDB Cloud using the `observability/v2/metrics` endpoint. To learn more about query parameters and usage, see: Example: ```shell curl --location 'https://api.skysql.com/observability/v2/metrics' \ --header 'X-API-KEY: {{SKYSQL_API_KEY}}' ``` Refer to the [Observability section of the MariaDB Cloud API docs](https://apidocs.skysql.com/#/Observability) for the full list of parameters and responses. ### API Documentation For the complete, detailed API reference (including request/response formats, error codes, etc.), please see the official MariaDB Cloud API docs here: * [MariaDB Cloud Observability (Logs + Metrics) Endpoints](https://apidocs.skysql.com/#/Observability). * [Prometheus HTTP API](https://prometheus.io/docs/prometheus/latest/querying/api/). ### Appendix #### Table A. Key Observability Metric Series We export the following metrics as part of the [metrics](#metrics) endpoint: | Metric | | ------------------------------------------------------------ | | mariadb\_galera\_evs\_repl\_latency\_avg\_seconds | | mariadb\_galera\_evs\_repl\_latency\_max\_seconds | | mariadb\_galera\_evs\_repl\_latency\_min\_seconds | | mariadb\_galera\_status\_info | | mariadb\_galera\_state\_comments | | mariadb\_galera\_wsrep\_cluster\_status | | mariadb\_galera\_wsrep\_connected | | mariadb\_galera\_wsrep\_desync\_count | | mariadb\_galera\_wsrep\_flow\_control\_paused\_ns | | mariadb\_galera\_wsrep\_flow\_control\_recv | | mariadb\_galera\_wsrep\_flow\_control\_sent | | mariadb\_galera\_wsrep\_local\_bf\_aborts | | mariadb\_galera\_wsrep\_local\_cert\_failures | | mariadb\_galera\_wsrep\_local\_recv\_queue | | mariadb\_galera\_wsrep\_local\_state | | mariadb\_galera\_wsrep\_ready | | mariadb\_galera\_wsrep\_received | | mariadb\_galera\_wsrep\_received\_bytes | | mariadb\_galera\_wsrep\_replicated | | mariadb\_galera\_wsrep\_replicated\_bytes | | mariadb\_global\_status\_aborted\_clients | | mariadb\_global\_status\_aborted\_connects | | mariadb\_global\_status\_buffer\_pool\_pages | | mariadb\_global\_status\_bytes\_received | | mariadb\_global\_status\_bytes\_sent | | mariadb\_global\_status\_commands\_total | | mariadb\_global\_status\_created\_tmp\_disk\_tables | | mariadb\_global\_status\_created\_tmp\_files | | mariadb\_global\_status\_created\_tmp\_tables | | mariadb\_global\_status\_handlers\_total | | mariadb\_global\_status\_innodb\_data\_read | | mariadb\_global\_status\_innodb\_data\_written | | mariadb\_global\_status\_innodb\_num\_open\_files | | mariadb\_global\_status\_innodb\_page\_size | | mariadb\_global\_status\_open\_files | | mariadb\_global\_status\_open\_table\_definitions | | mariadb\_global\_status\_open\_tables | | mariadb\_global\_status\_opened\_files | | mariadb\_global\_status\_opened\_table\_definitions | | mariadb\_global\_status\_opened\_tables | | mariadb\_global\_status\_queries | | mariadb\_global\_status\_questions | | mariadb\_global\_status\_rows\_read | | mariadb\_global\_status\_rows\_sent | | mariadb\_global\_status\_select\_full\_join | | mariadb\_global\_status\_select\_full\_range\_join | | mariadb\_global\_status\_select\_range | | mariadb\_global\_status\_select\_range\_check | | mariadb\_global\_status\_select\_scan | | mariadb\_global\_status\_slave\_running | | mariadb\_global\_status\_slow\_queries | | mariadb\_global\_status\_sort\_merge\_passes | | mariadb\_global\_status\_sort\_range | | mariadb\_global\_status\_sort\_rows | | mariadb\_global\_status\_sort\_scan | | mariadb\_global\_status\_table\_locks\_immediate | | mariadb\_global\_status\_table\_locks\_waited | | mariadb\_global\_status\_table\_open\_cache\_hits | | mariadb\_global\_status\_table\_open\_cache\_misses | | mariadb\_global\_status\_table\_open\_cache\_overflows | | mariadb\_global\_status\_threads\_cached | | mariadb\_global\_status\_threads\_connected | | mariadb\_global\_status\_threads\_created | | mariadb\_global\_status\_threads\_running | | mariadb\_global\_status\_uptime | | mariadb\_global\_status\_wsrep\_cert\_deps\_distance | | mariadb\_global\_status\_wsrep\_cluster\_conf\_id | | mariadb\_global\_status\_wsrep\_cluster\_size | | mariadb\_global\_status\_wsrep\_cluster\_status | | mariadb\_global\_status\_wsrep\_connected | | mariadb\_global\_status\_wsrep\_flow\_control\_paused | | mariadb\_global\_status\_wsrep\_last\_committed | | mariadb\_global\_status\_wsrep\_local\_recv\_queue | | mariadb\_global\_status\_wsrep\_local\_recv\_queue\_avg | | mariadb\_global\_status\_wsrep\_local\_recv\_queue\_max | | mariadb\_global\_status\_wsrep\_local\_recv\_queue\_min | | mariadb\_global\_status\_wsrep\_local\_send\_queue | | mariadb\_global\_status\_wsrep\_local\_send\_queue\_avg | | mariadb\_global\_status\_wsrep\_local\_send\_queue\_max | | mariadb\_global\_status\_wsrep\_local\_send\_queue\_min | | mariadb\_global\_status\_wsrep\_local\_state | | mariadb\_global\_status\_wsrep\_ready | | mariadb\_global\_status\_wsrep\_replicated | | mariadb\_global\_status\_wsrep\_replicated\_bytes | | mariadb\_global\_variables\_gtid\_current\_pos | | mariadb\_global\_variables\_innodb\_buffer\_pool\_size | | mariadb\_global\_variables\_innodb\_log\_buffer\_size | | mariadb\_global\_variables\_key\_buffer\_size | | mariadb\_global\_variables\_max\_connections | | mariadb\_global\_variables\_open\_files\_limit | | mariadb\_global\_variables\_query\_cache\_size | | mariadb\_global\_variables\_read\_only | | mariadb\_global\_variables\_table\_open\_cache | | mariadb\_info\_schema\_engine\_table\_count\_total | | mariadb\_info\_schema\_table\_size | | mariadb\_security\_users\_without\_passwords | | mariadb\_slave\_status\_exec\_master\_log\_pos | | mariadb\_slave\_status\_last\_io\_errno | | mariadb\_slave\_status\_last\_sql\_errno | | mariadb\_slave\_status\_read\_master\_log\_pos | | mariadb\_slave\_status\_relay\_log\_pos | | mariadb\_slave\_status\_seconds\_behind\_master | | mariadb\_slave\_status\_slave\_io\_running | | mariadb\_slave\_status\_slave\_sql\_running | | mariadb\_up | | mariadb\_xpand\_activity\_core0 | | mariadb\_xpand\_activity\_til | | mariadb\_xpand\_capacity\_disk\_system\_avail\_bytes | | mariadb\_xpand\_capacity\_disk\_system\_max\_bytes | | mariadb\_xpand\_capacity\_disk\_system\_usage\_ratio | | mariadb\_xpand\_capacity\_disk\_total\_usage\_percent | | mariadb\_xpand\_cluster\_nodes\_in\_quorum | | mariadb\_xpand\_cluster\_total\_nodes | | mariadb\_xpand\_cluster\_uptime\_seconds | | mariadb\_xpand\_containers\_rows | | mariadb\_xpand\_cpu\_load | | mariadb\_xpand\_io\_disk\_latency\_seconds | | mariadb\_xpand\_io\_network\_bytes | | mariadb\_xpand\_io\_network\_latency\_seconds | | mariadb\_xpand\_memory\_bm\_bytes | | mariadb\_xpand\_memory\_reserved\_bytes | | mariadb\_xpand\_memory\_total\_bytes | | mariadb\_xpand\_memory\_working\_bytes | | mariadb\_xpand\_qps | | mariadb\_xpand\_rebalancer\_jobs\_queued | | mariadb\_xpand\_rebalancer\_rebalancer\_rebalance | | mariadb\_xpand\_rebalancer\_rebalancer\_reprotects | | mariadb\_xpand\_rebalancer\_rebalancer\_reranks | | mariadb\_xpand\_rebalancer\_rebalancer\_softfail\_reprotects | | mariadb\_xpand\_rebalancer\_rebalancer\_splits | | mariadb\_xpand\_rebalancer\_underprotected\_slices | | mariadb\_xpand\_response\_time\_seconds | | mariadb\_xpand\_sessions | | mariadb\_xpand\_sessions\_time\_in\_state | | mariadb\_xpand\_sessions\_trx\_age | | mariadb\_xpand\_stats\_Com\_alter\_cluster | | mariadb\_xpand\_stats\_Com\_delete | | mariadb\_xpand\_stats\_Com\_delete\_seconds | | mariadb\_xpand\_stats\_Com\_insert | | mariadb\_xpand\_stats\_Com\_insert\_seconds | | mariadb\_xpand\_stats\_Com\_select | | mariadb\_xpand\_stats\_Com\_select\_seconds | | mariadb\_xpand\_stats\_Com\_set\_option | | mariadb\_xpand\_stats\_Com\_show\_slave\_status | | mariadb\_xpand\_stats\_Com\_show\_status | | mariadb\_xpand\_stats\_Com\_show\_variables | | mariadb\_xpand\_stats\_Com\_update | | mariadb\_xpand\_stats\_Com\_update\_seconds | | mariadb\_xpand\_stats\_connections | | mariadb\_xpand\_tps | | mariadb\_xpand\_wals\_avg\_sync\_time\_seconds | | maxscale\_modules | | maxscale\_server\_active\_operations | | maxscale\_server\_adaptive\_avg\_select\_time | | maxscale\_server\_connection\_pool\_empty | | maxscale\_server\_connections | | maxscale\_server\_max\_connections | | maxscale\_server\_max\_pool\_size | | maxscale\_server\_persistent\_connections | | maxscale\_server\_reused\_connections | | maxscale\_server\_routed\_packets | | maxscale\_server\_total\_connections | | maxscale\_service\_connections | | maxscale\_threads\_count | | maxscale\_threads\_current\_descriptors | | maxscale\_threads\_errors | | maxscale\_threads\_event\_queue\_length | | maxscale\_threads\_hangups | | maxscale\_threads\_max\_queue\_time | | maxscale\_threads\_reads | | maxscale\_threads\_stack\_size | | maxscale\_threads\_total\_descriptors | | maxscale\_threads\_writes | | maxscale\_up | | maxscale\_uptime\_seconds | | process\_resident\_memory\_bytes | *This page is: Copyright © 2026 MariaDB. All rights reserved.*