Deployment

Deploy MariaDB AI RAG as a Docker stack by downloading the compose file and config template, setting a license key and model credentials, then launching with docker compose up.

MariaDB AI RAG is an all-in-one, enterprise-ready solution that handles the entire Retrieval-Augmented Generation (RAG) pipeline, including document parsing with layout extraction, chunking, embedding generation, and easy-to-use retrieval APIs backed by hybrid search (vector + full-text search), with optional reranking before sending the relevant context to a foundation model for answer generation.

MariaDB AI RAG deployed as a containerized stack using Docker. For a quick evaluation, you can also leverage a MariaDB vector store deployed in a container alongside the AI RAG stack to get an end-to-end setup faster.

As of AI RAG 1.1 release, native binary-based deployments (such as .deb or .rpm packages) are no longer available.

Prerequisites

Ensure your environment meets the following requirements before starting the deployment:

Hardware: Minimum 4 CPU cores, 8 GB RAM, and 20 GB free disk space.
Operating System: 64-bit Linux
Software: Docker Engine and Docker Compose must be installed.
Mandatory Credentials:
- MariaDB License Key: A valid key is required for the application to pass the startup check.
- Model Provider API Keys: Credentials for chosen model providers (e.g., Google Gemini or OpenAI).
Database: A MariaDB 11.8+ instance is required for native vector search support.

Setup & Launch Instructions

Prepare Your Directory

Create a dedicated deployment folder on your host machine and navigate into it:

mkdir mariadb-rag-deployment
cd mariadb-rag-deployment

Obtain Configuration Files

Download the following essential files from the public AI RAG GitHub repository and place them in your new folder:

docker-compose.yml: The blueprint defining all services in the stack.
config.env.template: The template containing all necessary environment variables.

Create Local Storage Directory

Manually create these folders to prevent Docker from assigning 'root' permissions, which can cause access errors later:

mkdir uploaded_files
mkdir logs

Obtain the MariaDB License Key

The application performs a mandatory validation check at startup and will fail to start if this key is missing, invalid, or expired. Each license is valid for 30 days from the day of generation.

To get your license, follow these steps:

Navigate to the MariaDB License Portal.
Login with your MariaDB ID.
Click Get License under the RAG Trial card.
Click Generate License. You may then View, Copy, or Download the license information.
Paste this key into your config.env.secure file as MARIADB_LICENSE_KEY.

MariaDB trial license keys are valid for 30 days from the date of issue.

Configure Database and Security Keys

For a full breakdown of every variable in config.env.secure, see the AI-RAG Configuration Guide.

Open config.env.secure in a text editor to set your environment variables.

Security & Unified Authentication

To enable unified authentication across the API and MCP server, set these three variables to the same secure string

SECRET_KEY
JWT_SECRET_KEY
MCP_AUTH_SECRET_KEY

Secure String Generation

You may generate a secure 32-character string to assign to the three variables using openssl rand -hex 32

Using a unique, randomly generated key prevents "replay attacks" and ensures that only your authorized MCP Gateway can trigger heavy-lifting tasks in the RAG API.

Model Provider Configuration

Specify the models and API keys for your chosen providers.

LLM Settings

LLM_PROVIDER: Choose your provider (e.g., gemini, openai, or ollama).
LLM_MODEL: The specific model name (e.g., gemini-2.5-flash-lite).
GEMINI_API_KEY or OPENAI_API_KEY: Your provider-specific API key.

Embedding Settings

EMBEDDING_PROVIDER: Provider for vectorizing text (e.g., openai, gemini).
embedding_model: Specific embedding model (e.g., gemini-embedding-001).

Model Provider Setup Examples

Choose your preferred provider by configuring the following variables:

LLM_PROVIDER=gemini
LLM_MODEL=gemini-2.5-flash-lite
GEMINI_API_KEY=your_gemini_key_here
EMBEDDING_PROVIDER=gemini
embedding_model=gemini-embedding-001

Reranking & Search Accuracy

Reranking ensures that the most relevant document chunks are sent to the AI model, significantly reducing "hallucinations".

RERANKING_ENABLED: Set to true to activate the secondary search pass.
RERANKING_MODEL_TYPE: Defaults to flashrank (local/CPU efficient). Cloud options include cohere.
RERANKING_MODEL_NAME: The specific model used for scoring. Default: ms-marco-MiniLM-L-12-v2.

Database Configuration

These variables define how the application connects to your MariaDB 11.8+ instance.Common Connection Variables:

DB_PORT: The port your database is listening on (Default: 3306).
DB_USER: The username for database access.
DB_PASSWORD: The password for the specified user.
DB_NAME: The name of the database schema.

SSL Mandatory for Cloud Databases

For cloud-hosted instances, you must set DB_SSL_ENABLED=true to ensure secure, encrypted communication over the public internet.

Update the DB_HOST variables based on your database location:

Set DB_HOST=mariadb. The hostname must match the container name provisioned with Docker Compose. Also set your preferred DB_USER, DB_PASSWORD, and DB_NAME for database that will be provisioned along AI RAG stack.

Admin User Configuration

Set the admin credentials that will be used to access the AI RAG APIs:

ADMIN_EMAIL - Admin username
ADMIN_PASSWORD - Admin user password

Launch the Application

Run the following command to start the service

docker compose -f docker-compose.yml --env-file config.env.secure up -d

Verify and Access

Check the status of your services:

Check status

docker compose -f docker-compose.yml ps

Once services are Up or Healthy, access the interactive Swagger UI to view all API documentation and test endpoints:

URL: http://localhost:8000/docs

PreviousManagement NextNetwork and Firewall Requirements

Last updated 2 months ago

Was this helpful?