Transforming Databases for the AI Era: The Vector Revolution

I recently joined MariaDB as VP of Product Management, coming from a background in Artificial Intelligence (AI) Engineering and Product Management in databases and AI platform development. In my career, I have seen quite a few transformations in how we store, process, analyze, and more recently use AI in more and more processes – both personal and in business. Recently, we announced a new capability for MariaDB Server – vector operations for use with vector embeddings and GenAI applications. AI is fast becoming part of our lives, and MariaDB is committed to investing in AI related capabilities to help organizations maximize AI’s effectiveness. In this blog, I explain AI adoption trends and how it is impacting the database market today. I’ll also share why I’m particularly excited for MariaDB’s upcoming vector search capabilities as well as my predictions around what’s next for databases in this era of AI.

AI/ML Adoption Trends

The adoption of AI and machine learning (ML) in enterprises is rapidly increasing.

Source: https://www.linkedin.com/posts/gartner-for-high-tech_gartnerht-genai-techceo-activity-7206341174737940480-xAH_/

According to Gartner, 75 percent of US companies are increasing their AI budgets. This trend is expected to continue given the proliferation of AI both from the consumer side (OpenAI’s ChatGPT) and the enterprise side using chatbots and building sophisticated AI Agent systems.

Key trends in AI adoption include:

1. GenAI enabling democratization: GenAI tools have become widely available, leading to increased adoption across various industries. An example is how coding in many of its forms is becoming democratized by users who do not know how to code, but can create applications
2. AI for Workplace Productivity: AI is being increasingly used to automate repetitive tasks and enhance productivity in the workplace via AI Agents use cases – more and more of these use cases will be enabled over time
3. Multimodal AI: AI systems that can process multiple types of data (text, audio, video, images) are becoming more prevalent and will lead to a greater need for multi-modal data management

Vector Search Market Growth

While it is difficult to determine specific market size estimations for vector search, the increasing importance of vector databases in AI applications indicate growth will be significant in the years ahead. Vector databases are becoming crucial for powering applications based on large language models (LLMs) like OpenAI’s GPT4 or Google’s Gemini. The landscape is evolving rapidly, with a growing set of vector database solutions becoming increasingly available, indicating an expanding market. What is also indicative of usage of vector embeddings is the incorporation of vector support into traditional databases, such as the new capabilities available in MariaDB Server.

Why Vector Embeddings Matter Now

We are living in a captivating time in the technology space – while vector embeddings themselves are not new, given the rise of LLMs, the application of vector embeddings has reached an inflection point. Every day, more and more businesses are converting their raw data into vector representations, whether that be text, image, audio, or video data.

There is a fundamental difference between how vector search vs. traditional relational database search works. Traditional relational databases shine at exact matches and range queries, for example – finding all customers in New York who spent over $500 last month. Vector related operations are quite different, as they shine at finding similarities and patterns, for example – finding products that customers might like based on their behavior or identifying unusual transaction patterns that could indicate an anomaly.

Why MariaDB’s Vector Approach

Here are some of the things I’m excited about regarding MariaDB’s approach:

• Simplify your data stack – keep things simple and leverage what you already have:
  • No need to maintain separate vector databases
  • Reduction in complex data synchronization processes / reduce operational overhead
  • Use current security and access controls
• Native use of vector embeddings, not an add-on capability
  • Seamless integration with existing SQL operations
  • No plug in required, thus no additional management required
  • Fully open source

MariaDB’s approach stands out by natively integrating vector capabilities directly into its core database engine, keeping everything contained and simple within a database that is already used for other business applications. This makes it simple to use vector embeddings with existing relational data and easy to get started.

Predictions Around AI and Vector Database Capabilities

We’re only at the beginning of the AI and vector revolution. As embedding models become more sophisticated and new applications emerge, the ability to efficiently store and query vectors will become as fundamental as indexing text or processing transactions.

For 2025 and beyond, I predict we’ll see new patterns emerge within the vector database space:

• Hybrid queries combining traditional filters with vector similarity will become increasingly more important to align to the best information for AI at the right time
• As AI democratizes image, video, and audio generation – usage of vector embeddings will only increase and thus having robust vector capabilities will be important for new AI applications to utilize
• Vector embeddings may increasingly be used in pre-processing and post-processing data pipelines as AI Agent usage starts to proliferate, thus a way to work with vectors to store and retrieve data will only become more crucial to enable over time

Starting Your Vector Journey with MariaDB

If you’re beginning to explore vector capabilities – look for areas in your application where finding similarities or patterns could add value. This might be improving search results, adding recommendation features or detecting unusual patterns. Not all vector related functionality is tightly coupled with GenAI – there are many other ways to use vectors in other AI/ML applications, such as in clustering and classifications models for data analytics.

The advantage of having vector operations integrated into your database, such as with MariaDB Server, is that you can experiment at your own pace and you can add vector capabilities beside your existing functionality, measure the impact, and scale up as needed.

Vector search is now included as part of MariaDB Community Server 11.7 RC and will be available in our Enterprise Edition soon. Download MariaDB Server with vector search today.