Data Loading with load_from_s3
Overview
MariaDB Enterprise ColumnStore includes a stored procedure called columnstore_info.load_from_s3, which can load data from a plain-text file containing delimiter-separated values (such as CSV or TSV) stored on S3-compatible cloud object storage.
Compatibility
MariaDB Enterprise ColumnStore 23.02
System Variables
Before you import data with the columnstore_info.load_from_s3 stored procedure, the authentication credentials and the region can be configured using system variables:
columnstore_s3_key
columnstore_s3_secret
columnstore_s3_region
For example, the following statements show how to set the system variables for your current session:
Import Data
To import data with the columnstore_info.load_from_s3
stored procedure, use the CALL
statement:
Replace
'BUCKET_URL'
with the URL of your bucket. The protocol in the URL must bes3: for AWS S3 or gs:
for Google Cloud StorageReplace
'FILE_NAME'
with the file name to load from. The file must be a plain-text file containing delimiter-separated values, such as a comma-separated values (CSV) or tab-separated values (TSV) file. The supported file format is similar to the plain-text file formats supported bycpimport
andLOAD DATA [LOCAL] INFILE
. Please note that this stored procedure can't load dump files created by mariadb-dumpReplace
'DATABASE_NAME'
with the database to import intoReplace
'TABLE_NAME'
with the table name to import intoReplace
'TERMINATED_BY'
with the field terminator used in the file, similar to the-s command-line
option forcpimport
Replace
'ENCLOSED_BY'
with the quotes used in the file, similar to the-E command-line
option for cpimportReplace
'ESCAPED_BY'
with the escape character used in the file, similar to the-C command-line
option forcpimport
All parameters are mandatory.
For example, to load a comma-separated values (CSV) file from AWS S3:
When the stored procedure completes, it returns JSON containing the status of the operation. If the JSON shows an error or "success": false, check your table to see if some or all of your data was loaded, because many errors are non-fatal.
Permissions
When the data file is stored in Amazon S3, the AWS user only requires the s3:GetObject
action on the bucket.
For example, the AWS user can use a user policy like the following:
This page is: Copyright © 2025 MariaDB. All rights reserved.
Last updated
Was this helpful?