Step 5: Bulk Import of Data
This page details step 5 of a 5-step procedure for deploying Single-Node Enterprise ColumnStore Enterprise ColumnStore 23.02 with Local storage.
This step bulk imports data to Enterprise ColumnStore.
Interactive commands are detailed. Alternatively, the described operations can be performed using automation.
Import the Schema
Before data can be imported into the tables, create a matching schema.
On the primary server, create the schema:
For each database that you are importing, create the database with the CREATE DATABASE statement:
CREATE DATABASE inventory;
For each table that you are importing, create the table with the CREATE TABLE statement:
CREATE TABLE inventory.products ( product_name varchar(11) NOT NULL DEFAULT '', supplier varchar(128) NOT NULL DEFAULT '', quantity varchar(128) NOT NULL DEFAULT '', unit_cost varchar(128) NOT NULL DEFAULT '' ) ENGINE=Columnstore DEFAULT CHARSET=utf8;
Import the Data
Enterprise ColumnStore supports multiple methods to import data into ColumnStore tables.
MariaDB Enterprise ColumnStore includes cpimport, which is a command-line utility designed to efficiently load data in bulk.
To import your data from a TSV (tab-separated values) file, on the primary server run cpimport:
$ sudo cpimport -s '\t' inventory products /tmp/inventory-products.tsv
LOAD DATA INFILE
To import your data from a TSV (tab-separated values) file, on the primary server use LOAD DATA INFILE statement:
LOAD DATA INFILE '/tmp/inventory-products.tsv' INTO TABLE inventory.products;
Import from Remote Database
MariaDB Enterprise ColumnStore can also import data directly from a remote database. A simple method is to query the table using the SELECT statement, and then pipe the results into cpimport, which is a command-line utility that is designed to efficiently load data in bulk.
To import your data from a remote MariaDB database:
$ mariadb --quick \ --skip-column-names \ --execute="SELECT * FROM inventory.products" \ | cpimport -s '\t' inventory products
This page was step 5 of 5.
This procedure is complete.