MariaDB ColumnStore 1.1.0 Beta Release Notes

Release date: 14th September 2017

MariaDB ColumnStore 1.1.0 is a Beta release of MariaDB ColumnStore. This is the first release of the MariaDB ColumnStore 1.1 series. The MariaDB ColumnStore 1.1 series provides several new features and improvements over the MariaDB ColumnStore 1.0 release.

MariaDB ColumnStore 1.1.0 is a Beta release.

For an overview of MariaDB ColumnStore see MariaDB ColumnStore Architectural Overview

New Features

1. MariaDB ColumnStore 1.1.0 is based on MariaDB Server 10.2.8
2. The Window functions have been re-implemented with MariaDB Server 10.2.8 code.
3. C++ API to programmatically load data into PM nodes. See Link to CS API Doc] for usage.
4. Text and Blob Data Types.
5. User defined distributed aggregate and window functions.
6. ColumnStore Backup/Restore Tool. See Link to CS Backup Tool Doc for usage.
7. MariaDB Server Audit Plugin Integration.
8. Built-in data redundancy using GlusterFS integration for installations that use local disks for data storage on PMs. See Link to Doc for using GlusterFS with MariaDB ColumnStore.
9. Several performance improvements in string handling, memory utilization and general area.

Please provide feedback in JIRA for anything that is not working as expected so that we can fix it before we make the release available for the larger community. For general "how to questions" ask questions here or subscribe to mariadb-columnstore@googlegroups.com

Bugs and issues fixed

• MCOL-1 - Query Failed after a redistributeDB while ddl/dml/queries were active
• MCOL-267 - TEXT and BLOB data types are not supported
• MCOL-317 - replace drizzle with maria client library
• MCOL-318 - switch to using os distribution version of snappy
• MCOL-353 - with 6 PMs, cpimport has an long lag time
• MCOL-356 - BLOB data type not supported
• MCOL-377 - columnstore queries show as vtable query rather than original query in audit log
• MCOL-379 - system down and high memory alarm set after install
• MCOL-397 - binary package install doesnt check for package dependencies
• MCOL-400 - On a 2PM combo stack with query stats enabled, one of the PMs returned an error when queries is executed
• MCOL-468 - update default replication configuration
• MCOL-480 - warning error reported after upgrade to 1.0.6
• MCOL-505 - Performance improvements to ExeMgr
• MCOL-507 - More performance improvements to ExeMgr
• MCOL-511 - native write data api
• MCOL-513 - analyze and implement thread pools and memory buffers for performance optimization
• MCOL-515 - GROUP BY in a cross engine subquery generates bad results
• MCOL-518 - backup (cold) and restore tool
• MCOL-519 - productize glusterfs support and add tools to automate
• MCOL-522 - support pre-installed software in postConfigure and addModule - phase I
• MCOL-523 - support user defined aggregate functions
• MCOL-529 - DBRM message queue clients need to be pooled
• MCOL-533 - table_usage() has a calculation error
• MCOL-534 - postCfg upgrade output refers to calpont
• MCOL-538 - Argument in alias circumvents /.my.cnf option file
• MCOL-547 - enforce limitation of postCfg needing to be run on pm1
• MCOL-550 - Possible mem leak and crash in columnstore's mysqld
• MCOL-553 - 'Too many open files' errors during DBT3 performance test
• MCOL-561 - Can't create view by using ColumnStore's windowing funcion SQL
• MCOL-566 - script remote_command.sh doesn't work on SUSE
• MCOL-574 - Cross Engine step tries to use a bad UDS path for localhost
• MCOL-602 - RPATH set incorrectly
• MCOL-605 - Crash when running table_usage
• MCOL-618 - innodb query crash only happens on columnstore server
• MCOL-622 - MariaDB 10.2 create front end for 'VAR_SAMP' window function
• MCOL-623 - MariaDB 10.2 create 'STDDEV_SAMP' Windows function
• MCOL-626 - addModule should not be alowed on single-server installs
• MCOL-628 - getSystemResourceUsage doesnt work on non-root install
• MCOL-636 - Performance improvement with string handling
• MCOL-642 - Add BLOB/TEXT detection
• MCOL-647 - debian package install error from server package
• MCOL-650 - stored procedures with queries invoked over odbc prepared statements take very long
• MCOL-652 - Derived tables in a join subquery cause error
• MCOL-653 - sum function provides different results between innodb and columnstore
• MCOL-655 - Debugging symbols are missing
• MCOL-656 - Error in LIKE behaviour
• MCOL-657 - Support NULL safe equals (<=>)
• MCOL-664 - TEXT columns need to support the same functions as VARCHAR
• MCOL-675 - calsettrace(1) can cause a mysqld crash
• MCOL-677 - Columnstore executes join on incompatible types
• MCOL-679 - Crash on prepared statement when no parameters bound
• MCOL-683 - ADDDATE & DATE_FORMAT combo breaks datetime
• MCOL-684 - Prepare statement with parameter substitution asserts in debug build
• MCOL-686 - Using BETWEEN together with date functions in WHERE clause 100x slower than InfiniDB
• MCOL-693 - Segmentation Fault with non-string SP parameter
• MCOL-696 - REVERSE may reverse all strings that are the same.
• MCOL-699 - mcsadmin display issue for getModuleDisk
• MCOL-701 - Need better error reporting -- follow up on MCOL-698
• MCOL-702 - multi node package install with non login su won't start
• MCOL-703 - postConfigure should check for local rpm / bins existing
• MCOL-707 - GROUP_CONCAT function on TEXT/TEXT got ColumnStore stuck in 'Join or subselect exceeds memory limit' error
• MCOL-710 - Adding a column failed if the column being added is of TEXT/LONGTEXT/BLOB/LONGBLOB
• MCOL-712 - TEXT/LONGTEXT columns returned NULL when COUNT() MIN() MAX() functions used in the project list
• MCOL-715 - Amazon Install - error when setting up UM storage as 'io1' volume type
• MCOL-719 - Unexpected results using LEAST or GREATEST on aggregate
• MCOL-729 - Columnstore Cluster Test Tool - add check for mariadb-libs base install
• MCOL-730 - cross engine join query select on decimal bad precision/scale
• MCOL-734 - Error messages referenced an invalid OAM command unassignPmDbrootConfig
• MCOL-736 - transaction gets autocommitted if non columnstore query executed
• MCOL-744 - primproc crashes sometimes in test001 regression test runs during drop table
• MCOL-773 - system not startup up via postConfigure/startsSystem - remote_command issue
• MCOL-793 - ORDER BY NULL breaks subsequent query
• MCOL-794 - query cache not work with all engine with columnstore
• MCOL-799 - INSERT...SELECT with window functions fail
• MCOL-812 - Cross engine join and a filter where the string contains a quote character
• MCOL-815 - Logs going to system journal in Ubuntu instead of log files
• MCOL-819 - mysqld not shutdown by shutdownsystem commands, sometimes
• MCOL-829 - Implement stored procedure INSERT...SELECT
• MCOL-830 - CrossEngineSupport can not retrieve multi byte fields normally
• MCOL-833 - could not open file for OID after a outage recover from pm2 PrimProc
• MCOL-834 - PrimProc thread leak if ExeMgr dies
• MCOL-835 - Killing MariaDB connections can crash ExeMgr
• MCOL-837 - postConfigure - WARNING: Mismatch between FilesPerColumnPartition check should be removed
• MCOL-838 - ExeMgr StringStore crash
• MCOL-839 - UDAnF: if CREATE FUNCTION hasn't run, mysqld aborts
• MCOL-845 - postConfigure creates a list of dbroots and assigns them device nodes, it starts at 'g' and increments, but it skips 'k'. system fails to start
• MCOL-849 - Process Manager leaving files called '+' lying around
• MCOL-850 - ProcMgr crashes when too many 'getactivealarm' calls are made.
• MCOL-857 - PrimProc thread leak on ByteStream advance
• MCOL-860 - temp file error when using yum localinstall
• MCOL-864 - Ubuntu columnstoreClusterTester.sh incorrectly raises issue for snappy package
• MCOL-879 - With querystats enabled, mysqld crashes when process a query
• MCOL-887 - Merge MariaDB 10.2.8
• MCOL-890 - group_contact returns garbage
• MCOL-892 - 1.0.11 upgrade failed when base directory is nfs mounted

Upgrade

Multi version upgrades are not supported, please upgrade versions prior to 1.0.11 before upgrading to 1.1.0:

• 1.0.11 GA to 1.1.0 Beta upgrade procedure

Known issues and limitations

There are a number bugs and known limitations within this version of MariaDB ColumnStore, the most serious of these are listed below.

• MCOL-73: Wide table formatted display causes frontend to return error
  • MariaDB ColumnStore supports wide tables storage
  • Displaying the query results on a large number of columns without formatting the column works
  • Displaying the query results on a large number of columns with formatting causes error at MariaDB Server level
• MCOL-271 empty string values are treated as NULL. This means you cannot insert empty values into a NOT NULL string column.
• MCOL-364: In a multi UM configuration where the default storage engine has been set to columnstore replicated tables are not created as columnstore tables. Avoid overriding the default storage engine and specify engine=columnstore on all table DDL.
• MCOL-365: Log files created by load data infile remain in the bulk/data/log and /tmp directories. If storage is a concern these can safely be removed.
• MCOL-463 : gluster storage option in installer fails with an error. The installer option to install optimized for gluster storage will fail with an error. Manually set up gluster volumes can be used with the 'External' storage option.
• MCOL-540 : In a non root Ubuntu install with local query enabled, the PM servers crash and restart on table creation.
• MCOL-892 : 1.0.11 upgrade failed when base directory is nfs mounted
• The current logging default generates full verbose debug logs. This can be controlled by making logging configuration changes as described here.
• While Millisecond and Microsecond storage is supported for datetime, time and timestamp columns, at this time the query results cannot return millisecond and microseconds.
• UTF-8 Limitation
  • UTF-8 must be declared at the table level if the instance has been set up with a UTF-8 profile. Tables created with a non-matching character set will yield indeterminate results.
  • Viewing SQL output should be done using client software that supports UTF-8 character sets.
  • UTF-8 characters are not supported in object names.
• Known security issues and fixes are documented here.

Documentation

MariaDB ColumnStore Documentation

Packaging

RPM, Debian, and binary packages are provided for the Linux distributions supported by MariaDB ColumnStore 1.1.0 Beta version.

• The supported OS for the GA version are CentOS 6, CentOS 7, Debian 8.6, Debian 9.1, RedHat 6, RedHat 7, SUSE 12, and Ubuntu 16.0.4.
• Packages can be downloaded here
• An Amazon AWS AMI Image is available for this release, please search for AMI name "MariaDB-ColumnStore-1.1.0". AMI specific installation instructions can be found here.
• Certified to run in Google Cloud Environment in the GA OSs.

Source code

The source code of MariaDB ColumnStore is tagged at GitHub with a tag, which is identical with the version of MariaDB ColumnStore. For instance, the tag of version X.Y.Z of MariaDB ColumnStore is columnstore-X.Y.Z. Further, master always refers to the latest released non-beta version.

The source code is available at these locations

• Storage Engine - Source code for engine specific processes on UM and PM node
• MariaDB Server - Source code based on MariaDB Server 10.2.8 modified to support the ColumnStore storage engine