MyRocks and data compression

You are viewing an old version of this article. View the current version here.

MyRocks supports several compression algorithms.

Supported compression algorithms

Supported compression algorithms can be checked like so:

MariaDB [test]> show variables like 'rocksdb%compress%';
+-------------------------------------+------------------------------------+
| Variable_name                       | Value                              |
+-------------------------------------+------------------------------------+
| rocksdb_supported_compression_types | Snappy,Zlib,LZ4,LZ4HC,ZSTDNotFinal |
+-------------------------------------+------------------------------------+

Another way to make the check is to look into #rocksdb/LOG file in the data directory. It should have lines like:

2019/04/12-14:08:23.869919 7f839188b540 Compression algorithms supported:
2019/04/12-14:08:23.869920 7f839188b540         kZSTDNotFinalCompression supported: 1
2019/04/12-14:08:23.869922 7f839188b540         kZSTD supported: 1
2019/04/12-14:08:23.869923 7f839188b540         kXpressCompression supported: 0
2019/04/12-14:08:23.869924 7f839188b540         kLZ4HCCompression supported: 1
2019/04/12-14:08:23.869924 7f839188b540         kLZ4Compression supported: 1
2019/04/12-14:08:23.869925 7f839188b540         kBZip2Compression supported: 0
2019/04/12-14:08:23.869926 7f839188b540         kZlibCompression supported: 1
2019/04/12-14:08:23.869927 7f839188b540         kSnappyCompression supported: 1

Checking compression settings

Compression is set on a per-column family basis. To check current compression settings for a column family one can use a query like so:

select * from information_schema.rocksdb_cf_options 
where option_type like '%ompression%' and cf_name='default';

The output will be like:

+---------+-----------------------------------------+---------------------------+
| CF_NAME | OPTION_TYPE                             | VALUE                     |
+---------+-----------------------------------------+---------------------------+
| default | COMPRESSION_TYPE                        | kSnappyCompression        |
| default | COMPRESSION_PER_LEVEL                   | NUL                       |
| default | COMPRESSION_OPTS                        | -14:32767:0               |
| default | BOTTOMMOST_COMPRESSION                  | kDisableCompressionOption |
| default | TABLE_FACTORY::VERIFY_COMPRESSION       | 0                         |
| default | TABLE_FACTORY::ENABLE_INDEX_COMPRESSION | 1                         |
+---------+-----------------------------------------+---------------------------+

Current column family settings will be used for the new SST files.

Checking data compression

A query to check what compression is used in the SST files that store the data for a given table (test.t1):

select
  SP.sst_name, SP.compression_algo
from
  information_schema.rocksdb_sst_props SP,
  information_schema.rocksdb_ddl D,
  information_schema.rocksdb_index_file_map IFM
where
  D.table_schema='test' and D.table_name='t1' and
  D.index_number= IFM.index_number and
  IFM.sst_name=SP.sst_name;

Example output:

+------------+------------------+
| sst_name   | compression_algo |
+------------+------------------+
| 000028.sst | Snappy           |
| 000028.sst | Snappy           |
| 000026.sst | Snappy           |
| 000026.sst | Snappy           |
+------------+------------------+

Comments

Comments loading...
Content reproduced on this site is the property of its respective owners, and this content is not reviewed in advance by MariaDB. The views, information and opinions expressed by this content do not necessarily represent those of MariaDB or any other party.