Comments - TokuDB Differences

9 years, 11 months ago Jean Weisbuch

Here is a simple one liner to check the compression format of the TokuDB data files using the tokuftdump tool :

mysql -sNe 'SELECT dictionary_name, internal_file_name FROM information_schema.tokudb_file_map WHERE dictionary = "main" OR dictionary LIKE "key-%"' |perl -F'\t' -ane '@out=qx(tokuftdump --nodata --rootnode /var/lib/mysql/$F[1]); foreach $ftdump (@out) { if($ftdump=~/^ compression_method=([0-9]+)$/) { print $F[0]."\t".$1."\n"; last } }'

(don't forget to ajust to your datadir path if its not /var/lib/mysql)

The integer corresponds to the compression format, 0 is uncompressed, 8 is ZLIB and 10 is LZMA, the other format correspondances can be found on make_tdb.cc

 
9 years, 10 months ago Jean Weisbuch

On MariaDB5.5.37, the column "dictionary" has changed for "table_dictionary_name", so the right command is now :

mysql -sNe 'SELECT dictionary_name, internal_file_name FROM information_schema.tokudb_file_map WHERE table_dictionary_name = "main" OR table_dictionary_name LIKE "key-%"' |perl -F'\t' -ane '@out=qx(tokuftdump --nodata --rootnode /var/lib/mysql/$F[1]); foreach $ftdump (@out) { if($ftdump=~/^ compression_method=([0-9]+)$/) { print $F[0]."\t".$1."\n"; last } }'

Also note that the ZLIB compressed tables are having the value of 11 and not 8 as stated previously.

 
8 years, 1 month ago Sergei Khomutov

Hi, it does not work for me. Versions : tokudb_version - 5.6.26-74.0 , version mariaDB - 10.1.10-MariaDB-1jessie-log.

I've added --header to tokuftdump command. Finally it works for me :

 mysql -sNe 'SELECT dictionary_name, internal_file_name FROM information_schema.tokudb_file_map WHERE table_dictionary_name = "main" OR table_dictionary_name LIKE "key-%"' |perl -F'\t' -ane '@out=qx(tokuftdump --nodata --header --rootnode /var/lib/mysql/$F[1]); foreach $ftdump (@out) { if($ftdump=~/^ compression_method=([0-9]+)$/) { print $F[0]."\t".$1."\n"; last } }' 

And i noticed that changing compression method changes size of table on disk. But command

 SELECT 
    table_name AS `Table`, 
    round(((data_length) / 1024 / 1024), 2) `Table  size in MB` ,
    round(((index_length) / 1024 / 1024), 2) `Index  size in MB` 
FROM information_schema.TABLES 
WHERE table_schema = <table>
order by `Table  size in MB` desc; 

shows the same size of tables for different compression methods. Why ?

 
Content reproduced on this site is the property of its respective owners, and this content is not reviewed in advance by MariaDB. The views, information and opinions expressed by this content do not necessarily represent those of MariaDB or any other party.