LevelDB storage engine

Basic feature list

• single-statement transactions
• secondary indexes
• HANDLER implementation with extensions to support atomic multi-put (kind of like multi-statement transactions)
• binlog XA on the master to be crash safe
• crash-proof slave replication state
• (almost) non blocking schema change
• full test coverage via mysql-test-run
• hot backup
• don't know yet whether we want a LevelDB instance per mysqld, per schema or per table

Implementation overview

Multiple instances

Current solution is to use one LevelDB instance for mysqld process, and then prefix LevelDB keys with 'dbname.table_name', 'dbname.table_name.index_name', or some equivalent.

That way, we'll be able to store arbitrary number of tables/indexes in one LevelDB instance.

Transactions

LevelDB supports

• read snapshots
• batch updates when you have just those, there is no easy way to support full transactional semantics in the way it is required of SQL engine.

The hard requirement is only to have single-statement transactions. Perhaps, we could also put limits on the size of the transaction.

Non-blocking schema change

• There is a requirement that doing schema changes does not block other queries from running.

• Reclaiming space immediately after some parts of data were dropped is not important.

Possible approaches we could use:

• Record format that support multiple versions. That way, adding/modifying/ dropping a non-indexed column may be instant. Note that this is applicable for records, not keys.
• Background creation/dropping of indexes.

Hot backup

Hot backup will be made outside of this project. The idea is to hard-link the files so that they can't be deleted.

SQL Command mapping for LevelDB

INSERT

SELECT

UPDATE

DELETE

ALTER TABLE

Other details

• The target version is MySQL 5.6 (good, because LevelDB API uses STL and 5.6-based versions support compiling with STL).
• It is ok to make changes to LevelDB.