Tumblr (now a Yahoo! web property after a USD$1.1 billion acquisition) is one of the largest users of MySQL on the Web. In 2012, they had over 60 billion relational rows with over 21 terabytes of data. Just over a year later in 2013, they are processing 54 terabytes of data with 229,208,178,189 rows of data!

It is for this reason alone that Tumblr built Jetpants, an automation toolkit to handle sharding of large MySQL database topologies. They use Jetpants to scale writes and eliminate single points of failure.

Most of Tumblr's instances are MySQL or Percona Server 5.1 or 5.5, with plans to migrate to 5.6. However for merging shards, MariaDB 10.0 acts as an aggregator node, as it supports multi-source replication. It works by exporting data from each shard, importing all that data into the MariaDB aggregator node and then pushing it out to the new master. You can find out more information as to how this architecture works based on a presentation given by one of Tumblr's three MySQL engineers, Bob Patterson Jr.

Comments

Comments loading...