April 27, 2016

Automated server management with opensource tools

In a recent presentation by Balazs Pocze from Gawker Media LLC made clear that there are many automation strategies for server management that would be worth packaging up into an extended solution like MariaDB Enterprise.

Automation is good because there are way less "ops people" than "dev people" at many companies. Man-hours are expensive, so you don't want to waste them on routine tasks. Automated tasks also mean that there is a lot less possibility to make mistakes. "Everybody needs automation," says Pocze.

Gawker still uses physical infrastructure, powered by Linux, though there is some migration to the cloud. Cobbler is their Linux installation server of choice, where the hosts are defined, and there is no manual DHCP or DNS management; this tool not only manages but also defines their environment.

For provisioning the hosts, Gawker's tool of choice is Puppet. Puppet manages all the hosts as well as defines the hosts, and Pocze said, "If it doesn't exist in Puppet, it doesn't exist at all". From a developer enablement standpoint, MariaDB Enterprise provides the MariaDB Enterprise Chef Cookbook, which we could say is as close as can be.

In a bit of a twist, they also make use of Ansible for running commands on hosts; all commands are organised into playbooks, and it is all about agentless installation and management. They do create modules and contribute them, and all nodes are managed over SSH.

To wrap it all up, they use Jenkins for continuous integration. The demo of how Gawker's Kinja is maintained by Jenkins, with the Ansible playbooks for slave management was also a highlight.

It would be great if more shared what their backend infrastructure and ops ran like. Because its clear that there's plenty of opportunity to have tooling finesse.

Don't forget to check out banyek on github as well as gawkermedia on github. During the talk, an audience member said it might also be a great idea to checkout Rundeck - a job scheduler and runbook automation, something I have not heard of before but will definitely look at in the near future.

About Colin Charles Charles

Colin Charles is the Chief Evangelist at Percona. He was previously on the founding team of MariaDB Server in 2009, and had worked at MySQL since 2005, and been a MySQL user since 2000. Before joining MySQL, he worked actively on the Fedora and OpenOffice.org projects. He's well known within open source communities in APAC, and has spoken at many conferences.

Read all posts by Colin Charles Charles