my name is mwd

At camvine, like most web companies, we run a lot of servers. You reach this point over a matter of time – you don’t start with a lot of servers on day one, you gradually end up there over a matter of time. As such, you don’t start on day one thinking about how to manage running so many servers, it just kinda happens. As such, you build them all by hand, and then one day, when stepping through the list of wiki instructions on how to configure your n-th web server, you throw your hands up and scream “No! There has to be a better way!”

And of course, there is. Lots of them.

There’s lots of automated server configuration tools out there, and I’m sure all will have their fans, but I just wanted to take a moment here to recommend Puppet, which was recommend to me a few months about by Matt Biddulph of Dopplr (or indeed now Nokia).

Basically, Puppet comes in two halves, the puppet client software that you install on new machines, and the puppet master software that you install on a server. On the puppet master you write a set of module manifests that say things like “if this machine is to be a web server install apache2 and mysql and this set of configuration files” and then some node manifests that say “this machine is a web server” or such. Once you’ve specified all your configuration information on the master, all you need to do to configure a new server is install the puppet client, authenticate it with the master, and tell puppet to make it so. Puppet will also update machines every half hour, so if something gets accidentally removed, of something added to the master, things will just get sorted out on the clients.

There’s a couple of points in Puppet that really appeal to me. Firstly, you can specify file templates, which will then be filled out before installation on nodes based on variables you specify in manifests. So I can have template configuration files for my website that get filled with the correct IP address etc. upon deployment. This means I can deploy test servers, real servers, etc. all from the same single template. The second part is how well it works with debian packages. Simply by moving all our custom bits of website code, back end programs, and so on into debian files, then setting up our own private debian repository we have a central place for all these things, and to get machines to update just means updating the debian repository.

There’s lots more fun that can be had with it, but you can find that out by playing with Puppet if you’re interested. I certainly find it very easy to setup and use, and it not only reduces the time it takes us to generate new servers, so we can do more interesting things, but it also means if the person who sets up the servers usually isn’t reachable, there’s not need to worry – all that server configuration should be distilled into Puppet, so you can just install that and let it do the work.