Russ Garrett

Although I work in the Operations room at Last.fm which has its fair share of monitoring screens, sometimes you’ll see me wishing for the good old days: when men were men and control rooms had hundreds of gauges, dials, and lights.

A while ago I decided to take a small step towards that reality, so I hit up eBay and picked up a variety of old-school Bakelite panel gauges (many of which didn’t work). Throw in an Arduino, a maze of wires and some resistors, and I had myself a plan.

I’ve now had this setup for a number of months, and I have to say it’s massively useful. The gauges show a variety of key metrics on how Last.fm’s architecture is performing within a couple of seconds of realtime. That’s a lot quicker than our main monitoring systems which tend to take between three and five minutes to notice something going awry. The gauges are easy to read at a glance, and I generally notice if something’s going horribly wrong because I can see the gauges twitch out of the corner of my eye.

If you’re interested, you can get hold of the (Ruby and Arduino)  code which powers the gauges here on Github, complete with a lame ASCII art diagram. When I get round to it, I’d like to get them mounted onto a nice wooden panel, possibly with some warning lights.

I’ve got a while to go before I can rival Battersea Control Room A, though…

Tags: arduino, monitoring, operations, sysadmin