X-47 Industries

Exploring tools and infrastructure for systems administration.

Cascadia & Test Driven Systems Administration

There’s a spirit of adventure at conferences. It’s putting on a pith helmet and encountering big, hairy beasts as strange, new industry problems are described. It’s foraging deep into the jungle of new ideas and bringing back delicious, new fruits. It’s discovering new friends and allies, similarly interested in mapping the trails.

Seattle hosts Cascadia IT Conference, and poetry aside, my experience there was fruitful. It’s a journey I’ve made twice now, and each has been rewarding. This year the big idea I bagged was offered up by Paul English: Test Driven Systems Administration.

Developers offer us new code to deploy, yet we refuse to host it unless it passes certain qualifications. As systems administrators, we want to know it operates to a minimum standard, and is free of previously identified bugs. Yet, what similar discipline do we practice demonstrating we hold ourselves to the same standard? This the first reason we should operate our systems with testing in mind: uniform standards of quality.

More importantly, testing should give us peace of mind. How often do we take the offered code and just implement it, trusting that we’ve logged in to the right server, or that dependencies are in place? Testing reduces failures. It’s a lesson we should have learned from The Checklist Manifesto; this is a much-needed reminder.

The most intriguing advantage from the testing world is the freedom to refactor. Unless we specify a minimum operating environment (e.g. services on ports, page availability, or even, requests per second), how do we guarantee that the changes we make do not harm the production environment? And certainly, better admins verify that the website is still up after a change, but an automated test takes the drudgery out of this.

To most shops today these practices seem a fabulous, outrageous tale, but let us return from the wilds with these better practices that, to me, resonate with common sense. I’m glad to have visited the Cascadia IT Conference this year, to find these words, and I look forward to returning next year.