RAD Test Scripting

The other day, some developer coworkers and I were talking about ways to make it easier for our Test group to create and maintain test scripts. Test scripts are a very important piece of the software development puzzle. But test scripts don’t find new defects. Test scripts are really good at finding breakage. That’s why scripts are run on new builds as BVT’s or release candidates as part of a larger regression suite. For me, few things are worse than uncontrolled breakage to keep a project from hitting deadlines.

To find new defects, you really can’t beat good test cases and exploratory testing. We wanted to find ways to keep our Test group working on finding new defects, not struggling to maintain scripts. Our current test scripting tool is just what it claims to be: A scripting tool. The problem with scripting is that it’s programming and programming brings a whole set of issues to the party. We’d like to hide those issues from most of the testers and only have a small team worry about them, if possible.

The genius idea we hit upon was a sort of RAD system for creating test scripts. Ah yes, well I did say we are developers. The general idea is to componentize the creation of scripts. Testers could piece together various script components to create test cases and suites. Of course, there would be a nice Windows application to allow testers to build their projects. We would save the project in an intermediate format and “compile” to native script that our scripting tool would execute.

We even found an additional benefit of the system: By maintaining our test suites in an intermediate format, we avoid script tool vendor lockin. When we switch to a new test scripting tool, we just need to write a new “compiler” to convert to the new script language. Vendor lockin is a real problem. How many commerical test scripting tools do you know use a common, portable language? It can make switching vendors very costly.

We are currently prototyping a system. I’ll let you know how it goes.