Last year I worked a lot on finding better techniques for automated functional testing.
I was investigating topics like how to solve problems of automated web UI testing (especially ASP.NET), how to structure automated functional tests or how to ensure the preconditions of functional tests. We even created small tools to support this, but altogether it was not a big success.
Finally, we started to use cucumber and this was the first tool where I felt that it supported the concept we need. However, we had some issues with cucumber, too:
- Using it with .NET projects (through IronRuby) was quite slow, and since the IronRuby documentation is not too detailed, we continually encountered ruby/.net conversion issues that were hard to debug and solve.
- We had our continuous build infrastructure, but it was not prepared for running cucumber tests.
- The development team did not have sufficient ruby knowledge.
Considering these problems, we finally decided to move forward: to implement a small .NET running environment for cucumber, where the step definitions are written in .NET. Since I’ve gathered quite a lot of experience with Visual Studio SDK, I came to the idea to implement this tool as a single-file generator that generates NUnit test fixture files from the cucumber file, which can be executed by NUnit runner, also on the build server, without extra configuration.
We resolved to implement the bindings as .NET methods, having [Given], [When] or [Then] attributes.
[Given(“I have entered (.*) into the calculator”)]
public static void GivenIHaveEnteredANrIntoTheCalculator(int nr)
{
//…
}
The tool contains a runtime part as well that finds the matching step definition method during the test executions and executes them. The first implementation used an Oslo parser, but finally we had to rewrite it to use ANTRL, because of performance issues.
The tool worked surprisingly well and we started to use it in a project where we had our cucumber tests already; we just had to rewrite the step definitions to .NET, since the cucumber files were compatible.
In the meanwhile, TechTalk decided to make an open-source project out of the in-house tool. But life is not easy, we have quickly realized that the names we have used so far (“NCucumber” or “Cucumber for .NET”) cannot really be used as a “trademark”. The cucumber team – somehow understandable – wanted to keep the name “cucumber” for products which are direct ports of cucumber. Although we are compatible in the cucumber file (gherkin) level, we don’t have an own execution environment (NUnit does it for us). So we have chosen the new name: SpecFlow.
The first version of SpecFlow is already available, so you can try it out (the source-code is also coming next week). To learn more, visit the website:
http://www.specflow.org
Improved intellisense for steps, including binding information and examples from the feature files in the project (you can also see if the step was used in a scenario outline)
