Many teams that are currently in a transformation process towards using agile and bdd have to face the problem that the new specification artefacts and reports have to be integrated into the existing tool chain. In the case of SpecFlow, many of such teams use Microsoft Test Manager (MTM, part of Team Foundation Server / Visual Studio Team Services). As the manual test activities (especially exploratory testing) also play an important role in agile testing and BDD, MTM might still have its place in the new tool chain. Also MTM has an important feature that it allows to run automated tests on different virtual lab environments (e.g. on different operating systems). As the structure of test cases in MTM are somewhat similar to the structure of Gherkin scenarios (title, tags, steps), it is an obvious idea to try integrating SpecFlow to MTM. I have worked on a prototype to test  this idea that I would like to present in this post.

Update 12^nd August 2016: Yesterday the first public version of SpecSync has been released on NuGet. For detailed introduction, usage and licensing, please check out the project
site, the Getting Started guide or the documentation.

So the core idea for the prototype was to integrate SpecFlow with MTM in a way that

• it does not compromise the BDD process and the collaboration
• allows manual testing of Gherkin scenarios (in case automation is not efficient or if some automated checks should be included in a manual testing activity)
• allows execution of automated scenarios in lab environments

The prototype concept

Based on the goals above, the general concept is that the Gherkin scenarios are defined and edited in feature files just like before. However, once the scenario is automated and the described functionality is implemented, the scenario can be synchronized as a new test case in MTM. The linkage between the scenario and the test case is preserved as a tag in the scenario (e.g. @tc123 links the scenario to the test case #123). Based on the link the test case can be updated once the scenario has been modified. This update process can be implemented as a scheduled job, to avoid outdated test cases.

 

The synchronized scenario is linked to the test method generated from the scenario by SpecFlow so the automated tests can also be executed from MTM on different lab environments. If MTM is configured to use SCVMM (System Center Virtual Machine Manager), you can even automatically provision new virtual machines based on templates, in order to run the tests.

 

The scenario steps are represented as test case steps that might also contain DataTable and DocString (multi-line string) parameters. As steps are stored by TFS as a formatted text, even additional formatting (e.g. highlight keywords or parameters) can be applied.

 

Also MTM has a concept of parametrized tests for defining data driven tests (multiple iteration of the same test with different data). Scenario outlines are synchronized as such parametrized tests.

How it works

Instead of providing a long and detailed explanation of how this works in practice, have a look at this video that shows the prototype in action.

Next steps

My goal is to gather experience with the prototype on how this can fit into a BDD process. For this purpose I would like to cooperate with teams or enterprises who are interested in such integration. Please contact me if you are one of these.

My current assumption is that – within the core concept – there will be no single way of how such integration can work, but it has to be adapted with the specific needs and existing processes of the companies. So currently I rather see this integration as a custom solution than a boxed tool. It is also my goal to evaluate this assumption.

Update: I have created a site page for the tool that you can find here.