by Gáspár on March 19, 2012
After the successful debut of the full day event of the Cucumber BDD tool and the ecosystem (obviously SpecFlow as well), this year CukeUp! is here again! [read more…]
by Gáspár on March 5, 2012
Today was an exciting day. It started with Visual Studio 11 beta install and finished with uploading a working SpecFlow VS11 package to github downloads. Cool.
Currently I have created a hand-made package for v1.8.1, to test whether the integration components work well on VS11. Later I’ll include the integration package to the standard deployment process.
Currently the things seem to work well. The “Run SpecFlow Scenarios” feature does not seem to work with the MsTest runner, but with the ReSharper EAP this works too.
If you see any problems with the integration, please post it here.
It’s yet unclear to me, how the installation process will look like in the future. On one hand it’s tempting to put the VS11 (and maybe also the VS2010) integration to the VS gallery, but this way the file association (.feature) cannot be set. Currently the SpecFlow installer installs the tools and the runtime for users who don’t use NuGet. If the installation is done through the gallery, we need to find a solution for this too. If you have suggestions on this, please let me know.
by Gáspár on October 26, 2011
We have released SpecFlow v1.8 recently (just published 1.8.1 with a few fixes) and I would like to take the opportunity to give a subjective summary of the new features and also some outlook for the upcoming releases.
Maybe the most interesting (most wanted) feature is to make scenario execution more convenient. Now it is here: in the context menu of the editor, the project, folder or file nodes of the solution explorer, you will find new commands to run or debug the scenarios of the current context: the scenario of the cursor position (or all from the file if you stand in the header or between scenarios) or the scenarios in the files matching to the selected solution explorer node.
Since SpecFlow does not run the scenarios directly, but you run them with a unit test runner, this feature is a little bit tricky and needs a short explanation.
When you select the “Run SpecFlow Scenarios” command, SpecFlow tries to run the selected scenarios with your favorite unit test runner. You can either tell SpecFlow which one is your favorite or let it find out automatically based on some built-in ranking. This can be done in the SpecFlow section of the Visual Studio Tools/Options dialog.
From the picture you can also see the list of supported unit test runners (ReSharper (5 and 6), MsTest and SpecRun). The TDD.NET is in the list, but the gateway is not yet implemented. Contributions for the TDD.NET gateway or any other that is not listed yet is welcome.
As you’ve probably heard, Richard Lawrence has announced that the Cuke4Nuke project has been retired and we decided to merge our efforts to further improve SpecFlow.
We have been working for a while with Richard, Paul, Christian, Jonas and Darren for the preparation of this and we have started to implement features to be better compatible with Cucumber.
Currently we are at half-way. We have consolidated the terms (e.g. using “hook”) and validated/fixed some obvious compatibility problem. The remaining stuff is mainly concerning the table manipulation API, that is currently covered by the Assist namespace in SpecFlow. Regarding this, we would like to
SpecFlow has an evolving architecture so there are always improvements here as the features grow. 1.8 is maybe sill an interesting milestone, because we have fully refactored the class-dependency model of the runtime code and have replaced the custom object factory solution with a DI tool.
This step was important, because this consolidated and solved issues with the context injection feature (e.g. disposing injected objects) and also because this has opened up new ways for customizing SpecFlow. Here is an example for the customization:
Let’s say you want to implement a custom way of reporting SpecFlow user errors (e.g. for doing something special for pending steps). For this you need to create a class that implements the IErrorProvider interface. You can let SpecFlow use your custom provider by changing the default dependencies in the configuration like this:
Also the context injection feature can resolve the SpecFlow classes as well, so for example if you need to apply the same conversion logic as SpecFlow uses in your code (including step argument conversions), you can just let an IStepArgumentTypeConverter instance be injected to your step definition class like this:
Also a new improvement in the SpecFlow solution that now the specifications and the integration tests have got a new platform. The projects TechTalk.SpecFlow.Specs and TechTalk.SpecFlow.IntegrationTests contain SpecFlow tests for SpecFlow itself. The existing non-unit tests have been converted to the new model and I have also added a couple of additional tests, but the coverage is not yet close to complete. My hope is that with the established environment, it will be easier for the contributors to also provide SpecFlow end-to-end tests for the new features.
There is a similar work also in Cucumber, the cucumber-features. The goal is to make the necessary step definitions that SpecFlow can also run (and pass) these scenarios. But this is something to do still.
If you have followed the posts and tweets of TechTalk, you might know that we are also working on an integration test runner tool, that provides smarter (better, faster, more integrated) feedback for integration test execution.
The SpecRun beta has been updated to be compatible with SpecFlow 1.8.1. Take a look at the website: http://www.specrun.com or just grab the nuget package into your project:
Install-Package SpecRun -ProjectName MyProject
This post is really getting too long, so just a few bullet point for the future ideas:
Don’t forget, that the VS integration is independent from the project dependencies, so you don’t have to wait until all your projects can upgrade to 1.8. Go ahead and download the installer for 1.8.
Please also take a look at the documentation. It’s getting better and better, but if you see any glitches in it: it’s a wiki, so just fix it!
Have fun!
by Gáspár on September 2, 2011
We have started the SpecFlow project in 2009 to provide a tool for .NET developers to perform BDD-style functional testing. SpecFlow uses unit testing frameworks, like NUnit, MsTest or MbUnit to execute the Gherkin scenarios. While this was a pragmatic choice, it has turned out that for executing functional integration tests, these unit test frameworks are not optimal (quite understandable, as they are unit testing frameworks and not integration testing frameworks). While they can execute the tests very well, they lack features like
In addition to that, we were missing a test runner that treats Gherkin features, scenarios and scenario outlines as first-class citizens.
To address these problems, we have decided to launch a new product that fits into the SpecFlow-SpecLog context. It has been publicly announced today, so it’s here: SpecRun
We have been working for this tool for a while now and have tested it in our projects. It is really promising. It seems that with parallel execution, the local test execution time (that is more critical from the cost perspective, as developers can’t progress while the tests are running) can be dropped to ca. 60%, that is almost half time. This and the smarter feedback about the errors were received very positively by the developers on the selected projects. They were more willing to run the tests locally reducing the long round-trips and they were able to detect the root cause of the errors better.
SpecRun is also using SpecRun for testing itself obviously, and our own dog food tastes pretty good!
SpecRun will be a commercial product (with free versions for OSS projects of course). This was a hard decision. We have a wonderful community around SpecFlow and the support of TechTalk, but even this way we could not afford the costs of building up so many new things. I think this is a good combination. While we believe that SpecRun is a great help and provides quick ROI for the users, it is not the main concern of SpecFlow. With the separation of the two aspects, we can have still our SpecFlow efforts focused on the main concerns (editor, runtime API, documentation) and also provide an extension for the smarter execution experience.
SpecRun is currently in a closed beta stage, but we are planning to have the commercial release soon. Have a look at our website: http://www.specrun.com, stay tuned on our twitter account: @specrun and contact info@specrun.com if you are interested in participating in the beta program.
by Gáspár on August 1, 2011
We have just released SpecFlow v1.7 over the weekend. In this release there are extended integration options for the Silverlight async testing framework and also for SharpDevelop 4.
In addition to that, we have tried to make the upgrade process easier for the SpecFlow users, especially if they work in bigger teams or on different projects in parallel.
The decision to upgrade a tool for a project is usually made carefully. You have to consider your close deadlines, check the breaking changes, and discuss it with the team.
If you work on multiple projects in parallel (e.g. you have to support and older project too), it can be quite hard to synchronize that all projects upgrade at the same time.
As SpecFlow is (was) a single msi installer and the different versions cannot be easily installed side-by-side, the upgrade process was sometimes challenging.
From v1.7 we have managed to have better separation of the components that are directly used by the project (the generator and the runtime) and the IDE integration. With this, you can upgrade SpecFlow on your machine and defer the upgrade for the individual projects.
For that to work, please ensure that your project is using SpecFlow 1.6 or later and it is setup as one of the following (see further advanced options in the documentation):
If you have this setup, you can safely install a newer SpecFlow version on your machine. The projects will still use the configured generator and runtime.
When you feel confortable to upgrade SpecFlow on your project as well, you can do that project-by-project (the easiest it to use NuGet upgrade).
Note 1: Please make sure that the SpecFlow version on the machine is the same or newer than the SpecFlow version used by the projects.
Note 2: This feature works currently with Visual Studio 2010 only.
SpecFlow is and open-source project sponsored by TechTalk. You can also support the project by contributing on github or by “adopting a feature” ;-). TechTalk also provides commercial support for SpecFlow.
Learn SpecFlow at the the Progressive .NET Tutorials: http://skillsmatter.com/event/open-source-dot-net/progressive-dot-net-tutorials-2011