Gáspár Nagy on software

coach, trainer and bdd addict, creator of SpecFlow; owner of Spec Solutions

Tracing Web Automation Test Errors with SpecFlow

by Gáspár on July 18, 2011

If you try to run your automated UI tests on the build server, you can quickly run into problems that are quite hard to analyze.

The most common scenario is that one of your assertion like “Then my name should be displayed” fails, because the web driver does not find your name in the resulting page. It works locally, but what’s the problem on the build server? Permission problems? Timing issues? Browser compatibility? Different locale? Web compilation or configuration errors? All of these are potential candidates, but it is hard to find out quickly which one.

We are using the following SpecFlow event binding to trace the content of the current page in case of an error. In many cases from the page content it’s easy to say what the problem was. 

[AfterScenario]
public void AfterWebScenario()
{
    if (ScenarioContext.Current.TestError != null)
    {
        Console.WriteLine("Page src for failing test: {0}",
            BrowserContext.SeleniumWebDriver.PageSource);
    }

    //... (if you stop your browser session after
    //     each scenario, this comes here probably, 
    //     to ensure that the tracing runs before)
}

Git & Dropbox for preparing live demos

by Gáspár on June 14, 2011

I have arrived from Oslo where we did two sessions with Jonas Bandi on NDC 2011 about SpecFlow.

As Jonas lives in Switzerland and I live in Hungary, we have used Dropbox to synchronize the content we are working on. This was fine for slides and other notes, but for the code demo we did not have the right toolset so far.

Actually I like to do demos where I modify and evolve the code during the session. But if you do so, you need to maintain different versions of the code base. You need a baseline to start the demo from and probably also a “done” version if something goes wrong. Additionally you need to be able to revert back to the baseline while practicing the demo. This begs for a SCC and Git is a perfect choice.

To have a little bit of “surprise” effect, I don’t like to publish my demos upfront so did not wanted to put it into a github repository. To be able to share the code with Jonas, this time we’ve tried to combine Git with Dropbox like this:

  • create a “bare” git repository into a dropbox folder
  • clone it to your working folder
  • work on the code & commit changes
  • push the changes to the dropbox folder (the origin)

Though I’m pretty sure that this is not a well scaling solution for real development, but for two persons it was just fine & simple.

Agile Developer Course in Vienna

by Gáspár on May 31, 2011

In summer everyone is on holidays, the projects are running on low frequency, so maybe it’s a good time to meet others and improve your agile development skills.

July 6-8, 2011 there will be a three-day course in Vienna instructed by me (in English) that focuses on agile engineering practices and team collaboration. The attendees can also practice behavior driven development (with SpecFlow) and TDD. This is all done through a lot of hands-on exercises and group discussions.

There are two days left for the discounted price!

Details & registration: http://www.techtalk.at/scrum-trainings/certified-scrum-developer.html

(Successful completion of the course can be credited towards the Certified Scrum Developer (CSD) certification of Scrum Alliance.)

Testing manually vs. SpecFlow

by Gáspár on May 6, 2011

No, I cannot answer this general question. Just had a nice story in our team and since I’m sure that Mátyás and Viktor will never blog this, I do it now on behalf of them.

We are working on a small project, where we have to implement a smaller module for an existing system. Interesting project, but not BDD at all. Especially there are no automated tests.

Since the project is really small and it has to fit into this bigger existing system, we decided not to introduce SpecFlow there (was a hard decision, you can imagine :).

We are near to finish the first sprint (well, we have introduced Scrum and user stories to the project still), so we started to think about the demo and that we should some manual testing. But how?

The majority of Sprint 1 was a background job that had to process some data depending on some complex rules. So we sat down after the daily standup yesterday and brainstormed the different cases we want to test. “Surprisingly” the cases were pretty much look like acceptance criteria illustrated with examples. As there were too many cases (11) for manual testing, we have selected the 5 most important to be tested.

We had two choices: either we try to find matching data in our test database and test the cases with them or we create test data for our own (with SQL scripts?).

Finally we have chosen the third option: decided to create our own test data, but through Gherkin & SpecFlow. It was a risky decision (budget, deadline, etc.), so we decided for a 1 dev, 1 day time-boxed spike to do it. This was at 9:30am.

To make the story short, at 17:11, the Gherkin files were created and bound for all 11 tests… and they were green (like Cucumber).

And the funniest thing is, that we’ve found a bug… a very tricky one that would not have shown up in the manual tests most probably!

This was very convincing! To be honest, I did not expect to be finished in one day. I was asking Mátyás today morning in the standup carefully, what happened. He said: “Yeah, I’m sitting next to Viktor and doing some pair programming”. I was asking back (trying to hold back my excitement): “And what happened with the SpecFlow tests?” He said very calmly (he is a cool guy): “Ah, yeah the SpecFlow tests are done already. They are all green”.

Not even mentioning the extra benefit, that we can show the automated tests to the customer so they can also create new test variants quickly.

So, thanks for the good job, guys!

P.S.: Just to improve your C# skills, I also share buggy code (without comment):

a ?? 0 + b ?? 0

Re: Rapid Feedback

by Gáspár on April 11, 2011

I’ve read yesterday evening the post of Attila Domokos about rapid feedback that made me thinking. Here is my reaction that didn’t fit to a regular blog post comment.

I think no company should afford wasted time and having a development environment that is determined to waste time is a usually forgotten, but important factor in this. The scenario that has been described by Attila is known to all of us. Well known. Though the story can end many different ways. For Attila, Rails was the “solution”. For me it is BDD and good practices. (I’m not against Rails at all. Rails is fine, but these problems have to be solved within .NET for most of us.) I’m absolutely agreeing with the idea however, to use the right tool (framework) for the right job, but it would be very sad if it would turn out that WPF development cannot be efficient. (I’m not sure though… I know another Attila who would immediately tell me that it cannot, but I’m still hoping.)

[BTW: I’m pretty sure that there are a lot of devs (not the ones working with me fortunately), who actually love this situation and use the compilation times to read the daily newspaper.] 

As a developer and developer coach, I always try to ensure that the team is taking the time (early enough) to setup a development environment where you can get rapid feedback. For a WPF-frontend .NET app, here are my suggestions (I’m not saying that we can always do all unfortunately):

  • Do BDD/ATDD in a test-first, outside-in way, so that usually you don’t have to start up the application for testing the functionality. (We do this with SpecFlow of course. :-)
  • Consider testing the functionality at the controller level (as well?), in order to get feedback about the implemented functionality without the slow UI automation. (At least until we have a good test framework where the automated UI testing is really easy and fast.)
  • Do TDD for your inner components. Ensure that the unit testing environment and practices are confortable and helpful. If the tests are executed only after the run-and-test round has been done, you did not win too much.
  • For WPF, (try to) make sure that your controls can be seen in the VS designer or blend (aka “blendable”), so that you don’t have to start the application for pixel proofing. There are infrastructure for design-time data, but introducing it late is very hard.
  • Implement shortcuts to your application that the run-and-see test can be also done more efficiently. For example, you can make a stub authentication/authorization provider that bypasses the login screen.
  • Optimize the build process to be fast – review and remove projects, post-build events, slow signing process, etc. from the build configuration that you use for development. (Be very careful with post-IL tools, like Code Contracts or PostSharp.)
  • Measure the difference and argue for faster development machines, solid-state drives.

Still, it can happen that you have to wait. Personally, I’m using this waiting time for “development”: Even if I’m not doing TDD, I always follow the practice of “making a small piece work” and then “making it better” (refactoring). So while it is building, starting up, etc. I’m already thinking about the refactoring possibilities. Try it! It’s a little bit exhausting, but fun!