In the first part of this article I described why it is important to provide more information on errors and how you can collect useful artifacts (screenshots, log files, etc.) in an [AfterScenario] hook. In this part I would like to explain how to write more informative assertions.

Writing informative good assertions is important for unit testing as well, but since unit tests are much faster, fully isolated and more focused, in many cases, a simple assertion statement, like Assert.AreEqual(42, result) is enough (ie result should be 42).

In integration tests, the result is influenced by much more contextual information, so the information “result should be 42” might not be enough if you need to investigate a failing test. Why 42?

The situation is even worse, if the assertion is not a simple comparison, but a more complex expression, e.g. “the result list should have an item where the Answer property is 42”. Such complex expressions are usually verified with the Assert.IsTrue() assertion method, like this:

Assert.IsTrue(resultList.Any(x => x.Answer == 42));

It is pretty easy to read this code, however, you will receive a message that “Assert.IsTrue failed” when it fails. In some frameworks you might also get the very useful “expected true got false” message as well. The problem is the following: although it is obvious that there was no item in the list with 42 as an answer, you don’t know if it was because the list was empty or there were items in it with different answer values.

I think you all know these kind of typical useless error messages. My favorites are:

• Sequence contains no matching element – a “Single” was called on a list that was either empty or had more than one element
• Value cannot be null. Parameter name: source – a LINQ operation (e.g. Where) was called on a list that was null
• Expected:<Lorem ipsum dolor sit amet, consectetur adipiscing elit.>. Actual:<Lorem ipsum dolor sit amet, consectetur adepiscing elit.> – the two strings are different, but hard to figure out where

The assertion methods usually provide an overload where you can also specify a message, which will be also included in the error message when the assertion fails. By this message, you can provide a reason or explanation that helps understanding what the reason could be for the failure. But unfortunately this will not help diagnosing issues, like the collection verification I have mentioned above.

The assertion methods packaged together with the unit test framework are usually not good enough.

Use better assertion libraries!

When looking at a statement like Assert.AreEqual(42, result), people think that it is a special method that communicates with the test runner and reports if the comparison failed. In reality, these methods simply throw exceptions and the test runner captures these. There is no special magic API between them. Why do the unit test frameworks contain assertion methods then? This is just a matter of convenience, so that by using one library you can immediately start writing (simple) assertions.

There are many very good independent assertion libraries available that work with any unit test framework. I usually use FluentAssertions (open source project lead by Dennis Doomen), that

• is available on NuGet
• provides surprisingly good error messages by default
• allows specifying the reason, including parameters
• provides a fluent interface that is much easier to read and write

A simple comparison can look like this with FluentAssertions:

result.Should().Be(42, "we multiplied {0} by {1}", op1, op2)

If this assertion fails, you will get the following error message:

Expected value to be 42 because we multiplied 6 by 7, but found 66.

The collection assertion example provides a very good error message, even without a custom reason message. This is how you would express it:

resultList.Should().Contain(x => x.Answer == 42);

and this is the provided error message if the list was… Well, I will figure it out.

Collection { MyApp.Response { Answer = 66 } } should have an item matching (x.Answer == 42).

And finally, together with a message, this is really nice:

resultList.Should().Contain(x => x.Answer == 42, "we expect an answer to the question of {0}", question);

that produces

Collection { MyApp.Response { Answer = 66 } } should have an item matching (x.Answer == 42) because we expect an answer to the question of Life, the Universe, and Everything.