Gáspár Nagy on software

coach, trainer and bdd addict, creator of SpecFlow

SpecFlow tips: input and output files

by Gáspár on January 20, 2014

Integration tests have to deal with file system files from time to time, even if the system under test does not use the file system. You might have test input files or generated logs, traces or other output files that you generate during the test execution.

Handling such files is not too hard, but there are a few common mistakes that may arise along the folder handling of these files that I would like to highlight in this post, so that the tests can be properly executed in different environments.

Input files

You might need to use input files for test input data, for files required by the application to run, or even as expected result files that you compare your actual results with. Does not matter why, you need to load them from some folder.

There are several options for that, but the best that works in every situation is to store them in the folder of the test assembly or in a subfolder below, so that you can access them by combining the test assembly folder with a relative path.

It is important that the relative path should not start with “..”, so the file is not above the test assembly folder. If the file is not inside this folder yet, make sure your build process copies it to the right place. In most cases, marking the file to “copy to output folder” just does the trick.

But how can you get the folder of the test assembly during test execution? The only simple  and predictable solution that works fine is to use the CodeBase property of the “current” executing assembly. As the codebase is not a file path, but a URI, the code is a bit complicated, so you’d better place it into a separate helper method (see attached gist).

Path.GetDirectoryName(new Uri(Assembly.GetExecutingAssembly().CodeBase).LocalPath)

Just for the sake of completeness, here are the other (bad) solutions:

  • Using the Location property of the test assembly – this will not work if the tests are running with shadow copy enabled.
  • Using a hard wired full path ;-) – this will work only on your machine.
  • Using a relative path that starts with “..” -this might not work on some build servers (TFS) that use a single compilation output folder for the entire solution.
  • Using an environment variable to get the path – this is not that bad, but can cause problems for parallel execution (if you need different input folders).
  • Specifying the input folder full path in the config file – this can also work, but it’s quite an effort to maintain the different setting in different places.
  • Using the current folder – it depends on the test runner whether the current folder is set to the folder of the test assembly. See the output section below.

Output files

The test execution can also produce output files, for example logs, traces or other files (e.g. screenshot) that can help error analysis. Sometimes you just need the files temporarily, but in some cases you even want to archive them as a part of the test execution result.

In any case – just like with the input files – you need a folder to save them to. Again, a lot of options are possible, but all the test runners I have seen so far assume that you put the result into the current folder. I.e. if you can reconfigure the output folder in the runner tool (e.g. in Visual Studio test runner), it will make sure that the current folder is properly set before executing your tests. So it is simple:

Directory.GetCurrentDirectory()

The only thing you might want to consider is whether you generate these files directly to this folder, or you create a subfolder with a timestamp for each execution. If you produce many files, the later option is better.

Temporary files

You might also want to set different rules for those output files that do not contain any meaningful information after the execution (not even if the test failed), but you need them for a technical reason.

I usually use the output folder for these, but you can certainly use the system temp folder too. In any case, try to delete them after the test has executed to reduce the IO footprint of your tests.

Conclusion

The solutions mentioned above work well with SpecFlow and SpecRun, but we have also used them with other integration test scenarios successfully. Whatever you use, always make sure that you encapsulate the test input/output folder handling to a separate helper class, so if it turns out you need to make smaller adaptions on the chosen strategy, you can do it in one place.

You can find such a helper class below. Feel free to use it. 

2 thoughts on “SpecFlow tips: input and output files

  1. larryq says:

    Thanks for the article. Maybe I’m staring right at it and not seeing, but where is the gist file located?