Gáspár Nagy on software

coach, trainer and bdd addict, creator of SpecFlow

Detailed Report for PartCover in TeamCity 5.1

by Gáspár on September 20, 2010

I was configuring a build with TeamCity (for SpecFlow that is hosted on github). After the initial success, I thought I would also enable code coverage in the build. I don’t really believe in code coverage numbers, but if it is easy (and free) to configure, than why not?

TeamCity supports NCover and PartCover out of the box. As I wanted to go for a free solution (SpecFlow is a free, open-source product), I used PartCover (v4.0.10801). PartCover seems to be a great tool, but it is not so easy to set it up with TeamCity. (You can find a lot of questions and discussions about it.) As I can see, there are some improvements in the later versions of PartCover and TeamCity, but still there are some problems.

If you just start with the build server configuration, you should read first Jeremy Skinner’s post. (On TeamCity 5.1, you don’t have to worry with the assembly renaming.) The rest comes here:

Problem: Sometimes you see no results at all

Maybe there are many reasons that can cause this, but for me the problem was an x86/x64 issue. On an x64 build controller, you have to make sure that the test execution and the coverage are all running with the same processor architecture. I’ve achieved success with x86 compilation only, with x64 I always encountered a “…\tmpE1BE.tmp exited with code -1” error. But I did not play with this so much. These are my working settings:

teamcity_partcover_settings

Problem: The reports provided with PartCover don’t work with TeamCity

You can find fixes for the reports in stackoverflow and also on Jeremy’s post, but altogether these reports are not too detailed, e.g. they don’t go into method level. Without knowing which method causes the low class coverage, the numbers are not too helpful (except for your boss). So this leads me to the third problem…

Problem: There is no detailed HTML report available

As I have mentioned above, this was quite a pain for me. So I  refreshed my old XSLT skills (again) and tried to put together a report that shows what I need:

  • global & assembly-level summary
  • top 10 uncovered method
  • detailed method-level report with source code line numbers (it is not possible to embed the source code file from XSLT unfortunately)
  • coverage values excluding the very simple members (simple property get/set, empty constructor, etc.)

The detailed report (for a small solution):

Class details with method-level numbers:

How to setup the detailed report
  1. download the XSLT from here: PartCoverFullReport.xslt
  2. copy the XSLT file in the “xslt” folder of PartCover on the build controller(s) (e.g.: C:\Program Files\PartCover\PartCover .NET 4.0\xslt)
  3. in the build configuration, specify the “Report XSLT” field as:C:\Program Files\PartCover\PartCover .NET 4.0\xslt\PartCoverFullReport.xslt=>index.html

9 thoughts on “Detailed Report for PartCover in TeamCity 5.1

  1. Gaspar, this seriously is AWESOME work.

    I am extremely grateful to you, and so will my team.

    Thanks a lot for such a valuable contribution!

  2. Josh Buedel says:

    Thanks for posting this. It's been helpful to me, particularly your xslt.

    Are you seeing the TeamCity “coverage delta” graphic? It normally appears on the build overview page, as a nice green graph. I am *not* seeing it. I'm also not seeing the partcover xml file show up in the artifacts, which I believe it does by default. I'm confident it is being generated, because I see it referenced in the log and I was able to capture it out of the agent's build temp once.

    Thanks for any insight you can provide.

  3. Gáspár says:

    Thx!

    Yes, I do see the coverage delta from TeamCity on the overview page, and I also see the XML in the artifacts. So there must be some other reason why it is not displayed for you. Do you have the same settings as I do (like on the screenshot in the post)? I'm using TeamCity Version 5.1.4 (build 13550), but I have seen that there is a newer version already, are you maybe using that already?

    (Strangely the numbers calculated by TeamCity are slightly different from the ones calculated by me, but I did not look after this too deeply.)

  4. Josh Buedel says:

    My settings are close to yours, though not identical. I don't think it matters though because I can confirm that the partcover xml is *definitely* being created.

    I've narrowed it down to some specific weirdness. When I use *your* xslt, all works as expected. When I use *my* xslt, it does not work at all. Both xslts produce valid html when run via the VS2010 xml tools. I'm thinking the java xslt engine might not like my xslt perhaps. Anyways, I'll take it up with JetBrains on the support forum. Thanks!

  5. Gáspár says:

    Yes, that can happen. I'm also not too familiar with the java xslt, but what I have noticed that instead of msxsl:node-set(…) it is better to use exsl:node-set(…) that works on both platforms (xmlns:exsl=”http://exslt.org/common”). There are also other useful cross-platform functions on http://exslt.org/, but I did not checked them.

  6. litlit says:

    Hi Gaspar,

    Thanks for posting this.

    I'd just like to ask if what's the license for your xslt file?

  7. Gáspár says:

    Good point. I have added a “Simplified BSD License” header to the file. So you can use/modify it for free, just keep the original copyright notice.

  8. Aykut K says:

    Is it possible to view specflow reports in teamcity?

  9. Gáspár says:

    I haven't done such integration yet. Basically you would need to call specflow.exe from a command line build step to convert the unit test results to specflow reports. If you had success with it, just post it here.