Running MSTEST.exe /publish on a TeamBuild server, what are the prerequisites?

Question

Similarly to http://stackoverflow.com/questions/954943/how-to-use-mstest-in-continous-integration-without-vs, I want to run mstest.exe on a TeamBuild server.

My context is http://stackoverflow.com/questions/1059230/trapping-error-status-in-msbuild/1059672#1059672 - i.e., I'm only trying to use mstest.exe /publish to upload the results in to the TFS repository. Thus the full rigmarole in http://www.shunra.com/shunrablog/index.php/2009/04/23/running-mstest-without-visual-studio/ is (you'd hope) likely to be overkill, esp as MSTEST.exe, as covered in http://social.msdn.microsoft.com/Forums/en-US/tfsgeneral/thread/e4575890-8f88-413c-a7f3-0d09d3b9cb01 suggests all I need to do is install Team Explorer.

I've installed VS2008 Team Explorer SP1, and mstest.exe /? still isnt telling me about the publish options.

Recall that it says only Team Explorer is required of mstest.exe /? for publishing:

The following options are also available if Team Explorer is installed:

/publish:[server name] Publish results to the Team Foundation Server.

/publishbuild:[build name] The build identifier to be used to publish test results.

/publishresultsfile:[file name] The name of the test results file to publish. If none is specified, use the file produced by the current test run.

/teamproject:[team project name] The name of the team project to which the build belongs. Specify this when publishing test results.

/platform:[platform] The platform of the build against which to publish test results.

/flavor:[flavor] The flavor of the build against which to publish test results.

I'm going down the procmon.exe path to figure out what's missing, but ideally someone in the know would step in and answer:-

Should mstest.exe /publish only require Team Explorer as stated, or does it require VSTT and Team Explorer?

Answer 1

I've read on the MSDN forums that the /publish functionality of mstest requires purchasing either VS Team Developer or VS Team Test edition. Any lesser editions such as standard/professional or team explorer don't have it.

I'd really like to see some more evidence though, as it seems fairly unreasonable that with "standard" VS2008 you can write and run unit tests, but you arbitrarily can't run them on your build machine :-(

To answer the question:

Should mstest.exe /publish only require Team Explorer, or does it require VSTT and Team Explorer?

It appears that it requires VSTT (which I assume means visual studio team-tester edition)

Answer 2

The /publish option of MSTest.exe does a license check to ensure that a "Team" version of Visual Studio is installed (i.e. the Developer or Test edition) before it does the publish part of the code.

This restriction appears to have been introduced to allow the basic unit testing capabilities in standard versions of Visual Studio but requires you to have one of the more expensive Team editions of Visual Studio to enable the "Team" related features in testing - such as to publish your test results to TFS.

In total there are 3 missing features if you do not have a Team edition of Visual Studio with Team Epxlorer installed.

• Publishing to TFS
• Code coverage
• Connect to remote agent.

If you have the Development Edition installed then you get publish and codecoverage but not the remote agent agent capabilities (i.e. for doing Load lesting).

The Test Edition and Visual Studio Team Suite have everything.

If you would like to see an example of a custom MSBuild task that uses MSTest.exe to publish unit test data from a build server (in this example JUnit test data), then take a look at the Teamprise Build Extensions that I wrote. The source code for these is available under the permissive MS-PL open source license.