How do I tell MSTEST to run all test projects in a Solution?

Question

I need to know how to tell MSTEST to run all test projects in a solution file. This needs to be done from the command line. Right now I have to pass it a specific project file, I'm trying to get it to run from a SOLUTION file.

I'm hoping this is possible, because in Visual Studio, hitting Ctrl+R, A, runs ALL tests in the currently opened solution.

The way I've interpretted the help files, you have to pass in each DLL specifically.

I want to run this from the command line from my CruiseControl.NET server, so I can write other utilities to make this happen. If there is a wierd way of getting this to happen through some OTHER method, let me know.

How do I tell MSTEST to run all test projects for a solution?

<exec>
    <!--MSTEST seems to want me to specify the projects to test -->
    <!--I should be able to tell it a SOLUTION to test!-->
    <executable>mstest.exe</executable>
    <baseDirectory>C:\projects\mysolution\</baseDirectory>
    <buildArgs>/testcontainer:testproject1\bin\release\TestProject1.dll 
    /runconfig:localtestrun.Testrunconfig 
    /resultsfile:C:\Results\testproject1.results.trx</buildArgs>
    <buildTimeoutSeconds>600</buildTimeoutSeconds>
</exec>

Answer 1

I would just write a target that calls it the way you want, then whip up a batch file that calls the target that contains all the DLL's to be tested.

Unless you're adding test projects all the time, you'll very rarely need to modify it.

Answer 2

Why not just have msbuild output all the test assemblies to a folder.

Try setting OutputPath,OutputDir,OutDir properties in msbuild to accomplish this.

then have mstest execute against all assemblies in that folder.

Answer 3

You could enforce some convention on the naming and location of test projects, then you could run MSTest on, say, all *Test.dll below the location of your solution.

As far as I know, there is no way to tell a test project from a 'normal' DLL project based soleley on a solution file. So, an alternative could be to analyze the project files and/or .vsmdi files to find the test projects, but that could be rather tricky.

Answer 4

I don't know directly but this is where VSMDI [fx:spits in a corner] can help. In your solution add all the tests to the VSMDI. And then pass the VSMDI to mstest using /testmetadata.

However I would suggest that you follow the conventions above. And use a naming convention and dump that out from the SLN file using say a for loop in the command script

Answer 5

To elaborate on VladV's answer and make things a bit more concrete, following the suggested naming convention running your tests can be easily be automated with MSBuild. The following snippet from the msbuild file of my current project does exactly what you asked.

<Target Name="GetTestAssemblies">
 <CreateItem
        Include="$(WorkingDir)\unittest\**\bin\$(Configuration)\**\*Test*.dll"
  AdditionalMetadata="TestContainerPrefix=/testcontainer:">
       <Output
           TaskParameter="Include"
     ItemName="TestAssemblies"/>
    </CreateItem>
</Target>
<!-- Unit Test -->
<Target Name="Test" DependsOnTargets="GetTestAssemblies">
 <Message Text="Normal Test"/>
<Exec 
 WorkingDirectory="$(WorkingDir)\unittest"
 Command="MsTest.exe @(TestAssemblies->'%(TestContainerPrefix)%(FullPath)',' ') /noisolation /resultsfile:$(MSTestResultsFile)"/>
 <Message Text="Normal Test Done"/>
</Target>

Furthermore integrating MsBuild with CruiseControl is a piece of cake.

Answer 6

i know this thread is quite Old, but its still high on Google so i thought i might help one or two. Anyway, since there is no satisfactory solution for this. I've written an msbuild task for this. details can be found here: http://imistaken.blogspot.com/2010/08/running-all-tests-in-solution.html