How do I get Team Build to run MbUnit tests?

Question

I am having trouble getting Team Build to execute my MbUnit unit tests. I have tried to edit TFSBuild.proj and added the following parts:

<Project ...>
  <UsingTask TaskName="MbUnit.MSBuild.Tasks.MbUnit" AssemblyFile="path_to_MbUnit.MSBuild.Tasks.dll" />
  ...
  ...
  <ItemGroup>
    <TestAssemblies Include="$(OutDir)\Project1.dll" />
    <TestAssemblies Include="$(OutDir)\Project2.dll" />
  </ItemGroup>
  <Target Name="Tests">
    <MbUnit
      Assemblies="@(TestAssemblies)"
      ReportTypes="html"
      ReportFileNameFormat="buildreport{0}{1}"
      ReportOutputDirectory="." />
  </Target>
  ...
</Project>

But I have yet to get the tests to run.

Answer 1

The way ItemGroups in MSBuild work is that they are evaluated at the very start of the MSBuild scripts, before any targets are ran. Therefore if the assemblies don't exist yet (which they will not because they have not been built yet) then the ItemGroups will not find any files.

The usual pattern in MSBuild to work around this is to re-call MSBuild again at this point so that when the item groups get evaluated in the inner MSBuild execution, the assemblies will exist.

For example, something like:

  <PropertyGroup>
    <TestDependsOn>
      $(TestDependsOn);
      CallMbUnitTests;
    </TestDependsOn>
  </PropertyGroup>

  <Target Name="CallMbUnitTests">
    <MSBuild Projects="$(MSBuildProjectFile)"
             Properties="BuildAgentName=$(BuildAgentName);BuildAgentUri=$(BuildAgentUri);BuildDefinitionName=$(BuildDefinitionName);BuildDefinitionUri=$(BuildDefinitionUri);
                         BuildDirectory=$(BuildDirectory);BuildNumber=$(BuildNumber);CompilationStatus=$(CompilationStatus);CompilationSuccess=$(CompilationSuccess);
                         ConfigurationFolderUri=$(ConfigurationFolderUri);DropLocation=$(DropLocation);
                         FullLabelName=$(FullLabelName);LastChangedBy=$(LastChangedBy);LastChangedOn=$(LastChangedOn);LogLocation=$(LogLocation);
                         MachineName=$(MachineName);MaxProcesses=$(MaxProcesses);Port=$(Port);Quality=$(Quality);Reason=$(Reason);RequestedBy=$(RequestedBy);RequestedFor=$(RequestedFor);
                         SourceGetVersion=$(SourceGetVersion);StartTime=$(StartTime);Status=$(Status);TeamProject=$(TeamProject);TestStatus=$(TestStatus);
                         TestSuccess=$(TestSuccess);WorkspaceName=$(WorkspaceName);WorkspaceOwner=$(WorkspaceOwner);
                         SolutionRoot=$(SolutionRoot);BinariesRoot=$(BinariesRoot);TestResultsRoot=$(TestResultsRoot)"
             Targets="RunMbUnitTests"/>
  </Target>

  <ItemGroup>
    <TestAssemblies Include="$(OutDir)\Project1.dll" />
    <TestAssemblies Include="$(OutDir)\Project2.dll" />
  </ItemGroup>
  <Target Name="RunMbUnitTests">
    <MbUnit
      Assemblies="@(TestAssemblies)"
      ReportTypes="html"
      ReportFileNameFormat="buildreport{0}{1}"
      ReportOutputDirectory="." />
  </Target>

Hope that helps, good luck.

Martin.

Answer 2

Above suggestion didn't help me a lot, but I found some documentation for Team Build and adjusted my build script to override the AfterCompile target:

(EDIT: Now that I have a better understanding of Team Build, I have added some more to the test runner. It will now update the Build Explorer/Build monitor with build steps with details about the test run)

<Project ...>
  <UsingTask TaskName="MbUnit.MSBuild.Tasks.MbUnit" AssemblyFile="path_to_MbUnit.MSBuild.Tasks.dll" />
  ...
  ...
  <Target Name="AfterCompile">
    <ItemGroup>
      <TestAssemblies Include="$(OutDir)\Project1.dll" />
      <TestAssemblies Include="$(OutDir)\Project2.dll" />
    </ItemGroup>

    <BuildStep
      TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
      BuildUri="$(BuildUri)"
      Message="Running tests (cross your fingers)...">
      <Output TaskParameter="Id" PropertyName="StepId" />
    </BuildStep>

    <MbUnit
      Assemblies="@(TestAssemblies)"
      ReportTypes="html"
      ReportFileNameFormat="buildreport{0}{1}"
      ReportOutputDirectory="." />

    <BuildStep
      TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
      BuildUri="$(BuildUri)"
      Id="$(StepId)"
      Message="Yay! All tests succeded!"
      Status="Succeeded" />
    <OnError ExecuteTargets="MarkBuildStepAsFailed" />
  </Target>

  <Target Name="MarkBuildStepAsFailed">
    <BuildStep
      TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
      BuildUri="$(BuildUri)"
      Id="$(StepId)"
      Message="Oh no! Some tests have failed. See test report in drop folder for details."
      Status="Failed" />
  </Target>
  ...
</Project>

Answer 3

You don't need to call MSBuild again to have your ItemGroup populated, there is a easier way. Re-calling MSBuild has its downsides, like passing all Teambuild-parameters on to make TeamBuild-tasks work. We use the CreateItem task from MSBuild to dynamically generate a ItemGroup with all our Test DLLs:

<Target Name="AfterCompile">
<CreateItem Include="$(OutDir)\*.Test.dll">
           <Output
               TaskParameter="Include"
               ItemName="TestBinaries"/>
</CreateItem>
</Target><!--Test run happens in a later target in our case, we use MSTest -->