Can I dynamically determine all solutions, projects and configurations on a build/trigger event and call them all with MSBuild (from command line/script)?

Question

We're still a little early in setting up our build automation and just using a bat file for now for Win32 C++ solutions. We have about 4 solutions and each has a couple vcproj files.

Each time a new solution or configuration is added I have to update the bat file to reflect the new solution or configuration to call with MSBuild.

I thought perhaps it would be easier to write a tool that parses all the sln files in a given path (and its subtree) and then parse all project files it references for configurations and then call all the builds that way.

Is there an easy way to do this?

Really this is 2 questions:

1. How can I tell MSBuild just to build all solutions inside a subtree? (I can do a search on them - that is a simple tool I think to write)

2. How can I tell MSBuild to build all configurations of a solution/vcproj?

We're using MSBuild, bat files, vc2008, c++

Answer 1

The Microsoft build framework can be accessed via APIs. But i am not sure if one can do it only via a batch file. IMHO the easiest way is to automate the build by writing an addin (using VB.NET perhaps) to the VS IDE. This addin can be invoked from a batch file. We have automated VC++ builds like this using the addin approach. In our case there was no other way because we had to automate the moc process of Qt also. I think it will be more flexible also.

The build system (atleast in VS 2005) that i worked on was quite buggy. Refer to this link, it will help you in understanding the pitfalls and also provides code-snippets on how to traverse the solution files. Once the addin is installed, a small exe can be created which invokes the build via addin. This exe can inturn be called from a batch file.

Sample VS Automator exe that calls addin methods

''' Summary: Console App to automate MSVS build with special regard to Addin
Imports System
Imports Extensibility
Imports EnvDTE
Imports EnvDTE80
Imports System.Reflection
Imports System.Windows.Forms
Imports System.IO
'''<summary>Module class that automates launching MSVS in silent mode. It disables all user actions. A hidden window is used for MSVS automation. This should work for VS 2008 as well.</summary>
'''<remarks>An STA Thread is used for this console app. Refer http://msmvps.com/blogs/carlosq/archive/2007/10/11/frustrations-with-command-line-add-ins-for-visual-studio.aspx for details </remarks>
Module VSAutomaton
    Private Enum VisualStudioSolVersion
        Unknown = 0
        VSNET2002 = 1
        VSNET2003 = 2
        VS2005 = 3
        VS2008 = 4
    End Enum
    <STAThread()> _
    Sub Main(ByVal args() As String)
        Const ADDIN_PROGID As String = "AddIn-Name.Connect"
        Const ADDIN_METHOD As String = "SyncSolutionBatch" ' name of your method in addin code
        Dim dte As EnvDTE.DTE = Nothing
        Dim dteType As Type
        Dim commandLineAddIn As AddIn-Name.Connect = Nothing
        Dim solutionFullFileName As String
        Dim solutionFolder As String
        Dim solutionName As String
        Dim logFullFileName As String
        Dim buildLogFile As String = Nothing
        Dim buildConfig As String = Nothing
        Dim connectObject As Object = Nothing
        Dim connectObjectType As Type
        Dim version As VisualStudioSolVersion
        Dim progID As String
        Dim executableName As String
        Dim addIn As EnvDTE.AddIn
        Dim msgFilter As MessageFilter.MessageFilter = Nothing

        Try
            msgFilter = New MessageFilter.MessageFilter
            If args.Length = 0 Then
                executableName = IO.Path.GetFileName(System.Reflection.Assembly.GetExecutingAssembly.Location)
                ReportError("Usage: " & executableName & " solution_file_name.sln")
            Else
                solutionFullFileName = args(0) ' project solution file
                If Not IO.File.Exists(solutionFullFileName) Then
                    ReportError("Solution file '" & solutionFullFileName & "' does not exist.")
                Else
                    solutionFolder = IO.Path.GetDirectoryName(solutionFullFileName)
                    solutionName = IO.Path.GetFileNameWithoutExtension(solutionFullFileName)
                    logFullFileName = IO.Path.Combine(solutionFolder, solutionName & ".log")
                    If IO.File.Exists(logFullFileName) Then
                        IO.File.Delete(logFullFileName)
                    End If
                    version = GetSolutionVersion(solutionFullFileName)
                    If version = VisualStudioSolVersion.Unknown Then
                        ReportError("The format version of the solution file is not supported.")
                    Else
                        progID = GetVisualStudioProgID(version)
                        dteType = System.Type.GetTypeFromProgID(progID)
                        If dteType Is Nothing Then
                            ReportError("Could not find the ActiveX Server for ProgID '" & progID & "'. Likely the proper version of Visual Studio is not installed.")
                        Else
                            dte = DirectCast(System.Activator.CreateInstance(dteType), EnvDTE.DTE)
                            dte.SuppressUI = True
                            dte.UserControl = False
                            addIn = GetAddInByProgID(dte, ADDIN_PROGID)
                            If addIn Is Nothing Then
                                ReportError("The Add-in " & ADDIN_PROGID & " was not found in Visual Studio.")
                            Else
                                addIn.Connected = True
                                connectObject = addIn.Object
                                connectObjectType = connectObject.GetType
                                ' So a copy of the same DLL is necessary in the same dir as this app. exe
                                connectObjectType.InvokeMember(ADDIN_METHOD, Reflection.BindingFlags.InvokeMethod Or Reflection.BindingFlags.Static Or Reflection.BindingFlags.Public, Nothing, connectObject, New String() {solutionFullFileName})
                            End If
                        End If
                    End If
                End If
            End If

        Catch ex As Exception
            ReportError(ex.ToString)
        Finally
            If Not (dte Is Nothing) Then
                Try
                    dte.Quit() 
                Catch ex As Exception
                End Try
            End If
            If Not (msgFilter Is Nothing) Then
                ' this is a tricky aspect. We do not want leaks but .NET can sometimes be extra smart
                msgFilter.Dispose() 'If the GC decides to re-collect the garbage from this app, then a crash may result
                ' but this is the drawback of indeterministic destruction semantics
            End If
        End Try
    End Sub

    Private Sub ReportError(ByVal msg As String)
#If DEBUG Then
        MsgBox(msg)
#End If
        Console.WriteLine(msg)
    End Sub
    Private Function GetAddInByProgID(ByVal dte As EnvDTE.DTE, ByVal addinProgID As String) As EnvDTE.AddIn
        Dim addinResult As EnvDTE.AddIn = Nothing
        Dim addin As EnvDTE.AddIn
        For Each addin In dte.AddIns
            If addin.ProgID = addinProgID Then
                addinResult = addin
                Exit For
            End If
        Next
        Return addinResult
    End Function
    Private Function GetSolutionVersion(ByVal solutionFullFileName As String) As VisualStudioSolVersion
        Dim version As VisualStudioSolVersion = VisualStudioSolVersion.Unknown
        Dim solutionStreamReader As IO.StreamReader = Nothing
        Dim firstLine As String = Nothing
        Dim format As String
            Try
                solutionStreamReader = New IO.StreamReader(solutionFullFileName)
                firstLine = solutionStreamReader.ReadLine()
                format = firstLine.Substring(firstLine.LastIndexOf(" ")).Trim
                Select Case format
                    Case "7.00"
                        version = VisualStudioSolVersion.VSNET2002
                    Case "8.00"
                        version = VisualStudioSolVersion.VSNET2003
                    Case "9.00"
                        version = VisualStudioSolVersion.VS2005
                    Case "10.00"
                        version = VisualStudioSolVersion.VS2008
                End Select
            Finally
                If Not (solutionStreamReader Is Nothing) Then
                    solutionStreamReader.Close()
                End If
            End Try
        Return version
    End Function
    Private Function GetVisualStudioProgID(ByVal version As VisualStudioSolVersion) As String
        Dim progID As String = ""
        Select Case version
            Case VisualStudioSolVersion.VSNET2002
                progID = "VisualStudio.DTE.7"
            Case VisualStudioSolVersion.VSNET2003
                progID = "VisualStudio.DTE.7.1"
            Case VisualStudioSolVersion.VS2005
                progID = "VisualStudio.DTE.8.0"
            Case VisualStudioSolVersion.VS2008
                progID = "VisualStudio.DTE.9.0"
        End Select
        Return progID
    End Function

End Module

Sample batch file to invike the VS automator exe:

@echo off
:: --Usage:       $>BatchFileName.bat "<project_name(.sln)>" {Release | Debug} [ Make ]
::                Please remember the "double-quotes". 


REM -- check for blank input
if x%1%x == xx goto InputError
if x%2%x == xx goto InputError

echo Automating MSVS-Build for %1% ...
echo .

set arg1=%1%
REM -- remove quotes
for /f "useback tokens=*" %%a in ('%arg1%') do set match=%%~a
set slnFile=%match:~-4%
if %slnFile% == .sln goto lbl_FileOK 

:lbl_FileOK

REM build configuration and output file
set SOLFILE=%1%
set BUILDCONFIG=%2%
set CLEANWSOBJS=%3%


REM -- Read necessary registry entries
REM --- Read MSVS installation dir
regedit /e A$B$C$.bxt "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\8.0\Setup\VS"
find "VS7CommonDir" <A$B$C$.bxt>A$B$C$.bat
goto :1st


:1st
for /F "tokens=1* delims==" %%A in ('TYPE A$B$C$.bat ^| find "VS7CommonDir"') do set vscomdir=%%B
set vscomdir=%vscomdir:"=%
REM -- Initialize the MSVS environment
set VSENV="%vscomdir%Tools\vsvars32.bat"
call %VSENV% > nul

REM -- remove quotes
for /f "useback tokens=*" %%a in ('%SOLFILE%') do set str=%%~a
set LastFolder=%str%
REM -- Extract the project name
if "%LastFolder:~-1%"=="\" set LastFolder=%LastFolder:~0,-1%
for %%a in ("%LastFolder%") do set LastFolder=%%~nxa
set flname=%LastFolder:.shared=.sln%
set tmpfile=%solPath%\%flname%

REM --- Check if the target '.sln' already exists, if yes delete
if EXIST %NEWSOLFILE% DEL /Q %NEWSOLFILE%
     REM -- use the addin functionality 
    VSAutomator.exe %SOLFILE%

    REM --- create log file as projectname_buildconfig.log
    set tmplog=%NEWSOLFILE:.sln=%
    set OUTLOGFILE=%tmplog%_%BUILDCONFIG%.log

    REM -- Now build the newly ready .sln file
    echo .
    echo Building Solution file - %NEWSOLFILE%, Output log file - %OUTLOGFILE%
    echo .

    if x%CLEANWSOBJS%x == xMakex goto JustBuild1
    devenv.com /useenv %NEWSOLFILE% /CLEAN %BUILDCONFIG% /OUT %OUTLOGFILE% > nul

:JustBuild1
    devenv.com /useenv %NEWSOLFILE% /BUILD %BUILDCONFIG% /OUT %OUTLOGFILE% > nul

Please remember that the code presented above may not be perfect as i have referred it from my POC that i did when i had a similar automation problem.

Answer 2

Being a PowerShell fan the following reformatted one-liner might be of help:

Get-ChildItem -Recurse -Include *.sln | 
ForEach-Object {  
 $Solution = $_.FullName
 Get-Content $_ | 
 ForEach-Object { 
  if($_ -match '\{[^\}]+[^=]+= ([^\{\s]*)$') { 
   $matches[1] 
  }
 } | Sort-Object -Unique | 
 ForEach-Object { 
  $config =  ([string]$_).Split([char]'|')
  & "$env:windir\Microsoft.NET\Framework\v3.5\msbuild.exe" $Solution /p:Configuration="$($config[0])" /p:Platform="$($config[1])" 
 }
}

This script can be saved as a ps1 file or pasted as a function in your profile. To explain what it does:

1. find all .sln files in and below the current directory
2. parse the sln extracting the configuration and platform values
3. for each unique configuration platform combination call msbuild with the given solution

--edit: Forgot Split-String is part of PSCX, it's better to use [string] I hope this helps. If you still would like to use MsBuild, it does support a recursive form of the file glob operator via the ".\**\*.sln" syntax see here for details. MsBuild also provides the MsBuild task which can be used to build a set of solutions. I do not know how you can get all 'known' configurations from a solution easily in msbuild. Therefore I choose the PowerShell route.