I hate it when this happens... type a long question, only to find something that I missed minutes later.
I wonder if this is it:
I guess my question now is -- can I actually use %system.agent.work.dir% in my .wixproj? I'll try it now.
I can see some merit in trying to have the msbuild community tasks in SVN (to reduce the requirements of a build machine) but personally I'd just install them on the server. Important part: update your documentation for your build server to list installing that as a pre-requisite. For me, I use the community tasks in basically every msbuild across several projects, and deployment scripts, so keeping them all in SVN is a bit redundant. I also have WiX installed on the build server.
I do something similar, but rather than a before build target, I have an msbuild project file roughly like this:
<Target Name="Build">
<CallTarget Targets="UpdateVersionNumbers"/>
<MSBuild Projects="Project.sln" Targets="Build"/>
</Target>
My UpdateVersionNumbers target grabs the SVN revision, and then uses a regex and FileUpdate task to change the 4th part of the version number to the SVN revision.
Then I run the normal build of the solution file, and included in that it builds the .wixproj. Pretty simple, really.
To answer some of your other questions though:
wix30\MSBuildCommunityTasks. TeamCity executes msbuild with the working dir as the root, and on top of that, it doesn't matter where you or other developers do their checkout to - the paths are all just relative. You can pass parameters in teamcity to msbuild using the Build Parameters -> System properties. So eg, add a property called agentHome that has a value of %system.agent.home.dir% and then you can reference it in your msbuild file as $(agentHome). Note if you're also invoking msbuild again as in my example above, you'd have to do:
<MSBuild Projects="Project.sln" Properties="agentHome=$(agentHome)" Targets="Build"/>
to actually pass that variable into Project.sln. I think but I'm not positive that msbuild running on a .sln file will also pass all the properties to all the individual projects, so you can actually access it in your before build event.
Side note on installers (I recently went through the same thing you did): I settled on WiX 3.0, and though there's quite the learning curve to it, it works well. We started a new stream of development, and started using VS2010 and .NET 4. Well, WiX 3.0 is not compatible with VS2010, so we needed WiX 3.5 (which is still in Beta - though the state of it seems much better now than it was a few months ago, but they are still lagging. Such is the nature of open source, sometimes). I had some pain getting WiX 3.0 and 3.5 to install together, but did finally figure it out.
The frustration of this though (between incompatibility, flaky betas (from a few months ago), and the overall slow progress) really turned me off WiX (no offense to the guys working on WiX, but I just want an installer and I don't want to get that involved. Note, they are very frustrated too). Add to that the product I need it for next requires a much more complex installer, and WiX just seems like way too much work. I just built up my installer using AdvancedInstaller, and it only took a couple days, and so far has been working well (though we're just in early dev stages now, but starting continuous deployment and testing). AI's pricing is reasonable (we're still in trial version now) but I'll be buying in the next couple days.
I'm SURE the time I spent learning AI was MUCH less than I'd spend learning WiX, and then trying to make it do everything I need. It is a bit unavoidable to learn SOMETHING about how Windows Installer works, but you don't have to get too in-depth.