TeamBuild will build everything in the solution. If you want less things built, then give it a solution with less projects in it. But if you are using project references; you need at least all the projects that have those. The solution file under source control should be the one you want TeamBuild to use. You can have as many as you want for your own personal use, but those don't need to be source controlled.
I wouldn't use a Web Deployment Project if you are using TeamBuild. TeamBuild replaces the deployment project. You insert a post build task into the msbuild script to copy the files to your test environment.
EDIT:
<Target Name="AfterDropBuild">
<!-- Copy output assemblies -->
<Message Text="SiteFilesToPublish location is $(DropLocation)\$(BuildNumber)" Importance="Low" />
<CreateItem Include="$(DropLocation)\$(BuildNumber)\Default\_PublishedWebsites\IC.Web\**\*.*" Exclude="*.pdb" >
<Output ItemName="SiteFiles" TaskParameter="Include" />
</CreateItem>
<Copy
SourceFiles="@(SiteFiles)"
DestinationFiles="@(SiteFiles ->'\\genweb.lab.dev\Test Lab\Independence Compliance\%(RecursiveDir)%(Filename)%(Extension)')"
ContinueOnError="true" />
</Target>