Ok. here's my scenario. On the build server I run the clean and build which seems to work fine. The same thing on the local doesn't work.
The breaking point somewhere in the build process is the place where Project.Web gets the calls to GetXapOutputFile() to the Silverlight project (the one with xap file) and when it tries to build the xap file project it can't find its dependencies and fails the build. Differences between two builds boils down to these lines
Project "FinalVersion\Src\Client.Web\Client.Web.csproj" (3:3) is building "Client\Client\ConsoleClient.csproj" (18:2) on node 0 (GetXapOutputFile target(s)). Primary reference "Client.Messages". Could not find dependent files. Expected file "Client.Messages\bin\Build\Client.Messages.dll" does not exist.
And the other one(in the build server) finds this references and goes ahead..