views:

205

answers:

2

We use CCNET 1.4.4.83 on Win 2003 SP2. A few days ago CCNET started to report build failures with no specific reason. The project contains a single exec task which runs build.bat. The batch does all the job (msbuild, fxcop, db scripts, tests) but in the server log i can see that not all the commands were executed from build.bat. It looks like something kills the process in the middle but i'm sure it's not timeout issue. I tried to disable anti-virus software - no luck. The service is running under a local admin account. If i run build.bat manually on the CCNET machine it always succeeds.

Here is the log fragment:

2009-11-10 11:26:48,584 [288:DEBUG] [trunk C:\integration\trunk\src\build.bat]     60 Warning(s)
2009-11-10 11:26:48,584 [288:DEBUG] [trunk C:\integration\trunk\src\build.bat]     0 Error(s)
2009-11-10 11:26:48,584 [288:DEBUG] [trunk C:\integration\trunk\src\build.bat] 
2009-11-10 11:26:48,584 [288:DEBUG] [trunk C:\integration\trunk\src\build.bat] Time Elapsed 00:00:23.73
2009-11-10 11:26:48,724 [trunk:INFO] Task execution failed
2009-11-10 11:26:48,724 [trunk:INFO] Task output: 
<buildresults>
 -- the batch output here--
</buildresults>

2009-11-10 11:26:48,803 [trunk:INFO] Merging file: c:\integration\trunk\Artifacts\Tests1.xml
2009-11-10 11:26:48,803 [trunk:INFO] Merging file: c:\integration\trunk\Artifacts\FxCop.xml
2009-11-10 11:26:48,803 [trunk:WARN] File not Found: c:\integration\trunk\Artifacts\FxCop.xml
2009-11-10 11:26:49,428 [trunk:INFO] Emailing "trunk Build Failed" to ....
2009-11-10 11:26:55,506 [trunk:INFO] Integration complete: Failure - 10/11/2009 11:26:48

Any ideas to where to look at would be appreciated!!

+1  A: 

It's very hard to track down issues like this when the whole build process is done via a single CCNET task like this. It may not be an option, but I would recommend splitting the build process out from the single .bat file, and create separate CCNET tasks - it will make it much easier to see what is going on when things starting behaving oddly.

e.g. separate tasks for:

  • msbuild
  • fxcop
  • dbscripts
  • tests

If the problem is within the tests somewhere, that would soon become obvious as the tests task would fail but the other 3 would have succeeded.

Other than that, I'd suggest trying to log debug/progress info out along the way - trying to find the point at which it actually fails and go from there

AdaTheDev
Good idea and i'll try it if can't find another solution. Some time ago i found that shell scripts are much easier to go with than tweaking ccnet config. The log is set to DEBUG but there is nothing useful for me in it.
UserControl
I know what you mean, though once you're used to tweaking ccnet config it's not too bad. I find the granularity of smaller CCNET tasks is well worth it.
AdaTheDev
Spent almost the whole day and ended up with your solution :(
UserControl
+1  A: 

Try running the console version of CCNET and enable DEBUG logging.

  • If this also succeeds then you have something in your build script that the service is not allowed to do.
  • If not then it should at least make the problem easier to diagnose.
GraemeF
Thanks for the idea! Tried it and the console version builds the project just fine. But i can't imagine what is illegal from the service point of view. The only commands in the batch are: msbuild.exe, svn.exe, nunit-console.exe, fxcopcmd.exe plus a few calls to other batches (tried to comment them to localize the problem - no luck though). And the last changes i made to the batch were a week before the problem.
UserControl