Visual Studio Unit Test - Weird behaviour

Question

Has anyone seen this very strange behaviour before?

1. I've got a solution whith 70 unit tests. All of them pass on my dev machine.
2. Whenever I commit my changes, our continuous integration process kicks in and the build box will eventually run the same 70 unit tests.
3. There is only ONE test in the build box that fails all the time.
4. The error is in one line that only gets a record from our unit test db. (I know it sucks having unit test to rely on data but please don't focus on this as it's not relevant now)
5. The most weird thing is when I logon myself to the build box, open up the same visual studio solution and manually kick off the unit tests. Result: ALL PASS!

Has anyone ever had this weird situation? I'm guessing there is some weird thing going on with Cruise Control.NET and MSTest?

Answer 1

Surely your unit test runner produces a good log that shows the exact exception message or error? It's kinda pointless to guess at it but an "access denied" kind of error would be an obvious candidate. Setup whatever dbase engine you use (you forgot to mention that too) to give the user account that runs the tests on the build grunt access to the tables.

Answer 2

As said in another answer, it doesn't make too much sense to guess about it when there are detailed logs around...

But because I had this situation several times, here's a guess anyway: The account, which is used by the CI server to run the tests, may not have appropriate permissions in the database. This would also explain, why the same test succeeds when you run it manually (then with your user account)...

HTH! Thomas

Answer 3

Hi all ... thanks for your inputs but it wasn't anything related to credentials at all. I've found out that other tests that were running before that particular one were leaving my unit test database in an inconsistent state, therefore causing errors to the test in question. It's not a good practice to have your unit tests relying on data, so unless you are extremely bound to it like myself, this is what a recommend to everyone: DO NOT RELY ON DATA TO DO YOUR UNIT TESTING !!!! Make sure you have all the good stuff in place, specifically a good IOC/dependency injector container so your classes are loosely coupled and you can mock up any interface you may want to unit test easily!