views:

484

answers:

12

I'm working on a large project where, even with 10s of 1000s of automated tests and 100% code coverage, we're getting a ridiculous number of errors. About 95% of errors we get are NullReferenceExceptions.

Is there any way to enforce null-checking at compile time?

Barring that, is there any way to automagically enforce null-checking in unit tests without having to write the tests for null cases myself?

A: 

neither of those is possible with C# 3. you would have to use something like Spec#... i think C#4 may have some of that built into it, but i'm not sure about that.

spec#: http://research.microsoft.com/en-us/projects/specsharp

Derick Bailey
+3  A: 

100% code coverage means nothing.

It is a false sense of security.

The only thing you're measuring is that you're executing all the lines of code.

Not:

  • That those lines of code are all the lines of code that should've been there
  • That those lines of code are operating correctly (are you testing all edge cases?)

For instance, if your procedure to deal with a fire contains 1 step "run out of the building", then even if that happens in 100% of the cases, perhaps a better procedure would be to "alert the fire department, try to stop the fire, then run out if all else fails".

There is nothing built into C# that will help you with this without you specifically going in and adding code, either code contracts (.NET 4.0) or specific IF-statements (<4.0).

Lasse V. Karlsen
+17  A: 

You should look into Code Contracts. The static checker is only available for the higher-end VS editions, but that's basically what you're after.

There are plenty of resources online, and <plug> you can also read a prerelease version of the chapter on Code Contracts from the 2nd edition of C# in Depth - download chapter 15 for free. </plug> (The chapter is slightly out of date with respect to the latest and greatest build of Code Contracts, but nothing huge.)

Jon Skeet
+1 Code Contracts will definitely stop your null references cold at compile time. You will not be able to build until you eliminate all possibilities of a null value being passed into a particular method/class. Also check out Pex which goes hand-in-hand with Code Contracts.
masenkablast
@Jon Skeet: I'm wrong or code contracts only works if the developper use Requires.Something in the code ? So if there is a developper mistake of checking using contracts, it will pass at compile time ? I think that Juliet want to check this after developpment time, when testing or building.
Yoann. B
@Yoann: Well yes, you have to express the contract in the code. How else are you going to distinguish between APIs which *can* accept nulls and those which can't? But the static checker *does* perform checking at compile time of the callers of the API.
Jon Skeet
I still wish they'd done it the same way as Spec#. Code Contracts are probably the best solution here but they are so... verbose.
Aaronaught
@Jon: That's why i adviced custom code analysis rules, but not sure it's possible to build custom rules checking null references.
Yoann. B
THANK YOU for a clear, constructive answer. The linked chapter is great. Will it remain available? I would like to link to it...
Pascal Cuoq
@Aaronaught: Agreed. I'm going to look at mixing PostSharp with Code Contracts to at least allow it to be done on attributes. @Yoann: So what advantage would the custom code analysis rules have over using code contracts? @Pascal: I don't know whether chapter 15 will stay free, to be honest. You could always suggest people buy the book ;)
Jon Skeet
+1, +answer: interesting solution. I've run it by my team, but it doesn't seem like they want to add anymore frameworks to existing code -- I'll keep pushing for code contracts and hopefully we'll see an improvement in our code.
Juliet
@Juliet - Says the coders who can't be bothered to check for nulls...
ChaosPandion
A: 

You cannot have null checking at compile time as at compile time the objects are only Types and only at runtime the Types are converted to instances that have a concrete value ... here null.

Obalix
+1  A: 

This isn't a technical solution, but a social one. Simply make it unacceptable in your environment to access a reference type without checking for null when the reference type has been modified in any way by outside code (another method call, etc). Unit Testing doesn't replace good old-fashioned code review.

Payton Byrd
A: 

Maybe you should take a look at custom code analysis checkin policies for TFS

http://weblogs.asp.net/uruit/archive/2009/03/17/writing-custom-rules-for-tfs-2008-code-analysis-check-in-policy.aspx

Yoann. B
A: 

The .NET framework was looking to enforce compile time null reference checks by using an ! modifier.

public void MyMethod(!string cannotBeNull)

But alas, we don't have compile time checking. Your best bet is to minimize the amount occurrences for external callers to pass null values and then enforce null checks on public facing methods:

public class ExternalFacing
{
  public void MyMethod(string arg)
  {
     if (String.IsNullOrEmpty(arg))
        throw new ArgumentNullException(arg);

     implementationDependency.DoSomething(arg);
   }
}

internal class InternalClass
{
    public void DoSomething(string arg)
    {
         // shouldn't have to enforce null here.
    }
}

Then apply the appropriate unit tests to the External class to expect ArgumentNullExceptions.

bryanbcook
+1  A: 

Is there any way to enforce null-checking at compile time?

Nope. The compiler cannot determine if the run-time reference variable is pointed to null.

And ruling out null producing statements (sets and returns) isn't enough either. Consider:

public class Customer
{
  public List<Order> Orders {get;set;}
}
  //now to use it
Customer c = new Customer;
Order o = c.Orders.First(); //oops, null ref exception;
David B
A: 

I may be wrong, but I think FxCop has a rule that suggests you add null reference checks to your code. You could try running your assemblies through the tool and see what it has to say.

Grant Palin
A: 

Check out Gendarme, it can be run post-build alongside your tests (possibly before them, if you wish) and has a few rules relating to null checks. You can also fairly trivially write your own.

Noon Silk
+1  A: 

1) I think, the Resharper can suggest you to check some critical places in your code. For example, it suggests to add the [null reference check code] and adds it if you allow.

Try it. It will increase your experience if you need, of course.

2) Use "Fail Fast" pattern (or assert, assertions) in your code at the early stage of development application

igor
+1  A: 

Defensive programming can only get you so far... maybe its just better to catch the exception and deal with it like any other.

CurtainDog
In "dealing" with the exception, be sure to deal with why it happened. Why is this reference never being set? Was an exception thrown before it could be set? That happened to me today, and it was necessary to track down the reason for it (a missing resource causing an `ArgumentNullException`, which was logged and ignored).
John Saunders
Certain things, in particular io operations, you can never be sure will work. If yanking out a cable somewhere causes a method to return null (probably bad practice but you can't always get what you want) then you may as well catch it as an exception.
CurtainDog