C# / Visual Studio: production and test code placement

views:

134

answers:

+4 Q:

C# / Visual Studio: production and test code placement

Hi,

In JavaLand, I'm used to creating projects that contain both production and test code. I like this practice because it simplifies testing of internal code without artificially exposing the internals in a project's published API.

So far, in my experiences with C# / Visual Studio / ReSharper / NUnit, I've created separate projects (i.e., separate DLLs) for production and test code. Is this the idiom, or am I off base? If this idiomatically correct, what's the right way to deal with exposing classes and methods for test purposes?

Thanks,

-Patrick

+4 A:

Testing internal code is easy: use [InternalsVisibleTo] to make your production assembly "trust" your test assembly. The test assembly will then have access to all the production assembly's internal members.

The MSDN documentation linked above gives an example. It's very simple.

You should definitely separate the two out as you're doing though, IMO. You don't really want your test code (or data) to end up in production assemblies.

EDIT: Just to respond to Steven's point about testing internals: I think it's entirely reasonable to test internals. I treat unit tests as white box tests, not black box. There's definitely a place for testing only the public API - particularly for integration tests - but I find there are plenty of places where it makes sense to test internal members. Heck, without that ability, how are you going to test internal types? Sometimes a relatively small public API is layered on top of a complex internal implementation - and testing units of that implementation can be tricky through just the public API.

It does make the testing potentially more brittle, in that internal implementation changes can require test changes which wouldn't be needed if you were only testing public APIs... but it can also make the production code more robust, as you're more likely to write copious tests for corner cases etc if it's easy to do so.

Moderation in all things, basically.

Jon Skeet 2010-05-26 06:09:13

@Jon - Could you give an example of a test for **internal type**?

Gutzofter 2010-05-26 14:03:56

@Gutzofter: Sure. As an example, IIRC Microsoft actually shipped an initial version of BigInteger (which is public in .NET 4) in .NET 3.5 as an internal type, presumably to support some other APIs. It makes more sense to unit test the functionality of `BigInteger` directly rather than through the dependent APIs. I can imagine similar situations for other libraries.

Jon Skeet 2010-05-26 14:22:57

@Jon - Unit testing a library that you don't own, really could be a code smell, IMO. I could see a situation where you're spiking to understand how a library works or identify a bug in third-party software, but as a unit test for your production code, could be a serious coupling issue don't you think?

Gutzofter 2010-05-26 15:08:26

@Gutzofter: I'm not suggesting unit testing a library you don't own. I'm suggesting that if you were in Microsoft's position, you'd be unit testing BigInteger from a different assembly, but BigInteger would still be internal. I was using it as an example of where you might want a type to be internal, but you'd still want to unit test it. To reverse the question: under what circumstances do you use internal types, and why don't you want to unit test them?

Jon Skeet 2010-05-26 15:34:00

@Jon - Why would you ever want to get access to a member of an object? For me software has to be malleable because of the contexts in which we use these objects. IOW, we need to be able to extend the behavior of objects. Using internal types creates friction in extending objects. If we set a member to internal then we have to treat the assembly as a unit. The assembly becomes the object under test. Which IMO is an integration test (black box).

Gutzofter 2010-05-26 16:09:23

@Gutzofter: Why would we want to access a member? To call it (if it's a method), fetch it or set it (if it's a property), subscribe to it (if it's an event) etc. So are you saying you never ever use the internal modifer at all? If so, then clearly the question isn't relevant to you - but I would suggest that there are times when it's entirely appropriate to make types internal. (I also don't think that all classes need to be extensible - I'd favour classes being sealed by default.) I don't see why making a member internal means you have to treat the assembly as a unit...

Jon Skeet 2010-05-26 16:29:28

@Jon - As your example stated, in 3.5 marked internal 4.0 public for `BigInteger`. For some reason, the context in which the APIs were used was not sufficient enough to allow access to `BigInteger`, a more direct access was required. The problem is we had to wait for Microsoft to make the changes to the library. Two solutions: implement your own `BigInteger` or use reflection to get access to `BigInteger`, until 4.0. I admit, never having never implemented a huge library before, I've not seen a need for restrictive access to the underlying code. is this for defensive coding?

Gutzofter 2010-05-26 17:29:26

@Gutzofter: Public APIs are hard to get right - you can't easily change them afterwards. There are plenty of times when you want to be able to use some functionality without making it publicly available to everyone - thus requiring that it stays the same forever. I haven't looked at the API for BigInteger in 3.5 and 4, but I woudn't be at all surprised to see that they're different. Indeed, MS has explicitly said that they *wanted* BigInteger to get into 3.5 to start with, but they weren't happy enough with the API to make it public. What do you think they should have done? (Continued.)

Jon Skeet 2010-05-26 17:37:23

Option 1: Delay the entire release of .NET 3.5 until they were happy with BigInteger. Not feasible, IMO. Option 2: Say "to hell with it" and make it public anyway. This leads to two suboptions: Option 2a: fix the API in .NET 4 and break everyone using it from 3.5. Option 2b: Live forever with a bad API. Option 3: Don't ship BigInteger at all in .NET 3.5, and remove any other classes that were using it. Option 4: Ship it internally in .NET 3.5 but don't unit test it. Option 5: Ship it internally, unit test with InternalsVisibleTo. I know which option *I* think is the most palatable.

Jon Skeet 2010-05-26 17:39:30

Great response! Thanks for the breakdown in options. I didn't even look at it from a deployment standpoint. Which should be, as programmers our primary focus.

Gutzofter 2010-05-26 18:07:33

@Gutzofter: Designing an API is certainly a hard task. Very few people do it well. There's a great talk by Joshua Bloch which is worth watching: http://www.youtube.com/watch?v=aAb7hSCtvGw

Jon Skeet 2010-05-26 18:22:55

Test code in the production code? I have no idea how Java handles that, but it sounds quite scary to me :-)

It's "the norm" in DotNetLand to usually have one test assembly per main assembly, which in my mind is good for several reasons:

Separation of code - I'm not sure how Java handles this, but what's to stop you accidentally shipping test code in your app if you have them all in the same project? This is hopefully just my ignorance of how Java works from a testing perspective than a real concern.
Testing from the "outside" - If you're forced to test from the outside of the code your testing it should force you to think more about whether you're testing the behavior of the code under test, rather than the implementation. You can work around internal state using things like InternalsVisibleTo, but whenever you reach for things like that it might well be a sign you're testing implementation details, not externally observable behaviour which leads to brittle tests.

The second point may sound like "teaching you to suck eggs" a little, but it's a trap I've seen many devs fall into.

Steven Robbins 2010-05-26 06:12:33

You will find answers to your questions in this post (how to unit test private methods). In essence, if you are not able to separate your test code from your production code, you should reconsider your design (and that is true for Java, too).

Doc Brown 2010-05-26 06:13:20

You are correct in assuming that the creation of separate assemblies/projects to isolate your test code from your production code is the norm in .NET land. I really encourage the practice when mentoring people about unit testing because I don't want test code mixed in amongst my production code.

Generally speaking, I am extremely reluctant to test private/protected methods of my SUT. The great thing about TDD/UnitTesting is that you describe external behaviour and leave the implementation details flexible and "black boxed". You are then free to make as many refactorings as you feel are appropriate and as long as they do not break the external behaviours, the tests should not need to be changed.

If you are in some way tied to the implementation details (inheritance, access to privates/protected methods from within the production assembly), your tests will need to be refactored as frequently as the internal details of the class change.

Occasionally, you may have an internal class/method that you want to test. In this case, you can use the [assembly:InternalsVisibleTo("Test Assembly Name")] attribute in your production AssemblyInfo.cs file and allow your test assembly to see the internals of your production assembly.

InternalsVisibleToAttribute on MSDN

It should be noted that if you are new to .NET, internal is a different access qualifier than private and protected. An internal class is "public" within the assembly that houses it.

Hope this answers your questions.

Cheers, Dave

Dave White 2010-05-26 06:15:51

I don't think unit testing is universally regarded as black box testing. As I mentioned in my answer, I regard *unit* tests as white box, but *integration* tests as mostly black box. I don't think that's a *terribly* unusual way of thinking.

Jon Skeet 2010-05-26 16:30:43

I would agree with you that testing internal classes and methods is feasible and often appropriate. My point was that internal is different than private/protected and if you are trying to test private/protected members or classes, you're going to end up with brittle tests and this isn't a good thing. If you need to change the accessibility characteristics of your class to write tests or use reflection to test your code, this is a "smell" that I really strive to avoid/eliminate.

Dave White 2010-05-28 16:40:29

ansaurus

tags:

views:

answers:

C# / Visual Studio: production and test code placement

related questions