Why can't I test Data Annotations in Silverlight 4 DLL within a normal unit test project?

Question

Here's the setup:

1. A Silverlight DLL containing some very basic class definitions. These classes have DataAnnotations via System.ComponentModel.DataAnnotations

2. A "normal" C# unit test library. It references the Silverlight DLL via its binary - not via its project. (And yes, I always rebuild the silverlight DLL manually before running tests.)

3. A "Silverlight Unit Test" project via the Silverlight Toolkit.

If I try to run validation tests on my classes in the Silverlight DLL they always pass validation without any errors, even though I know I have validation errors. (i.e. I set a field to null that is required, and the validation routine through Validator.TryValidateObject always passes it.)

However, if I create the exact same unit test in my Silverlight Unit Test project, the Validator works correctly and errors out the unit test.

I have it working via the browser / silverlight based unit tester, but I'm frustrated trying to figure out why I can't just run these silverlight class tests with all the rest of my unit tests. Is there some particular reason why? Or is there some additional work I can do to allow these unit tests to run properly in my "normal" test DLL?

Thanks