When running a Silverlight application in a browser, the silverlight application may only make requests to URLs from the same domain as the application was installed from. For instance, if your silverlight app is installed from http://yoursite.com/yourapp.xap
, you can only make WebClient requests to other URLs on http://yoursite.com/
. Your options here are either create a proxy script that requests the WoW armory from your server, and have your silverlight app hit that proxy, or run the silverlight out-of-browser and request full trust.
Edit: Generally the best option is a cross-domain policy file as explained here. Sadly, It does not appear that wowarmory.com implements the cross-domain policy file.