First a note - when you run under IIS 6 and 7, AppDomains can be in separate worker processes if you allow IIS to use multiple processes per AppPool. To put it in proper historical context, AppDomain just emulates what IIS AppPool mechanism would do to any binary. Allows for a lot of scalability and in many cases it doesn't really matter if you have a few copies of a "singleton" - that can improve perf as well.
If you really, really, and I mean really :-) have the need for a server-level singleton there is a way to do it, without paying the cost of remoting, but only if you know your COM or WinNT API.
WinNT route will be the lightest resource-wise but you'll have to do complete WinNT work to the point that you'll start asking why are you still in C# and not C++ - shared memory, events or WinNT mutexes etc.
COM route will require you to force out of proc invocation (proper registry keys) and you may want to go with AutoDual instead of AutoDispatch - first to buy perf and second to make sure that .NET has the minimal chance of trerating that dll as "it's own". The gola is to make it look and quack like any other out of proc COM
Important question to ask yourself is - why do I actually need it? Very different tactics needs to be used if the purpose is caching, vs. fast event processing (in which case MSMQ will serve well - it's now under WCF umbrella).