Yes, they are all shared per AppDomain which is why, in general, they should not be used!
They should not be used, in general, because they are so unlikely to be used properly. Also because there are safer alternatives, like HttpContext.Cache, or even Session state.
Still, if you encapsulate all access to these static members, and if you handle locking correctly, then you'll have a safe implementation that may then turn out to be a bottleneck, with all threads contending for the shared resource. It's really better to do without.
Also, you seem to mean ASMX web services, but you should specify ASMX or WCF.