A distinction should be made between the Singleton pattern, and its implementation. Most if not all common Singleton implementations suffer from the mentioned problems with classloading as well as serialization, thread safety etc. See https://www.securecoding.cert.org/confluence/display/java/CON23-J.+Address+the+shortcomings+of+the+Singleton+design+pattern?focusedCommentId=41058653 for a quite thorough overview.
However, the Singleton pattern in its broadest sense can be any service guaranteed to be unique within a certain context. The uniqueness of a Singleton can only be specified relative to a given scope like a classloader, JVM, container or cluster; the level of uniqueness depends on the implementation, which should be chosen based on the application's requirements.
Two very common requirements leading to the use of Singletons are dependency injection (c.q. using a factory) and in-memory caching. In both cases there are good frameworks that hide the Singleton aspect from the client and offer a sufficient level of uniqueness within for instance an enterprise application container. For dependency injection Spring, Guice or Pico come to mind. For caching I know Ehcache as leading solution, but there are certainly more out there. (fun trivia: The name 'ehcache' is a palindrome)
In general, use of Singletons is 'frowned upon', and seen as an anti-pattern. On the other hand, services like dependency injection and caching require uniqueness to work. So we are fooling ourselves a bit if we proclaim to be anti-Singleton and at the same time use Spring or Ehcache or the like.
The fear of Singletons in my opinion stems from the many poor implementations that are possible, and abundant. And even if the Singleton implementation itself is 'safe', calling it directly (via static access) throughout the application leads to tight coupling and poor testability.
An improvement you can make if you have a Singleton factory in your application, is to refactor its clients so that they don't call the factory whenever they need a dependency, but provide a private field and a public setter allowing the dependency to be injected. You can then centralize the initialization of the clients, maybe in the same factory, and leave the client code clean, loosely coupled and testable (you can inject mock dependencies now). This could also be a first step towards introducing a dependency injection framework like Spring.
I hope somewhere in this rather long and rambling post I helped answer your question! (-;