Adding a library to the classpath won't cause it to get loaded btw, not any class in one.
Classes are typically loaded as needed (maybe the JVM will preload some common classes but that's usually as far as it goes).
The only real need to reduce the number of classes/librarie is therefore reduction in maintenance (even if it's not used you're going to have to keep track of versions etc.) and possibly download times and deployment size.
Those can be important factors of course, but typically won't affect runtime performance.
The main benefit of logging classloading is to detect whether maybe an incorrect version of a class is being loaded and from where if you're getting weird problems.
For example you're expecting a class to be loaded from a library you've explicitly included but instead it's being loaded from a lib/ext directory where it was put by some unscrupulous 3rd party application (Adobe has a tendency to put things there, maybe others too, but it also can happen with applications running inside appservers getting common classes from the appserver at times which may conflict with different versions of those same classes distributed with the application).