In my opinion your problem here is a lack of control about how this library is produced and used by other projects. If I were you (which I'm not :) I'd set about developing the library through a unit test co-project where new functionality can be developed and tested independently. Once that functionality has been implemented and tested to be working within your unit test parameters manually copy the assembly into a "library" folder of the web project that the required the extension of the library in the first place (this folder holds all your compiled assemblies used by that project).
Even better would be to maintain a version system in which you tag the new version of the library so as to keep track of the exact source revision that it's using.
The reason I suggest what may seem like a cumbersome methodology of working is that your current practice makes your existing websites quite brittle as a change made in the library for one site may in fact break one of the other sites... and as the amount of sites you have increases you can't be forever retro testing new versions of the shared library against the existing sites.
It's also for these reasons that I don't recommend using the GAC either.