Classes will only be loaded as required (referenced by other classes via import
etc.)
The headache with multiple frameworks is that you have to manage their shared dependencies. e.g. Framework A requires Logging Framework X, but Framework B requires Logging Framework Y.
These problems aren't insurmountable, but you have to keep track of them. When you upgrade Framework A, you may well end up with a ripple effect, in which you have to update a corresponding dependency. That then requires another framework component update, and so on.
e.g. Framework A gets upgraded, and requires an update to Log4J. THat then forces you to update Framework B to a version compatible with your new Log4J, and so on.
If you have multiple framework requirements, this may in fact point to a requirement to subdivide your application accordingly (into different deployables/services etc.).