If the question is, "Will external libraries INHERENTLY make my app slower than if I wrote the same code myself?", the answer is generally, "Yes, but not very much."
It will take the JVM some time to load an external library. It's likely that the library has functions or features that you aren't using, and loading these or reading past them will take some time. But in most cases this difference will be trivial, and I wouldn't worry about it unless you are in a highly constrained environment.
If what you mean is, "Can I write code that will do the same function faster than an external library?", the answer is, "Almost certainly yes, but is it worth your time?"
The odds are that any external library you use will have all sorts of features that you don't need but are included to accomodate the needs of others. The authors of the library don't know exactly what every user is up to so they have to optimize in a general way. So if you wrote your own code, you could make it do exactly what you need and nothing more, and be optimized to exactly what you are up to.
Whether it's worth the trouble in your particular case is the big question.