Your patched library is GPL for sure and most likely so is your application. If you're fine with that, then there's no problem. If you would like to use a different license with your application, it depends how you want to understand what GPL means by "derived work".
If your application does not depend much on the library (you did say it's a small library) and there are other (non-GPL) libraries doing the same thing that you could just as well use (or the user of the application could use), then... well ... you might not need to license the app under GPL, mostly because it is arguable that doesn't really derive from that library. But that's really loose definition.
If the library was LGPL, then you could license your app which way you wanted, but your patched library would be under LGPL. And because the original authors of the library chose GPL instead of LGPL, it can be assumed that they wanted the apps using their library also be under GPL. This is the usual counterargument to the view in the previous paragraph.
Of course, you can also ask the authors of the original library if they grant you license to the library with more permissive license (like LGPL).