@Chris Becke's comment:
"think for just a moment about that statement. What part of doing it in a single (complicated) step is going to make it any faster? The image decoding routines still need to decode every pixel in order for the filter routines to filter them."
That isn't always the case. For example, when decoding a JPEG you can ask the JPEG library to give you a 1/2, 1/4, 1/8 size image (or something like that; it's a while since I've looked in detail) which it can do without having to decode the extra detail at all, due to the way JPEG works. It can be much quicker than a full decode + scale.
(Obviously you may need to scale a bit afterwards if the smaller image isn't the exact size you want.)
(Sorry I can only post this reply as a comment due to no reputaton. First time I've tried to post anything here. If someone wants to repost this or something similar as a comment and delete my answer, feel free!)