278k is less than many images that appear on websites, and YUI is only as heavy as the component stack that you have requested. The common core package yahoo-dom-event.js is 37k (12k gzipped), so you are requesting at least a few other modules. That said, even across a slow connection, 278k should not cause a 3 minute delay. I suspect there is something else causing the performance issue.
There are many tactics for improving page performance. As MontyBongo suggests, use the Yahoo! CDN. If you are hosting the files yourself, make sure you have gzip enabled on your server. One benefit to using the CDN is the ability to use the Combo handler, which bundles all modules into a single script file. If self hosted, you should look into creating manual rollups of the files you are using to emulate this. Also, look into the YUI Loader to bring in module files asynchronously. http://developer.yahoo.com/yui/yuiloader/
Otherwise, look at http://developer.yahoo.com/performance/ and ask on yuilibrary.com/forum for advice. Please note that you should always supply some code for reference when asking for advice to improve it. Paste code snippets inline, link to larger snippets on http://pastie.org or http://gist.github.com, link to the page in question, or (best of all) link to a reduced page that illustrates in isolation the issue you want to address.