Something you should definitely NOT do is concatenate all your JavaScript into a single file. If you ever make a change to your codebase, that file is recreated... and redistributed to every visitor. HTTP overhead is fairly insignificant, so unless you are loading hundreds upon thousands of unique files, loading 20 different files versus loading 1 big one won't show much of a difference except to users with exceptionally slow connections (who will be waiting for the big file anyway, so they won't notice an extra second or two from the HTTP overhead).
ToonMariner's suggestion to use hosted code (particularly off the Google Code repos) is a good one - it saves you having to host the file, it allows users who have encountered that file to take advantage of caching (improving the apparent load speed of your site), and it won't get included in your concatenated file if you make a change. Even if you choose to maintain your whole app in one big file, you should look into using this, since you can avoid packaging jQuery and that's a savings of 50+kb.
Furthermore, your concern about interpreting bellsAndWhistlesPlugin() is correct - the this
in the bellsAndWhistlesPlugin function is simply an empty list (though I should hope the plugin does a $(this).each call to iterate over elements and returns early, since there are no elements... otherwise, you may want to revisit your plugins!). You can eliminate this problem by removing page-specific code from your full application.js file and putting it in an inline <script> element on the page itself, where it belongs anyway, or rewriting the plugin to make it return early if there are no matching elements.
Just make sure you enable caching for resources loaded from a /js directory and you won't have issues with reloading libraries - only ones that have changed. You can use an Expires header or a Last-modified header; Expires won't necessarily force an update unless the person reloads or the cache has expired, and Last-modified invokes an HTTP overhead for each file, which is problematic for a larger number of files. You'll have to evaluate the tradeoffs for your application.
If you are really, truly, seriously interested in maximum efficiency, you can rewrite your application using GWT. That technically guarantees maximum cross-browser portability, maximum code efficiency, eliminates dependency on jQuery libraries, will execute faster, and produces much smaller filesizes. All in one file, I might add, but the tradeoffs for getting a static compiler to put out maximum efficiency JavaScript are worth it... if you are willing to rewrite the whole thing in GWT.
The question you have to ask yourself is: who is my average user? What kind of connection does s/he have? Does my app need to run on mobile devices? If your average user has a fast connection, don't worry about it - they'll load your page fast enough whichever way you choose. If you need to work on mobile devices, or your intended audience has a slow connection speed, consider caching your large libraries that change very infrequently and using external repos where available (e.g. jQuery), then package the rest of your app into one big file. HTTP overhead for mobile devices and slow internet is significant enough to warrant doing this.