With all the awesome jQuery plug-ins out there, it's so tempting to just start including them all over the place to enhance the look of our website.
I'm looking for some opinions on whether and when I should be concerned about the sizes of these scripts.
Assume they're being cached on the web server level, so I guess the potential concerns are on bandwidth for the scripts being served from the web server, as well as performance on the client browser that needs to download all these scripts.
There are two issues here as you mentioned.
The first is server load. If this is an issue, you can use the Google AJAX Libs APIs, and get your libs directly from Google's CDN.
Second issue is client side loading time. Well, you should decide who your target audience is, and how much you are wiling to let the user wait. As a rule of thumb, most websites today consume ~300KB of bandwidth per page, including scripts, stylesheets and images.
How long is a piece of string?
It depends on the bandwidth of your target audience, the bandwidth of the server(s), and the relative value the JS provides.
You have to put it in perspective. See how long the loading your page is and what percentage of that is javascript. Usually it's not that big.
Since js usually adds niceties to a site, it might not matter if it loads last, or relatively slowly, so long as it does not block other elements from being loaded.
In the way I see it, if you need the functionality, you'll have to load the code so it's unavoidable. If it's decorative, then load that decorative js last, and it's better to have it a bit later than to not have it at all.
As others have said for jQuery itself it is best to use any public CDN hosted copy, I would also recommend looking at using a tool to bundle your plugins / custom scripts into a package(s) that make sense. These will vary depending on your server type but 2 common tools are ASP.NET ScriptManager Control - Script Combining - What's the big deal? and Bundler - Bundler Now Supports Css And .less. On the latter post there is more information about this topic in general.
I don't think you should worry too much about it. Usually, the images will consume most of your bandwidth and not Javascript files.
If you want to speed up your site, reducing the number of HTTP requests in your page is the place to start. This will probably give a much better performance boost. Try compressing all your .js files into just one and use CSS sprite techniques to display your images.