If I annotate the column with a size
of 128MB, I feel like I'll be wasting
a lot of space for the small and
mid-sized objects. Is there a variable length blob type I can use?
LOB (CLOB or BLOB) don't have a size (as far as I know in most database). The type is meant to hold information that is clearly larger than "regular" data. Most database will use a different storage scheme for such data, e.g. store them in another segment as the main data of the table. I don't think the size parameter is considered by Hibernate.
Will hibernate take care of all of
this for me behind the scenes without
wasting space?
Hibernate can not do much and it's the database that will manage the efficient storage of LOB. LOB have are a bit slower than other types (because of the special storage), but they are still very well managed by databases.
the length of the byte[] can vary a
lot (from ~10KB all the way up to
~100MB)
What's the best way to go about this?
10KB is already big compared to the size of regular data in a row (e.g. a numeric, or a few characters).
It would be more problematic if you had for instance strings that can be short (e.g 20 chars), but also very long (e.g. 3000 characters). In this case you would pay the price of the LOB for small strings (which could be stored in the main data of the table). But 10KB is anyway relatively large, so the usage of a LOB is really justified.
Notes:
You might consider to use a stream (e.g. InputStream) instead of a byte array. This might improve the performance of your application, and you won't be constrained by memory (what happens when you have 20 concurrent users which access a 100MB LOB?).
You might also consider asking your DBA how to tune the LOB storage. Most database have parameters for this. Here is for instance the Oracle LOB Performance Guideline.