When working on my current project, I had a similar problem of just not being able to find a solid, reliable growing text area plugin. I ended up settling on growfield, but I can't say that I'm too happy with it.
One problem that I ran into that I was able to address was that growfield (in addition to almost every other plugin) insisted on adding an extra line to the text area at all times. This is rough development code, and currently only addresses Safari and Firefox, but here's some code I added to the end of growfield's getDummyHeight() function:
if ($.browser.safari) return h-sr.lh*2+sr.pt+sr.pb;
if ($.browser.mozilla) return h-sr.lh*2+sr.pt*2+sr.pb+5;
It uses browser detection (which is bad practice), and is probably dependent on our layout and CSS, but you might be able to do some tweaking to get it work well in your design.
In case you were wondering (growfield is a bit cryptic), h refers to the textarea height, sr.lh refers to the line height, sr.pt refers to the top padding and sr.pb refers to the bottom padding. The full list of variables is in the prepareSizeRelated() function.