The Django apps come with their own features and design. If your requirements don't match 100% with the features of the contib app, you end up customizing and tweaking the app. I feel this involves more effort than just building your own app to fit your requirements.
What do you think?
It all depends. We had a need for something that was 98% similar to contrib.flatpages. We could have monkeypatched it, but we decided that the code was so straightforward that we would just copy and fork it. It worked out fine.
Doing this with contrib.auth, on the other hand, might be a bad move given its interaction with contrib.admin & contrib.session.
Most of the apps in django.contrib are written very well and are highly extensible.
Don't like quite how comments works? Subclass the models and forms within it, adding your own functionality and you have a working comment system that fits your sites schema, with little effort.
I think the best part when you extend the contrib apps is you're not really doing anything hacky, you're just writing (mostly) regular Python code to add the functionality.
I'd also check out third-party re-usable apps before building my own. Many are listed on Django Plug(g)ables, and most are hosted on Google Code, GitHub or BitBucket.