I asked a similar question about 6 months ago when I was first starting out in Django.
http://stackoverflow.com/questions/1133550/is-it-worth-it-using-the-built-in-django-admin-for-a-decent-sized-project
For that particular application I opted not to use Django admin, and that was a very smart decision in hindsight. Since then, I've generally not used it, but sometimes there are situations when it's great. For me, it really depends on the users. If we are building a custom data-driven app for a client and are working off a feature set provided by them, I would never want to use the Django admin. In those cases, almost certainly they will have changes that could be a real pain to try to get working in the admin. And if it's an evolving project, these changes will become more and more of a hack and you'd probably end up having to start ripping out pieces into non-Django admin parts of the site, at which point there are now two interfaces for doing stuff.
However, if the clients are more of the mindset of accepting the way the app works, then the Django admin would be fine, assuming there's generally a 1:1 correspondence between your tables and the data they're dealing with.
As Brian Luft said, it's really easy to create interfaces for CRUD apps, so if you sense you'll need any customization in the future, it might be easiest just to write your own from the start. You can always keep the django admin around for your own needs as a super-user. That's usually what I do, so I can easily have table-level access to change fields that might not be shown in the normal user admin.