If terms (admin OR user created) are duplicates, I don't see a way to delete them from within the Drupal browser interface.
This becomes a problem during, say, adding products to an ecommerce site that breaks their products up into categories. The person adding the product sees multiple duplicate categories (which are terms within a Vocabulary) and doesn't know which one to choose.
You can delete ALL terms in a lists by virtue of deleting a Vocabulary, but that's not an ideal solution.