Smart pointer are most useful when it is difficult to predict when an object should be deleted. For example: if you create an object in one point, and the object might be delete in another very distant point, or more importantly, might be deleted by several different places, smart pointers is the best solution. So basically, unless you can say for sure when an object should be deleted, and it will always be safe to do so (i.e. no other object holds a pointer to that object) use smart pointers.
Another point of view, which some friends use, is that smart pointers are so cheap (i.e. the processing cost of using them is very small), smart pointer should always be used when objects are allocated on the heap (i.e. using new). That way, you'll never have to worry about memory leakage or double-frees, etc.