STL containers have value semantics. When you push an object into an STL container, the STL container keeps it's own copy of the object, and when the object (internal copy) is removed from the container it is destroyed.
If you used a container of a proxy type, as raw pointers, smart pointers (shared_ptr, weak_ptr), or adapters (as boost::reference_wrapper), then the STL container will destroy the proxy but not the type. Choosing one over the others is usually a matter of how you want to deal with resources.
The most common idiom is using raw pointers, but they don't explicit who is in charge of destruction (the code that pulls from the container should delete the pointer, or the resource is handled somewhere else?).
Modern usage moves towards the shared_ptr approach, as it dilutes the ownership problem. The object will be guaranteed to be alive when you take it out of the container, and if nobody else holds a shared_ptr then the object will automatically be deleted when the local shared_ptr goes out of scope. Using a weak_ptr will keep the ownership in the original code, but will allow you to check for validity of the pointer (if it was deleted) before usage. This could allow you to avoid performing the operation on an object that will be removed right away.
The problem with the shared_ptr/weak_ptr approach is that it forces you to use shared_ptr to hold the original resource. This means that you will not be able to put a pointer into a subobject (member attribute) of another class without redesigning the class to hold the attribute through a shared_ptr, and that will have other implications (the attributes will no longer be contiguous in memory, more dynamic allocation operations will be required...)
A technique that is hardly seen is using adapters as boost::reference_wrapper<>. A reference wrapper is a proxy object that contains a reference to the original object and is itself copyable. The advantage over plain raw pointers is that reading the code it is clear that the resource is managed outside of the queue: the code that pulls data from the queue does not need to delete the object. The advantage over the smart pointer approach is that you do not need to redesign other parts of your system to use smart pointers. The disadvantage is that, as in the raw pointer approach, you must ensure that the lifetime of the referred object outlives the reference in the container manually.