Actually...
C++ has something called the "as if" principle. All the guarentees made referenced in all of these answers only refer to the observable behavior. The compiler is allowed to ellude, reorder, add, etc.. any function call, as long as the observable behavior is as if it had executed as originally written. This also applies to destructors.
So, technically, your observation is correct: the compiler is allowed to destruct the object earlier, if it detects it is not used, and there are no observable side effects from the destructor or any function it calls. But, you are guarenteed to not be able to tell this is happening outside of a debugger, because if you were able to tell, the compiler would no longer be able to do it.
It's more likely the compiler uses this power to do something useful like completely ellude a trivial destructor rather than actually reorder destructor calls, however.
Edit: Someone wanted a reference... 1.9/5, along with footnote 4 of the C++0x draft standard (this isn't a new rule, I just don't have the C++03 standard handy. It's also present in the C standard, AFAIK)
1.9/5:
A conforming implementation executing a well-formed program shall produce the same observable behavior
as one of the possible execution sequences of the corresponding instance of the abstract machine with the
same program and the same input. However, if any such execution sequence contains an undefined operation,
this International Standard places no requirement on the implementation executing that program with that
input (not even with regard to operations preceding the first undefined operation).
Footnote 4:
This provision is sometimes called the “as-if” rule, because an implementation is free to disregard any requirement of this
International Standard as long as the result is as if the requirement had been obeyed, as far as can be determined from the
observable behavior of the program. For instance, an actual implementation need not evaluate part of an expression if it can
deduce that its value is not used and that no side effects affecting the observable behavior of the program are produced.
My reading (and what I thought was the general understanding) was that this is what enables the compiler free hand to do whatever it wants (ie, enables optimizations), as long as the observable behavior is that of the original written source - including moving around destructors, not destructing objects at all, inventing destructors, etc.