const is about program semantics and not about implementation details. You should mark a member function const when it does not change the visible state of the object, and should be callable on an object that is itself const. Within a const member function on a class X, the type of this is X const *: pointer to constant X object. Thus all member variables are effectively const within that member function (except mutable ones). If you have a const object, you can only call const member functions on it.
You can use mutable to indicate that a member variable may change even within a const member function. This is typically used to identify variables used for caching results, or for variables that don't affect the actual observable state such as mutexes (you still need to lock the mutex in the const member functions) or use counters.
class X
{
int data;
mutable boost::mutex m;
public:
void set_data(int i)
{
boost::lock_guard<boost::mutex> lk(m);
data=i;
}
int get_data() const // we want to be able to get the data on a const object
{
boost::lock_guard<boost::mutex> lk(m); // this requires m to be non-const
return data;
}
};
If you hold the data by pointer rather than directly (including smart pointers such as std::auto_ptr or boost::shared_ptr) then the pointer becomes const in a const member function, but not the pointed-to data, so you can modify the pointed-to data.
As for caching: in general the compiler cannot do this because the state might change between calls (especially in my multi-threaded example with the mutex). However, if the definition is inline then the compiler can pull the code into the calling function and optimize what it can see there. This might result in the function effectively only being called once.
The next version of the C++ Standard (C++0x) will have a new keyword constexpr. Functions tagged constexpr return a constant value, so the results can be cached. There are limits on what you can do in such a function (in order that the compiler can verify this fact).