Testing pointers for validity (C/C++)

Question

Is there any way to determine (programatically, of course) if a given pointer is "valid"? Checking for NULL is easy, but what about things like 0x00001234? When trying to dereference this kind of pointer an exception/crash occurs.

A cross-platform method is preferred, but platform-specific (for Windows and Linux) is also ok.

Update for clarification: The problem is not with stale/freed/uninitialized pointers; instead, I'm implementing an API that takes pointers from the caller (like a pointer to a string, a file handle, etc.). The caller can send (in purpose or by mistake) an invalid value as the pointer. How do I prevent a crash?

Answer 1

AFAIK there is no way. You should try to avoid this situation by always setting pointers to NULL after freeing memory.

Answer 2

There isn't any portable way of doing this, and doing it for specific platforms can be anywhere between hard and impossible. In any case, you should never write code that depends on such a check - don't let the pointers take on invalid values in the first place.

Answer 3

Take a look to this and this question. Also take a look to smart pointers.

Answer 4

IsBadReadPtr(), IsBadWritePtr(), IsBadCodePtr(), IsBadStringPtr() for Windows.
These take time proportional to the length of the block, so for sanity check I just check the starting address.

Answer 5

On Win32/64 there is a way to do this. Attempt to read the pointer and catch the resulting SEH exeception that will be thrown on failure. If it doesn't throw, then it's a valid pointer.

The problem with this method though is that it just returns whether or not you can read data from the pointer. It makes no guarantee about type safety or any number of other invariants. In general this method is good for little else other than to say "yes, I can read that particular place in memory at a time that has now passed".

In short, Don't do this ;)

Raymond Chen has a blog post on this subject: http://blogs.msdn.com/oldnewthing/archive/2007/06/25/3507294.aspx

Answer 6

There are no provisions in C++ to test for the validity of a pointer as a general case. One can obviously assume that NULL (0x00000000) is bad, and various compilers and libraries like to use "special values" here and there to make debugging easier (For example, if I ever see a pointer show up as 0xCECECECE in visual studio I know I did something wrong) but the truth is that since a pointer is just an index into memory it's near impossible to tell just by looking at the pointer if it's the "right" index.

There are various tricks that you can do with dynamic_cast and RTTI such to ensure that the object pointed to is of the type that you want, but they all require that you are pointing to something valid in the first place.

If you want to ensure that you program can detect "invalid" pointers then my advice is this: Set every pointer you declare either to NULL or a valid address immediately upon creation and set it to NULL immediately after freeing the memory that it points to. If you are diligent about this practice, then checking for NULL is all you ever need.

Answer 7

Preventing a crash caused by the caller sending in an invalid pointer is a good way to make silent bugs that are hard to find.

Isn't it better for the programmer using your API to get a clear message that his code is bogus by crashing it rather than hiding it?

Answer 8

Setting the pointer to NULL before and after using is a good technique. This is easy to do in C++ if you manage pointers within a class for example (a string):

class SomeClass
{
public:
    SomeClass();
    ~SomeClass();

    void SetText( const char *text);
    char *GetText() const { return MyText; }
    void Clear();

private:
    char * MyText;
};


SomeClass::SomeClass()
{
    MyText = NULL;
}


SomeClass::~SomeClass()
{
    Clear();
}

void SomeClass::Clear()
{
    if (MyText)
        free( MyText);

    MyText = NULL;
}



void SomeClass::Settext( const char *text)
{
    Clear();

    MyText = malloc( strlen(text));

    if (MyText)
        strcpy( MyText, text);
}

Answer 9

Why do you keep voting down valid, answers to this question??

Anyway

Update for clarification: The problem is not with stale/freed/uninitialized pointers; instead, I'm implementing an API that takes pointers from the caller (like a pointer to a string, a file handle, etc.). The caller can send (in purpose or by mistake) an invalid value as the pointer. How do I prevent a crash?

You can't make that check. There is simply no way you can check whether a pointer is "valid". You have to trust that when people use a function that takes a pointer, those people know what they are doing. If they pass you 0x4211 as a pointer value, then you have to trust it points to address 0x4211. And if they "accidentally" hit an object, then even if you would use some scary operation system function (IsValidPtr or whatever), you would still slip into a bug and not fail fast.

Start using null pointers for signaling this kind of thing and tell the user of your library that they should not use pointers if they tend to accidentally pass invalid pointers, seriously :)

Answer 10

I have seen various libraries use some method to check for unreferenced memory and such. I believe they simply "override" the memory allocation and deallocation methods (malloc/free), which has some logic that keeps track of the pointers. I suppose this is overkill for your use case, but it would be one way to do it.

Answer 11

In general its impossible to do. Here's one particularly nasty case:

struct Point2d {
    int x;
    int y;
};
struct Point3d {
    int x;
    int y;
    int z;
};
void dump(Point3 *p)
{
    printf("[%d %d %d]\n", p->x, p->y, p->z);
}

Point2d points[2] = { {0, 1}, {2, 3} };
Point3d *p3 = reinterpret_cast<Point3d *>(&points[0]);
dump(p3);

On many platforms, this will print out:

[0 1 2]

You're forcing the runtime system to incorrectly interpret bits of memory, but in this case it's not going to crash, because the bits all make sense. This is part of the design of the language (look at C-style polymorphism with struct inaddr, inaddr_in, inaddr_in6), so you can't reliably protect against it on any platform.

Answer 12

Technically you can override operator new (and delete) and collect information about all allocated memory, so you can have a method to check if heap memory is valid. but:

1. you still need a way to check if pointer is allocated on stack ()

2. you will need to define what is 'valid' pointer:

a) memory on that address is allocated

b) memory at that address is start address of object (e.g. address not in the middle of huge array)

c) memory at that address is start address of object of expected type

Bottom line: approach in question is not C++ way, you need to define some rules which ensure that function receives valid pointers.

Answer 13

Firstly, I don't see any point in trying to protect yourself from the caller deliberately trying to cause a crash. They could easily do this by trying to access through an invalid pointer themselves. There are many other ways - they could just overwrite your memory or the stack. If you need to protect against this sort of thing then you need to be running in a separate process using sockets or some other IPC for communication.

We write quite a lot of software that allows partners/customers/users to extend functionality. Inevitably any bug gets reported to us first so it is useful to be able to easily show that the problem is in the plug-in code. Additionally there are security concerns and some users are more trusted than others.

We use a number of different methods depending on performance/throughput requirements and trustworthyness. From most preferred:

• separate processes using sockets (often passing data as text).

• separate processes using shared memory (if large amounts of data to pass).

• same process separate threads via message queue (if frequent short messages).

• same process separate threads all passed data allocated from a memory pool.

• same process via direct procedure call - all passed data allocated from a memory pool.

We try never to resort to what you are trying to do when dealing with third party software - especially when we are given the plug-ins/library as binary rather than source code.

Use of a memory pool is quite easy in most circumstances and needn't be inefficient. If YOU allocate the data in the first place then it is trivial to check the pointers against the values you allocated. You could also store the length allocated and add "magic" values before and after the data to check for valid data type and data overruns.

Answer 14

Regarding the answer a bit up in this thread:

IsBadReadPtr(), IsBadWritePtr(), IsBadCodePtr(), IsBadStringPtr() for Windows.

My advice is to stay away from them, someone has already posted this one: http://blogs.msdn.com/oldnewthing/archive/2007/06/25/3507294.aspx

Another post on the same topic and by the same author (I think) is this one: http://blogs.msdn.com/oldnewthing/archive/2006/09/27/773741.aspx ("IsBadXxxPtr should really be called CrashProgramRandomly").

If the users of your API sends in bad data, let it crash. If the problem is that the data passed isn't used until later (and that makes it harder to find the cause), add a debug mode where the strings etc. are logged at entry. If they are bad it will be obvious (and probably crash). If it is happening way to often, it might be worth moving your API out of process and let them crash the API process instead of the main process.

Answer 15

There is no way to make that check in C++. What should you do if other code passes you an invalid pointer? You should crash. Why? Check out this link: http://blogs.msdn.com/oldnewthing/archive/2006/09/27/773741.aspx

Answer 16

Regarding your update, it's not a very good policy to accept arbitrary pointers as input parameters in a public API. It's better to have "plain data" types like an integer, a string or a struct (I mean a classical struct with plain data inside, of course; officially anything can be a struct).

Why? Well because as others say there is no standard way to know whether you've been given a valid pointer or one that points to junk.

But sometimes you don't have the choice - your API must accept a pointer.

In these cases, it is the duty of the caller to pass a good pointer. NULL may be accepted as a value, but not a pointer to junk.

Can you double-check in any way? Well, what I did in a case like that was to define an invariant for the type the pointer points to, and call it when you get it (in debug mode). At least if the invariant fails (or crashes) you know that you were passed a bad value.

// API that does not allow NULL
void PublicApiFunction1(Person& in_person)
{
  assert(in_person != NULL);
  assert(in_person.Invariant());

  // Actual code...
}

// API that allows NULL
void PublicApiFunction2(Person& in_person)
{
  assert(in_person == NULL || in_person.Invariant());

  // Actual code (must keep in mind that in_person may be NULL)
}

Answer 17

As others have said, you can't reliably detect an invalid pointer. Consider some of the forms an invalid pointer might take:

You could have a null pointer. That's one you could easily check for and do something about.

You could have a pointer to somewhere outside of valid memory. What constitutes valid memory varies depending on how the run-time environment of your system sets up the address space. On Unix systems, it is usually a virtual address space starting at 0 and going to some large number of megabytes. On embedded systems, it could be quite small. It might not start at 0, in any case. If your app happens to be running in supervisor mode or the equivalent, then your pointer might reference a real address, which may or may not be backed up with real memory.

You could have a pointer to somewhere inside your valid memory, even inside your data segment, bss, stack or heap, but not pointing at a valid object. A variant of this is a pointer that used to point to a valid object, before something bad happened to the object. Bad things in this context include deallocation, memory corruption, or pointer corruption.

You could have a flat-out illegal pointer, such as a pointer with illegal alignment for the thing being referenced.

The problem gets even worse when you consider segment/offset based architectures and other odd pointer implementations. This sort of thing is normally hidden from the developer by good compilers and judicious use of types, but if you want to pierce the veil and try to outsmart the operating system and compiler developers, well, you can, but there is not one generic way to do it that will handle all of the issues you might run into.

The best thing you can do is allow the crash and put out some good diagnostic information.

Answer 18

Addendum to the accpeted answer(s):

Assume that your pointer could hold only three values -- 0, 1 and -1 where 1 signifies a valid pointer, -1 an invalid one and 0 another invalid one. What is the probability that your pointer is NULL, all values being equally likely? 1/3. Now, take the valid case out, so for every invalid case, you have a 50:50 ratio to catch all errors. Looks good right? Scale this for a 4-byte pointer. There are 2^32 or 4294967294 possible values. Of these, only ONE value is correct, one is NULL, and you are still left with 4294967292 other invalid cases. Recalculate: you have a test for 1 out of (4294967292+ 1) invalid cases. A probability of 2.xe-10 or 0 for most practical purposes. Such is the futility of the NULL check.

Answer 19

You know, a new driver (at least on Linux) that is capable of this probably wouldn't be that hard to write.

On the other hand, it would be folly to build your programs like this. Unless you have some really specific and single use for such a thing, I wouldn't recommend it. If you built a large application loaded with constant pointer validity checks it would likely be horrendously slow.