views:

114

answers:

4

I would like to switch my application to LARGEADDRESSAWARE. One of issues to watch for is pointer arithmetic, as pointer difference can no longer be represented as signed 32b.

Is there some way how to find automatically all instances of pointer subtraction in a large C++ project?

If not, is there some "least effort" manual or semi-automatic method how to achieve this?

+1  A: 

This is only a problem if you have 2 pointers that are more than 2000 million bytes (2GB) apart. This means that you:

  • either have very large arrays (> 2GB)
  • or you are subtracting pointers that point to totally different structures

So look for these special cases.

I think that in most cases this is not a problem.

Patrick
"or you are subtracting pointers that point to totally different structures" - this is something which can happen easily. We are sometimes using pointers to objects as secondary keys when sorting them to make sure sorting order is stable. As for "look for these special cases" - how can I look for them other then review every single piece of the code?
Suma
Comparing the pointers is not a problem, subtracting them is, and for the sort, comparing the pointers should be sufficient.Also, it's not a good idea to use pointers as the final comparison. I used to do this in the past as well, but this could give problems (problems can become hard to reproduce since the pointers can be different at every run of your application; trust me, I had this problem myself several times). Try to find another characteristic (e.g. a line number, a database key, ...) to get a stable sort.
Patrick
> "Comparing the pointers is not a problem, subtracting them is,"Unfortunately, when implementing comparison function for qsort, returning difference seems to be quite natural, and I have seen it several times already.Nice comment, unfortunately the main questions is still not answered - how to find the cases? The codebase is huge, it is impossible to find all cases just by remembering them. Some systematic approach is needed, either automated, or manual (systematic code review). The solution can be compile time, a dedicated tool, or runtime, with some instrumentation.
Suma
Fair point: it's possible that you end up with `a<b<c<d` but `d-a < 0`
MSalters
Notice that if you subtract two pointers, you will get the 'offset' in number of elements in between the pointers. This works fine for pointers pointing to elements in an array, but gives strange results if the two pointers are unrelated. The result may not be integer. Suppose your pointers are pointing to a class of 1000 bytes. If ptr1 points to 50000, and ptr2 points to 60100, then a subtract will probably return 10, but that's incorrect. See https://www.securecoding.cert.org/confluence/display/seccode/ARR36-C.+Do+not+subtract+or+compare+two+pointers+that+do+not+refer+to+the+same+array.
Patrick
You can pretty much strike the first point. The OP wants to enable LARGEADDRESSAWARE, which means that it's not enabled yet... Which means that the process' address space is restricted to 2GB.
jalf
A: 

As our code already compiles with GCC, I think perhaps the fastest way might be:

  • build a GCC
  • create a custom modification of GCC so that it prints warning (or error) whenever pointer subtraction is detected
  • build the project and gather all warnings about pointer subtraction

Here is the outline of changes which need to be done to GCC for this:

Add your warnings into:

  • c-typeck.c (pointer_diff function)
  • cp/typeck.c (pointer_diff function).

Besides of directly detecting pointer subtraction, another thing to do can be to detect cases where you first convert pointers to integral types and then subtract them. This may be more difficult depending on how is your code structured, in out case regexp search for (.*intptr_t).-.-(.*intptr_t) has worked quite well.

Suma
+1  A: 

Compile the code with a 64 bit compiler and Wp64 turned on.

Because pointers are 64bit wide, but int, long, DWORD etc. stay 32 bit wide, you get warnings for shorting a ptrdiff_t to a int32_t

Christopher
This answer is not relevant, because it catches problems that may arise if you use the wrong type to store pointers, but it doesn't warn you if you are making "dangerous" pointer subtractions/comparisons.
Matteo Italia
Nice idea. It will not find cases where the pointer subtraction is assigned to ptrdiff_t, still those cases are a problem as well, as ptrdiff_t (int on Win32) is not enough to represent pointer difference - but those can be searched for by text searching for ptrdiff_t. The real problem is there are too many such warnings in the code, solving each of them seems quite a lot of work and vast majority of them is not related LARGEADDRESSAWARE at all (we have no intention to port the code to Win64).
Suma
Are you claiming that there's a compilation mode where ptrdiff_t isn't large enough to hold the difference between two pointers? That sounds… well, plain broken, given that that's the standard _definition_ of that type. (I would expect it to be of a different size in the different compilation modes.)
Donal Fellows
+2  A: 

PC-Lint can find this kind of problem.

Look at http://gimpel-online.com/MsgRef.html, error code 947:

Subtract operator applied to pointers -- An expression of the form p - q was found where both p and q are pointers. This is of special importance in cases where the maximum pointer can overflow the type that holds pointer differences. For example, suppose that the maximum pointer is 3 Gigabytes -1, and that pointer differences are represented by a long, where the maximum long is 2 Gigabytes -1. Note that both of these quantities fit within a 32 bit word. Then subtracting a small pointer from a very large pointer will produce an apparent negative value in the long representing the pointer difference. Conversely, subtracting a very large pointer from a small pointer can produce a positive quantity.

Patrick