tags:

views:

823

answers:

6
void problem3(void) {
    int overflowme[16];
    int x = (int) problem3; // x is the address of the first instr for problem3
    printf("hello world\n");
    overflowme[17] = x;

I'm wondering what does the (int) do in C programming.

+2  A: 

It's an explicit cast. You are casting the value of problem3 to an integer and then assigning that integer value to x.

Note that this does not actually change the value of problem3.

Matthew Jones
+14  A: 

It's a typecast, and tells the compiler "Ignore the type that problem3 really has, and deal with it as if it were typed as an int".

In this example, problem3 has a function pointer type, so normally the compiler would reject the program (Using a function pointer when an integer is expected is normally a programmer error). The typecast forces a different interpretation - the programmer is stepping in and saying "I know what I'm doing".

Adam Wright
and just because they say they know what they're doing doesn't mean they actually do. This is a great way to shoot yourself in the foot. In this case, especially when you recompile for a 64 bit system.
rmeador
It should also be noted that the semantics of pointer->int cast is entirely implementation defined, and the implementation need not guarantee that it produces a meaningful result; in particular, it may well be that `sizeof(int) != sizeof(void(*)(void))` - this is true in practice on most 64-bit OSes, including Windows and Linux (and excepting those few that use ILP64) - so the comment in the code is, generally speaking, wrong.
Pavel Minaev
It's generally speaking wrong anyway, since `problem3` isn't necessarily what the comment says even before the cast. A function pointer isn't required to be the address of the first instruction of the function. It's an abstract representation, and the calling convention can specify whatever it likes as the value. In practice, of course, the address or something very like it is needed to perform the call, so a function pointer at a bare minimum does incorporate the address, and usually is just the address.
Steve Jessop
A: 

It means that problem3 is converted to type int before assigning to the int x

Raj More
Not to be picky but problem3 isn't a variable, it's a function name that evaluates to a function pointer value in this context.
Graphics Noob
tx for being picky, i had completely missed that.
Raj More
Why will `x` always be zero just because the function has a specific type? X is still assigned the address of the function (or tried to).
Johannes Schaub - litb
@litb: i answered like it was problem3(). that's what i get for switching from c to vb
Raj More
A: 

It's a typecast ie. it converts the variable/constant following it into the specified type. Here, a void (*) (void) type is converted into an int (thing in the braces)

Aviral Dasgupta
actually, the type is `void (*)(void)` - you have one `void` too many
Christoph
fixed it -- thanks...
Aviral Dasgupta
A: 

As others have noted this is just explicit cast. It just changes type of variable into int type.

But from code you posted it looks like this function is preparing for some kind of buffer overflow or something. What is the rest of this function ?

Michal Sznajder
It appears to attempt to overwrite the return address on the stack with the address of problem3( ); if successful, this would just put the program into an loop that will eventually underflow the stack. It will likely fail on many systems because of padding forced by alignment restrictions on the stack pointer, and may on others where the return address is not passed on the stack (ARM, for example). There's probably at least one system out there where it succeeds, though.
Stephen Canon
And even if not, it possibly serves as step 1 in a "beginners' guide to why stack overflow bugs end up in your box being pwned". Next step is to see that what the attacker ideally wants is to write the value of `overflowme+something` to `overflowme[17]` (where the function problem3 has copied some malicious input data into the buffer, and the attacker has arranged for that data to contain machine instructions at offset `something`). And why that's hard, but attackers can get there the long way round, and what the counter-measures are that OSes can take, and so on.
Steve Jessop
+1  A: 

It's a type cast - it's a form of converting the type of the operand (problem3 in your example) to another type.

In C (and in C++ when a 'C-style cast is used), the cast can perform one of several things:

  • do nothing but change the type of something without changing the form of the data. For example, when you cast a pointer to an int.
  • perform a conversion as part of the cast operation. For example, when casting a float to an int, the data is actually transformed from the form used to represent floating point values (usually an exponent/mantissa form) to a plain old integer (with any fractional part lost)

Because the different forms of casting can be confusing or unclear as to what's happening (or intended to happen), C++ added several specific casting operators:

  • reinterpret_cast<>() which corresponds to the first form described above
  • static_cast<>() which corresponds to the second form (even if the conversion doesn't result in a change of the internal data format)
  • const_cast<>() which is a special case of casting that is able to remove the const or volatile qualifiers that might be applied to an object
  • dynamic_cast<>() which is entirely new to C++ and has no similar functionality in C. This operator is used to safely 'downcast' a base object type to one of its derived types.

Because they're inherently dangerous, casts are generally considered bad form. When you perform a cast operation, you're subverting the compiler's ability to perform type checking. However, there are times when it might be necessary or very useful, and you'll see it used often in C code.

In your example, problem3 is a pointer to a function, and the cast is 'converting' the address for that function to an int. It's then storing that address-as-int into the array, but actually one array element past the end of the array (which is a no-no). On many platforms that invalid array element is where the return address for the the function is stored, so what will happen is when the problem3() function returns, it'll return to itself and run again (ad-infinitum - sort of).

It'll eventually underflow the stack because the new, 'hacked' run of problem3() won't have a function call that put a return address on the stack - it'll just trash whatever else was on the stack before it and return to itself again, repeating the process until the stack underflows which will likely cause a processor exception.

Michael Burr