tags:

views:

330

answers:

5

Below is code which includes a variadic function and calls to the variadic function. I would expect that it would output each sequence of numbers appropriately. It does when compiled as a 32-bit executable, but not when compiled as a 64-bit executable.

#include <stdarg.h>
#include <stdio.h>

#ifdef _WIN32
#define SIZE_T_FMT "%Iu"
#else
#define SIZE_T_FMT "%zu"
#endif


static void dumpargs(size_t count, ...) {

    size_t i;
    va_list args;

    printf("dumpargs: argument count: " SIZE_T_FMT "\n", count);

    va_start(args, count);

    for (i = 0; i < count; i++) {

        size_t val = va_arg(args, size_t);
        printf("Value=" SIZE_T_FMT "\n", val);
    }
    va_end(args);
}

int main(int argc, char** argv) {

    (void)argc;
    (void)argv;

    dumpargs(1, 10);
    dumpargs(2, 10, 20);
    dumpargs(3, 10, 20, 30);
    dumpargs(4, 10, 20, 30, 40);
    dumpargs(5, 10, 20, 30, 40, 50);

    return 0;
}

Here is the output when compiled for 64-bit:

dumpargs: argument count: 1
Value=10
dumpargs: argument count: 2
Value=10
Value=20
dumpargs: argument count: 3
Value=10
Value=20
Value=30
dumpargs: argument count: 4
Value=10
Value=20
Value=30
Value=14757395255531667496
dumpargs: argument count: 5
Value=10
Value=20
Value=30
Value=14757395255531667496
Value=14757395255531667506

Edit:

Please note that the reason the variadic function pulls size_t out is because the real-world use of this is for a variadic function that accepts a list of pointers and lengths. Naturally the length argument should be a size_t. And in some cases a caller might pass in a well-known length for something:

void myfunc(size_t pairs, ...) {
    va_list args;
    va_start(args, count);

    for (i = 0; i < pairs; i++) {
        const void* ptr = va_arg(args, const void*);
        size_t len = va_arg(args, size_t);
        process(ptr, len);
    }
    va_end(args);
}

void user(void) {
    myfunc(2, ptr1, ptr1_len, ptr2, 4);
}

Note that the 4 passed into myfunc might encounter the problem described above. And yes, really the caller should be using sizeof or the result of strlen or just plain put the number 4 into a size_t somewhere. But the point is that the compiler is not catching this (a common danger with variadic functions).

The right thing to do here is to eliminate the variadic function and replace it with a better mechanism that provides type safety. However, I would like to document this problem, and collect more detailed information as to exactly why this problem exists on this platform and manifests as it does.

+4  A: 

The problem is that you're using size_t to represent the type of the values. This is incorrect, the values are actually normal 32 bit values on Win64.

Size_t should only be used for values which change size based on the 32 or 64 bit-ness of the platform (such as pointers). Change the code to use int or __int32 and this should fix your problem.

The reason this works fine on Win32 is that size_t is a different sized type depending on the platfrom. For 32 bit windows it will be 32 bits and on 64 bit windows it will be 64 bit. So on 32 bit windows it just happens to match the size of the data type you are using.

JaredPar
Perhaps my example didn't have an appropriate real-world signature: I ran into this problem with my variadic function that accepts a list of pointer/length pairs. Naturally the length argument should be size_t. However, in some cases well-known lengths would be passed into the function, and fail.
Jared Oberhaus
The question is, why does it work with the first three values? Is it something with the ABI? I know that on linux, some registers are used for passing arguments, could it be that the three were passed in registers and the rest (which doesn't work) on the stack?
jpalecek
+1, but note size_t is not "for values which change size on the 32 or 64-bit-ness", it's for things that are size_t. Ptrdiff_t, various sizes of int, etc., will change size, but don't use size_t for that!
dwc
@J. Oberhaus: it's OK to use size_t, as long as you ensure the calling site will pass a size_t as well, ie. write (size_t)10 instead of 10. When you pass 10 in C, it's an int. The same applies to pointers, if you want to pass NULL ptr, it must be (void*)0, not 0.
jpalecek
@jpalecek That was one of my burning questions as to why: does this happen on all/most compilers? Does the C spec say anything about a guarantee, or does it ignore this issue and assume 100% of the arguments are correct?
Jared Oberhaus
+1  A: 

The reason for this is because size_t is defined as a 32-bit value on 32-bit Windows, and a 64-bit value on 64-bit Windows. When the 4th argument is passed into the variadic function, the upper bits appear to be uninitialized. The 4th and 5th values that are pulled out are actually:

Value=0xcccccccc00000028
Value=0xcccccccc00000032

I can solve this problem with a simple cast on all the arguments, such as:

dumpargs(5, (size_t)10, (size_t)20, (size_t)30, (size_t)40, (size_t)50);

This does not answer all my questions, however; such as:

  • Why is it the 4th argument? Likely because the first 3 are in registers?
  • How does one avoid this situation in a type-safe portable manner?
  • Does this happen on other 64-bit platforms, using 64-bit values (ignoring that size_t might be 32-bit on some 64-bit platforms)?
  • Should I pull out the values as 32-bit values regardless of the target platform, and will that cause problems if a 64-bit value is pushed into the variadic function?
  • What do the standards say about this behavior?

Edit:

I really wanted to get a quote from The Standard, but it's something that's not hyperlink-able, and costs money to purchase and download. Therefore I believe quoting it would be a copyright violation.

Referencing the comp.lang.c FAQ, it's made clear that when writing a function that takes a variable number of arguments, there's nothing you can do for type safety. It's up to the caller to make sure that each argument either perfectly matches or is explicitly cast. There are no implicit conversions.

That much should be obvious to those who understand C and printf (note that gcc has a feature to check printf-style format strings), but what's not so obvious is that not only are the types not implicitly cast, but if the size of the types don't match what's extracted, you can have uninitialized data, or undefined behavior in general. The "slot" where an argument is placed might not be initialized to 0, and there might not be a "slot"--on some platforms you could pass a 64-bit value, and extract two 32-bit values inside the variadic function. It's undefined behavior.

Jared Oberhaus
+1  A: 

If you are the one writing this function, it is your job to write the variadic function correctly and/or correctly document your function's calling conventions.

You already found that C plays fast-and-loose with types (see also signedness and promotion), so explicit casting is the most obvious solution. This is frequently seen with integer constants being explicitly defined with things like UL or ULL.

Most sanity checks on passed values will be application-specific or non-portable (e.g. pointer validity). You can use hacks like mandating that pre-defined sentinel value(s) be sent as well, but that's not infallible in all cases.

Best practice would be to document heavily, perform code reviews, and/or write unit tests with this bug in mind.

HUAGHAGUAH
+2  A: 

A variadic function is only weakly type checked. In particular, the function signature does not provide enough information for the compiler to know the type of each argument assumed by the function.

In this case, size_t is 32-bits on Win32 and 64-bits on Win64. It has to vary in size like that in order to perform its defined role. So for a variadic function to pull arguments out correctly which are of type size_t, the caller had to make certain that the compiler could tell that the argument was of that type at compile-time in the calling module.

Unfortunately 10 is a constant of type int. There is no defined suffix letter that marks a constant to be of type size_t. You could hide that fact inside a platform-specific macro, but that would be no clearer than writing (size_z)10 at the call site.

It appears to work partially because of the actual calling convention used in Win64. From the examples given, we can tell that the first four integral arguments to a function are passed in registers, and the rest on the stack. That allowed count and the first three variadic parameters to be read correctly.

However it only appears to work. You are actually standing squarely in Undefined Behavior territory, and "undefined" really does mean "undefined": anything can happen. On other platforms, anything can happen too.

Because variadic functions are implicitly unsafe, a special burden is placed on the coder to make certain that the type of each argument known at compile time matches the type that argument will be assumed to have at run time.

In some cases where the interfaces are well known, it is possible to warn about type mismatch. For example, gcc can often recognize that the type of an argument to printf() doesn't match the format string, and issue a warning. But doing that in the general case for all variadic functions is hard.

RBerteig
+4  A: 

So basically, if a function is variadic, it must conform to a certain calling convention (most importantly, the caller must clean up args, not the callie, since the callie has no idea how many args there will be).

The reason why it starts happening on the 4th is because of the calling convention used on x86-64. To my knowledge, both visual c++ and gcc use registers for the first few parameters, and then after that use the stack.

I am guessing that this is the case even for variadic functions (which does strike me as odd since it would make the va_* macros more complicated).

On x86, the standard C calling convention is the use the stack always.

Evan Teran
+1 for answering "why the 4th parameter".
Jared Oberhaus