views:

76

answers:

5

I have a project that consists of a bunch of dynamically loaded modules. Originally, everything was always built with MSVC 2003, but lately I've been working on getting it to work with GCC. Everything has been going pretty smoothly, except for one problem. For 64-bit code, GCC and MSVC don't agree about what a va_list is. For 32-bit, things seem to line up fine. The problem the 64-bit mismatch causes is when a module built with one compiler has a public function with a va_list parameter and that function is called from a module built by the other compiler.

The spec says nothing about what a va_list is, outside of Section 7.15 Variable arguments <stdarg.h>, paragraph 3:

The type declared is

va_list

which is an object type suitable for holding information needed by the macros va_start, va_arg, va_end, and va_copy.

That paragraph just means this is all compiler dependent stuff - so, is there a way to make these two compilers agree on the contents of a 64-bit va_list? For least impact to my system, making GCC match the MSVC va_list would be best, but I'll take any solution I can get.

Thanks for helping out!

Edit:

I did some 32-bit testing, and I have problems there too, which surprised me since there are supposedly no ABI differences between any 32-bit Intel platforms. The MSVC codebase I'm using defines all of the variadic function macros as:

typedef char *va_list;
#define intsizeof(n)    ((sizeof(n) + sizeof(int) - 1) &~(sizeof(int) - 1))
#define va_start(ap, v) (ap = (va_list)&(v) + intsizeof(v))
#define va_arg(ap, t)   (*(t *) ((ap += intsizeof(t)) - intsizeof(t)))
#define va_end(ap)      (ap = (va_list)0)

I've simplified a bit from the real project, but this is the code I was using for my test. With GCC, this code definitely isn't correctly getting my arguments. Maybe it is just a bug, as Zack suggests below?

Edit again:

I get working results for the following 32-bit test application with -O0, -O0, and -O2, but not -O3, -Os, and -Oz:

typedef char *va_list;
#define intsizeof(n)    ((sizeof(n) + sizeof(int) - 1) &~(sizeof(int) - 1))
#define va_start(ap, v) (ap = (va_list)&(v) + intsizeof(v))
#define va_arg(ap, t)   (*(t *) ((ap += intsizeof(t)) - intsizeof(t)))
#define va_end(ap)      (ap = (va_list)0)

int printf(const char *format, ...);

int f(int n, ...)
{
  int r = 0;
  va_list ap;

  va_start(ap, n);  
  while (n--)
    r = va_arg(ap, int);
  va_end(ap);

  return r;
}

int main(int argc, char **argv)
{
  int r;

  r = f(1, 1, 2, 3, 4, 5);
  printf("%x\n", r);

  r = f(2, 1, 2, 3, 4, 5);
  printf("%x\n", r);

  r = f(3, 1, 2, 3, 4, 5);
  printf("%x\n", r);

  r = f(4, 1, 2, 3, 4, 5);
  printf("%x\n", r);

  r = f(5, 1, 2, 3, 4, 5);
  printf("%x\n", r);

  return 0;
}
A: 

Because there is no standard on how va_args have to be handlded, if you need this functionality to be consistant in a cross compiled platform, you'd probably be better off rolling your own version. We didn't do this and have been burned multiple times recently as a result when supporting additional targets for our codebase. I'd love to be wrong though if others have a better solution :)

Michael Dorgan
OK, so how do I roll my own version? I wrote one that works for IA32, but gcc doesn't seem to be spilling the arguments the way I want it to for Win64.
Carl Norum
A: 

Try running it through an assembly level debugger such as ollydbg, as your problem might not be with the va_args, but rather with the way the compiler is expecting the args to be passed(gcc could be expecting them in the linux format, where as msvc is using win64 __fastcall), if anything this will give a little more light to the situation. Another, rather hacky approach, is try fiddling with the definitions used for the 64 bit args, such as importing the msvc macros into the gcc header(use a project local copy of course), see if that remedies anything.

Necrolis
Variadic function calls between modules built with the different compilers, so the ABI is matching up. It's just the actual internal implementation of the `va_list` that's biting me.
Carl Norum
+1  A: 

Since there doesn't seem to be an ABI for va_list (or at least MSVC and GCC don't agree on an ABI), you'll probably need to marshal those parameters yourself.

The most straightforward way I can think of to work around this problem off the top of my head is to marshal your variable parameters in a dynamically allocated block of memory and pass a pointer to that block.

Of course, this has the drawback of completely changing the interface to the functions that are currently using va_args.

Michael Burr
But I'd have to know how many parameters there *are* to stick them in a dynamic block. That means every function that wants to do `va_start` and then call another function with the `va_list` has to know how to identify them - in the case of `printf`-like functions, that means a lot of extra parsing, where the entire point of passing the `va_list` is to consolidate all of that format string parsing in a single function (like `vsprintf`).
Carl Norum
+3  A: 

Since MSVC defines the Win64 ABI, you have found a bug in GCC. Please report it in GCC bugzilla.

Zack
A `va_list` isn't part of the ABI is it? Anyway my longer-term plan is to use clang/llvm, so I guess I should check that out sooner than later. It's quite possible that GCC is ok in more recent versions - I'm stuck on a kind of 'special' old GCC in this case.
Carl Norum
Positively `va_list` is part of the ABI. The ABI's purpose is to ensure all the compilers for a given target CPU+OS generate compatible code, and that includes variadic functions. ... I have seen a bunch of Win64 support patches for GCC go by pretty recently, so this may well work with the current release or at least the development tree.
Zack
@Zack, variadic functions work fine - just passing the `va_list` itself doesn't work. Thanks for the sanity check - I'll do some more testing and see what goes on.
Carl Norum
That, too, really is supposed to work - consider `vfprintf` and similar - but there are some gotchas. Try taking the address of the `va_list` object and passing that, rather than passing the object itself. (Short version of the gotchas: after you pass a va_list object by value to another function, it is in an indeterminate state in the caller, and the only portable thing to do with it is immediately call va_end on it. This is not the case if you pass a pointer to the object instead.)
Zack
Thanks @Zack. I just did some 32-bit tests, and there's a mismatch there too. I've been reading a lot of ABI and calling convention documentation, and I don't see anything explicit about `va_list` anywhere. As far as MSVC is concerned, a `va_list` is a pointer type anyway, so it's passing by reference anyway, right?
Carl Norum
`va_list` is indeed a pointer in everyone's x86-32 ABI, but I don't know if it still is in Win64 (it isn't in x86-64/ELF). The 32-bit tests should have worked.
Zack
Oh, and ... passing a pointer to a pointer is very much not the same as passing a pointer.
Zack
@Zack - oh I see what you mean now. I will try to come up with a smaller test case to try that out.
Carl Norum
@Zack - thanks for the help, and sorry about that IA32 detour. I tried clang/llvm and everything works fine, so it definitely points to a bug in the compiler. I'll file with the appropriate people.
Carl Norum
A: 

What types are you using? Strictly speaking, WinXX only defines behavior for characters, strings, pointers, and integers for varargs (see documentation for wsprintf in user32.dll). So, if you pass down floating point values or structures, the results are technically unspecified for the Windows platform.

MSN
There's no floating point of any kind in this project, so that should be ok. I would be surprised if structures were getting flung around, but I can look into that. In general only integer types and strings are getting printed.
Carl Norum
I think this answer is just plain wrong, btw... that `user32.dll`'s version of `wsprintf` doesn't mention "%f" and friends in its documentation does *not* mean that you can't pass floating point through varargs on win32 in general. And the C standard does require passing both floating point and structures through varargs to work.
Zack
@Zack, I updated my answer to reflect that this is not specified for the Windows platform. It does mean that you shouldn't pass those types to wsprintf(...) which is a user32.dll function. It doesn't mean anything for a language implemented on top of Windows, such as C or C++. So if you want compatibility between compilers, you should stick to types that the platform forces you to support. In this case, you are lucky enough that wsprintf was exported from user32.dll.
MSN
@MSN: WinXX cannot claim to be a conforming implementation of C (and Microsoft definitely does make that claim) if it doesn't support passing floating point types and structures through varargs. And, unless there's another implementation of `wsprintf` somewhere that everyone gets instead of that one, that one probably *does* support floating point, and you're basing all your assertions on an oversight in the documentation. I sent their documentation-feedback address a query about it.
Zack
@Zack, WinXX is not a conforming implementation of C. There are bindings for the C language which have semantics that are defined by the WinXX platform. `wsprintf` is one of those functions that uses varargs that is specified to only take pointers, strings, and integers. If you google "wsprintf windows floating point", you'll see that it doesn't support floating point values. It is basically a deprecated function, but it does implicitly specify the ABI for varargs for integers and pointers. The C standard does not specify a particular ABI for varargs on any platform, in case you are wondering.
MSN
@Zack, I see what you mean now. The x64 ABI is technically defined here (http://msdn.microsoft.com/en-us/library/dd2wa36c.aspx), although I could not find a canonical Windows x64 ABI. I suppose the Visual C++ one is the canonical one.
MSN