views:

994

answers:

5

I have a DLL compiled with D2007 that has functions that return AnsiStrings.

My application is compiled in D2009. When it calls the AnsiString functions, it gets back garbage.

I created a little test app/dll to experiment and discovered that if both app and dll are compiled with the same version of Delphi (either 2007 or 2009), there is no problem. But when one is compiled in 2009 and the other 2007, I get garbage.

I've tried including the latest version of FastMM in both projects, but even then the 2009 app cannot read AnsiStrings from the 2007 dll.

Any ideas of what is going wrong here? Is there a way to work around this?

+10  A: 

The internal structure of AnsiStrings changed between Delphi 2007 and Delphi 2009. (Don't get upset; that possibility has been present since day 1.) A Delphi 2009 string maintains a number indicating what code page its data is in.

I recommend you do what every other DLL on Earth does and pass character buffers that the function can fill. The caller should pass a buffer pointer and a number indicating the size of the buffer. (Make sure you're clear about whether you're measuring the size in bytes or characters.) The DLL function fills the buffer, writing no more than the given size, counting the terminating null character.

If the caller doesn't know how many bytes the buffer should be, then you have two options:

  • Make the DLL behave specially when the input buffer pointer is null. In that case, have it return the required size so that the caller can allocate that much space and call the function a second time.

  • Have the DLL allocate space for itself, with a predetermined method available for the caller to free the buffer later. The DLL can either export a function for freeing buffers that it has allocated, or you can specify some mutually available API function for the caller to use, such as GlobalFree. Your DLL must use the corresponding allocation API, such as GlobalAlloc. (Don't use Delphi's built-in memory-allocation functions like GetMem or New; there's no guarantee that the caller's memory manager will know how to call Free or Dispose, even if it's written in the same language, even if it's written with the same Delphi version.)

Besides, it's selfish to write a DLL that can only be used by a single language. Write your DLLs in the same style as the Windows API, and you can't go wrong.

Rob Kennedy
It's good to know that 2009 changed the ansistring definition, but rewriting DLLs makes the port to 2009 all the more onerous, and given the amount of code involved, will probably kill efforts to port to 2009 in our shop. Are there any other work arounds?
Zartog
@Zartog: This has nothing to do with porting to Delphi 2009, and everything with having used improper data types in the DLL API. Trying to use the DLL from .NET or another language != Delphi would result in similar problems. Cleaning up the code base would be a good idea, even without a port to Delphi 2009. See "Technical Debt", for example http://martinfowler.com/bliki/TechnicalDebt.html
mghie
+2  A: 

OK, so haven't tried it, so a big fat disclaimer slapped on this one.

In the help viewer, look at the topic (Unicode in RAD Stufio) ms-help://embarcadero.rs2009/devcommon/unicodeinide_xml.html

Returning the Delphi 2007 string to Delphi 2009, you should get two problems.

First, the code page mentioned by Rob. You can set this by declaring another AnsiString and calling StringCodePage on the new AnsiString. Then assign that to the old AnsiString by calling SetCodePage. That should work, but if it doesn't there is hope still.

The second problem is the element size which will be something completely mad. It should be 1, so make it 1. The issue here is that there is no SetElementSize function to lean on.

Try this:

var
  ElemSizeAddr: PWord; // Need a two-byte type
  BrokenAnsiString: AnsiString; // The patient we are trying to cure
...
  ElemSizeAddr := Pointer(PAnsiChar(BrokenAnsiString) - 10);
  ElemSizeAddr^ := 1; // The size of the element

That should do it!

Now if the StringCodePage/SetCodePage thing didn't work, you can do the same as above, changing the line where we get the address to deduct 12, instead of 10.

It has hack scribbled all over it, that's why I love it.

You are going to need to port those DLLs eventually, but this makes the port more manageable.

One final word - depending on how you return the AnsiString (function result, output parameter, etc) you may need to first assign the string to a different AnsiString variable just to make sure there is no trouble with memory being overwritten.

Cobus Kruger
The SetCodePage idea won't work. The memory returned by the DLL doesn't have space allocated to hold a code page, so the EXE can't set it. The value returned by the DLL is not an AnsiString at all, as far as the EXE is concerned, so it is not valid to use any AnsiString operations on it. Subtracting 10 from the returned address puts you into unallocated memory. It does not belong to the string at all. (Well, it's allocated, but it's used for bookkeeping by the memory manager.)
Rob Kennedy
Note the "One final word" bit in my answer. Get the value returned from the DLL, assign it to another AnsiString, which has memory allocated but receives garbage. And then fix this second AnsiString.Did I word it better this time?
Cobus Kruger
String assignment just increments the reference count, unless it also tries to do a conversion between the source and destination code pages (but remember that the source code page is garbage). If you assign to a RawByteString, and then call UniqueString on it, then you might be able to get a valid string. But you'll still be unable to free the source string. The EXE will subtract the wrong amount from the pointer to get the base address, so the EXE's memory manager will try to free the wrong value.
Rob Kennedy
Cobus: Well, thanks for the attempt. It seems like a conversion could be done, but I don't think that's going to be a realistic option for me.I confess, I'm a little disappointed that Embarcadero apparently decided not to provide a backward compatible type that could be used to ease transisition. That is going to make things pretty difficult for me.
TrespassersW
Rob:You're right - that did slip by me. But all's not lost.Do it like this:var MyNewStr: AnsiString;MyNewStr := UniqueString(BrokenAnsiString);And then fix MyNewStr. I did say untested and big disclaimer, heh?TrespassersW: Yeah, the hack gets worse the more you need it. You could of course create a set of overloads for the imports and lessen the pain that way. How much that could help will depend on the number of functions and how they're defined.The alternative is lots of manual labour, but that is probably the right way.
Cobus Kruger
Oh, and one last thing about freeing the string returned by the DLL. One could probably do it yourself if the reference count is 1. It still is only a block of memory in the end. But yeah, that makes the hack even nastier.
Cobus Kruger
No, it's not "only a block of memory." It's a block of memory with an address owned by a particular memory manager. In this case, it's owned by the DLL's memory manager, with all the underlying data structures to go with it. You can't free it yourself without having access to the memory manager's internals. The DLL's memory manager isn't necessarily the same as the host application's.
Rob Kennedy
A: 

You'll likely just need to convert the DLL to 2009. According to Embarcadero, the conversion to 2009 is 'easy' and should take you no time at all.

Darian Miller
A: 

Your DLL should not be returning AnsiString values to begin with. The only way that would work correctly in the first place is if both DLL and EXE were compiled with the ShareMem unit, and even then only if they are compiled with the same Delphi version. D2007's memory manager is not compatible with D2009's memory manager (or any other cross-version use of memory managers), AFAIK.

Remy Lebeau - TeamB
FastMM has made ShareMem obsolete. Ever since it was incorporated into Delphi, you've been able to share AnsiStrings between DLLs without ShareMem. I didn't decide the API of these DLLs. So all these comments about what should have been done differently by someone else years ago aren't really all that helpful. I appreciate their intent, but I'm more interested in solutions at this point.
TrespassersW
One thing to consider is that Delphi incorporates a default build of FastMM with default settings applied. Users can override it with a full build of FastMM that has been customized to the user's needs. How would that affect the abilty to share memory between a DLL and EXE if they are not using the same FastMM build?
Remy Lebeau - TeamB
If you include your own FastMM, there's a backwards compatible option that you can set to make sure it will still work with the built-in one. But you're right, if your app or dll used its own FastMM without setting that option then things would break.
TrespassersW
A: 

I agree with Rob and Remy here: common Dlls should return PAnsiChar instead of AnsiStrings.

If the DLL works OK compiled with D2009, why simply doesn't stop compiling it with D2007 and start compiling it with D2009 once and for all?

Fabricio Araujo
You've misunderstood. These DLLs (there are hundreds of them) have not been converted. Converting them would be an enormous task.
TrespassersW