Delphi DLL that is compatible with other programming languages

Question

I want to build a DLL that exports functions that return a string. This DLL should work with other programming languages!! I have found all kind of nasty solutions/hacks to this, best one is to make my function return Pchar then call another function contained in the same DLL ( let's call it ReleaseMemory) to release the memory reserved for PChar.

Anyway, recently I have discovered FastShareMem library. It says it can do exactly what I want WITHOUT calling the ReleaseMemory. On the other side FastMM seems to do the same AS LONG as both the DLL and the application uses FastMM as memory manager. This instantly kill the chance of using FastMM as memory manger for my universal DLL. Right?

====================

FastShareMem (http://www.codexterity.com/fastsharemem.htm), Delphi 7, Windows XP 32 bits, Windows 7 64 bits

Answer 1

If you return a Delphi string, then your DLL will not work with other programming languages because no other programming languages use Delphi's string type. It doesn't matter how you allocate memory if the types aren't the same. If you're working with text, follow the model of the Windows API and use plain old character pointers.

The solution you found — to return a pointer and then provide another function for your DLL to free the memory — is not a hack and is not nasty at all. It's a perfectly ordinary solution, and nobody using your DLL will bat an eye when they see it. The FormatMessage API function uses a similar model: It allocates a string for you, and it specifies that strings it allocates must be freed with LocalFree.

It doesn't really matter what memory manager you use as long as you're consistent and consumers of your DLL can use it. One approach is to specify Windows API functions for allocating and freeing strings, such as LocalAlloc and LocalFree, or SysAllocString and SysFreeString. Another way is to never allocate anything at all — if the caller needs you to return a string, the caller provides the buffer and tells you how big it is. If the buffer is too small, then you return the required size so the caller can re-allocate the buffer and re-call the function. For an example of that, see GetLongPathName.

FastSharemem provides a long explanation of how Delphi's memory manager works, and then it says you can avoid all the trouble by simply using that unit in your program. But remember what I said above: consumers of your DLL need to be able to use the same memory manager you use. When the consumer of your DLL isn't written in Delphi, then it can't use the FastSharemem unit. FastSharemem is fine in a homogenous Delphi environment, but it suffers all the same pitfalls as any other memory manager when used in a mixed environment.

Answer 2

What happens is basically this. Every piece of code compiled separately (DLL or EXE) contains its own code which allocates memory from the system and manages it, called the memory manager. Simply speaking, when that piece of coded is initialized, it allocates a big block of memory from the system. Later, when it does GetMem or allocates strings, arrays et cetera, memory manager marks parts of that big block as used. When you FreeMem/deallocate them, they're marked as unused.

Now imagine you have EXE and DLL, both with their own memory managers. EXE calls DLL procedure, DLL allocates a string (PChar), thus marking a part of it's grand memory block as used. It then returns the pointer to the EXE, which uses it and later decides to free. EXE gives the pointer to its own memory manager and asks to free it, but its not even from EXE's grand memory block! EXE's memory manager does not know how to "Free" someone else's memory.

That's why you need to call DllReleaseString(), thus returning the borrowed memory pointer to the DLL and letting DLL's own internal memory manager free it.

Now, what Sharing Memory Managers do is, they connect to each other. Memory manager in your DLL and memory manager in your EXE know how to talk to each other, and when you give DLL's memory pointer to EXE's memory manager, it understands it's from DLL and lets DLL memory manager release it. Of course that's possible only when BOTH DLL and EXE memory managers are built from the same memory manager code (or else they wouldn't recognize each other!). If your DLL memory manager is a sharing one, and your EXE memory manager is something else, DLL memory manager will not be able to "ask" EXE to release memory, and EXE memory manager will not even try (it's not sharing).

Therefore, if you want your DLL to be universal, you cannot rely on memory managers talking to each other. Your DLL might be used with EXEs or DLLs which rely on a different memory manager, maybe written in a different language altogether. Having sharing memory managers is possible only when you control all parts of your project and can explicitly setup one and the same manager everywhere.

Answer 3

You are mixing two different scenarios:

1. Delphi applications using Delphi DLLs
2. Any application using Delphi DLLs

In the first scenario, unless you mix Delphi version or do something weird, the memory manager is the same, and the compiler as well. Thereby there are ways to share the memory manager, and then the compiler is able to handle allocations/deallocations properly. That's the scenario in which both FastMM and FastShareMem works - and only this one.

In the second scenario, the application and the DLL will use different memory managers, maybe very different ones, and there is usually no way to share one. In such a situation the best approach is never to return a PChar allocated inside the DLL, even if you provide a deallocation function, because you can't be sure what the calling language would do later with your PChar on its own, and if the caller has any chance to call the proper deallocation routine before the compiler/interpreter. COM works somewhat in the way you said, but it enforces memory allocation/deallocation through its own memory manager, thereby it's safe. You can't enforce it with plain DLLs, thereby it is not safe.

The best approach is let the calling language pass you a buffer large enough, and write your PChar there. Of course you need to have some way to tell the caller what size the buffer should be. That's the way Windows itself works, and there are good reasons why they made this choice.

Answer 4

Hello, I'm the author of FastSharemem and I'd like to contribute my 2 cents' worth. Rob is right, FastSharemem assumes that the modules will all be written in Delphi. Passing data between modules in different languages can be tricky, especially with dynamic data like strings.

This is why the Windows API is often a pain to work with when dealing with complex data structures, and it's also one reason why Microsoft's COM (OLE) provides its own memory management functions and special types; the goal is binary compatibility between modules compiled from different source.

So, since Windows has already done it before, you could use one of the two ways that Windows does it. Either:

1) Expose a C-style API (PChars etc) and specify API in painstaking detail. You could either expose memory-allocation routines that clients would need to call, or have clients do the allocation. The Windows API does both at different times. Clients may also need an SDK to talk to your module conveniently, and remember to use the stdcall calling convention uniformly.

Or,

2) Use COM types and to pass data in and out. Delphi has excellent, almost transparent COM support. For example, for strings you could use COM's BSTR (WideString in Delphi).

Hope this helps.