Duplicating file handles between processes error

Question

A DLL has the following shared variables (I'm using MinGW):

int iCount __attribute__((section(".str"), shared)) = 0;
HANDLE hMainFile __attribute__((section(".shr"), shared)) = NULL;
HANDLE hProcess __attribute__((section(".shr"), shared)) = NULL;

and the global variable:

HANDLE hFile = NULL;

This is how I'm handling my DLL_PROCESS_ATTACH:

case DLL_PROCESS_ATTACH:
  if(!iCount)
  {
    hMainFile = CreateFile("Hello.txt", GENERIC_WRITE, 0, NULL, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
    hFile = hMainFile;
    hProcess = GetCurrentProcess();
    iCount = 1;
  }
  else
  {
    DuplicateHandle(hProcess, hMainFile, GetCurrentProcess(), &hFile, 0, FALSE, DUPLICATE_SAME_ACCESS);
  }
  break;

As you can see, the first instance of the DLL will create the file and set the shared file handle. The rest of the DLL instances should duplicate the original file handle to one that is compatible for its instance. However, DuplicateHandle is always giving an error of "The handle is invalid". I'm confused because I don't know which handle it's talking about. I have confirmed that the shared variables are indeed the same between all instances. Can somebody tell me what I'm doing wrong here?

Answer 1

You can't share handles like that. Handle sharing is Win32 is much more restrictive than you might imagine.

To successfully share a handle in Win32, you need to do the following:

• When calling the API to open the object (CreateFile for files) pass in a SECURITY_ATTRIBUTES structure, with bInheritHandle set to TRUE.
• Only once all the sharable objects have been created, launch the child process with CreateProcess, ensuring that the bInheritHandles parameter is TRUE.

This will duplicate the handles in the child process, ensuring that the duplicated handles have the same values.

This only works if you can abide by the restriction that the second process MUST be launched by the first process, and only after all potentially shared objects have been opened.

However, in the more general case where the processes might be started in any order, or need to share access to objects opened after the other process is launched: You cannot directly share the handles in that case - you MUST open the objects by namein both processes - obviously using flags that are compatible with the object being opened multiple times.

In the case of CreateFile: FILE_SHARE_READ|FILE_SHARE_WRITE as the fSharingMode parameter would be necessary to open "Hello.txt" twice.