How to create a "Shell IDList Array" to support drag-and-drop of virtual files from C# to Windows Explorer?

Question

I started trying to implement drag-and-drop of virtual files (from a C# 4/WPF app) with this codeproject tutorial. After spending some time trying to figure out a DV_E_FORMATETC error, I figured out I need to support the "Shell IDList Array" data format. But there seems to be next to zero documentation on what this format actually does.

After some searching, I found this page on advanced data transfer which said that a Shell IDList Array was a pointer to a CIDA structure. This CIDA structure then contains the number of PIDLs, and a list of offsets to them. So what the hell is a PIDL? After some more searching, this page sort of implies it's a pointer to an ITEMIDLIST, which itself contains a single member that is a list of SHITEMIDs.

I did some more reading, I still can't tell what multiple PIDLs are for, but there's one SHITEMID for each "level" in the path. At least that's one mystery solved.

My next idea was to try dragging a file from another application with virtual files (WinSCP). I just got a MemoryStream back for this format. At least I know what class to provide for the thing, but that doesn't help at all for explaining what to put in it. I tried examining this MemoryStream based on the formats in the links above, but I've had no success. I just got back garbage data, and one of the 'cb' fields told me it was 18000 bytes long when the whole stream was only 539 bytes.

In addition, upon further reading, this page seems to imply that the data contained in a PIDL actually winds up being a path, and examining the contents of said MemoryStream in a hex editor yielded a path inside my local Temp directory (split into parts, anyway).

It seems that WinSCP just uses a shell extension to handle dropping on explorer, something I really don't want to resort to. However, it has an alternate mode where it aborts the dragdrop until after it transfers to a temp folder - this is acceptable to me, but I haven't the faintest idea how to do it.

My questions are now:

1. What is a valid "abID" member for a SHITEMID? These virtual files only exist with my program; will the thing they are dragged to pass the "abID" back later when it executes GetData? Do they have to be system-unique?
2. How can I manage the abort-and-redo-later drag and drop thing that WinSCP does?
3. If I have to implement a shell extension for this, how would I even go about that? I'm sure I can easily find explanations of shell extension theory but how would I support custom PIDLs or whatever?

Any help or even links that explain what I should be doing would be greatly appreciated.

Edit: So here's how I actually did it in the end:

I scrapped most of the code from the CodeProject tutorial above, keeping the functions for creating the FileGroupDescriptor. I then reimplemented the .Net IDataObject interface (and actually, I didn't have to use the COM IDataObject interface at all). I then am forced to synchronously download the file in the background, and pass a MemoryStream back out from GetData(). Conveniently, the actual copying is in the background, but the waiting for data is in the foreground. Thanks, explorer. Any reasonably large files are slow, but for now it "works" which is more than I can say for the past few weeks.

I did try passing the PipeStream class I use internally for transfers, but explorer isn't happy with this:

• It tries to Seek, despite this class having CanSeek be false.
• It ignores the size I send in the FileGroupDescriptor and just downloads whatever's in the stream right now.

So, I'm not sure the ideal case is actually possible. Still, thanks for all of your help. I'm giving the bounty to JPW because his replies eventually set me on the right path.

Answer 1

Disclaimer: I'm neither an expert in Windows Shell programming nor in COM programming, and especially not in COM programming using .NET.

Disclaimer 2: Shell programming is rather complex and Windows Explorer is likely to crash. (Be sure to read the information on MSDN on debugging shell extensions.)

This is what I can provide to answer your questions:

1. The abID value of a SHITEMID is dertermined by the folder which contains the corresponding item. Therefore there a no valid or invalid information as this data is only meaingful to the folder. Windows Explorer will pass this data in order to identify a specific item. abID must not be system-unique but it will probably be unique in its containing folder (though this is not guaranteed). Under certain circumstances the SHITEMID may be persisted, so in some cases it must be valid after a reboot (because of this memory references in abID should be avoided).
2. As I understand it, IDataObject::GetData will only be called if the actual data is required (and i may be wrong with this). So it should be fine, if you extract the data to a temp file in GetData and return a PIDL to that file (see http://support.microsoft.com/kb/132750/en-us, though I remember that there was a convinient method for this offered by the shall, but I can't remember its name). Unfortunately I can't see a good way on when to delete the temp file (as a last resort you could do this when the IDataObject is destroyed, but this will result in may temp files which are not needed anymore).
3. You would probably need to support custom PIDLs (including a custom PIDL manager) and implement a custom IShellFolder. Most documentation on shell extensions, however, is for C++ and many functions are not documented very well or not documented at all. For more information on PIDLs I would recommend reading http://msdn.microsoft.com/en-us/library/cc144090.aspx . Furthermore you should search for "Shell namespace extension" on the Internet as there might be some information on the topic. Also have a look into the examples provided by Windows SDK on this topic.

I hope some information will be usefull to you.

Edit: You could also implement QueryGetData and EnumerableFormatEtc appropriately to indicate that your object does not support the format in question. Maybe Windows Explorer tries a different format then.

Edit 2: Though you've already settled on a solution, here is a some more information on how to drag and drop asynchronously (for large files), maybe you (or at least somebody looking for an answer to a simulary problem, who reads this) find it useful: http://msdn.microsoft.com/en-us/library/bb776904(VS.85).aspx#async

Answer 2

Have you seen this article? How to drag a virtual file from your app into Windows Explorer The source code is in c++ but it should help you to get the idea how all the stuff works.