Persist a pidl (ITEMIDLIST)

Question

I'd like to persist pidls between sessions, so that my application can remember the users' folder selections, wherever they may be in the namespace, even if they're not file-system folders.

I have a feeling that the way to do this is to write out the binary contents of the ITEMIDLIST itself, but I can't confirm this for sure, since these contents are supposed to be opaque, and are up to the provider. I don't know if after a reboot, or even in another process, if this data is valid. It could contain pointers, for all I know.

What is the proper way to persist and later reconstruct a pidl?

Update:

Jerry Coffin has suggested a pair of functions that seem to do exactly what I asked. One question remains, however.

As Joel Spolsky points out, Raymond Chen seems to imply that saving the binary contents of the ITEMIDLIST is indeed the proper way to persist a pidl, from which one can infer that ILSaveToStream and ILLoadFromStream are helper functions that do just that.

I can't find documentation that proves this, however. Since this project is in C#, I'd prefer to avoid having to interop up an IStream for the IL... functions and just persist the binary data myself if possible. Can anybody confirm that this is correct?

Solution notes:

Looking at the docs for ILSaveToStream and ILLoadFromStream, I see that these functions didn't even exist until version 5.0 of the shell (Windows 2000). So how was this done prior to Win2K? After some testing, I conclude that, as I suspected and Joel Spolsky postulated, writing out the raw ITEMIDLIST is the way to go.

A simple implementation in C# follows:

unsafe{
    byte* start = (byte*)pidl.ToPointer();
    byte* ptr   = start;
    ushort* length;

    do{
        length = (ushort*)ptr;
        ptr += *length;
    }while(*length != 0);

    byte[] rtn = new byte[ptr + 2 - start];
    Marshal.Copy(pidl, rtn, 0, rtn.Length);
    return rtn;
}

Of course, this could be done without pointers using Marshal.ReadInt16:

int offset = 0;
int length;

do{
    length = Marshal.ReadInt16(pidl, offset);
    offset += length;
}while(length != 0);

byte[] rtn = new byte[offset + 2];
Marshal.Copy(pidl, rtn, 0, rtn.Length);
return rtn;

It only costs a few more clock cycles, but it still requires full trust, so it doesn't really buy much besides staying away from scaaaary-looking pointers.

Reconstructing the pidl is even easier, since the total length of the data is already known, and it doesn't even need any pointers:

byte[] itemidlist = ReadPidl();
IntPtr pidl = Marshal.AllocCoTaskMem(itemidlist.Length);
Marshal.Copy(itemidlist, 0, pidl, itemidlist.Length);

Persisting and reconstructing pidls in this way worked in all of my tests across processes, and, in limited scenarios, even across machines. I haven't yet tested across reboots, as I'm loath to close everything and restart my machine, but given the apparent cross-machine compatibility, I'm confident in this solution.

I'm accepting Joel Spolsky's answer as the solution, but do want to give a caveat for future passersby: Joel talks about writing out a SHITEMID structure, but this is not the whole story. An ITEMIDLIST (which is what a pidl points to) is actually a null-terminated list of these variable-length SHITEMID structures, and the whole list must be persisted. This is why the code above executes a loop to determine the total length. It jumps from element to element in this list, reading the length of each element to find out the offset to the next one. Only after an element length of zero is read is the length of the entire list known.

Answer 1

According to Raymond Chen you can persist a pidl -- or, more specifically, a SHITEMID structure, just by writing out length of the item and then the bytes.

Notice that this struct is a typical Windows variable length struct, with a cb ("count of bytes") element specifying the length of the structure in bytes, followed by the rest of the data. In other words, to write the structure, you need to write cb bytes. To read it, you need to allocated cb bytes of memory and set the cb field.

Be careful not to use sizeof(SHITEMID) because the way it's declared assumes only one byte for the abID field so that won't be big enough.

Answer 2

ILSaveToStream and ILLoadFromStream (and possibly ILLoadFromStreamEx). There is one caveat though: ILLoadFromStream and ILLoadFromStreamEx are deprecated, and if Microsoft has said what's supposed to replace them, I'm not aware of it.

Answer 3

There isn't standard on the internal of pidls. After all, pidl is only used to locate items. As long as the pidl is recognized (or fail predictably) by the PIDL generator & resolver (Windows Explorer or shell namespace extensions), Explorer.exe is happy. Nobody says a PIDL must be persistable across machine reboots or user desktops.

In one case you can make an educated guess if the pidl is persistable across machine reboots is to check the SFGAO_CANLINK attribute. By declaring this attribute, the PIDL resolver declares you can drag an item in the folder to create a desktop shortcut which contains the PIDL of the item. The resolver must handle persistence and backward compatibility in the PIDL otherwise double clicking the shortcut would crash the Explorer process. Buggy programming happens so don't take this for granted.

In the dark old days pidl are just persisted in byte arrays, or when passing across COM objects are needed, passed in VARIANT as safe arrays of bytes.