Pretty bizarre. The iterator correctly implements IDisposable, it calls FindClose(). The AllDirectories options could be a source of trouble since FindFileFirst/Next only allows iterating a single directory. But I'm seeing the iterator doing the right thing, it only keeps a single handle open while iterating the directory structure.
The MSDN article specifically mentions "if there is an open handle that remains on one of the enumerated directories or files". FindFileFirst/Next won't leave a handle open. But sloppy user code that reads files while enumerating does. "a delete operation on a file or directory" is relevant too, I think the behavior changed in Vista. A DeleteFile() can succeed but the file won't actually disappear until all handles on the file are closed.
We need somebody to volunteer and not implement this code on XP. I reckon we'll find someone soon :)