views:

221

answers:

5

I'm using MSVE, and I have my own tiles I'm displaying in layers on top. Problem is, there's a ton of them, and they're on a network server. In certain directories, there are something on the order of 30,000+ files. Initially I called Directory.GetFiles, but once I started testing in a pseudo-real environment, it timed out.

What's the best way to programatically list, and iterate through, this many files?

Edit: My coworker suggested using the MS indexing service. Has anyone tried this approach, and (how) has it worked?

+1  A: 

I've worked on a SAN system in the past with telephony audio recordings which had issues with numbers of files in a single folder - that system became unusable somewhere near 5,000 (on Windows 2000 Advanced Server with an application in C#.Net 1.1)- the only sensible solution that we came up with was to change the folder structure so that there were a more reasonable number of files. Interestingly Explorer would also time out!

The convention we came up with was a structure that broke the structure up in years, months and days - but that will depend upon your system and whether you can control the directory structure...

Ronnie
(I'm now at a new company, but I do remember parts of this project.)The problem is, AFAIK, MSVE had a specific folder structure that it was expecting - and everything for one layer went into one folder. We didn't look into it enough to know whether we'd be able to aggregate multiple folders so that MSVE could look in all of them, or look in specific folders at specific levels.
Adam V
A: 

None. .NET relies on underlying Windows API calls that really, really hate that amount of files themselves.

As Ronnie says: split them up.

Stu
A: 

You could use DOS?

DIR /s/b > Files.txt
GateKiller
A: 

Definitely split them up. That said, stay as far away from the Indexing Service as you can.

Greg Hurlman
We never got around to trying to use the indexing service, fortunately. And the team I was part of is now down to one person. So I think this part of the project has dropped way down on the priority list.
Adam V
A: 

You could also look at either indexing the files yourself, or getting a third part app like google desktop or copernic to do it and then interface with their index. I know copernic has an API that you can use to search for any file in their index and it also supports mapping network drives.

lomaxx