ansaurus

Question

Fast Linux File Count for a large number of files

Answer 1

+1 A:

Did you try find? For example:

find . -name "*.ext" | wc -l

igustin 2009-09-15 13:12:00

This will *recursively* find files under the current directory.

mark4o 2009-09-15 14:01:00

On my system, `find /usr/share | wc -l` (~137,000 files) is about 25% faster than `ls -R /usr/share | wc -l` (~160,000 lines including dir names, dir totals and blank lines) on the first run of each and at least twice as fast when comparing subsequent (cached) runs.

Dennis Williamson 2009-09-15 14:01:23

If he want only current directory, not the whole tree recursively, he can add -maxdepth 1 option to find.

igustin 2009-09-15 14:47:30

Answer 2

+2 A:

You could try if using opendir() and readdir() in Perl is faster. For an example of those function look here

Peter van der Heijden 2009-09-15 13:16:36

usage: perl -e 'opendir D, "."; @files = readdir D; closedir D; print scalar(@files)'

glenn jackman 2009-09-15 14:11:23

Answer 3

+12 A:

By default ls sorts the names, which can take a while if there are a lot of them. Also there will be no output until all of the names are read and sorted. Use the ls -f option to turn off sorting.

ls -f | wc -l

Note that this will also enable -a, so ., .., and other files starting with . will be counted.

mark4o 2009-09-15 13:55:32

+1 And I thought I knew everything there was to know about `ls`.

mobrule 2009-09-15 13:58:53

ZOMG. Sorting of 100K lines is nothing - compared to the `stat()` call `ls` does on every file. `find` doesn't `stat()` thus it works faster.

Dummy00001 2010-07-19 20:03:29

`ls -f` does not `stat()` either. But of course both `ls` and `find` call `stat()` when certain options are used, such as `ls -l` or `find -mtime`.

mark4o 2010-07-19 23:46:37

ansaurus

tags:

views:

answers:

Fast Linux File Count for a large number of files

related questions