How can I get the last changed directory in Perl?

Question

Apache version 2.2.11 (Unix) Architecture x86_64 Operating system Linux Kernel version 2.6.18-164.el5

Ok, here is what I have working. However, I may not be using File::Util for anything else in the rest of the script.

My directory names are 8 digits starting at 10000000 . I was comparing the highest found number with stat last created as a double check but, overkill I believe.

Another issue is that I did not know how to slap a regex in the list_dir command so only 8 digits eg m!^([0-9]{8})\z!x) could reside in that string. Reading the man, the example reads ....'--pattern=\.txt$') but, my futile attempt: '--pattern=m!^([0-9]{8})\z!x)') well, was just that.

So, would there be a "better" way to grab the latest folder/directory?

use File::Util;
my($f) = File::Util->new();
my(@dirs) = $f->list_dir('/home/accountname/public_html/topdir','--no-fsdots');
my @last = (sort { $b <=> $a } @dirs); 
my $new = ($last[0]+1);
print "Content-type: text/html\n\n";
print "I will now create dir $new\n";

And.. How would I ignore anything not matching my regex?

I was thinking an answer may reside in ls -d as well but, as a beginner here, I am new to system calls from a script (and if in fact that's what that would be? ;-) ).

So, more specifically: Best way to open a directory, return the name of the latest 8 digit directory in that directory ignoring all else. Increase the 8 digit dir name by 1 and create the new directory. Whichever is most efficient: stat or actual 8 digit file name. (directory names are going to be 8 digits either way.) Better to use File::Util or just built in Perl calls?

Thanks to everyone in advance. I have learned so much here.

Answer 1

Best way to open a directory, return the name of the latest 8 digit directory in that directory ignoring all else. Increase the 8 digit dir name by 1 and create the new directory. Whichever is most efficient: stat or actual 8 digit file name?

First, I should point out that having about 100,000,000 subdirectories in a directory is likely to be very inefficient.

1. How do you get only the directory names that consist of eight digits?

   use File::Slurp;
   my @dirs = grep { -d and /\A[0-9]{8}\z/ } read_dir $top;
   

2. How do you get the largest?

   use List::Util qw( max );
   my $latest = max @dirs;
   

Now, the problem is, between the determination of $latest and the attempt to create the directory, some other process can create the same directory. So, I would use $latest as the starting point and keep trying to create the next directory until I succeed or run out of numbers.

#/usr/bin/perl

use strict;
use warnings;

use File::Slurp;
use File::Spec::Functions qw( catfile );
use List::Util qw( max );

sub make_numbered_dir {
    my $max = 100_000_000;
    my $top = '/home/accountname/public_html/topdir';
    my $latest = max grep { /\A[0-9]{8}\z/ } read_dir $top;

    while ( ++$latest < $max ) {
        mkdir catfile($top, sprintf '%8.8d', $latest)
            and return 1;
    }
    return;
}

If you try to do it the way I originally recommended, you will invoke mkdir way too many times.

As for how you use File::Util::list_dir to filter entries:

#/usr/bin/perl

use strict;
use warnings;

use File::Util;

my $fu = File::Util->new;

print "$_\n" for $fu->list_dir('.',
    '--no-fsdots',
    '--pattern=\A[0-9]{8}\z'
);

C:\Temp> ks
10001010
12345678

However, I must point out that I did not much like this module in the few minutes I spent with it, especially the module author's obsession with invoking methods and functions in list context. I do not think I will be using it again.

Answer 2

What are you doing? It sounds really weird and fraught with danger. I certainly wouldn't want to let a CGI script create new directories. There might be a better solution for what you are trying to achieve.

How many directories do you expect to have? The more entries you have in any directory, the slower things are going to get. You should work out a scheme where you can hash things into a directory structure that spreads out the files so no directory holds that many items. Say, it you have the name '0123456789', you create the directory structure like:

 0/01/0123456789

You can have as many directory levels as you like. See the directory structure of CPAN, for instance. My author name is BDFOY, so my author directory is authors/id/B/BD/BDFOY. That way there isn't any directory that has a large number of entries (unless your author id is ADAMK or RJBS).

You also have a potential contention issue to work out. Between the time you discover the latest and the time you try to make the next one, you might already create the directory.

As for the task at hand, I think I'd punt to system for this one if you are going to have a million directories. With something like:

ls -t -d -1 [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] | head -1

I don't think you'll be able to get any faster than ls for this task. If there are a large number of directories, the cost of the fork should be outweighed by the work you have to do to go through everything yourself.

I suspect, however, that what you really need is some sort of database.