Sync File Modification Time Across Multiple Directories

Question

I have a computer A with two directory trees. The first directory contains the original mod dates that span back several years. The second directory is a copy of the first with a few additional files. There is a second computer be which contains a directory tree which is the same as the second directory on computer A (new mod times and additional files). How update the files in the two newer directories on both machines so that the mod times on the files are the same as the original? Note that these directory trees are in the order of 10s of gigabytes so the solution would have to include some method of sending only the date information to the second computer.

Answer 1

I think rsync (with the right options) will do this - it claims to only send file differences, so presumably will work out that there are no differences to be transferred.

--times preserves the modification times, which is what you want.

See (for instance) http://linux.die.net/man/1/rsync

Also add -I, --ignore-times don't skip files that match size and time

so that all files are "transferred' and trust to rsync's file differences optimisation to make it "fairly efficient" - see excerpt from the man page below.

-t, --times This tells rsync to transfer modification times along with the files and update them on the remote system. Note that if this option is not used, the optimization that excludes files that have not been modified cannot be effective; in other words, a missing -t or -a will cause the next transfer to behave as if it used -I, causing all files to be updated (though the rsync algorithm will make the update fairly efficient if the files haven't actually changed, you're much better off using -t).

Answer 2

I would go through all the files in the source directory tree and gather the modification times from them into a script that I could run on the other directory trees. You will need to be careful about a few 'gotchas'. First, make sure that your output script has relative paths, and make sure you run it from the proper target directory, which should be the root directory of the target tree. Also, when changing machines make sure you are using the same timezone as you were on the machine where you generated the script.

Here's a Perl script I put together that will output the touch commands needed to update the times on the other directory trees. Depending on the target machines, you may need to tweak the date formats or command options, but this should give you a place to start.

#!/usr/bin/perl

my $STARTDIR="$HOME/test";

chdir $STARTDIR;
my @files = `find . -type f`;
chomp @files;

foreach my $file (@files) {
   my $mtime = localtime((stat($file))[9]);
   print qq(touch -m -d "$mtime" "$file"\n);
}

Answer 3

The other approach you could try is to attach the remote directory using NFS and then copy the times using find and touch -r.

Answer 4

The following command will make sure that TEST2 gets the same date assigned that TEST1 has

touch -t `stat -t '%Y%m%d%H%M.%S' -f '%Sa' TEST1` TEST2

Now instead of using hard-coded values here, you could find the files using "find" utility and then run touch via SSH on the remote machine. However, that means you may have to enter the password for each file, unless you switch SSH to cert authentication. I'd rather not do it all in a super fancy one-liner. Instead let's work with temp files. First go to the directory in question and run a find (you can filter by file type, size, extension, whatever pleases you, see "man find" for details. I'm just filtering by type file here to exclude any directories):

find . -type f -print -exec stat -t '%Y%m%d%H%M.%S' -f '%Sm' "{}" \; > /tmp/original_dates.txt

Now we have a file that looks like this (in my example there are only two entries there):

# cat /tmp/original_dates.txt 
./test1
200809241840.55
./test2
200809241849.56

Now just copy the file over to the other machine and place it in the directory (so the relative file paths match) and apply the dates:

cat original_dates.txt | (while read FILE && read DATE; do touch -t $DATE "$FILE"; done)

Will also work with file names containing spaces.

One note: I used the last "modification" date at stat, as that's what you wrote in the question. However, it rather sounds as if you want to use the "creation" date (every file has a creation date, last modification date and last access date), you need to alter the stat call a bit.

'%Sm' - last modification date
'%Sc' - creation date
'%Sa' - last access date

However, touch can only change the modification time and access time, I think it can't change the creation time of a file ... so if that was your real intention, my solution might be sub-optimal... but in that case your question was as well ;-)