How to rename large number of files

Question

I have a directory with files like this

a.JPG
b.JPG
c.JPG

I would like to perform something like this

git mv a.JPG a.jpg

I tried using xargs and other tools but nothing seems to work.

Answer 1

Maybe rename them to *.somethingelse and then rename back to *.jpg

Answer 2

Use the standard Linux rename(1) utility. Once you've renamed the files, git add them.

Answer 3

Whether or not you can just change the case of the file will depend on your filesystem. Even if it works on your filesystem, you could cause problems for other people updating. You'll be best renaming them, committing them, then renaming them back. Change everything to *.tmp with the following bash script:

for i in *.JPG; do mv $i ${i%.JPG}.tmp; done

Then move them all in git. You can use a similar command, but I would recommend checking out guess-renames which will help with the move.

Then rename them all back to *.jpg with a similar process.

Answer 4

The core of the solution will be to use a tool/method that will automate the bulk rename. You can either use mv in combination with git add or just git mv. In either case you may have to take extra steps if you are using a case insensitive filesystem. So before we tackle the bulk renaming, it may be useful to discuss how case is handled a bit.

Case Sensitivity

Some systems (or system+filesystem combinations—like the default variant of the HFS+ filesystem on Mac OS X^*) are case preserving, but case insensitive. On such systems, you may need to be careful when making renames that involve only changing the case of a name. The usual workaround is to use a temporary name that differs by more than just case as a “bridge” between the two names that differ by case alone (e.g. mv foo.JPG tmp && mv tmp foo.jpg).

^{* It is possible to use case sensitive file systems on Mac OS X (including a case sensitive variant of HFS+).}

From here on, I will assume a case insensitive filesystem.

The mv command on Mac OS X can handle case-change-only renames in a single step. It will give an “overwrite?” prompt if run with the -i option, and it will skip the rename if given the -n option. It is only succeeding through the “enough rope to hang your self” default operation of many parts of Unix-like systems.

The git mv command is a bit more paranoid about the situation. It refuses the operation (“destination exists” error) unless given -f/--force option.

# this will succeed, though it may fail/prompt if mv is aliased to use -n/-i
mv foo.JPG foo.jpg

# this will succeed
mv -f bar.JPG bar.jpg

# this will succeed but give a warning
git mv -f quux.JPG quux.jpg

Bulk Rename Options

Perl rename

The desired operation is simple enough to do with a bit of shell scripting, but you could get the Perl rename utility (the one Jordan Lewis mentions) if you needed to do something that was a lot more complicated. You could try the rename from Debian's perl package, or if you feel up to using CPAN, you could install File::Rename, which includes the rename program.

ksh, bash, zsh, dash

The -ef used below is not POSIX compatible. Likewise, while -e is specified in POSIX, it is not pure-Bourne compatible. Both of them are widely supported though.

for f in *.JPG; do
    ff="${f%.JPG}.jpg"
    test -e "$f" || continue        # possible when not using nullglob
    test "$f" != "$ff" || continue  # possible when using nocaseglob
    if test -e "$ff" &&
       ! test "$f" -ef "$ff"; then  # possible on a case sensitive filesystem
        echo "skipping <$f>: destination <$ff> exists and is distinct" 1>&2
        continue
    fi

    # "mv" with "git rm" and "git add"
    mv -f "$f" "$ff"     &&
    git rm --cached "$f" &&
    git add "$ff"
done

The last section (mv, git rm, git add) could be replaced with just git mv:

    # "git mv"
    git mv -f "$f" "$ff"

If you are very concerned with how the rename might fail on case insensitive systems, then you could use a temp name:

    # temp-based "mv" with "git rm" and "git add"
    t="$ff.tmp"; while test -e "$t"; do t="$t.tmp"; done
    mv -n "$f" "$t"      &&
    mv -n "$t" "$ff"     &&
    git rm --cached "$f" &&
    git add "$ff"

Or with git mv:

    # temp-based "git mv"
    t="$ff.tmp"; while test -e "$t"; do t="$t.tmp"; done
    git mv "$f" "$t"  &&
    git mv "$t" "$ff"

zsh/zmv

This one needs -f for both zmv and git mv.

zsh -c 'autoload zmv && $0 $@' zmv -fp git -o 'mv -f' '(*).JPG' '$1 x.jpg'

---

Now that you have them all renamed and updated in Git's index you can commit them.

But will other Git users using case sensitive filesystems be able to check them out?

git checkout After a Case-Only Rename

If there are other users of your history, they will probably still have the JPG files and when they eventually checkout (a descendent of) your commit with the jpg files. What will happen for them?

No matter what happens, there is no need for “rename to temp, commit, rename to final, commit”. git checkout does not apply commits in sequence when moving between commits. It really works by “merging” the index and working tree from HEAD to the new commit. This effectively means that it “jumps” directly to the new commit while dragging along non-conflicting changes found between HEAD and the index/working-tree.

Internally, Git views renames as a deletion and an addition. I did not find any documentation that described the behavior of git checkout with respect to the order of deletions and additions, so I looked at the source code. git checkout processes all deletions before any updates/additions (cmd_checkout -> switch_branches -> merge_working_tree (-> reset_tree) -> unpack_trees -> check_updates).

You can test this out right after your rename commit:

git checkout HEAD~ # note: detached HEAD
# verify that the original names are back in place
git checkout -     # back to your branch
# verify that the new names are in place again

The git blame on the file seemed to indicate a likely commit: Make unpack-tree update removed files before any updated files, which was first released in Git 1.5.6-rc0 (2008-06-18). So, though undocumented(?), this behavior was implemented specifically to support case insensitive filesystems.

Thanks, Linus!