How to find duplicate files with same name but in different case that exist in same directory in Linux?

Question

How can I return a list of files that are named duplicates i.e. have same name but in different case that exist in the same directory?

I don't care about the contents of the files. I just need to know the location and name of any files that have a duplicate of the same name.

Example duplicates:

/www/images/taxi.jpg
/www/images/Taxi.jpg

Ideally I need to search all files recursively from a base directory. In above example it was /www/

Answer 1

Try:

ls -1 | tr '[A-Z]' '[a-z]' | sort | uniq -c | grep -v " 1 "

Simple, really :-) Aren't pipelines wonderful beasts?

The ls -1 gives you the files one per line, the tr '[A-Z]' '[a-z]' converts all uppercase to lowercase, the sort sorts them (surprisingly enough), uniq -c removes subsequent occurrences of duplicate lines whilst giving you a count as well and, finally, the grep -v " 1 " strips out those lines where the count was one.

When I run this in a directory with one "duplicate" (I copied qq to qQ), I get:

2 qq

For the "this directory and every subdirectory" version, just replace ls -1 with find . or find DIRNAME if you want a specific directory starting point (DIRNAME is the directory name you want to use).

This returns (for me):

2 ./.gconf/system/gstreamer/0.10/audio/profiles/mp3
2 ./.gconf/system/gstreamer/0.10/audio/profiles/mp3/%gconf.xml
2 ./.gnome2/accels/blackjack
2 ./qq

which are caused by:

pax> ls -1d .gnome2/accels/[bB]* .gconf/system/gstreamer/0.10/audio/profiles/[mM]* [qQ]?
.gconf/system/gstreamer/0.10/audio/profiles/mp3
.gconf/system/gstreamer/0.10/audio/profiles/MP3
.gnome2/accels/blackjack
.gnome2/accels/Blackjack
qq
qQ

---

Update:

Actually, on further reflection, the tr will lowercase all components of the path so that both of

/a/b/c
/a/B/c

will be considered duplicates even though they're in different directories.

If you only want duplicates within a single directory to show as a match, you can use the (rather monstrous):

perl -ne '
    chomp;
    @flds = split (/\//);
    $lstf = $f[-1];
    $lstf =~ tr/A-Z/a-z/;
    for ($i =0; $i ne $#flds; $i++) {
        print "$f[$i]/";
    };
    print "$x\n";'

in place of:

tr '[A-Z]' '[a-z]'

What it does is to only lowercase the final portion of the pathname rather than the whole thing. In addition, if you only want regular files (no directories, FIFOs and so forth), use find -type f to restrict what's returned.

Answer 2

I'm sorry, i don't have enough reputation to leave a comment yet.

The other answer is great, but instead of the "rather monstrous" perl script i suggest

perl -pe 's!([^/]+)$!lc $1!e'

Which will lowercase just the filename part of the path.

Edit: In fact the entire problem can be solved with:

find | perl -ne 's!([^/]+)$!lc $1!e; print if 1 == $seen{$_}++'

Answer 3

I believe

ls | sort -f | uniq -i -d

is simpler, faster, and will give the same result

Answer 4

Using Duplicate Finder 2009 to find and remove duplicate files... Fast and Easy to use software. http://www.duplicate-finder-pro.com

Answer 5

// Duplicate files with same name in same directory // Duplicate files are copies of original files that unnecessarily occupy precious hard disk space,but we should be careful while deleting these files as these may lead to human errors. I can suggest one duplicate file remover that is the nice optimizer.It will scan files based on each criteria or extensions chosen by the user.Use ASO's (Advanced system optimizer) duplicate file remover . Advanced system optimizer is a windows optimization tool. Try downloading it from download.com or CNET . Use http://download.cnet.com/Advanced-System-Optimizer/3000-2094_4-10147659.html?tag=mncol to download.

You can find your duplicates from network as well using this.