How to sort by line length, then reverse alphabetically

Question

I have a large (600 odd) set of search and replace terms that I need to run as a sed script over some files. The problem is that the search terms are NOT orthogonal... but I think I can get away with it by sorting by line length (i.e. pull out the longest matches first, and then alphabetically within each length. So given an unsort set of:

aaba
aa
ab
abba
bab
aba

what I want is a sorted set such as:

abba
aaba
bab
aba
ab
aa

Is there a way of doing it by say prepending the line lenght and sorting by a field?

For bonus marks :-) !!! The search and replace is actually simply a case of replacing term with _term_ and the sed code I was going to use was s/term/_term_/g How would I write the regex to avoid replacing terms already within _ pairs?

Answer 1

This will sort a file by line length, longest lines first:

cat file.txt | (while read LINE; do echo -e "${#LINE}\t$LINE"; done) | sort -rn | cut -f 2-

This will replace term with _term_ but won't turn _term_ into __term__:

sed -r 's/(^|[^_])term([^_]|$)/\1_term_\2/g'
sed -r -e 's/(^|[^_])term/\1_term_/g' -e 's/term([^_]|$)/_term_\1/g'

The first will work pretty well except it will miss out on _term and term_, mistakenly leaving those alone. Use the second if that's important. Here's my silly test case:

# echo here is _term_ and then a term you terminator haha _terminator and then _term_inator term_inator | sed -re 's/(^|[^_])term([^_]|$)/\1_term_\2/g'
here is _term_ and then a _term_ you _term_inator haha _terminator and then _term_inator term_inator
# echo here is _term_ and then a term you terminator haha _terminator and then _term_inator term_inator | sed -r -e 's/(^|[^_])term/\1_term_/g' -e 's/term([^_]|$)/_term_\1/g'
here is _term_ and then a _term_ you _term_inator haha __term_inator and then _term_inator _term__inator

Answer 2

Just pipe your stream through this kind of script :

#!/usr/bin/python
import sys

all={}
for line in sys.stdin:
    line=line.rstrip()
    if len(line) in all:
        all[len(line)].append(line)
    else:
        all[len(line)]=[line]

for l in reversed(sorted(all)):
    print "\n".join(reversed(sorted(all[l])))

And for the bonus mark question : again, do it in python (unless there really is a reason not to, but I'd be pretty curious to know it)

Answer 3

You could compact it all into one regexp:

$ sed -e 's/\(aaba\|aa\|abba\)/_\1_/g'
testing words aa, aaba, abba.
testing words _aa_, _aaba_, _abba_.

If I understand your question correctly, this will solve all your problems: No "double replacement" and always matching the longest word.

Answer 4

This does the sort by length first, then reverse alpha bit

for mask in `tr -c "\n" "." < $FILE | sort -ur`
do
    grep "^$mask$" $FILE | sort -r
done

The tr usage replaces each character in $FILE with a period - which matches any single character in grep.

Answer 5

You can do this in a one-line Perl script:

perl -e 'print sort { length $b<=>length $a || $b cmp $a } <>' input

Answer 6

$ awk '{print length($1),$1}' file |sort -rn
4 abba
4 aaba
3 bab
3 aba
2 ab
2 aa

i leave you to try getting rid of the first column yourself