Move line(s) to follow another line in a file

Question

I got a file that has a line in the file like this:

check=('78905905f5a4ed82160c327f3fd34cba')

I'd like to be able to move this line to follow a line that looks like this:

files=('somefile.txt')

The array though at times that can span multiple lines, for example:

files=('somefile.txt'
       'file2.png'
       'another.txt'
       'andanother...')

text
in between

check=('78905905f5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

The array/line always ends in a ) and no text in between will contain a closed parenthesis.

I got some advice that awk can do this:

awk '/files/{
    f=0
    print $0
    for(i=1;i<=d;i++){ print a[i]  }
    g=0
    delete a # remove array after found
    next
}
/check/{ f=1; g=1 }
f{ a[++d]=$0 }
!g' file

This will only span one line though. I was told to expand the search:

awk '/source/ && /\)$/{
    f=0
    print $0
    for(i=1;i<=d;i++){ print a[i]  }
    g=0
    delete a # remove array after found
    next
}
/md5sum/ && /\)$/{ f=1; g=1 }
f{ a[++d]=$0 }
!g'

Just learning awk so I'd appreciate help with this. Or if there is another tool that can do this, I'd like to hear about it. Someone told me that 'ed' these types of capabilities.

Answer 1

To answer your last question first, yes, awk is the typical Unix tool for this, other candidates are the incredibly powerful Perl, Python, or .. my favorite .. Ruby. One advantage of awk is that it's always there; it's part of the base system. Another way to solve this kind of problem is with an editor script that controls ed(1) or ex(1).

Ok, new program for the revised question. This program will move the "check" lines either up or down as necessary so that they follow the "files" lines.

BEGIN {
  checkAt = 0
  filesAt = 0
  scanning = 0
}

/check=\(/ {
  checkAt = NR
  scanning = 1
}

/files=\(/ {
  filesAt = NR
  scanning = 1
}

/)$/ {
  if (scanning) {
    if (checkAt > filesAt) {
      checkEnd = NR
    } else {
      filesEnd = NR
    }
    scanning = 0
  }
}

{
  lines[NR] = $0
}

END {
  for (i = 1; i <= NR; ++i) {
    if (checkAt <= i && i <= checkEnd) {
      continue
    }
    print lines[i]
    if (i == filesEnd) {
      for (j = checkAt; j <= checkEnd; ++j) {
        print lines[j]
      }
    }
  }
}

Answer 2

I looked in to doing this with Awk, but it looked like you wouldn't really get anything clever out of it, it would just be the same logic, but with some Awk pain to go with it, so I did it in Perl :)

#!/usr/bin/perl

open(IN, $ARGV[0]) || die("Could not open file: " . $ARGV[0]);

my $buffer="";

foreach $line (<IN>) {
        if ($line =~ /^check=/) {
                $flag = 1;
                $buffer .= $line;
        } elsif ($flag == 1 && $line =~/\)/) {
                $flag = 0;
                $buffer .= $line;
        } elsif ($flag == 1) {
                $buffer .= $line;
        } elsif ($flag == 0 && $line =~ /^files=/) {
                $flag = 2;
                print $line;
        } elsif ($flag == 2 && $line =~ /\)/) {
                $flag = 0;
                print $line;
                if (length($buffer) > 0) {
                        print $buffer;
                        $buffer = "";
                }
        } else {
                print $line;
        }

}

And the output :)

Chill:~ rus$ cat test check=('78905905f5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

text in between

files=('somefile.txt'
       'file2.png'
       'another.txt'
       'andanother...')

asdasdasd

check=('78905905f5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

text in between

files=('somefile.txt'
       'file2.png'
       'another.txt'
       'andanother...')

asdsd

check=('78905905f5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

text in between

files=('somefile.txt'
       'file2.png'
       'another.txt'
       'andanother...')

Chill:~ rus$ ./t.pl test

text in between

files=('somefile.txt'
       'file2.png'
       'another.txt'
       'andanother...') check=('78905905f5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

asdasdasd


text in between

files=('somefile.txt'
       'file2.png'
       'another.txt'
       'andanother...') check=('78905905f5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

asdsd


text in between

files=('somefile.txt'
       'file2.png'
       'another.txt'
       'andanother...') check=('78905905f5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

ta da ?! :D

Answer 3

@todd, I seem to have left you in the lurch after providing you the awk solution haven't i. ? :). here's another method, this time not using method of flags. there are some loose ends (hint: check the patterns p,q and output again) that i leave it to you to tidy up.

gawk 'BEGIN{
    RS="check=[(]"
    q="files=(.*\047)"  # pattern to replace files= part
    p=".*(files=(.*\047)).*" # to get the whole files= part to variable
}
NR>1{
    b=gensub(p, "\\1","g",$0) # get the files=part to var b
    printf "%s\n\n",b    
    printf "check=("
    gsub(q,"",$0)
    print $0
}' file

NB: gensub is specific to gawk so if you have gawk, then that's alright

output

$ more file
check=('5277a9164001a4276837b59dade26af2'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

text in between one

files=('somefile1.txt'
       'file1.png'    
       'another1.txt' 
       'andanother1...')

asdasdasd blah blah

check=('78905905f5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

text in between  two

files=('somefile2.txt'
       'file2.png'    
       'another2.txt' 
       'andanother2...')

asdsd blaasdf aslasdfaslj aslfjsldfsa 123e12

check=('78905905fblah blah5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'         
       '3f8b60b6fbb993c18442b62ea661aa6b')        

text in between

files=('somefile3.txt'
       'file3.png'    
       'another3.txt' 
       'andanother3...')

$ ./shell.sh
files=('somefile1.txt'             
       'file1.png'                 
       'another1.txt'              
       'andanother1...'            

check=('5277a9164001a4276837b59dade26af2'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

text in between one

)

asdasdasd blah blah


files=('somefile2.txt'
       'file2.png'
       'another2.txt'
       'andanother2...'

check=('78905905f5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

text in between  two

)

asdsd blaasdf aslasdfaslj aslfjsldfsa 123e12


files=('somefile3.txt'
       'file3.png'
       'another3.txt'
       'andanother3...'

check=('78905905fblah blah5a4ed82160c327f3fd34cba'
       '5277a9164001a4276837b59dade26af2'
       '3f8b60b6fbb993c18442b62ea661aa6b')

text in between

)