help with unix tar and grep loop

Question

Hi

I would like some help in creating a loop that will take one of my files extension .tar.gz unzip it untar it and search the files inside (with extension .tlg) using grep -a >> output.text.

In the outout.text i will require the matching data as well as the name of the file and parent tar it came from

one this search has been performed i would like the untared files to be deleted and the preocess to continue on the next tar file until all tars have been checked.

I can't untar all at one as i dont have the disk space for this

Can anyone help ?

thanks

Answer 1

You could loop over the tars, extract them, then grep them; something like this should work:

match="somestring"
mkdir out/
for i in *.tar.gz; do
 mkdir out/${i} # create outdir
 tar -C out/${i} -xf ${i} # extract to sub-dir with same name as tar; 
                          # this will show up in grep output
 cd out
 grep -r ${match} ${i} >> ../output.text
 cd ..
 rm -rf out/${i} # delete untarred files
done

be careful, as the contents of the $i variable are passed to rm -rf and has the power to delete stuff for good.

Answer 2

To avoid creating temporary files, you can use GNU tar's --to-stdout option.

The code below is careful about spaces and other characters in paths that may confuse the shell:

#! /usr/bin/perl

use warnings;
use strict;

sub usage { "Usage: $0 pattern tar-gz-file ..\n" }

sub output_from {
  my($cmd,@args) = @_;
  my $pid = open my $fh, "-|";
  warn("$0: fork: $!"), return unless defined $pid;
  if ($pid) {
    my @lines = <$fh>;
    close $fh or warn "$0: $cmd @args exited " . ($? >> 8);
    wantarray ? @lines : join "" => @lines;
  }
  else {
    exec $cmd, @args or die "$0: exec $cmd @args: $!\n";
  }
}

die usage unless @ARGV >= 2;
my $pattern = shift;
foreach my $tgz (@ARGV) {
  chomp(my @toc = output_from "tar", "-ztf", $tgz);
  foreach my $tlg (grep /\.tlg\z/, @toc) {
    my $line = 0;
    for (output_from "tar", "--to-stdout", "-zxf", $tgz, $tlg) {
      ++$line;
      print "$tlg:$line: $_" if /$pattern/o;
    }
  }
}

Sample runs:

$ ./grep-tlgs hello tlgs.tar.gz 
tlgs/another.tlg:2: hello
tlgs/file1.tlg:2: hello
tlgs/file1.tlg:3: hello
tlgs/third.tlg:1: hello

$ ./grep-tlgs ^ tlgs.tar.gz 
tlgs/another.tlg:1: blah blah
tlgs/another.tlg:2: hello
tlgs/another.tlg:3: howdy
tlgs/file1.tlg:1: whoah
tlgs/file1.tlg:2: hello
tlgs/file1.tlg:3: hello
tlgs/file1.tlg:4: good-bye
tlgs/third.tlg:1: hello
tlgs/third.tlg:2: howdy

$ ./grep-tlgs ^ xtlgs.tar.gz 
tar: xtlgs.tar.gz: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Child returned status 2
tar: Exiting with failure status due to previous errors
./grep-tlgs: tar -ztf xtlgs.tar.gz exited 2 at ./grep-tlgs line 14.