Why doesn't my awk one-liner work when I call it from Perl?

Question

Dear all,

I have no problem using the following command of AWK as a stand alone command, without any error:

$ awk '$9 != "NTM" && $9 != ""' myfile.txt  | less -Sn

But when I apply them inside Perl's script for qsub (i.e. running job in linux cluster) command, like this:

use strict;
use Data::Dumper;
use Carp;
use File::Basename;

my $path = "/mypath/"; 
my @files = glob($path."*.txt");

foreach my $file ( @files ) {
print "$file\n";
    wy $base = basename($file,".fc-gn_".$type);
    my $nn = $path.$base.".out";

print "$file $nn\n";

    open PIPE, "| qsub" or die $!; 
    print PIPE <<EOF;
#!/bin/sh 
#PBS -N CLEAN
#PBS -l nodes=1:ppn=2 
#PBS -r n 

awk '$9 != "NTM" && $9 !=""' $file > $nn


EOF

}

It gave the following error

awk: cmd. line:1:  != "NTM" &&  !=""
awk: cmd. line:1:  ^ syntax error

What's the right way to do it?

Answer 1

Putting a "\" character before all of your "$9" variables will fix it. They're being parsed by Perl itself.

The following script:

print <<EOF;
awk '$9 != "NTM" && $9 !=""' $file > $nn
EOF

outputs:

awk ' != "NTM" &&  !=""'  >

but:

print <<EOF;
awk '\$9 != "NTM" && \$9 !=""' \$file > \$nn
EOF

outputs:

awk '$9 != "NTM" && $9 !=""' $file > $nn

However, since you want $file and $nn to be interpreted by Perl, the actual line should be:

awk '\$9 != "NTM" && \$9 !=""' $file > $nn

Answer 2

you can use the backslash solution, or make it more visually appealing by using:

print <<'EOF'

(and no backslash), also if you like Perl, you might want to give a shot at gearman, it's really cool.