how to pre-construct awk statement to pass to awk on command line?

Question

I have a shell script that constructs an awk program as a string then pass that string to awk. This is because I want to use values of shell variables in the awk program.

My code looks like this:

awk_prog="'{if (\$4~/$shell_var/) print \$1,\$2}'"
echo $awk_prog
awk $awk_prog $FILENAME

However, when I pass the string to awk, I always get the error:

'{if ($4~/regex/) print $1,$2}'
awk: '{if
awk: ^ invalid char ''' in expression

What does that error message mean? I tried the -F: switch but it does not help. How can I settle this issue?

Thank you.

Answer 1

This is caused by shell quoting. The following will work:

awk_prog="{ if (\$4 ~ /$shell_var) print \$1, \$2 }"
echo "$awk_prog"
awk "$awk_prog" $FILENAME

When you run awk '{ print }' foo from the command line, the shell interprets and removes the quotes around the program so awk receives two arguments - the first is the program text and the second is the filename foo. Your example was sending awk the program text '{if ...}' which is invalid syntax as far as awk is concerned. The outer quotes should not be present.

In the snippet that I gave above, the shell uses the quotes in the awk_prog= line to group the contents of the string into a single value and then assigns it to the variable awk_prog. When it executes the awk "$awk_prog"... line, you have to quote the expansion of $awk_prog so awk receives the program text as a single argument.

Answer 2

There's another way to get your shell variable into awk -- use awk's -v option:

awk -v pattern="$shell_var" '$4 ~ pattern {print $1, $2}' "$FILENAME"

Use -v multiple times if you have several variables to pass to awk.

If you truly want to hold your awk program in a shell variable, build it up using printf:

awk_script="$( printf '$4 ~ /%s/ {print $1, $2}' "$shell_var" )"
awk "$awk_script" "$FILENAME"

Note the use of quotes in the printf command: single quotes around the template to protect the dollar signs you want awk to interpret, double quotes for shell variables.

Answer 3

Another (IMO simpler) solution which (I think) addresses what you are intuitively trying to do is simply to use eval. You want the shell to behave as if you had literally typed:

awk '{if ($4~/foo/) print $1,$2}' path

(where foo and path are the literal contents of $shell_var and $FILENAME). To make that happen, just slap an eval on the front of your last line (and perhaps quotes for good measure, but they aren't necessary in this case) so that your last line is:

eval "awk $awk_prog $FILENAME"