expanding variables in shell script

Question

I have a query that I need to loop.

query="select '$dbserver' as server;"

while read dbserver username password dbname type
do
mysql -h$dbserver -u$username -p$password $dbname -Be"$query" >> /home/develop/myreport.csv
done < $dblist

The following line expands correctly.

mysql -h$dbserver -u$username -p$password $dbname -Be"select '$dbserver' as server;" >> /home/develop/myreport.csv

But when I take out the query and save it in a variable as shown above, it stops working as expected and gives blank value for "dbserver". The line mentioned in the loop above does not work. How to correct this problem?

Answer 1

Put the query assignment inside the loop. Before the loop your variables are not defined yet (i.e. blank)

Answer 2

while read -r dbserver username password dbname type
do
 query="select '$dbserver' as server;"
 mysql -h${dbserver} -u${username} -p${password} ${dbname} -Be"${query}" >> /home/develop/myreport.csv
done < $dblist

Answer 3

If you want the '$dbserver' in the query expanded inside the loop, you'd probably write:

while read dbserver username password dbname type
do
    query="select '$dbserver' as server;"
    mysql -h$dbserver -u$username -p$password $dbname -Be"$query"
done < $dblist  > /home/develop/myreport.csv

As originally written, the query string is evaluated before any value is assigned to $dbserver, which is why you got the empty string in the output.

Note that the output redirection was done just once - on the done line rather than each time in the loop (which means that you don't need append any more).

---

Getting the query created outside the loop is normally something you can do, using eval. However, because the value of the $dbserver is enclosed inside single quotes, this turns out to be hard. If the DBMS you are using permits the use of double quotes around strings (contrary to the SQL standard), then this works with an eval:

query='select \"$dbserver\" as server;'
echo "$query"
while read dbserver username password dbname type
do
    echo 1: "$query"
    eval echo 2: "$query"
    qval=$(eval echo "$query")
    echo mysql -h$dbserver -u$username -p$password $dbname -Be"$qval"
done

You can then adapt this to use single quotes by replacing the '"' with the ''\''' sequence:

query='select \'\''$dbserver\'\'' as server;'
echo "$query"
while read dbserver username password dbname type
do
    echo 1: "$query"
    eval echo 2: "$query"
    qval=$(eval echo "$query")
    echo mysql -h$dbserver -u$username -p$password $dbname -Be"$qval"
done

That is, however, the sort of quote sequence that sends sane people from the room screaming - excuse me a moment while I vacate the premises noisily! [...later...] That's better!

Explanation:

• The overall string is inside single quotes.
• There are no escape characters inside such a string.
• Therefore, the first backslash is just a backslash.
• The next 4 characters are the sequence '\''.
• The first of these quotes terminates the current single-quoted string.
• The backslash suspends the special meaning of the next character, so that the string contains an actual single quote from the second single quote in the sequence.
• The third single quote starts a new single-quoted string.
• So, after processing the first sequence of backslashes and quotes, the string contains a backslash and a single quote.
• $dbserver is just normal text at this point.
• We then have a repeat of the previous sequence, ending up with a second backslash-quote pair in the string.
• Everything is normal to the last single quote on the line.

The eval process runs an extra lot of expansion on the string. The backslash-quote pairs are replaced by just quote; the current value of $dbserver is inserted. This can then be passed to the command as an ordinary argument.

The difficulty with eval is ensuring that you don't get unexpected side-effects. This is doubly complex with MySQL which uses back-quotes to enclose keywords used as tokens. That notation interacts diabolically with eval, of course. However, with single quotes around the whole query and backslash-backquote in place of each backquote, you can do it:

query='select \'\''$dbserver\'\'' as server, \`ls\` as column;'
echo "$query"
while read dbserver username password dbname type
do
    echo 1: "$query"
    eval echo 2: "$query"
    qval=$(eval echo "$query")
    echo mysql -h$dbserver -u$username -p$password $dbname -Be"$qval"
done

I don't think it can be recommended, though.