How to repace ${} variables in a *nix text file

Question

I want to pipe the output of a "template" file into mysql, the file having variables like ${dbName} interspersed. What is the commandline utility to replace these instances and dump the output to stdout?

Answer 1

Are you open to using perl? If so, that would be my suggestion. Although there are probably some sed and/or awk experts that probably know how to do this much easier. If you have a more complex mapping with more than just dbName for your replacements you could extend this pretty easily but you might just as well put it into a standard perl script at that point.

perl -p -e 's/\$\{dbName\}/testdb/s' yourfile | mysql

A short perl script to do something slightly more complicated (handle multiple keys):

#!/usr/bin/env perl
my %replace = ( 'dbName' => 'testdb', 'somethingElse' => 'fooBar' );
undef $/;
my $buf = <STDIN>;
$buf =~ s/\$\{$_\}/$replace{$_}/g for keys %replace;
print $buf;

If you name the above script as replace-script, it could then be used as follows:

replace-script < yourfile | mysql

Answer 2

/bin/sh

create a small shell script that sets the variables, and then parse the template using the shell itself.

Like so (edit to handle newlines correctly :P):

template.txt:

the number is ${i}
the word is ${word}

script.sh:

#!/bin/sh

#set variables
i=1
word="dog"
#read in template one line at the time, and replace variables
#(more natural (and efficient) way, thanks to Jonathan Leffler)
while read line
do
    eval echo "$line"
done < "./template.txt"

output:

#sh script.sh 
the number is 1
the word is dog

Answer 3

It can be done in bash itself if you have control of the configuration file format. You just need to source (".") the configuratiopn file rather than subshell it. That ensures the variables are created in the context of the current shell (and continue to exist) rather than the subshell (where the variable disappear when the subshell exits).

$ cat config.data
    export parm_jdbc=jdbc:db2://box7.co.uk:5000/INSTA
    export parm_user=pax
    export parm_pwd=never_you_mind

$ cat go.bash
    . config.data
    echo "JDBC string is " $parm_jdbc
    echo "Username is    " $parm_user
    echo "Password is    " $parm_pwd

$ bash go.bash
    JDBC string is  jdbc:db2://box7.co.uk:5000/INSTA
    Username is     pax
    Password is     never_you_mind

If your config file cannot be a shell script, you can just 'compile' it before executing thus (the compilation depends on your input format).

$ cat config.data
    parm_jdbc=jdbc:db2://box7.co.uk:5000/INSTA # JDBC URL
    parm_user=pax                              # user name
    parm_pwd=never_you_mind                    # password

$ cat go.bash
    cat config.data
        | sed 's/#.*$//'
        | sed 's/[ \t]*$//'
        | sed 's/^[ \t]*//'
        | grep -v '^$'
        | sed 's/^/export '
        >config.data-compiled
    . config.data-compiled
    echo "JDBC string is " $parm_jdbc
    echo "Username is    " $parm_user
    echo "Password is    " $parm_pwd

$ bash go.bash
    JDBC string is  jdbc:db2://box7.co.uk:5000/INSTA
    Username is     pax
    Password is     never_you_mind

In your specific case, you could use something like:

$ cat config.data
    export p_p1=val1
    export p_p2=val2
$ cat go.bash
    . ./config.data
    echo "select * from dbtable where p1 = '$p_p1' and p2 like '$p_p2%' order by p1"
$ bash go.bash
    select * from dbtable where p1 = 'val1' and p2 like 'val2%' order by p1

Then pipe the output of go.bash into mysql and voila, hopefully you won't destroy your database :-).

Answer 4

Sed!

Given template.txt:

The number is ${i}
The word is ${word}

we just have to say:

cat template.txt | sed -e "s/\${i}/1/" | sed -e "s/\${word}/dog/"

Answer 5

template.txt

Variable 1 value: ${var1}
Variable 2 value: ${var2}

data.sh

#!/usr/bin/env bash
declare var1="value 1"
declare var2="value 2"

parser.sh

#!/usr/bin/env bash

# args
declare file_data=$1
declare file_input=$2
declare file_output=$3

source $file_data
eval "echo \"$(< $file_input)\"" > $file_output

./parser.sh data.sh template.txt parsed_file.txt

parsed_file.txt

Variable 1 value: value 1
Variable 2 value: value 2