how to read STDOUT to a series of variables

Question

I have a python script called mythicalPythonBindings.py. It outputs something like

Establishing database connection
chanid = 1952
starttime = 2010-09-29 08:12:00
endtime = 2010-09-29 08:30:00
basename = 1952_20100929081200.mpg
stars = 0.0

I need to read this information into variables of the same name in a bash script. My bash script looks like this

#! /bin/bash

/usr/local/bin/mythicalPythonBindings.py --filename='1952_20100929081200.mpg' --DBHostName='192.168.1.110' --DBName='mythconverg' --DBUserName='mythtv' --DBPassword='mythtv' --output=./ReturnFile.txt --display|
   while read line
     do 
     case "$line" in
         "basename = "* ) 
              echo $line
              basename=`echo "$line" |sed s/'basename = '//g`
              echo $basename
         ;;
     esac
   done
echo $basename
#doing more stuff 

The problem is that when setting variables inside the piped output from mythicalPythonBindings.py, it does not export the variables. The statement "echo $basename"(inside the piped output) works, but the seccond "echo $basename"(outside the piped output) does not work.

How can I set these variables for use in the rest of the script?

edit: I need this variable to be used in other commands without saving the data to a file which must be deleted later.

Answer 1

#!/bin/sh
./testpyscri.sh | (
   while read line
     do 
     case "$line" in
         "basename = "* ) 
    echo $line
              basename=`echo "$line" |sed s/'basename = '//g`
                echo $basename
         ;;
     esac
   done
   echo and also $basename
)

I made a couple of changes to easily reproduce and demo the result, but the only important change was the addition of the single group of parens.

so ross$ ./testpyscri.sh  | ./pyscri.sh 
basename = 1952_20100929081200.mpg
1952_20100929081200.mpg
and also 1952_20100929081200.mpg

Update:

It should be easy to understand the above design pattern which can in fact execute an arbitrarily complex script on either end of the pipeline. But perhaps you would like an different sort of design. Try something like this:

so ross$ expand < test.sh
#!/bin/sh
# here is an alternate design pattern
if [ x$1 != x-a ]; then
  echo abc | ./$0 -a
  exit 0
fi
read x
echo $x
so ross$ ./test.sh
abc
so ross$ 

Answer 2

Two key changes:

#! /bin/bash
/usr/local/bin/mythicalPythonBindings.py --filename='1952_20100929081200.mpg' \
    --DBHostName='192.168.1.110' --DBName='mythconverg' --DBUserName='mythtv' \
    --DBPassword='mythtv' --output=./ReturnFile.txt --display |
{
while read var equals value
do
    if [ $equals = "=" ]
    then
        eval $var=\"$value\"
    fi
done

echo basename=$basename
echo starttime=$starttime
echo endtime=$endtime
echo stars=$stars
echo chanid=$chanid
# ...rest of script...
}

1. Use the shell's built-in splitting facility, plus eval to set the variable without needing to know what the variable is (and without a separate assignment for each different variable). The test ignores the first line of the output.
2. Use the I/O grouping facility of the { ... } to capture the rest of the script so that the variables set in the loop are available after the loop.

You could use parentheses instead of the braces; in this case, it amounts to the same thing. In general, the I/O grouping does not start a sub-shell, which can be useful to avoid repeated redirects while executing a block of code, while still allowing variables set in the block of code to be seen outside the block.

Answer 3

The while loop creates a subshell when something is piped into it. To avoid that you can redirect process substitution into the done:

pythonfunc () {
    /usr/local/bin/mythicalPythonBindings.py --filename='1952_20100929081200.mpg' --DBHostName='192.168.1.110' --DBName='mythconverg' --DBUserName='mythtv' --DBPassword='mythtv' --output=./ReturnFile.txt --display
}

while ...
do
    ...
done < <(pythonfunc)
echo $basename

I would use

read var equals value
# do some validation
declare $var=$value

Using declare in this way allows you to avoid using eval.

Answer 4

Here's another version using eval It assumes:

1. Python outputs variables and values separated by =.
2. Uses Awk with -F '=' to break down inputs into variable and value
3. Removes leading and trailing white-space characters on variables and values using Sed.

Bash code:

#!/bin/bash

while read line
do
    var=$(echo $line | awk -F '=' '{print $1}' | sed 's/^[ \t]*//;s/[ \t]*$//')
    val=$(echo $line | awk -F '=' '{print $2}' | sed 's/^[ \t]*//;s/[ \t]*$//')

    eval $var="$val"
    echo $python_var
done

Examples usage:

>> echo "python_var = 1234" | ./script.sh 
>> 1234