Getting parameters from a file via shell script into python script in the right format

Question

I have the following shell script:

#! /bin/sh

while read page_section
  page=${page_section%%\ *}
  section=${page_section#* }     #NOTE: `%* }` is NOT a comment

  wget --quiet --no-proxy www.cs.sun.ac.za/hons/$page -O html.tmp & wait

#  echo ${page_section%%\ *} # verify correct string chopping
#  echo ${page_section#* }   # verify correct string chopping

  ./DokuWikiHtml2Latex.py html.tmp $section & wait
done < inputfile

And an input file like this:

doku.php?id=ndewet:tools:tramonitor TraMonitor
doku.php?id=ndewet:description Implementation -1
doku.php?id=ndewet:description Research\ Areas -1

The script downloads a number of webpages spesified in inputfile and must then pass the rest of line (eg. "Implementation -1" or "Research\ Areas -1") to the python script.

Now for the sticky bit. When the third line of this example file is processed it passes "Research\ Areas" to the python script as two separate arguments, as confirmed by:

>>> print sys.argv
['./DokuWikiHtml2Latex.py', 'html.tmp', 'Research', 'Areas', '-1']

How can I get a multi word section, like "Research Areas" from the input file into a single argument for the python script? I've tried escaping the '\', and also doing

./DokuWikiHtml2Latex.py html.tmp `echo ${section#* }`

among other things, but to no avail.

The number at the end of an input line is another argument, but optional.

Answer 1

Just let read do the parsing stuff:

while read page section rest
do
    echo "Page: $page"
    echo "Section: $section"
done < inputfile

For handling the optional argument elegantly, use an array:

while read -a fields
do
    wget --quiet --no-proxy "www.cs.sun.ac.za/hons/${fields[0]}" -O html.tmp
    unset "fields[0]"
    ./DokuWikiHtml2Latex.py html.tmp "${fields[@]}"
done < inputfile

Always quote your variables!

Answer 2

Normally multi-word arguments can be passed as one by using quotes, so:

doku.php?id=ndewet:description "Research Areas" -1

Answer 3

Put quotes around $section:

./DokuWikiHtml2Latex.py html.tmp "$section" & wait