reading a file line by line from a shell script

Question

I am facing a problem in a bash shell script. This script is supposed to execute another shell script (./script here) and the output of the script is redirected to a file(tmp). Then the file should be read line by line and for each of the lines the same script(./script) should be executed giving the line as argument and the result should be stored in a file(tmp1). Eventually these results should be appended to the first file(tmp).

I am pasting my script below:

./script $1 $2 > tmp
cat tmp | while read a
do
 ./script $a $2 >> tmp1
done

cat tmp1 | while read line
do
 ./script $line $2 >> tmp
done

I get the following error when I execute the script "./script: line 11: syntax error: unexpected end of file"

Can anyone please help me out in this??

Thanks a lot in advance.

Answer 1

The shell splits on whitespace normally. You can make it split on newlines by writing, towards the top,

IFS='
'

However, I think what you're trying to do might not be appropriate for shell, and might be better served by Perl or Ruby.

Answer 2

lose all the cats! they are unnecessary. And i suppose summ_tmp is an existing file?

#!/bin/bash
set -x
./wiki $1 $2 > tmp
while read -r a
do
    ./wiki $a $2 >> tmp1
done < summ_tmp

while read -r line
do
    ./wiki $line $2 >> tmp
done < tmp1

With what you are doing, you might want to refactor your "./script" to eliminate unnecessary steps. If its not too long, show what your "./script" does. Show your desired output and show examples of relevant input files where possible

Put set -x in your script (wiki and ./script) to help you debug.

Answer 3

Alternatively you could use xargs - this executes a command on every line in a file, which is exactly what you want.

You can replace

cat tmp1 | while read line
do
    ./script $line $2 >> tmp
done

with

xargs <tmp1 -n1 -IXXX ./script XXX $2 >>tmp

-n1 means read one line at a time from the input,

-IXXX means substitute XXX with the line that was read in - the default is to append it to the end of the command line.

Answer 4

What is on line 11 of ./script?

Answer 5

The script file has DOS/Windows line endings. Try this command, then run your script:

dos2unix ./script

You're probably editing the file using a Windows editor and that's adding \r (0x0d) to the end of each line. This can be removed by using dos2unix.