tags:

views:

501

answers:

3

How do I iterate through each line of a text file with Bash?

With this script

echo Start!
for p in (peptides.txt)
do
    echo $p
done

I get this output on the screen:

Start!
./runPep.sh: line 3: syntax error near unexpected token `('
./runPep.sh: line 3: `for p in (peptides.txt)'

(Later I want to do something more complicated with $p than just output to the screen.)


The environment variable SHELL is (from env):

SHELL=/bin/bash

"/bin/bash --version" output:

GNU bash, version 3.1.17(1)-release (x86_64-suse-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.

"cat /proc/version" output:

Linux version 2.6.18.2-34-default (geeko@buildhost) (gcc version 4.1.2 20061115 (prerelease) (SUSE Linux)) #1 SMP Mon Nov 27 11:46:27 UTC 2006

The file peptides.txt contains:

RKEKNVQ
IPKKLLQK
QYFHQLEKMNVK
IPKKLLQK
GDLSTALEVAIDCYEK
QYFHQLEKMNVKIPENIYR
RKEKNVQ
VLAKHGKLQDAIN
ILGFMK
LEDVALQILL
+5  A: 
cat peptides.txt | while read line
do
   # do something with $line here
done
Warren Young
In general, if you're using "cat" with only one argument, you're doing something wrong (or suboptimal).
JesperE
I have tried it and it works (as well as Bruno De Fraine's).
Peter Mortensen
Yes, it's just not as efficient as Bruno's, because it launches another program, unnecessarily. If efficiency matters, do it Bruno's way. I remember my way because you can use it with other commands, where the "redirect in from" syntax doesn't work.
Warren Young
There's another, more serious problem with this: because the while loop is part of a pipeline, it runs in a subshell, and hence any variables set inside the loop are lost when it exits (see http://bash-hackers.org/wiki/doku.php/mirroring/bashfaq/024). This can be very annoying (depending on what you're trying to do in the loop).
Gordon Davisson
no need cat. its useless
ghostdog74
+8  A: 

The correct syntax is:

while read p; do
  echo $p
done < peptides.txt
Bruno De Fraine
I have tried it and it works (as well as Warren Young's).
Peter Mortensen
How should I interpret the last line? File peptides.txt is redirected to standard input and somehow to the whole of the while block?
Peter Mortensen
"Slurp peptides.txt into this while loop, so the 'read' command has something to consume." My "cat" method is similar, sending the output of a command into the while block for consumption by 'read', too, only it launches another program to get the work done.
Warren Young
+1  A: 

Option 1a: While loop: Single line at a time: Input redirection

#!/bin/bash
filename='peptides.txt'
echo Start
while read p; do 
    echo $p
done < $filename

Option 1b: While loop: Single line at a time: Open the file, read from a file descriptor (in this case file descriptor #4).

#!/bin/bash
filename='peptides.txt'
exec 4<$filename
echo Start
while read -u4 p ; do
    echo $p
done

Option 2: For loop: Read file into single variable and parse. This syntax will parse "lines" based on any white space between the tokens. This still works because the given input file lines are single work tokens. If there were more than one token per line, then this method would not work as well. Also, reading the full file into a single variable is not a good strategy for large files.

#!/bin/bash
filename='peptides.txt'
filelines=`cat $filename`
echo Start
for line in $filelines ; do
    echo $line
done
semiuseless
For option 1b: does the file descriptor need to be closed again? E.g. the loop could be an inner loop.
Peter Mortensen
semiuseless