Problem with shell script

Question

Hi all,

This is a very simple bash script i wrote:

#!/bin/bash

ITEM_LIST=items.txt
LOG_FILE=log.log

TOTAL_ITEMS=$(wc -l ${ITEM_LIST} | awk '{ print $1 }')
let NOT_FOUND=0

cat ${ITEM_LIST} | while read item; do

    grep "${item}" ${LOG_FILE} > /dev/null
    FOUND=${?}
    if [ ${FOUND} -ne 0 ]; then
     let NOT_FOUND=NOT_FOUND+1
     echo "Item not found [${item}] Item not found number: ${NOT_FOUND}"
    fi

done

echo "Total items: ${TOTAL_ITEMS}"
echo "Total not found items: ${NOT_FOUND}"

I want to check if some items are present in a log file, count how much are not present and print some kind of reporting (last two echo). At this moment I'm running it from cygwin bash shell.

Consider these two sample files:

items.txt

first item
second item
third item
fourth item
fifth item

log.log

blahblah blah blah first item blah blah blah
second blah blah item
blah third item blah

Script's output:

[17:46:38]:/cygdrive/c/Temp/qpa# ./script2.sh 
Item not found [second item] Item not found number: 1
Item not found [fourth item] Item not found number: 2
Total items: 4
Total not found items: 0

Question:

Why is the script printing "Total not found items: 0"? It prints current total on every echo on the loop (both NOT_FOUND variable).

Are there some bad practices in this shell script? Where and why?

Thanks in advance.

Answer 1

The pipe you use here:

cat ${ITEM_LIST} | ...

Will execute the while loop afterwards in a sub-shell. But that means that the NOT_FOUND variable won't be updated in the parent shell, but only in the sub-shell you execute the loop in.

Include the echo commands at the end inside that sub-shell instance:

cat ${ITEM_LIST} | { 
  while read item; do
    grep "${item}" ${LOG_FILE} > /dev/null
    FOUND=${?}
    if [ ${FOUND} -ne 0 ]; then
        let NOT_FOUND=NOT_FOUND+1
        echo "Item not found [${item}] Item not found number: ${NOT_FOUND}"
    fi
  done

  echo "Total items: ${TOTAL_ITEMS}"
  echo "Total not found items: ${NOT_FOUND}"
}

That problem is also explained in a Bash FAQ item. Hope this helps.

As the FAQ explains, in that particular case, you can also rewrite it so it reads:

while read item; do
    grep "${item}" ${LOG_FILE} > /dev/null
    FOUND=${?}
    if [ ${FOUND} -ne 0 ]; then
        let NOT_FOUND=NOT_FOUND+1
        echo "Item not found [${item}] Item not found number: ${NOT_FOUND}"
    fi
done < ${ITEM_LIST}

Prefer the second option in this case, since it will get rid of one "useless use of cat" :)

Answer 2

Use expr to do the math

NOT_FOUND=0; NOT_FOUND=`expr ${NOT_FOUND} + 1`; echo ${NOT_FOUND}

1

@litb is right, you need to update NOT_FOUND in your main script, not in the sub shell spawned by the pipe command.