How to search for a time value in a log file using shell and awk?

Question

I have a script where a user provides a date, start time and end time. The script will take these values and scan a log file to search for key words in the log file that fall within the time range. When a user uses AM times (Example: 0700 to 0900 - this is 7am to 9am) as a range or PM times (Example: 1400 to 1600 - this is 2pm to 4pm) as a range the script works fine. when they use an AM start time (starting with a zero - 0700) and an end time (doesn't start with zero - 1000 (this is 10am) the script does not function properly.

Here is the line in question that I'm having problems with

echo "Number of ${shipmentTags[$i]} shipments processed in \
    log file log$logDate/$logFileName is:
    `cat $logDirectory/log$logDate/$logFileName | 
        awk -F":" '$1$2 >= '"$startTime"' && $1$2 <= '"$endTime"' {print $0}' |
        grep ${shipmentStrings[$i]} | 
        wc -l`" 

Thanks. I've copied here just some dummy data of what the log would look like.

07:00:01.124 dfsdfjsdflkjsdfkljsdflkjEDIRequestServiceModule/Routedasdfkl
07:05:02.123 fsldjfsdfskdfjsdfsdkfjEDIRequestServiceModule/Rouedsdfjsdfj
07:10:33.233 sdkjfasdflkjasdfaskdfjasdfkljEDIRequestServiceModule/Routed
09:30:02.222 sefsklfjsdfljksdfEDIRequestServiceModule/Routedasdfdf
14:00:12.222 sdfsdfsdfsdfsdfsdfEDIRequestServiceModule/Routed sdfsdfs
14:01:22.223 sdfsdfsdfsdfsdfsdfsdfEDIRequestServiceModule/Routed sdfsdfsdf
17:00:22.222 sdfsdfsdfsdfsdfsdfEDIRequestServiceModule/Routedsdfsdfsdf
18:00:33.333 sdfsdfsdfsdfsdfsdfsdfsdfsdfsdfsdf

If anyone wants to see the whole script, just reply and i can cut and paste it into here as well.

Answer 1

You should use AWK's variable passing. You won't have to do all that awkward quoting.

awk -F":" -v start=$startTIME -v end=$endTime '$1$2 >= start && $1$2 <= end {print $0}'

Don't use cat. You can either do:

awk ... filename

or

< filename awk ... 

Also, AWK can do the grepping and counting for you:

< $logDirectory/log$logDate/$logFileName awk -F":" -v start=$startTIME -v end=$endTime -v selection=${shipmentStrings[$i]} '$0 ~ selection { if ($1$2 >= start && $1$2 <= end) count++} END {print count}'

Also, you should probably use a variable and shorten your echo. And use $() instead of backticks:

shipmentcount=$(... awk ...)
echo "Number of ... is: $shipmentcount"

Or even let the END clause in the AWK script print the message:

< $logDirectory/log$logDate/$logFileName \
awk ... -v file=$logDirectory/log$logDate/$logFileName \
    '... END {print "Number of "selection" shipments ... "file" ... is: " count}'