file empties itself

Question

hello i have a shell script that sends out email. here is how it works:

• it reads the html from an url and put it in a file
• loop through all the emails and in the loop it reads from that html file

the problem is:

• the html file becomes empty after a few hundred email for no apparent reason. (i put du /thehtml.html in the email log.)

any one know why? thanks.

source:

/usr/local/bin/lynx --source http://www.site.com/email.php?message=1 > /root/lynx.html
read var < /root/lynx.html
if [  -z "$var" ]; then
     echo "" > /dev/null
else

         /usr/local/bin/lynx --source http://www.site.com/email.php?list=1 > /root/html_mail.html
         number=0
         OLDIFS = "$IFS"
         export IFS=","
         read mails < /root/html_mail.html
         for mail in $mails; do
              echo "To: $mail " > /root/finished_message.html
              cat /root/lynx.html >> /root/finished_message.html
              /usr/sbin/sendmail "$mail" < /root/finished_message.html
              echo ".*. $number. " - ". $mail .*." >> /bin/scripts/email.log
              du /root/lynx.html >> /bin/scripts/email.log
              number=$((number+1))
         done;
         IFS = "$OLDIFS"
         lynx --source http://www.site.com/email.php?done_with=1
    fi

Answer 1

It might be a race condition where 2 processes are trying to write to the file, which corrupts it. However, it's very hard to tell from your description. Why don't you try to reduce this to the simplest amount of code that reproduces the problem?

Answer 2

Try using mktemp for the filename.

I'm pretty sure you overwrite the file somewhere in your code - by running the code in parallel.

Try:

 SOURCE_HTML=`mktemp lynx-XXXX.html`
 lynx --source URL > $SOURCE_HTML
 # the rest ...

Also - is this some kind of spam generator?

Answer 3

Try putting this right after the else:

chmod -w /root/lynx.html

and see who complains.

Answer 4

Who starts the script, cron? To me it looks like you start an other instance of the script before the first one has finished sending the mails. One quick way to verify this is to add something like

echo "BEGIN mailer script" >>/bin/scripts/email.log

before the

/usr/local/bin/lynx --source http://www.site.com/email.php?list=1 > /root/html_mail.html

and

echo "END   mailer script" >>/bin/scripts/email.log

after done; before the IFS=

Next time the log shows zeroed out html_mail.html you most likely see something like

END
BEGIN
.*.
.*.
BEGIN
du zero size

in the log file. I do hope you don't see that because if you do, you may have sent out wrong or garbled messages in all of the overlap cases where html_mail.html has not been zeroed out.

As a side note I highly recommend phoku's suggestion of using unique temporary names for all temporary files created by scripts.