I start a cronjob at 1:00 A.M. to use curl to check for updates on the web for some files, save needed updates to a text file, and then read the text file and download off of that. I then output the curl responce to a log file so I can see what's happening in the morning. It seems my connection doesn't start for about 4 seconds after cron calls my script. i.e. The first few lines of my log are:
1:00:01 A.M. curl: (6) cannot resolve host 'somehost.com/dir/file.zip'
# which is the error I see when I have no internet connection
and repeated a few times until about 4 seconds after 1:00 when the files start downloading and I see the normal curl output. Unfortunately this means I always miss about 7-8 files at the top of my update list that never get downloaded.
The only cause I can think of is that even though I allowed my network adapter to wake my computer, it takes ~4 seconds to connect to the internet after my computer wakes up; in those same 4 seconds my script is running down my text file, calling curl, and failing to download anything.
Is there anything in curl that deals with this?
I know I can write a wake up script or pass the error urls back through the script to download later, but possible infinite loops could come up if a site is down or I actually don't have internet service and it'd be nice if I could just use curl to keep trying to wake my network.