Personalised bulk email programmatically without timing out

Question

I have a list of around 5,000 to 10,000 (individual user supplied) email addresses from people all over the world, each associated with their username and language codes. I also have a single message translated into the different languages of the users that I want to email. Now, I would like to send a single plain text email to each of the address, with the actual text of the email varying based on the user language, and personalised with the username of the person I'm emailing.

Because of the personalised requirement, and the fact that they will only be emailed once (per year or two with a overlapped but different user list), formal mailing list is probably (& preferably) out. 3rd party bulk email service is also out.

Ignoring programming time, what is the least manually time consuming way to do this in (preferably) PHP? (I am writing the script(s), but not necessarily the person that end up "pressing the button" to send it.) The ideal result is the person sending only having to type a single command to run the script (supplying the email list) and all the email will be sent with no more user intervention. This mean I am looking to avoid things like setting up cron jobs to run the script repeatedly until the email list is exhausted.

When this was done before a year ago, I wrote a PHP script that simply read in the email list line by line processing the username, email address, and language code and build the desired email text out of that before supplying it to PHPMailer to send individually. The problem I had was the script timing out and me not knowing where it got up to so that I can trim the email list at the right place to start again. I ended up manually splitting up the 1 email list into several sub-list that was short enough so that the script doesn't time out. How do I either avoid timing out, or keep track of where the script is up to email address wise so that it can be restarted manually and no person is sent emails more than once?

What other issues are there to take into account, such as avoiding blacklisting etc.?

Answer 1

PHP's not the right tool for the job here - you'll want something that runs independent of a web server (although potentially triggered by it) or you'll certainly run into timeouts.

For example, you could have PHP system() out to another script (Perl? Python?), which reads job information from a database then forks into the background to do its work.

Answer 2

You said no Cron scripts - do you not have access to Cron on your server? Or is your concern not wanting to manually set up the Cron job every time a mail job has to be sent out?

You might want to look at http://pear.php.net/package/Mail_Queue - it's something I've been using recently. You can have your user-called script dump all the emails at once into the mail queue. This is much faster than actually sending the emails on the fly since you won't have to deal with communication between servers, but rather just your database.

Then you have a Cron script that, every few minutes, calls the Mail_Queue::sendMailsInQueue command with a limit on the number of emails it sends per Cron call. You won't have to mess with the Cron script if that's your concern - if the queue is empty, it'll just exit. Your email will then trickle out of the server at a reasonable pace per call. This will prevent timeouts due to the limit on emails per cron call. It will also help avoid trouble with other mail servers, who might not be happy to suddenly be hammered by a lot of emails from you at once.

Answer 3

You need to read about the function set_time_limit and maybe ini_set('memory_limit', xxMB') for memory;

You can run a php cli script from your web process, which forks (pcntl_fork) then kills the parent (the parent cli process exits). The webserver thread running the script can then continue with other code or exit.

Now the cli child process (A) can fork (and become the parent) and monitor a child (B) which sends out the emails. If the child B dies, the now parent A process can fork again and the new child continues where the previous left off.

You have to keep track of who you sent email to in a file/db/shared memory or through paired sockets (socket_create_pair) with the parent process.

I hope you get the idea.