Framework or design pattern for mailing all users of a webapp

Question

My app takes care of user registration (with the option to receive email announcements), and can easily handle the actual template-based rendering of email for a given user. JavaMail provides the mail transport layer. But how should I design the application layer between the business objects (e.g. User) and the mail transport?

The straightforward approach would be a simple, synchronous loop: iterate through the users, queue the emails, and be done with it. "Queue" might mean sending them straight to the MTA (mail server), or to an in-memory queue to be consumed by another thread.

However, I also plan to implement features like throttling the rate of emails, processing bounced emails (NDRs), and maintaining status across application restarts. My intuition is that a good design would decouple this from both the business layer and the mail transport layer as much as possible. I wondered if others had solved this problem before, but after much searching I haven't found any Java libraries which seem to fit this problem. Standalone mail apps such as James or list servers are too large in scope; packages like Spring's MailSender or Commons Email are too small in scope (being basically drop-in replacements for JavaMail). For other languages I haven't found anything appropriate either.

I'm curious about how other developers have gone about adding bulk mailing to their applications.

Answer 1

One option is to use a hardware appliance. My company uses Strongmail, at least for marketing communications: http://www.strongmail.com/index.php. I don't know much about it, but I think it handles bulk e-mail concerns like do-not-contact lists, throttling, avoiding getting spam filtered, etc.

Answer 2

The approach I've been happiest with is to provide an interface to my application to "send" mails. In reality, the implementation of this interface simply queues the mail into a database for later processing. From the application's perspective, this interface is fast, as it performs very little actual work. Plus, the persistence survives server downtime, as you mentioned.

Another thread reads from the queue and takes it's sweet time sending mail up to it's configured throttle, and flags messages in the queue after successfully processing them (effectively dequeuing them without deleting them). This provides both a history of sent mail, and a reference when mail is bounced, etc. I delete from the queue 7 days after a successful send.

In terms of decoupling the solution from the mail transport layer... I've applied this approach to an automated Twitter client, and found it to be equally successful.