Email deliverability - Influencing factors

Question

[Our website] is very dependent on being able to successfully send email to its members. We are currently having trouble reaching all our members, especially hotmail users.

What do you recommend we do to improve our sending of email?

We are sending heavily user customized emails. So a third party solution would need a good api to support this.

Possible solutions:

• Would sending email through the app engine help for delivery rates?
• Does returnpath help? http://www.returnpath.net/

Update: Some good comments on how to improve and test our own email sending capabilities. Another option would be a third party solution.

We're sending updates on your networks activities, registration emails, new comment emails, new follower emails these type of things. Especially your networks activity is highly individual and problematic with most third party emailing solutions. Would need a very flexible email solution.

Are there sufficiently capable solutions out there?

Answer 1

We had a similar issue a while back.. You probably want to read up on Microsoft's Sender ID:

https://www.microsoft.com/mscorp/safety/technologies/senderid/default.mspx and look at the link called "Sender ID SPF Record Submission Form".

Answer 2

I'd in general be pessimistic about the IP reputation of platform-as-a-service type offerings. Testing Google AppEngine is on my to-do list, but I've there's been much talk about Amazon EC2 presenting a real problem -- these products are not very efficient preventing use by spammers, and reputation is taking a hit.

As for the practical steps of setting up outbound email, Jeff Atwood has a very nice and nearly comprehensive article on his blog.

What I'd certainly suggest is:

• Make sure your sending IP has a reverse DNS.
• Check your IP reputation for example at senderscore.org (though that's heavily US centric)
• Make sure bounces are handled on your side, and [email protected] is reachable
• Set up SPF and DKIM. SenderID if you want to.
• Sign up for all feedback loops at major mailbox providers / ISPs and act on spam complaints -- if your user complain, you're doing something wrong. Also, set a "friendly name" on your From: address, as some mailboxes will only display the local part -- " Update" is friendlier than only seeing "automatic" (Gmail does this).
• Watch the volume you send. If it's high from the start (>1000s/day to each major ISP) you may get blocked outright.

You'll find a lot of deliverability tips, most of the time from interested parties (email service providers). A relatively reputable resource is deliverability.com, backed in part by Return Path. Of course, going with a commercial email service provider might be a solution for you, but your use case is quite specific and you'll need real-time individual messaging, not marketing newsletters, if I understand you right.

I worked for a company that re-sold Return Path's tool -- so take this with a pinch of salt: It' won't help you get delivered. It can, however, be a valuable tool tracking down where your problems are. It is on the other hand expensive, and hiring a specialist that can go through your specific case might be more affordable. Or reading a lot and experimenting a lot yourself.

Answer 3

@chryss does a great job pointing out the important factors that need to be taken into consideration: -- reverse DNS, sender reputation, list management (ie, cleaning lists of addresses who have marked your email spam, invalid addresses, etc and keeping track of hard and soft bounces and acting accordingly to those events), SPF records, DKIM signatures, ISP feedback loops, ISP rate limits. Also, email content is important to keep in mind.

Generally speaking, this is all pretty complicated and annoying stuff to deal with, especially as your email volume increases.

In terms of IP reputation with PaaS systems, the key thing to remember is this: -- if you share an IP with someone who earns a poor reputation (say, a spammer on EC2), that reputation will negatively affect your deliverability. On the other hand, if you send from a dedicated IP, you have the opportunity to earn your own reputation - if you are a good sender, follow best practices, and your customers want the emails they expect to receive from you (which they should since it sounds like you are sending mostly transactional emails), you will maintain a great reputation and should enjoy good deliverability (granted all of the technical stuff mentioned above is taken care of).

We generally keep an eye on deliverability "chatter" online, and send out all the cool/useful stuff that we find on a daily basis through our twitter feed -- feel free to follow us: twitter.com/sendgrid. We are also beginning to ramp up our own blogging, so you can join the conversation if you like: blog.sendgrid.com.

Answer 4

Postmark and Sendgrid seem to offer a very decent api to use for sending email and improving deliverability. As a bonus stats are also handled by them.