Meaningful interaction with IIS SMTP Server in .Net

Question

Our business sends a newsletter to a vast number of subscribers every week. When the business was very young, before I joined, they used a "free" version of some mass mailer that took six hours to send 5K mails and fell foul of every reverse DNS check on the internet.

I upgraded this to a bespoke .Net widget that ran on the correct server and could send up to about 20k mails in half an hour with full DNS compliance. Unfortunately (or fortunately depending on your standpoint) our mail list has now outgrown this simple tool. In particular its lack of adequate throttling, it can make more mails than the server can comfortably send at once. I need to actually monitor how full the IIS SMTP server's available outgoing mail storage allocation is and throttle the load accordingly.

Unfortunately I can find no information on where a mail object goes when (or even if) it is turned into a mail. I can implement a filesystemwatcher if I have a place to watch, currently I don't. If no actual mail file is ever created I guess I will have to create one to implement the functionality but I need to know where to put it. It would also be more reassuring to allow the system to confirm sending somehow but I have no idea how to go about retrieving data from the system that says a mail has been sent.

Extensive Googling has proven vague on these points; so I was wondering if anyone here knew where I could get a guide to these problems, or could otherwise point me in the right direction.

Many thanks.

EDIT: In the end I gave up trying to measure throughput on the IIS SMTP server as a bad job. It just didn't seem to want to play. I'm now carrying out my logging in a separate location and just shunting it through to the SMTP server thereafter. I still don't know of anyone who really bothers trying to keep tabs on the doings of the IIS SMTP server and so this question as of this writing goes unanswered.

Oh well...

Answer 1

I would use the PerformanceCounter component to read the SMTP Service's Local Queue Length counter. That should keep you in control :-)

Answer 2

If your .net widget is bespoke, why not just throttle it's output to some (definable) throughput?

As an alternative you might be able to fiddle with some registry settings for the SMTP server.

http://blog.rednael.com/CommentView,guid,dc20366c-3629-490a-a8ee-7e8f496ef58b.aspx

Apparently there are also some WMI counters (SMTP Server\Remote Queue Length and SMTP Server\Remote Retry Queue Length) that will give you useful information.

http://www.tech-archive.net/Archive/Internet-Server/microsoft.public.inetserver.iis.smtp_nntp/2008-02/msg00011.html

Answer 3

Okay so I've been working on this project for ages now and I thought I might share my findings with the world.

The IIS SMTP Server

All mails created using the IIS SMTP server are sent, in the first instance, to the Pickup Directory. If you are sending one mail then you will have to operate in Matrix time to actually ever see it there because it will probably just go, straight away.

On an individual mail's way out of the door it passes through the queue folder in IIS.

If you wanted to watch the Performance Counter to monitor this process you ould look at the "Remote Queue Length". (The reason for this is that the "Local Queue Length" monitors mails sent "Locally" within the network. "Remote" in this instance refers to "Outside into the world". The specific definition of "Local" escapes me as we send no local mail but I imagine it means queued to go to mailboxes contained within the specific installation of IIS on the server or any local grouping thereof.)

From an Exchange point of view it seems to be the equivalent of mails sent within the Exchange Domain and those sent out of that domain into the wider world.

Anyhow. The Remote Queue Length doesn't tell the whole story. You also have to look at the Remote Retry Queue, the number of Current Outbound Connections and, for belt and braces sake the actual number of files in the queue directory.

Here's why:

• Remote Queue: All messages that have not yet been sent, however many times this has been tried. The number of mails currently assigned to any open connections are not counted as they are in a state of "being tried".
• Remote Retry Queue: All messages that have not yet been sent that have, at some point in the past, been assigned to an open connection for delivery. Obviously the delivery must have failed or the message would have been delivered. Any messages currently assigned to an open connection for a retry are not counted.
• Current Outbound Connections: Shows when the server is attempting to send queued mails, more than one message can be assigned to an outbound connection. Messages thus assigned are not counted in either the Remote Queue or the Remote Retry queue. Physical
• Files in the queue directory: This shows the number of mails still in the Queue directory. This will decrease as mails are successfully delivered.

Example: If you have 0 outbound connections and 50 mails in the Queue directory then the Remote Queue, Retry Queue and Physical files will all read at 50. When a retry flag is raised (this is a setting in IIS) the number of connections increases and the number of mails in the queues decreases. Until a mail is delivered the number of physical files remains the same. However as more than one mail can be sent on a current connection 1 connection may result in Remote Queue and Retry Queue lengths of 47 or lower. If, during the retry event, any mails are successfully delivered the number of physical files in the Queue directory will then decrease. When the connection closes the queue counters should all stabilise again.

Logging

It is possible with .Net's Mail library to Specify a Pickup directory separate from the IIS default. Here you can queue mails and get a bespoke service to occasionally move the mails into the IIS directory where the IIS service will take over and send out queued mails.

To do this you will be looking for the SmtpClient object's "DeliveryMethod" property which should be set to SmtpDeliveryMethod.SpecifiedPickupDirectory.

To actually set the SpecifiedPickupDirectory you should set the SmtpClient's PickupDirectoryLocation property.

When mails are delivered to this location they are stored as .eml files. The filename is a GUID. This means that multiple emails will be despatched in an essentially random order. You could, in theory, write code to address this situation if desired. The .eml file follows a standard format which can be read by opening the .eml in notepad. Parsing this will allow you to extract information for a log.

I hope this high level overview of the way the SMTP server in IIS works is of some assistance to someone in a similar position to the one I was in in March.