hello,
We send out emails to around 10k client and right now the process takes around 45 min or so. The application runs on the server and just sends out the whole batch. I'm wondering if introducing threads and splitting up the list would speed up the process. If so, how many threads are optimal?
EDIT:
actually it is one message that we usually send and we add all the recipients to the bcc. However, it still takes a long time. We send it out through our server on a fiber line.
You must profile your process before making any decisions.
Where is the time spent (creating the messages, retrieving recipients, submitting the messages to SMTP, SMTP sending messages, etc.)?
Do not guess or assume you know where the bottleneck it, especially when considering a parallel approach. You could easily make things worse (or have no impact).
This definitely sounds like something that could be easily re-written to run in parallel and take advantage of multiple cores (or even threads on a single core).
However, that might not result in much improvement. There's a good chance that your limiting factor is bandwidth or throughput from the smtp server, in which case no amount of code changes will help you. Before you do anything else, you need to stop and make sure that your c# code is really where the slowness is.
Depends on your SMTP server and network setup setup, as well as your code. Your bottleneck is probably communication with your SMTP server - is it possible to run the e-mail sending code and the SMTP server on the same computer?
Profile first and see if you can cut down on the 0.27 sec/email you're currently at.
My first reaction was that you should just split the data and farm it out to N different servers.
Some ideas:
If the message you are sending is the same for all clients, send one message using BCC. SMTP server will probably handle a single message better then 10k messages.
If your company has multple SMTP servers for backup/load balancing, consider splitting up the request across your multiple SMTP servers.
It depends on your smtp server but the server i have used in one of the projects had limitation. For example if you try to send 5 emails in less than 1 sec, it marks it as a spam.
so i have used System.Threading.Thread.Sleep after 2 sent mails.
and dont forget to increase Server.ScriptTimeout because it could take more time than the default value for all the mails.
Implement some your own SMTP server logic. You probably could speed it up to 10-100 times if would send the message simultaneously to several hosts. It looks that now all the bottleneck is in your SMTP server which is not able to send more then 3 email per second. Real limits are much higher. Or you could change your current SMTP sever for better one. Or fine tune existent. Are you sure it is not configured to allow no more than 3 concurrent sends?