This question is language independent.
I have an application that handles requests in a loop. During this loop for each request multiple actions are taken. These actions are sitting inside try / catch / log blocks. I am now extending this to notify administrators of severe errors via email.
This is all very easy, except for 1 thing. We are relying on the client/s to implement their own email delivery redundancy, and I know from experience there will always be 1 client who just has 1 smtp exchange server, and this is bound to go down from time to time.
So here is the dilemma
Scenario 1 (don't handle the error during failed send) - when I send an email to admin and smtp is down it will break the app (app will stop running, and additional loops will stop processing, because the error is unhandled) This means that the error reporting which was supposed to be beneficial to the app suddenly becomes the reason why 99/100 requests don't get processed because there was an issue with request 1.
Scenario 2 (handle the exception during failed send) - this means that I surround send code in try/catch/log blocks, great! the application processes all requests 99 of them , except 1, but the admin now has no notification of this 1 error via email because when it tried to send smtp was down, and that error was simply logged to the application log, the admin who doesn't check this log for days (even weeks) at a time now has no way to know that error took place.
So is there a win/win way to solve this problem or am I always going to be at a loss, and in the mercy of smtp being up. Remember it is out of our scope to manage email server redundancy.