how to collate output in python logging memoryhandler + smtphandler

Question

I have the logging module MemoryHandler set up to queue debug and error messages for the SMTPHandler target. What I want is for an email to be sent when the process errors that contains all debug statements up to that point (one per line). What I get instead is a separate email for every debug message.

This seems like it should be trivial, and part of the logging package, but I can't find anything about it, no examples, nothing on Google.

log = logging.getLogger()
log.setLevel(logging.DEBUG)
debug_format = logging.Formatter("%(levelname)s at %(asctime)s in %(filename)s (line %(lineno)d):: %(message)s")

# write errors to email
error_mail_subject = "ERROR: Script error in %s on %s" % (sys.argv[0], os.uname()[1])
error_mail_handler = logging.handlers.SMTPHandler(SMTP_HOST, 'errors@'+os.uname()[1], [LOG_EMAIL], error_mail_subject)
error_mail_handler.setLevel(logging.ERROR)
#error_mail_handler.setLevel(logging.DEBUG)
error_mail_handler.setFormatter(debug_format)

# buffer debug messages so they can be sent with error emails
memory_handler = logging.handlers.MemoryHandler(1024*10, logging.ERROR, error_mail_handler)
memory_handler.setLevel(logging.DEBUG)

# attach handlers
log.addHandler(memory_handler)
log.addHandler(error_mail_handler)

Related to this:

Do I need to add the error_mail_handler to the logger explicitly if it is a target of memory_handler anyway? Should error_mail_handler be set to DEBUG or ERROR target? Does it even need a target when it is being fed from memory_handler?

Would love to see some working code from anyone who has solved this problem.

Answer 1

You might want to use or adapt the BufferingSMTPHandler which is in this test script.

In general, you don't need to add a handler to a logger if it's the target of a MemoryHandler handler which has been added to a logger. If you set the level of a handler, that will affect what the handler actually processes - it won't process anything which is less severe than its level setting.