Parsing email with Python

Question

I'm writing a Python script to process emails returned from Procmail. As suggested in this question, I'm using the following Procmail config:

:0:
|$HOME/process_mail.py

My process_mail.py script is receiving an email via stdin like this:

From hostname Tue Jun 15 21:43:30 2010
Received: (qmail 8580 invoked from network); 15 Jun 2010 21:43:22 -0400
Received: from mail-fx0-f44.google.com (209.85.161.44)
by ip-73-187-35-131.ip.secureserver.net with SMTP; 15 Jun 2010 21:43:22 -0400
Received: by fxm19 with SMTP id 19so170709fxm.3
for <[email protected]>; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.103.84.1 with SMTP id m1mr2774225mul.26.1276652853684; Tue, 15
Jun 2010 18:47:33 -0700 (PDT)
Received: by 10.123.143.4 with HTTP; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)
Date: Tue, 15 Jun 2010 20:47:33 -0500
Message-ID: <[email protected]>
Subject: TEST 12
From: Full Name <[email protected]>
To: [email protected]
Content-Type: text/plain; charset=ISO-8859-1

ONE
TWO
THREE

I'm trying to parse the message in this way:

>>> import email
>>> msg = email.message_from_string(full_message)

I want to get message fields like 'From', 'To' and 'Subject'. However, the message object does not contain any of these fields.

What am I doing wrong?

Answer 1

I answer to myself.

I found a bug in the code that builds the messages. It's appending linebreaks between some lines, preventing the parser from working properly.

Answer 2

It looks like you have linefeeds without spaces prepended to the additional lines, which according to RFC 2822 §2.3.2 is illegal:

Each header field is logically a single line of characters comprising
the field name, the colon, and the field body. For convenience
however, and to deal with the 998/78 character limitations per line,
the field body portion of a header field can be split into a multiple
line representation; this is called "folding". The general rule is
that wherever this standard allows for folding white space (not
simply WSP characters), a CRLF may be inserted before any WSP. For
example, the header field:

    Subject: This is a test

can be represented as:

    Subject: This
     is a test

It should look something like this:

From hostname Tue Jun 15 21:43:30 2010
Received: (qmail 8580 invoked from network); 15 Jun 2010 21:43:22 -0400
Received: from mail-fx0-f44.google.com (209.85.161.44)
    by ip-73-187-35-131.ip.secureserver.net with SMTP; 15 Jun 2010 21:43:22 -0400
Received: by fxm19 with SMTP id 19so170709fxm.3
    for <[email protected]>; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.103.84.1 with SMTP id m1mr2774225mul.26.1276652853684; Tue, 15
    Jun 2010 18:47:33 -0700 (PDT)
Received: by 10.123.143.4 with HTTP; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)
Date: Tue, 15 Jun 2010 20:47:33 -0500
Message-ID: <[email protected]>
Subject: TEST 12
From: Full Name <[email protected]>
To: [email protected]
Content-Type: text/plain; charset=ISO-8859-1

ONE
TWO
THREE

Answer 3

You must ensure that the lines are not accidentally broken (as they are above, though it's hard to say if that was a copy-paste problem) -- with an intact message such as:

Received: (qmail 8580 invoked from network); 15 Jun 2010 21:43:22 -0400
Received: from mail-fx0-f44.google.com (209.85.161.44) by ip-73-187-35-131.ip.secureserver.net with SMTP; 15 Jun 2010 21:43:22 -0400
Received: by fxm19 with SMTP id 19so170709fxm.3 for <[email protected]>; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.103.84.1 with SMTP id m1mr2774225mul.26.1276652853684; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)
Received: by 10.123.143.4 with HTTP; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)
Date: Tue, 15 Jun 2010 20:47:33 -0500
Message-ID: <[email protected]>
Subject: TEST 12
From: Full Name <[email protected]>
To: [email protected]
Content-Type: text/plain; charset=ISO-8859-1

ONE
TWO
THREE

then

msg = email.message_from_string(msgtxt)
print msg['Subject']

prints TEST 12 as desired.