Parsing "From" addresses from email text

Question

I'm trying to extract email addresses from plain text transcripts of emails. I've cobbled together a bit of code to find the addresses themselves, but I don't know how to make it discriminate between them; right now it just spits out all email addresses in the file. I'd like to make it so it only spits out addresses that are preceeded by "From:" and a few wildcard characters, and ending with ">" (because the emails are set up as From [name]<[email]>).

Here's the code now:

import re #allows program to use regular expressions
foundemail = []
#this is an empty list

mailsrch = re.compile(r'[\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4}')
 #do not currently know exact meaning of this expression but assuming
 #it means something like "[stuff]@[stuff][stuff1-4 letters]"

        # "line" is a variable is set to a single line read from the file
# ("text.txt"):
for line in open("text.txt"):

    foundemail.extend(mailsrch.findall(line))

    # this extends the previously named list via the "mailsrch" variable
      #which was named before

print foundemail

Answer 1

if you can be reasonably sure that lines containing these email addresses start with whitespace followed by "From:" you can simply do this:

addresslines = []
for line in open("text.txt"):
    if line.strip().startswith("From:"):
        addresslines.append(line)

then later - or on adding them to the list - you can refine the addresslines items to give out exactly what you want

Answer 2

"[stuff]@[stuff][stuff1-4 letters]" is about right, but if you wanted to you could decode the regular expression using a trick I just found out about, here. Do the compile() in an interactive Python session like this:

mailsrch = re.compile(r'[\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4}', 128)

It will print out the following:

in 
  category category_word
  literal 45
max_repeat 1 65535 
  in 
    category category_word
    literal 45
    literal 46
literal 64 
in 
  category category_word
  literal 45
max_repeat 1 65535 
  in 
    category category_word
    literal 45
    literal 46
max_repeat 1 4 
  in 
    range (97, 122)
    range (65, 90)

Which, if you can kind of get used to it, shows you exactly how the RE works.

Answer 3

Try this out:

>>> from email.utils import parseaddr

>>> parseaddr('From: [email protected]')
('', '[email protected]')

>>> parseaddr('From: Van Gale <[email protected]>')
('Van Gale', '[email protected]')

>>> parseaddr('    From: Van Gale <[email protected]>   ')
('Van Gale', '[email protected]')

>>> parseaddr('blah abdf    From: Van Gale <[email protected]>   and this')
('Van Gale', '[email protected]')

Unfortunately it only finds the first email in each line because it's expecting header lines, but maybe that's ok?

Answer 4

I'd do it by expanding the regular expression you're using to include the extra text you want to match. So first, let me explain what that regex does:

[\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4}

• [\w\-] matches any "word" character (letter, number, or underscore), or a hyphen
• [\w\-\.]+ matches (any word character or hyphen or period) one or more times
• @ matches a literal '@'
• [\w\-] matches a word character or hyphen
• [\w\-\.]+ matches one or more word characters, hyphens, and/or periods
• [a-zA-Z]{1,4} matches 1, 2, 3, or 4 lowercase or uppercase letters

Now, to modify this for your purposes, let's add regex parts to match "From", the name, and the angle brackets:

From: [\w\s]+?<([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>

• From: matches the literal text "From: "
• [\w\s]+? matches one or more consecutive word characters or space characters. The question mark makes the match non-greedy, so it will match as few characters as possible while still allowing the whole regular expression to match (in this case, it's probably not necessary, but it does make the match more efficient since the thing that comes immediately afterwards is not a word character or space character).
• < matches a literal less-than sign (opening angle bracket)
• The same regular expression you had before is now surrounded by parentheses. This makes it a capturing group, so you can call m.group(1) to get the text matched by that part of the regex.
• > matches a literal greater-than sign

Since the regex now uses capturing groups, your code will need to change a little as well:

import re
foundemail = []

mailsrch = re.compile(r'From: [\w\s]+?<([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>')

for line in open("text.txt"):
    foundemail.extend([m.group(1) for m in mailsrch.finditer(line)])

print foundemail

The code [m.group(1) for m in mailsrch.finditer(line)] produces a list out of the first capturing group (remember, that was the part in parentheses) from each match found by the regular expression.

Answer 5

mailsrch = re.compile(r'[\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4}')

Expression breakdown:

[\w-]: any word character (alphanumeric, plus underscore) or a dash

[\w-.]+: any word character, a dash, or a period/dot, one or more times

@: literal @ symbol

[\w-][\w-.]+: any word char or dash, followed by any word char, dash, or period one or more times.

[a-zA-Z]{1,4}: any alphabetic character 1-4 times.

To make this match only lines starting with From:, and wrapped in < and > symbols:

import re

foundemail = []
mailsrch = re.compile(r'^From:\s+.*<([\w\-][\w\-\.]+@[\w\-][\w\-\.]+[a-zA-Z]{1,4})>', re.I | re.M)
foundemail.extend(mailsrch.findall(open('text.txt').read()))

print foundemail

Answer 6

Use the email and mailbox packages to parse the plain text version of the email. This will convert it to an object that will enable to extract all the addresses in the 'From' field.

You can also do a lot of other analysis on the message, if you need to process other header fields, or the message body.

As a quick example, the following (untested) code should read all the message in a unix style mailbox, and print all the 'from' headers.

import mailbox
import email

mbox = mailbox.PortableUnixMailbox(open(filename, 'rU'), email.message_from_file)

for msg in mbox:
   from = msg['From']
   print from

Answer 7

import email
msg = email.message_from_string(str)

# or
# f = open(file)
# msg = email.message_from_file(f)

msg['from']

# and optionally
from email.utils import parseaddr
addr = parseaddr(msg['from'])

Answer 8

Roughly speaking, you can:

from email.utils import parseaddr

foundemail = []
for line in open("text.txt"):
    if not line.startswith("From:"): continue
    n, e = parseaddr(line)
    foundemail.append(e)
print foundemail

This utilizes the built-in python parseaddr function to parse the address out of the from line (as demonstrated by other answers), without the overhead necessarily of parsing the entire message (e.g. by using the more full featured email and mailbox packages). The script here simply skips any lines that do not begin with "From:". Whether the overhead matters to you depends on how big your input is and how often you will be doing this operation.