Email threading

Question

I'm working at a helpdesk application where i have a standalone script that queries a mail server and parses the mail it finds there.
I'm facing the following issue: How do i figure it out in a reliable way what mail is in reply to what mail?
I could add something in the subject like "[ticket:21312]" and look for that but what if the user changes the subject? Is there another way? Can i do it by setting a custom mail header and look for that or the header will not be preserved between mail servers on reply back from user? What about when i send a message from my application to a non existing user or a user that has quota full and his server replies back with the usual standard message "the mail daemon at .... could not ...." then the subject will also be modified and i can't place the message correctly as a reply to an existing mail.

How does gmail do it? There the messages are sorted perfectly in almost all of the cases.

Answer 1

As you say custom headers might get lost and the subject might change. Use both. If one exists then you can identify the thread. I don't know of any better way to solve this.

Answer 2

in helpdesk email piping there are 3 basic methods:

a) include the id in the subject somewhere (works fine in practice)

b) have the id in the body somewhere

c) use an auto-generated email alias with the id, like "[email protected]". that can easily be handled by something like procmail or a script to pick out the id.

gmail might use a combination of the subject, In-Reply-To header (may not be defined) (References and Original-Message-ID headers possibly as well), and various heuristics, which work very well, but of course not necessarily bulletproof, and slightly more involved to implement. something like nestscape's original threading algorithm perhaps. though some have reported that gmail doesn't use the In-Reply-To header and relies mostly on the subject (as in this post).

Answer 3

If your message was sent with a Message-ID-Header any standards-conform mailer should add a In-Reply-To-Header referencing your Id. Additionally Referencesshould contain a list of all previous mails in this thread.

This works with most mail clients, to be safe for the bad clients you have to use the subject, the easy way is by adding the "[issue:123]" thing, a secondary fallback is to recognize the subject (after cutting of the "Re:" part in all the variations) for this it could help that you know most of your legitimate senders ...