Where does Thunderbird store the UID of the last message downloaded via POP?

Question

I use Thunderbird to receive email using POP3. I have Thurnderbird configured to leave email on the server. Lets say one day I uses POP3 to retreive (RETR) 10 email messages, then I logout for the night. Overnight 10 more messages are sent to my mailbox. When I fire up Thunderbird the next morning, the STAT command should show 20 messages. However, Thunderbird should not download the first 10 messages; it should start at message 11 (or the unique identifier or UID for message 11). Thunderbird will send a POP3 UIDL command, then compare the UID's to the UID of the last message Thunderbird retrieved yesterday. It will find that the last UID matches the UIDL list for message 10, then Thunderbird will RETR 11, RETR 12, and so on.

In my case, the POP3 STAT command shows that I have 5379 messsages on the POP server. I have already received about 5000 of them. For some reason Thunderbird wants to download all 5379 messages instead of starting at 5001. I am trying to debug this and was looking for the UID that Thunderbird thinks was the last message retrieved.

Does anyone know where Thunderbird (on Windows) stores the last UID, which it will use to compare to the UIDL (list)?

Is there a way to manually set it so I can force Thunderbird to start retrieving somewhere close to 5001?

--update--

Thunderbird has a file called popstate.dat that contains the UID, an timestamp (epoch), and a flag. The flag indicates the action that Thunderbird is to perform for the associated message.

Evidently, Thunderbird does not actually work like I described above. I think Thunderbird does the following. It sends a POP3 UIDL command to get a list of the UID's stored on the POP server. It then compares this list to the UID's stored in popstate.dat. Any UID's that are not already in popstate.dat are new messages to be retrieved. The UIDL command previously returned the message number and the associated UID. Thunderbird must then do an POP3 RETR command using the message number associated with the UID's that it has not yet retreived. Thunderbird must also look at the flag in popstate.dat and take any actions for the associated message. For example, the flag "d" tells Thunderbird to delete the associated message. The "f" flag means that Thunderbird has only a truncated part of the message and should retrieve the full message.

At some point Thunderbird updates the popstate.dat with the new messages. I think this happens in a batch update to popstate.dat after all the actions have been completed. That is, if there are 10 new messages to retrieve, popstate.dat is not updated until all 10 messages have been retrieved.

I think my problem resides on the server. Apparently our infrastructure upgraded to a new version of the POP server and new UIDs were assigned on the new version. My popstate.dat had all the old UIDs. The UIDL to the new POP server send a list of 5000+ UIDs that Thunderbird did not have listed in popstate.dat. So, Thunderbird proceeded to download all 5000+ messages. If the new POP server had retained the old UID's then Thunderbird would have seen that I already had retrieved most of the 5000+ messages and would have just downloaded the ones that I did not have. I think most people in my organization use Outlook and do not use POP3, and however the version update was done to the POP server did not cause a problem for those users. Seems like some extra care was needed to ensure the new server had the same UIDs as the old server. Live and learn!

Heading

Answer 1

Which version of server software there is?

http://courier.sourcearchive.com/documentation/0.60.0-2/pop3dserver_8c-source.html

00718 ** The UIDL of the message is really just its filename, up to the first MDIRSEP character

May be you just need to replace first part of UIDL in popstate.dat to new ones?

Answer 2

There is an extension for Thunderbird to find duplicates and delete them. e.g. based on message-id (which is usually generated by the original sender of the mail and thus hasn't changed with your infrastructure update).

Answer 3

(My update to the question was really an answer, as pointed out in a comment. So I am copying the text of the update into this anwer in hopes of showing it as answered.)

Thunderbird has a file called popstate.dat that contains the UID, an timestamp (epoch), and a flag. The flag indicates the action that Thunderbird is to perform for the associated message.

Evidently, Thunderbird does not actually work like I described above. I think Thunderbird does the following. It sends a POP3 UIDL command to get a list of the UID's stored on the POP server. It then compares this list to the UID's stored in popstate.dat. Any UID's that are not already in popstate.dat are new messages to be retrieved. The UIDL command previously returned the message number and the associated UID. Thunderbird must then do an POP3 RETR command using the message number associated with the UID's that it has not yet retreived. Thunderbird must also look at the flag in popstate.dat and take any actions for the associated message. For example, the flag "d" tells Thunderbird to delete the associated message. The "f" flag means that Thunderbird has only a truncated part of the message and should retrieve the full message.

At some point Thunderbird updates the popstate.dat with the new messages. I think this happens in a batch update to popstate.dat after all the actions have been completed. That is, if there are 10 new messages to retrieve, popstate.dat is not updated until all 10 messages have been retrieved.

I think my problem resides on the server. Apparently our infrastructure upgraded to a new version of the POP server and new UIDs were assigned on the new version. My popstate.dat had all the old UIDs. The UIDL to the new POP server send a list of 5000+ UIDs that Thunderbird did not have listed in popstate.dat. So, Thunderbird proceeded to download all 5000+ messages. If the new POP server had retained the old UID's then Thunderbird would have seen that I already had retrieved most of the 5000+ messages and would have just downloaded the ones that I did not have. I think most people in my organization use Outlook and do not use POP3, and however the version update was done to the POP server did not cause a problem for those users. Seems like some extra care was needed to ensure the new server had the same UIDs as the old server. Live and learn!