Please help with my problems retrieving email using POP3 protocol

Question

I am using C# and Microsoft .Net Compact Framework 1.0. I tried to implement the POP3 protocol using the System.Net.Sockets.NetworkStream and TcpClient classes. I am able to log in and receive email and save the attachments with some email servers. But for some, I keep running into problems.

I read the size of the email message by sending the List <Index> command. In some situations I get a size significantly smaller than the actual value. For example, for the same EMail:

Actual size: 577860, Returned size: 421096 using Exchange 2007
Actual size: 561005, Returned size: 560997 using Exchange 2003

Why don't I ever get the right size? The following the code I am using.

The size of the email never matches the size of the StringBuilder at the end of PopRead procedure. I am not able to read the email reliably because the Email size is not reliable and the DataAvailable property of the NetworkStream is sometimes false, even if there is more data that can be read.

I observed that the DataAvailable property is false more often when I am trying to connect to the email server over the air (using data-plan) than when using the internet connection of the computer through activesync.

If it helps, the email servers are Exchange 2003 and Exchange 2007.

private bool POPRead(StringBuilder strBuffer, long lngFetchMailSize)
{
   const int bufferSize = 1024;

    byte[] inb;
    if (enc == null)
    {
        enc = new ASCIIEncoding();
    }

    try
    {
        if (lngFetchMailSize > 0 && lngFetchMailSize < (32 * bufferSize))
        {
            // limit the size of the buffer as the amount of memory
            // on Pocket PC is limited.
            inb = new byte[lngFetchMailSize];
        }
        else
        {
            inb = new byte[bufferSize];
        }
        Array.Clear(inb, 0, inb.Length);
        bool bMoreData = true;
        long iBytesRead = 0L;
        int bytesReadInThisRound = 0;

        int numberOfTimesZeroBytesRead = 0;

        while (bMoreData)
        {
            bytesReadInThisRound = this.nsPOP.Read(inb, 0, inb.Length);
            iBytesRead += bytesReadInThisRound;

            if (bytesReadInThisRound == 0)
            {
                numberOfTimesZeroBytesRead++;
            }
            else
            {//If on a retry the data read is not empty, reset the counter.
                numberOfTimesZeroBytesRead = 0;
            }

            strBuffer.Append(enc.GetString(inb, 0, bytesReadInThisRound));
            Array.Clear(inb, 0, bytesReadInThisRound);
            // DataAvailable sometimes gives false even though there is
            // more to be read.
            bMoreData = this.nsPOP.DataAvailable;
            // Use this number (5), since some servers sometimes give the size
            // of the email bigger than the actual size.
            if ((lngFetchMailSize != 0 && !bMoreData)
                && (iBytesRead < lngFetchMailSize)
                && numberOfTimesZeroBytesRead < 5)
            {
                bMoreData = true;
            }
        }
    }
    catch (Exception ex)
    {
        string errmessage = "Reading email Expected Size: " + lngFetchMailSize;
        LogException.LogError(ex, errmessage, false, "oePPop.POPRead");
        Error = ex.Message + " " + errmessage;
        return false;
    }
    finally
    {
        GC.Collect();
    }
    return true;
}

The following procedure is used to get the size of the email message:

private long GetMailSize(int index)
{
    StringBuilder strBuffer = new StringBuilder();
    const string LISTError = "Unable to read server's reply for LIST command";
    if ((this.POPServer != null) && (this.nsPOP != null))
    {
        if (!this.POPWrite("LIST " + index))
        {
            return -1L;
        }
        if (!this.POPRead(strBuffer))
        {
            this.Error = LISTError;
            return -1L;
        }
        if (!this.IsOK(strBuffer))
        {
            return -1L;
        }
        string strReturned = strBuffer.ToString();
        int pos1 = strReturned.IndexOf(" ", 3);
        if (pos1 == -1)
        {
            this.Error = LISTError;
            return -1L;
        }
        int pos2 = strReturned.IndexOf(" ", (int)(pos1 + 1));
        if (pos2 == -1)
        {
            this.Error = LISTError;
            return -1L;
        }
        int pos3 = strReturned.IndexOf("\r\n", (int)(pos2 + 1));
        if (pos3 == -1)
        {
            this.Error = LISTError;
            return -1L;
        }
        long mailSize = 0;
        Int64.TryParse(strBuffer.ToString(pos2 + 1, pos3 - (pos2 + 1)).Trim(), out mailSize);
        return mailSize;
    }
    this.Error = NotConnectedError;
    return -1L;
}

Hope I gave all the information needed for solving the issue. Any help or pointer in the right direction would be a lot of help.

Thanks,
Chandra.

Answer 1

POP3 is a tricky protocol. So many different servers to deal with, many with their own obscure quirks. If this is a production app, I would seriously consider buying a 3rd party component that's been thoroughly tested.

Answer 2

Your error may lie in using the ASCIIEncoder the way you are.

From MSDN:

Data to be converted, such as data read from a stream, can be available only in sequential blocks. In this case, or if the amount of data is so large that it needs to be divided into smaller blocks, the application should use the Decoder or the Encoder provided by the GetDecoder method or the GetEncoder method, respectively.

Since you are decoding a little bit at a time, it is possible it is decoding portions of the stream incorrectly.

I would change your code to use a decoder, or to read the entire message at once, then decode it with the GetString() member.

As an additional sanity check you could use the message size that RETR <index> returns and see if it matches what LIST returns. If they don't match, I at least, would go with what RETR returns.

Answer 3

Since an email should end with a period, I would do this comparison for termination of the loop.

Instead of

 if ((lngFetchMailSize != 0 && !bMoreData)
       && (iBytesRead < lngFetchMailSize)
       && numberOfTimesZeroBytesRead < 5)
  {
      bMoreData = true;
  }

I would write it as

  if(!bMoreData 
    && strBuffer.ToString(strBuffer.Length - 5, 5) != "\r\n.\r\n")
  {
       bMoreData = true;
  }