Java: how to collect incoming data fragments into properly terminated strings for subsequent parsing?

Question

I just joined StackOverflow after having found many great answers here in the past. Here's my first question:

EDIT: I feel bad... my first question to StackOverflow turned out to be a "wild goose chase". The problem is in the data, not the code reading the data. I was looking for a solution in the wrong place! I will explain in a comment below.

Please disregard this question.

In my application, data arrives in fragments (it is serial data). I want to pass the data along to a method that will parse it. But I only want to pass along properly terminated lines. The lines will be terminated by '\r' (return char). Sometimes a fragment of data comes in that has multiple terminated lines. But sometimes a fragment of data comes in that doesn't yet have a terminator (the terminator will arrive later).

My goal is to be able to send only properly terminated lines (complete lines) to dataProcessor.logSerialData. (The code works without this requirement, but I could write much cleaner code if I could meet this requirement. The parsing logic -- not shown here -- would be easier to maintain.)

Here is my code that DOES work (but does not meet my new requirement):

case SerialPortEvent.DATA_AVAILABLE:
            try
            {
                int len = -1;
                while ((len = in.read(readBuffer)) > 0)
                {
                    if (len > 0)
                    {
                        String s = new String(readBuffer, 0, len);
                        if (dataProcessor != null)
                        {
                            dataProcessor.logSerialData(s);
                        }
                        else
                        {
                            Logger.log("ERROR: packetReceiver is null");
                        }
                    }
                }
            }
            catch (IOException ex)
            {
                Logger.log(ex);
                return;
            }

Here is an example of one of the many things I tried that does NOT work. It somehow causes errors in the data stream.

private static final String REGEX = ".*\\r";
private Pattern p = Pattern.compile(REGEX);

public void serialEvent(SerialPortEvent e)
{
    byte[] readBuffer = new byte[11000];//tried many values for buffer size

    switch (e.getEventType())
    {
        case SerialPortEvent.DATA_AVAILABLE:
            try
            {
                int len = -1;
                while ((len = in.read(readBuffer)) > 0)
                {
                    if (len > 0)
                    {
                        String input = new String(readBuffer, 0, len);
                        Matcher m = p.matcher(input);
                        int count = 0;
                        while (m.find())
                        {
                            count++;
                            String terminatedLine = m.group();
                            if (!terminatedLine.isEmpty())
                            {
                                completedLines.add(terminatedLine);
                            }
                        }
                    }
                }
            }
            catch (IOException ex)
            {
                Logger.log(ex);
                return;
            }
            for (String line : completedLines)
            {
                dataProcessor.logSerialData(line);
            }

Any suggestions? (In fact, any idea why the new code introduces errors into the data stream? Every variation of the working code that I have tried does not work correctly. I have tried regex and non-regex methods and any other things. The only code that works is the working example shown above. Serial data processing seems very tricky! I could use some help.

Answer 1

This should work as needed.

private String remains = "";
...

int len = -1;
while ((len = in.read(readBuffer)) > 0) {
    if (len > 0) {
        String input = remains + new String(readBuffer, 0, len);
        Matcher m = p.matcher(input);
        int last = 0;
        while (m.find()) {
            String terminatedLine = m.group();
            last = m.end(0);
            if (!terminatedLine.isEmpty()) completedLines.add(terminatedLine);
        }
        remains = input.substring(last);
    }
}

Answer 2

It is possible to wrap the InputStream into a Reader into a BufferedReader and read lines from that?

Answer 3

None of the various methods of reading by lines worked. The original method worked only because it did not look for line terminators.

Upon seeing that none of the "sure fire" methods worked predictably, I did further investigation. It seems that there is some inconsistency in the incoming data with regard to line terminators. Java sees terminators where there should not be any, and misses terminators that are there.

After spending several days on this, I think I'm just going to go back to the original method that worked without looking for line terminators.

Sorry for the bogus question.