Simple, Custom Parsing with c++

Question

Hi! I have been reading SO for some time now, but I truly cannot find any help for my problem.

I have a c++ assignment to create an IAS Simulator.

Here is some sample code...

0   1   a
1   2   b
2   c
3   1
10  begin
11  . load a, subtract b and offset by -1 for jump+
11  load M(0)
12  sub M(1)
13  sub M(3)
14  halt

Using c++, I need to be able to read these lines and store them in a "memory register" class that I already have constructed...

For example, the first line would need to store "1 a" in register zero.

How can I parse out the number at the line beginning and then store the rest as a string?

I have setup storage using a class that is called using mem.set(int, string);. int is the memory location at the beginning of the line and string is the stored instruction.

Edit: Some Clarifications:

• I must use standard libraries
• the grammar for the input file is here: http://www.cs.uwyo.edu/~seker/courses/2150/iascode.pdf
• The loader will overwrite duplicate line entries. That means the first line 11 in the sample will be overwritten by the second.

Answer 1

I'd suggest taking a look at the <ifstream> library.

Answer 2

Something like this may be a good use of the Boost.Spirit library. It's an EBNF parser generator in C++, like flex and yacc without the extra compilation steps.

Answer 3

#include <iostream> // #include <fstream> for file objects as others suggest
#include <string>
#include <map>
using namespace std;

map<int, string> my_program;

int line_num;
string line_text;
while ( cin >> line_num ) { // or any input stream such as a file
    getline( cin, line_text ); // standard function defined in <string>
    my_program[ line_num ] = line_text; // store line for next phase
}

This will read lines of the file until either the end is encountered, or a line which begins with something besides a number. Use cin.eof() to verify that the entire file was read, if you care.

Of course, since map sorts its contents, the lines will be in numerical order for the next phase.

Answer 4

Here's the easy part:

std::string text_to_parse;
unsigned int register_number;

void Parse_Line(std::istream& data_file)
{
  // Read in the register number.
  if(data_file >> register_number)
  {
    // Read the remaining line as a string {variable}
    getline(data_file, text_to_parse);

    //  Now do something with the text...
  }
  return;
}

An issue with your data file, is that it is not following an easy grammar or syntax. For example, you have two text lines that start with 11. Line 10 is not a "memory register" line, but an instruction line. Also, line 2 and 3 don't follow the same grammar as 0 and 1.

For more assistance, please post the grammar rules (preferable in BNF syntax or ASCII art).

Answer 5

Splitting the leading number and the rest of the line is not too difficult of a task. Use something like getline to read one line at a time from your input file, and store the line in a string char cur_line[]. For each line, try something like this:

• Declare a pointer char* pString and an integer int line_num
• Use the strstr function to find the first whitespace character, and assign the result to pString.
• Move pString forward one character at a time until it points to a non-whitespace character. This is the start of the string containing the "rest of the line".
• Use atoi on cur_line to convert the first entry in the string to an integer, and store the results in line_num
• Now, you should be able to call your function like mem.set(line_num, pString)

Interpreting those strings is going to be much more difficult, however...

Edit: As Mike DeSimone mentions, you can combine the strstr and atoi steps above if you use one of the strto* functions instead of atoi.

Answer 6

If the first part of the line is always a number, look at the strtoul function. From the man page:

strtoul -- convert a string to an unsigned long integer

LIBRARY

Standard C Library (libc, -lc)

SYNOPSIS

 #include <stdlib.h>
 unsigned long strtoul(const char *restrict str, char **restrict endptr, int base);

DESCRIPTION

The strtoul() function converts the string in str to an unsigned long value. The conversion is done according to the given base, which must be between 2 and 36 inclusive, or be the special value 0.

The string may begin with an arbitrary amount of white space (as determined by isspace(3)) followed by a single optional + or - sign. If base is zero or 16, the string may then include a 0x prefix, and the number will be read in base 16; otherwise, a zero base is taken as 10 (decimal) unless the next character is 0, in which case it is taken as 8 (octal).

The remainder of the string is converted to an unsigned long value in the obvious manner, stopping at the end of the string or at the first character that does not produce a valid digit in the given base. (In bases above 10, the letter A in either upper or lower case represents 10, B represents 11, and so forth, with Z representing 35.)

If endptr is not NULL, strtoul() stores the address of the first invalid character in *endptr. If there were no digits at all, however, strtoul() stores the original value of str in *endptr. (Thus, if *str is not \0 but **endptr is \0 on return, the entire string was valid.)

RETURN VALUES

The strtoul() function returns either the result of the conversion or, if there was a leading minus sign, the negation of the result of the conversion, unless the original (non-negated) value would overflow; in the latter case, strtoul() returns ULONG_MAX. In all cases, errno is set to ERANGE. If no conversion could be performed, 0 is returned and the global variable errno is set to EINVAL.

---

The key here is the endptr parameter. It sets a pointer to where you need to continue parsing. If endptr == str, then you know the line didn't start with a number.

I like the strto___ family of functions a lot more than the ato__ functions because you can set the base (including the context-sensing "base 0") and because the endptr tells me where to continue from. (And for embedded applications, strto___ is a lot smaller footprint than __scanf functions.)

EDIT: Sorry to miss your comment. To use endptr, write code like:

char* restOfLine = NULL;
unsigned long result = strtoul(lineBuffer, 10, &restOfLine);
if(restOfLine == NULL || restOfLine == lineBuffer)
{
     /* Handle error. */
}
else
{
    // Use result, and do further parsing starting at restOfLine.
}

Usually, the "handle error" clause returns or breaks or throws an exception or does something else to bail out of further processing, so you wouldn't need an explicit else clause.