What's the best way to explain parsing to a new programmer?

Question

I am a college student getting my Computer Science degree. A lot of my fellow students really haven't done a lot of programming. They've done their class assignments, but let's be honest here those questions don't really teach you how to program.

I have had several other students ask me questions about how to parse things, and I'm never quite sure how to explain it to them. Is it best to start just going line by line looking for substrings, or just give them the more complicated lecture about using proper lexical analysis, etc. to create tokens, use BNF, and all of that other stuff? They never quite understand it when I try to explain it.

What's the best approach to explain this without confusing them or discouraging them from actually trying.

Answer 1

Parsing is the process of analyzing text made of a sequence of tokens to determine its grammatical structure with respect to a given (more or less) formal grammar.

The parser then builds a data structure based on the tokens. This data structure can then be used by a compiler, interpreter or translator to create an executable program or library.

If I gave you an english sentence, and asked you to break down the sentence into its parts of speech (nouns, verbs, etc.), you would be parsing the sentence.

That's the simplest explanation of parsing I can think of.

That said, parsing is a non-trivial computational problem. You have to start with simple examples, and work your way up to the more complex.

Answer 2

Have them try to write a program that can evaluate arbitrary simple arithmetic expressions. This is a simple problem to understand but as you start getting deeper into it a lot of basic parsing starts to make sense.

Answer 3

I'd explain parsing as the process of turning some kind of data into another kind of data.

In practice, for me this is almost always turning a string, or binary data, into a data structure inside my Program.

For example, turning

":Nick!User@Host PRIVMSG #channel :Hello!"

into (C)

struct irc_line {
    char *nick;
    char *user;
    char *host;
    char *command;
    char **arguments;
    char *message;
} sample = { "Nick", "User", "Host", "PRIVMSG", { "#channel" }, "Hello!" }

Answer 4

Parsing is about READING data in one format so that you can use it to yours needs.

I think you need to teach them to think like this, so, this is the simplest way I can think of to explain parsing for someone new to this concept.

Generally, we try to parse data one line at a time because generally it is easier for humans to think this way, dividing and conquering, and also easier to code.

We call field to every minimun undivisible data. Name is field, Age is another field, and Surname is another field, for example.

In a line, we can have various fields. In order to distiguish them, we can delimite fields by separators or by the maximun length assign to each field.

For example: By separating fileds by comma

Paul,20,Jones

Or by space (Name can have 20 letters max, age up to 3 digits, Jones up to 20 letters)

Paul                020Jones               

Any of the before set of fields is called a record.

To separate between a delimited field record we need to delimit record. A dot will be enough (though you know you can apply CR/LF).

A list could be:

Michael,39,Jordan.Shaquille,40,O'neal.Lebron,24,James.

or with CR/LF

Michael,39,Jordan
Shaquille,40,O'neal
Lebron,24,James

You can say them to list 10 nba (or nlf) players they like. Then, they should type them according to a format. Then make a program to parse it and display each record. One group, can make list in a comma-separated format and a program to parse a list in a fixed size format, and viceversa.

Answer 5

What is parsing?

In computer science, parsing is the process of analysing text to determine if it belongs to a specific language or not (i.e. is syntactically valid for that language's grammar). It is an informal name for the syntactic analysis process.

For example, suppose the language a^n b^n (which means same number of characters A followed by the same number of characters B). A parser for that language would accept AABB input and reject the AAAB input. That is what a parser does.

In addition, during this process a data structure could be created for further processing. In my previous example, it could, for instance, to store the AA and BB in two separate stacks.

Anything that happens after it, like giving meaning to AA or BB, or transform it in something else, is not parsing. Giving meaning to parts of an input sequence of tokens is called semantic analysis.

What isn't parsing?

• Parsing is not transform one thing into another. Transform A into B, is, in essence, what a compiler does. Compiling takes several steps, parsing is only one of them.
• Parsing is not extract meaning from a text. Extract meaning from a text, is semantic analysis which is a step of the compiling process.

What is the simplest way to understand it?

I think the best way for understanding the parsing concept is to begin with the simpler concepts. The simplest one in language processing subject is the finite automaton. It is a formalism to parsing regular languages, such as regular expressions.

It is very simple, you have an input, a set of states and a set of transitions. Consider the following language built over the alphabet { A, B }, L = { w | w starts with 'AA' or 'BB' as substring }. The automaton below represents a possible parser for that language whose all valid words starts with 'AA' or 'BB'.

    A-->(q1)--A-->(qf)
   /  
 (q0)    
   \          
    B-->(q2)--B-->(qf)

It is a very simple parser for that language. You start at (q0), the initial state, then you read a symbol from the input, if it is A then you move to (q1) state, otherwise (it is a B, remember the remember the alphabet is only A and B) you move to (q2) state and so on. If you reach (qf) state, then the input was accepted.

As it is visual, you only need a pencil and a piece of paper to explain what a parser is to anyone, including a child. I think the simplicity is what makes the automata the most suitable way to teaching language processing concepts, such as parsing.

Finally, being a Computer Science student, you will study such concepts in-deep at theoretical computer science classes such as Formal Languages and Theory of Computation.