lexical-analyser

My flex/yacc program compiles differently on two different linux machines

One one machine, everything compiles fine. On another machine, it complains about the -ly option when I use gcc to create the output file. If I remove the -ly option, then it makes the program, but then it runs differently. What's happening. How can I get the program to run correctly on this linux machine? ...

Why Use Lexical Analyzers?

Hello, I'm building my own language using Flex, but I want to know some things: Why use lexical analyzers? There are going to help me in something? Are they obligatory? Thanks. ...

Insert text in the input file in Lex (with C)

Hey! I'm trying to help a friend in a college assignment, but i kind of forgot a lot of C an Lex. The thing is, we are trying to parse a HTML and a correspondent CSS file and add to a tag it's style. Eg: HTML <body> </body> CSS body{color:black;} modified HTML <body style="color:black;"> </body> All the regex are done and ...

How to use yylval with strings in yacc

I want to pass the actual string of a token. If I have a token called ID, then I want my yacc file to actually know what ID is called. I thing I have to pass a string using yylval to the yacc file from the flex file. How do I do that? ...

yylval and union

What is the purpose of union in the yacc file? Is it directly related to yylval in the flex file? If you don't use yylval, then you don't need to use union? ...

How to put a final summary message in a yacc program?

When I redirect input to my yacc program from an input file, after it finishes parsing the file I want the yacc parser to print a summary of what it did. I want it to do the same thing if I am entering input through the keyboard and then I press Ctrl-D. Is there a way to do that? ...

What does the default main() created by flex look like?

I want my flex/yacc program to do the same thing as what it already does, but I want to modify it a little. If I were to put a main() in my .l file, and have it do the same thing as if I didn't add a main(), then what would the code look like? ...

What constitutes a 'word' in vim ?

Let's say we have the following in vim atm: int main () { printf("hello"); return 0; } In vim, w moves a word to the right, but what exactly constitutes a 'word'? For example, if I have the cursor on p of printf, pressing w takes u to ( and pressing another w skips the " and puts the cursor on the h of hello. Why was the "...

is there a simple compiler for a small language

I am looking for a simple compiler that compiles a simple language, I need it to write a paper about it and to learn how compilers work, I am not looking for a sophisticated thing just a simple language (by simple I mean a small code because for example gcc is toooooo big). any help is appreciated. ...

lex/yacc and parser/scanner

lex and yacc are supposed to be used together. Which ones is the scanner and which one is the parser? Which one creates a scanner and which one creates a parser? ...

What to name these columns in my symbol table

I'm using yacc to make a symbol table for a made-up language grammar. Column 1 will have either "variable" or "function." Column 2 will have either "int" or "void." I was thinking of naming one column "type" but then I wouldn't know what to call the other one. ...

How to recognize words in text with non-word tokens?

I am currently parsing a bunch of mails and want to get words and other interesting tokens out of mails (even with spelling errors or combination of characters and letters, like "zebra21" or "customer242"). But how can I know that "0013lCnUieIquYjSuIA" and "anr5Brru2lLngOiEAVk1BTjN" are not words and not relevant? How to extract words a...

How can I modify the text of tokens in a CommonTokenStream with ANTLR?

I'm trying to learn ANTLR and at the same time use it for a current project. I've gotten to the point where I can run the lexer on a chunk of code and output it to a CommonTokenStream. This is working fine, and I've verified that the source text is being broken up into the appropriate tokens. Now, I would like to be able to modify the...

Character position in scanner using Lex/Flex

Hi there, In Lex/Flex is there a way to get the position in the character stream (from the start of the file) that a token appears at? Kind of like yylineno except that it returns the character position as an integer? If not, what's the best way to get at this? Do I need to keep my own counter? Thanks! ...

In the programming languages specifications, why is it that lexical analysis not translatable?

In all of the standard specifications for programming languages, why is it that you cannot directly translate the lexical analysis/layout to a grammar that is ready to be plugged into and working? I can understand that it would be impossible to adapt it for the likes of Flex/Bison, Lex/Yacc, Antlr and so on, and furthermore to make it ...

Programmatic parsing and understanding of language (English)

Hello, I am looking for some resources pertaining to the parsing and understanding of English (or just human language in general). While this is obviously a fairly complicated and wide field of study, I was wondering if anyone had any book or internet recommendations for study of the subject. I am aware of the basics, such as searching ...

Are back references possible in flex (lexical analyser)?

Hi, I'm used to play with regexp in languages where I can use parenthesis to capture references. The only thing near that in flex that I'm seeing is the yytext variable. But it's contents are the full matched regexp and not just some part of it. Isn't the use of back references in flex possible? If so, how can I do it? I can't find any...

Error compiling flex (the lexical analyzer)

I'm trying to install flex (the lexical analyzer, not the Adobe program) on my Windows computer. I have MSYS installed. I untar flex, ./configure it, but when I try to make it, I get this error: In file included from ccl.c:34: flexdef.h:94:19: error: regex.h: No such file or directory In file included from ccl.c:34: flexdef.h:1195: erro...

Big problem with regular expression in Lex (lexical analyzer)

Hi, I have some content like this: author = "Marjan Mernik and Viljem Zumer", title = "Implementation of multiple attribute grammar inheritance in the tool LISA", year = 1999 author = "Manfred Broy and Martin Wirsing", title = "Generalized Heterogeneous Algebras and Partial Interpretation...

Is it possible to call C# lexical/syntactic analyzers without compilation?

Considering this question of SO, where whole C# in-memory compiler is being called. When only lexical and syntactic analyzing is required: parse text as a stream of lexemes, check them and exit. Is it possible in current version of System.CodeDom.Compiler, if not - will it be? ...