I've been given a job of 'translating' one language into another. The source is too flexible (complex) for a simple line by line approach with regex. Where can I go to learn more about lexical analysis and parsers? ...
Can people point me to resources on lexing, parsing and tokenising with Python? I'm doing a little hacking on an open source project (hotwire) and wanted to do a few changes to the code that lexes, parses and tokenises the commands entered into it. As it is real working code it is fairly complex and a bit hard to work out. I haven't w...
Here's the deal. Is there a way to have strings tokenized in a line based on multiple regexes? One example: I have to get all href tags, their corresponding text and some other text based on a different regex. So I have 3 expressions and would like to tokenize the line and extract tokens of text matching every expression. I have actua...
Does there exist a parser that generates an AST/parse tree at runtime? Kind of like a library that would accept a string of EBNF grammar or something analogous and spit out a data structure? I'm aware of antlr, jlex and their ilk. They generate source code which could do this. (like to skip the compile step) I'm aware of Boost::Spirit...
I want to be able to predicate pattern matches on whether they occur after word characters or after non-word characters. In other words, I want to simulate the \b word break regex char at the beginning of the pattern which flex/lex does not support. Here's my attempt below (which does not work as desired): %{ #include <stdio.h> %} %x ...
What are some good tools for getting a quick start for parsing and analyzing C/C++ code? In particular, I'm looking for open source tools that handle the C/C++ preprocessor and language. Preferably, these tools would use lex/yacc (or flex/bison) for the grammar, and not be too complicated. They should handle the latest ANSI C/C++ defi...
What is the difference between Flex & Lex and Yacc & Bison. I searched the Internet wildly and I didn't find any solid answer. Can I install pure Lex and Yacc on Ubuntu, or I can install only flex and bison. I am confused. Is Lex or Yacc still being maintained by someone ? Are all of them free ? If Lex is not free why do I have it in...
Could somebody post a link to some excellent tutorials for lex and yacc? ...
I'm writing an assembler for a microprocessor I'm creating using lex/yacc. I'd like to implement labels in my assembler code, and I thought a good way to do this would be to have a dictionary of labels in the form {name:line#}. I could then check when inserting a label, if it's already defined, its an error. So how can I use a dictiona...
I'm using lex in my program and I've run into a problem I need some help with. My program accepts its input in the form of [something " something]. This is working correctly. However, I also need to accept the form [something"something]. Is there a way that I can have some sort of first case in lex that all input is run through (like ...
I created test.l, input to flex, which ends with the main function. When the main function is implemented as: int main(void) { yylex(); return 0; } I have no problem. I want to trick the parser into believing that the first character is always a semi-colon, so I implemented main as int main(void) { unput(';'); yylex(...
I have an application where I already have a parser for one sort of grammar and I need to add a second different grammar for another purpose. Is it possible to have more than one? And if so how do you get another entry point? Thanks david allan finch ...
Hi, I'm trying to figure out how I can display a message/prompt when using lex/yacc (flex/bison). For instance, main looks like so: int main(int argc, char *argv[]) { yyparse(); } Which calls yacc, which calls yylex(). This yields a blank line waiting on STDIN. How can i display a message like... message $ _ instead of _ Whe...
I want to create a read-eval-print loop using flex/bison parser. Trouble is, the flex generated lexer wants input of type FILE* and i would like it to be char*. Is there anyway to do this? One suggestion has been to create a pipe, feed it the string and open the file descriptor and send to the lexer. This is fairly simple but it feels ...
So, it seems like Happy is a robust replacement for yacc in Haskell. Is there an equally robust lexer generator to replace lex/flex? ...
I have as input blocks of text with commands and arguments, one per line, such as XYZ ARG1,ARG2,ARG3,...,ARGN And I want to verify that the arguments to XYZ are well formed for that particular command and execute the correct block of code if they are. There are something like ~100 commands, some of which have variable numbers of argum...
Forgive me, I'm completely new to parsing and lex/yacc, and I'm probably in way over my head, but nonetheless: I'm writing a pretty basic calculator with PLY, but it's input might not always be an equation, and I need to determine if it is or not when parsing. The extremes of the input would be something that evaluates perfectly to an e...
I am developing a simple translator from MathML to Latex, using Lex and Yacc. In my lex file containing the regex rules I have one defined for arithmetic operators [-+*=/]. I want to extended so that it would recognize plus-minus (+-) and invisible times ('&InvisibleTimes'), but I'm unfamiliar with regex and I need some help. ...
I'm using Flex and Bison for a parser generator, but having problems with the start states in my scanner. I'm using exclusive rules to deal with commenting, but this grammar doesn't seem to match quoted tokens: %x COMMENT // { BEGIN(COMMENT); } <COMMENT>[^\n] ; <COMMENT>\n { BEGIN(INITIAL); } "==" ...
i have worked with lex for excuting some code whenever some regular expression is found, Can yacc do something more than that?if yes what ...