How do I generate different yyparse functions from lex/yacc for use in the same program?

Question

Hi,

I want to generate two separate parsing functions from lex/yacc. Normally yacc gives you a function yyparse() that you can call when you need to do some parsing, but I need to have several different yyparses each associated with different lexers and grammars. The man page seems to suggest the -p (prefix) flag, but this didn't work for me. I got errors from gcc that indicated that yylval was not properly being relabeled (i.e. it claims that several different tokens are not defined). Does anyone know the general rpocedure for generating these separate functions?

thanks

Answer 1

How about a function pointer in surrounding code that swaps out parsers, presuming you don't want to change the parsing context in the middle of yylex() on the same buffer.

The function pointer can be set from including parsers with the prefix option, or ones loaded in from DSOs at runtime, a-la a plugin setup.

This has the advantage, which may or may not be useful to you, that your code which handles the AST can be blind to which parser/lexer combo was used to produce it.

Answer 2

If you are using GNU flex/bison, you can use the "reentrant" options, that permits to have multiple instances of the same (or also different) scanner. This option is present on recent flex version. IMHO it's the more clean way to have multiple scanners/parsers inside the same program. This article explains in depth the issue.

An alternate way is to use the flex's ability to generate a C++ class instead of a static scanner, but this force the C programmer to write some wrapper functions.

Regards

Answer 3

I had a same problem sometime ago and I ended up writing following header file:

#ifndef RENAME_FLEX_H
#define RENAME_FLEX_H

#define yy_create_buffer scan__create_buffer
#define yy_delete_buffer scan__delete_buffer
#define yy_init_buffer scan_init_buffer
#define yy_load_buffer_state scan_load_buffer_state
#define yy_switch_to_buffer scan_switch_to_buffer
#define yyin scan_in
#define yyleng scan_leng
#define yylex scan_lex
#define yyout scan_out
#define yyrestart scan_restart
#define yytext scan_text
#define yy_flex_debug scab_flex_debug
#define yywrap scan_wrap
#define yyrealloc scan_realloc
#define yyfree scan_free
#define yy_flush_buffer scan_flush_buffer
#define yypush_buffer_state scan_push_buffer_state
#define yypop_buffer_state scan_pop_buffer_state
#define yy_scan_buffer scan_scan_buffer
#define yy_scan_string scan_scan_string
#define yy_scan_bytes scan_scan_bytes
#define yyget_in scan_get_in
#define yyget_out scan_get_out
#define yyget_leng scan_get_leng
#define yyset_text scan_set_text
#define yyset_in scan_set_in
#define yyset_out scan_set_out
#define yyget_debug scan_get_debug
#define yyset_debug scan_set_debug
#define yylex_destroy scan_lex_destroy
#define yyalloc scan_alloc
#define yyget_text scan_get_text

#endif

and include it into .l, so from my .c file I can use scan_-prefixed symbols for second scaner, rather then yy-prefixed