yyparse is printing a leading tab

Question

In my bison/flex program, right after yyparse() is called, a leading tab is printed, but I don't know why. Can you see what's wrong?

This calls the bison code, and right after yyparse() returns, a tab is printed.

void parseArguments(int argc, char** argv)
130 {
131     int i;
132 
133     int sum = 0;
134     // calculate the length of buffer we need
135     for(i = 1; i < argc; i++)
136     {
137         sum += strlen(argv[i]) + 1;
138     }
139 
140     if(sum <= 0)
141         return;
142 
143     // make us a buffer and zero it out
144     char tempBuffer[sum];
145     memset(tempBuffer, 0, sum);
146 
147     // pointer to walk through our buffer
148     int pos = 0;
149 
150     // copy arguments into the buffer
151     for(i = 1; i < argc; i++)
152     {
153         memcpy(tempBuffer+pos, argv[i], strlen(argv[i]));
154         pos += strlen(argv[i]);
155         sprintf(tempBuffer+pos++, " ");
156     }
157 
158     // give our arguments to lex for parsing
159     configBuffer(tempBuffer);
160 


// use bison parsing
163     int returnVal = yyparse(); // after this returns a tab character has been printed


164     if(returnVal != 0)                                                                                            
   165     {   
   166         printf("yyparse failed!\n");                                                                              
   167     }                                                                                                             
   168

All my bison rules are just regular expressions paired with a return statement. The only code of interest in bison that I could see affecting this would be this:

64 %%
 65 void configBuffer(char* arguments)
 66 {
 67 #ifdef DEBUG
 68     printf("Given the buffer: %s\n", arguments);
 69 #endif
 70     yy_delete_buffer(YY_CURRENT_BUFFER);
 71 
 72     yy_scan_string(arguments);
 73 }

I tried the suggestions given by several people, but still not luck. Here is my full flex file:

%{
#include <string.h>
#include "CommandParser.tab.h"
%}

%%

\t {
    printf("TAB!\n");
}

" " {
    printf("SPACE!\n");
}

\n {
    return;
}

-p {
    return PRINTMODE; 
}

-x {
    return XORMODE;
}

-n {
    return NOTMODE;
}

-a {
    return ANDMODE;
}

-o {
    return ORMODE;
}

-r {
    return RANGEFLAG;
}

-l {
    return LENGTHFLAG;
}

0[xX][0-9a-fA-F]+ {
    int retVal = sscanf(yytext, "%x",&(yylval.int_val));
    if(retVal != 1)
        return;
    return NUMBER;
}

[0-9]+ {
    yylval.int_val = atoi(yytext);
    return NUMBER;
}

['"].*+['"] {
    yylval.string_val = strdup(yytext);
    return ARGUMENT;
}

[^ \t\n]+ {
    yylval.string_val = strdup(yytext);
    return ARGUMENT;
}


%%
void configBuffer(char* arguments)
{
#define DEBUG
#ifdef DEBUG
    printf("Given the buffer: %s:\n", arguments);
#endif
    yy_delete_buffer(YY_CURRENT_BUFFER);
    yy_scan_string(arguments);

}

Answer 1

Is the tab not handled in you lexer and therefore the default rule matching and echoed is being applied?

Put a extra match

\t { printf("TAB"); }

into the code before your end code section.

if that shows TAB instead of the \t, then turn the printf into an empty statement

\t { /*printf("TAB")*/; }

After lex posting Edit:

Ok, after testing your lex it would seem you are matching things correctly.

I used this code to test it

#include <stdio.h>
#include "CommandParser.tab.h"

YYSTYPE yylval;

int main(int argc, char* argv[])
{
    while(1)
    {
        printf("lex:%d\r\n",yylex());
    }
    return 0;
}

extern "C" int yywrap();

int yywrap ()
{
    return 1;
}

So with the input (via stdin)

-a<\ >-x<\t>-p<space>-c<\r>

I get

lex:103
SPACE!
lex:101
TAB!
lex:100
SPACE!
lex:108
lex:3

for this header file

#define PRINTMODE   100
#define XORMODE  101
#define NOTMODE  102
#define ANDMODE  103
#define ORMODE   104
#define LENGTHFLAG  105
#define RANGEFLAG   106
#define NUMBER   107
#define ARGUMENT    108
#define DEFUALT  0

typedef union {
    int int_val;
    char* string_val;
} YYSTYPE;

#ifdef __cplusplus
extern "C" int yylex();

extern "C" YYSTYPE yylval;
#else // __cplusplus
extern YYSTYPE yylval;
#endif // __cplusplus

So what I'd try next is replace the yyparse with this code and see what you get.

while(1)
{
    printf("lex:%d\r\n",yylex());
}

If you still get the tab printed it is somehow you lexer, otherwise it is somehow your parser/main program.

To find that out I'd replace the magic string building you do with a const string, and see what happen in that case. Basically binary search your code to find the problem spot.