ifstream fails to open in recursive calls

Question

hello. we are running into an odd issue when trying to parse an input file. the idea is that this file can include other files, which must be parsed as well. We are doing this recursively in a function defined as

int parse_inp(const char* filename)

The main file parses no problem, but recursive calls cannot open their file streams.

int parse_inp(const char* filename)
{
    char buffer[BUFFER_MAX+1];
    char* token;
    std::string tok;
    int keywordSection;
    bool end_of_file;
    int cardNum;

    ...

    int i;
    std::string tempop;
    double tempd1, tempd2;
    SetSegmentCard2 tempSetSegmentCard2;
    int offset;

    printf("%s\n", filename);
    std::ifstream inp;
    inp.clear();
    inp.open(filename, std::ios::in);
    if(!inp.good() || !inp.is_open())
    {
        char path1[256];
        getcwd(path1,256);
        printf("CWD: %s\n", path1);
        fflush(NULL);
        printf("Unable to open '%s'\n", filename);
        return 0;
    }
    std::set<std::string> unrecognized;
    std::string line;
    while(inp.good() && !inp.eof())
    {
        getline(inp, line);
        strcpy(buffer, line.c_str());

        if (isComments(buffer)) //skip the comments line
            continue;

        if (buffer[0]=='*') //this is a keyword line
        {
            token = strtok(buffer," \n");
            keywordSection = is_inp_keyw(token);

            if (keywordSection==0)
                unrecognized.insert(token);
            cardNum = 0;
            continue;
        }

        //a data line
        tempop="";
        char* found = NULL;
        char path_buffer[100] = "Dyna3DWriter\\";
        int pos = 0;
        switch(keywordSection)
        {
            case 0: //not recognized
                //end of last keyword, not recognizable word
                break;
            case 1: //KEYWORD
                //"KEYWORD didn't do anything
                break;
            case 2: //TITLE
                break;
            case 3: //INCLUDE
                token = strtok(buffer, "\n");
                inp.clear();
                parse_inp(token);
                break;
            ...
        }
    }
    if(inp.is_open())
    {
        inp.close();
        inp.clear();
    }
}

The recursive files never parse. I looked around a lot and most issues seemed to be either that the fail bit was set (which is why we are calling inp.clear() a lot), or that we are making the wrong assumption about the current working directory.

To test the second theory, we added in:

if(!inp.good() || !inp.is_open())
{
    char path1[256];
    getcwd(path1,256);
    printf("CWD: %s\n", path1);
    fflush(NULL);
    printf("Unable to open '%s'\n", filename);
    return 0;
}

And our working directory and file name are both correct. We see the same behavior when using fopen(filename, "r") --- a call to perror("fopen") results in:

fopen: no such file or directory

EDIT: Filled in more code

Answer 1

Are you sure the filename does not contain any garbage or bad character that would lead to this issue? If the error is file not found, that means the filename is wrong in some way.

Could it come from a bad declaration of buffer? We don't see it in your code.

Another possibility is that you use strtok again in your initialization before opening the file. You must avoid using strtok that is based on global storage for recursive method like this. You should use strtok_r instead.

Answer 2

If your recursive function is called very deeply you can easily overload the OS limit on the number of open files.

I used to run my Gentoo Linux with a per-process file limit of 250 with exceptions for programs that needed a lot more.

Windows has limits that vary depending on how much memory is available to the system and how many objects have already been created system-wide.

The smart way to do this is to have two functions. The first function is the one everyone else calls and it does the setup, including opening the file. The second function is the recursive function and it takes nothing but a reference to a std::ifstream object.

EDIT:

I see that I misunderstood your question, and you aren't recursively opening the same file. I will leave my above paragraph anyway.