Error with threads opening files

Question

I'm using C and pthread on a Linux machine, and I'm having trouble parallelizing a program.

I'm basically trying to take in a folder of data files, divide them into groups, each group handled by a thread, and run a function on each of the data file.

The way I'm doing this is I have a global char **filename variable, where filename[i] = filename of a data file. In the main function, I'll read in the filenames of all the data files (minus "." and "..") using scandir and put them in the filename variable. Then 4 (arbitrary number) threads are created each calling the Process function. In Process(), each thread only opens (using a FILE *fin declared in Process()) and works on a portion of the data files using a start_index and an end_index. For example, if there are 100 files, then each thread will handle filename[0] to filename[24], filename[25] to filename[49], filename[50] to filename[74] and filename[75] to filename[99] respectively. After they're done, there is a pthread_join in main() for all 4 threads.

I have checked that the filenames have been stored correctly, both in main() and Process(). However, I keep getting segmentation fault here, in Process():

for (i = start_index; i <= end_index ; i++)
   fin = fopen(filename[i], "rb"); <--- Seg fault

I don't really get why there should be an error since none of the threads are trying to open the same file.

Please advise.

Answer 1

I am guessing here, you are probably setting filename[i] to namelist[i]->d_name and then calling free(3) on the namelist[i]. After this the pointer to file name is invalid. Or the free(3) happens in main thread and races with processing threads. You really need to strdup(3) each string and only release the memory after you are really done with it.

I of course might be wrong since code given does not show how the strings are allocated.

Answer 2

What are the values of start_index and end_index, and hence of i, when the crash occurs?

If i is under control, the code should not crash - so that is the first thing I would check.

The show code fragment leaks file streams abominably because it overwrites fin on each iteration. I assume that is an artefact of reducing the test code to a minimum rather than the actual behaviour of the (as yet not) working program.