Appending to a global file handle, is it bad?

Question

Lets say there are multiple functions throughout my program that need to append data to a certain file. I open the file at the beginning of the program with a global file handle so I can append to it wherever I need to. (Note that I know I could pass the file handle as an argument to the functions but that is not the purpose of this question). Is it bad to open the file handle at the beginning of a program, and then close it at the end; or is it better to have a function say void AppendFile(char *data_to_append); and then open the file and append to it and close it in this same function? If the program dies the FD would still be in use is the only bad thing I see, but at the same time you are opening and closing the same file hundreds and hundreds of times if you use the function.

Answer 1

You are probably best to do a single open and a single close. Opening/Closing non stop will lead to a lot of wasted IOs.

You will probably want to protect this function though with a mutex so only one thread can be writing to the file handle at a time.

Do make sure that you close your file though eventually.

Answer 2

Yes.

Not the most helpful answer but I'm afraid that your question is too general and vague to give you anything detailed. If you have accesses to this FD scattered through your program then it suggests there is something wrong with your high-level design. Are these separate accesses related to each other in some way? Could they be combined into a smaller number of points that access the file? Do you have some kind of implied data-structure in your program that would better implemented in a class?

Answer 3

If your program is single threaded, it's okay. If it dies while the file handle's open, it may be closed by the operating system anyway. If it isn't, what's to guarantee it won't die inside your AppendFile function?

I'd suggest you do make that AppendFile function though. It will simplify the write process and you'll be able to change things about the file handle more easily than with a bunch of fwrite()s sprinkled about.

Answer 4

Depends on how much you care about your file not getting trashed or losing data. If the program crashes, there's no guarantee about what happens to all the writes that haven't been finished (not just done, but flushed and committed to disk) yet. If there are uncommitted writes, they could be tossed out or half-done. Closing the file guarantees those writes get committed.

If the writes will be infrequent, open/append/close is a better idea IMO -- but AppendFile could be made to work with an already open file handle, so it actually works better either way.

Add to that, if you use threads at all, you don't want random writes to the file -- you want to have some way to synchronize them. Having an AppendFile or similar function gives you that synchronization point -- you can add code in there to wait til another thread's finished. Just try doing that when you're directly writing to the file in a hundred different places.

Answer 5

Sometimes a database is a very good substitute for text files, expecially when the database was specifically designed to replace text files in the first place :)

Take a look at SQLite ( http://www.sqlite.org/ ).

Think of SQLite not as a replacement for Oracle but as a replacement for fopen()

Answer 6

If by file handle you mean a FILE * then opening at the beginning and closing at the end with lots of users should work as expected even if there are multiple threads as long as you do your writes with single stdio function calls on POSIX systems.

If by file handle you mean an integer returned by the operating system's open call then these are usually thread safe within a single call to write (or similar) since the operating system will lock the file while transferring data to buffers associated with that file.

If the program is a single threaded application then you don't have much to worry about either way.

If you were to go with repeatedly opening, appending, and closing the file you could run into trouble if using stdio's FILE * with multiple threads or if calls to AppendFile were somehow made recursively because the different FILE *s would not share buffers within the application and so as the file changed in one thread other threads might overwrite those changes.

A similar thing can happen with os file handles (integers returned by open) since the different calls to open will produce different file handles with will not share their seek position, so as the file grew the different file descriptors would end up with seek positions that weren't actually at the end of the file unless you can open the file in append only mode ( O_APPEND ) where the OS handles this for you.

Opening and closing the file over and over does generate a lot of extra work, either way.

Answer 7

global variables are usually not a good thing. For small programs it's doesn't matter though.

While still using a global file handle, consider accessing it only through a void AppendFile(char *data_to_append); function, where only AppendFile references the global file, instead of scattering it all over your code.

Opening/closing a file on every access can be a vaste if the file is accessed often.

Also, file handles are normally closed when your program ends (either dies, or exits normally), so you're not leaking anything if your program crashes.