Before calling fflush can I consider fwrite to be = a non-blocking write?
If not, why not and what are my alternatives?
views:
338answers:
3
A:
fwrite() is blocking. fwrite() may call fflush() internally at any time.
If all you need it to buffer, then buffer in your own array. fwrite's buffer is typically a few K.
Joshua
2010-03-22 15:47:51
Any way to force fwrite not to fflush() internally?Cause thats waht i need.. i just call "write" and it puts it in a buffer and returns immediately and maybe it starts to write maybe not but it doesnt halt my program till i call flush
John Michaels
2010-03-22 16:01:45
+3
A:
fwrite() may block. It uses (usually) an internal buffer with a maximum length. It will send the data (all or part of its internal buffer) when the buffer becomes full.
The setbuf() and setvbuf() functions let you alter the buffer maximum length, and actually provide the block for the buffer, but the details are implementation dependent, so you will have to read the documentation for your specific C library.
Conceptually, if you want guaranteed non-blocking writes under all conditions, then you need potentially infinite buffers, which can be somewhat expensive. You could make your own functions for buffering data (within a block of RAM, using realloc() to make it grow when necessary) and write out (with fwrite() and possible fflush()) only at the end. Alternatively, you could try to use non-blocking I/O in which write functions never block but may respond that they refuse to accept your data due to internal congestion. Non-blocking I/O is not part of the C standard itself (there is no f*() function for that) but can be found under various names on some systems (e.g. with fcntl() and write() on Unix systems).
Thomas Pornin
2010-03-22 15:59:56
It's also important to note that even if fwrite blocks, all it's definitely doing is handing its data over to the kernel/filesystem layer. The kernel will typically do buffering of its own rather than sync the data to disk immediately.
Andrew Medico
2010-03-22 16:07:56
+1
A:
Technically fwrite() is a blocking call in that it does not return until the procedure has completed. However the definition of completion for fwrite() is that the data you supply has been written to an internal file buffer. As a side effect some of that buffer may also be written to the disk as part of the fwrite() call but you cannot rely on that behavior. If you absolutely require the data to be on the disk you need to call fflush().
Andrew O'Reilly
2010-03-22 16:13:14
If this is true then it does exactly what i need. By non-blocking i just meant that it shouldnt block while writing to disk.. i just need to set a huge buffer
John Michaels
2010-03-22 16:18:50
Note that fflush only guarantees the data is moved from userland into the kernel. The kernel also buffers so if you lose power, you may still lose the data. If you truly want the data on disk, you would need to do a `fsync(fileno(fp));`
R Samuel Klatchko
2010-03-22 17:45:21
Well, as I said above, a side effect of calling fwrite() is that it might flush its internal buffer to disk. This indeterminacy will cause significant jitter in the length of time calls to fwrite() take. As Thomas Pornin said you can set the buffer size but details of when fwrite() will flush its buffer to disk are implementation dependent. If you truly want a deterministic non-blocking fwrite() equivalent you'll either need to defer that processing to a separate thread or write your data to intermediate staging area of memory and then call fwrite() once at the end.
Andrew O'Reilly
2010-03-22 17:46:20