writing to a Random access file in Erlang

Question

How does one write to a file after a certain amount of data is received?

I know the file can be opened and then it can be written to using file:pwrite(Position, Bin).

The thing I would like to know is if data is e.g. being downloaded and needs to be written to the file. How does one do this efficiently? e.g. using a buffer(how is a buffer written?)

Also Is the file kept open the entire time until the download is done or is it opened each time the buffer is reached and the data needs to be written?

Thanks

Answer 1

I think file:pwrite/3 has what you need.

Answer 2

The most efficient method is to append the data to an in-memory buffer (an IO list?) and write it to a file in one operation. If you wish to open the file and write each block of data, as it arrives, you can open the file in appendmode:

> file:write_file(FileName, DataToWrite, [append]).

Each write will append the data to the end of the file.

Answer 3

If you'd like erlang to take care of buffering file writes for you, you can open a file with the {delayed_write, Size, Delay} option. Then you can just io:write to it as much as you like and the VM will take care of batching those requests into one OS write call when the buffer reaches Size bytes, or after Delay milliseconds since the buffer started to fill.

This approach would allow you to issue io:write calls every time you receive data from the network but know that you won't be issuing system calls for every byte (if you got really pathological network behaviour).