tags:

views:

157

answers:

1
+4  Q: 

Erlang File I/O: Large binary files and gzip streaming

I have two questions regarding Erlang file i/o; what is the best way to achieve in Erlang:

  1. reading large binary files (many gigabytes) without copying the whole file into memory
  2. reading a gzipped binary file as a decompressed stream

Thanks!

+2  A: 
  1. See file:read/2 for sequential block access and file:pread/2,3 for random access.
  2. See compressed option in file:open/2.
Hynek -Pichi- Vychodil  2010-09-27 21:50:26
I might have to figure out something that's higher performance for production, but this works very well for prototyping.
Erlang  2010-09-29 11:43:26
Do you have measured or just guessing? I'm pretty satisfied with `file:read/2` when opened file using `[raw, binary, read_ahead]` and using block oriented processing. What performance you expect?
Hynek -Pichi- Vychodil  2010-09-29 14:26:10

related questions

Why is MPI considered harder than shared memory and Erlang considered easier, when they are both message-passing?
lists:usort for nth element in tuple
Gaining a better understanding of functional programming
Replace keys in a tuple in Erlang
Should a client handling process be added to the supervisor tree?
Where can I find free Erlang screencasts or tutorials, preferablly in a series format
Sieve of Eratosthenes in Erlang
What could make Erlang more popular and widespread?
Where can I find documentation for the Erlang shell?
Programming future web apps
Mnesia write fails
CouchDB Backups and Cloneing the Database
Mnesia table replication/sharing
Directory Layout for Erlang Services?
URL encode in Erlang
Can anyone recommend a primer to Erlang?
What libraries can I use to build a GUI with Erlang?
Concurrent Prime Generator
Erlang Multicast
Merging records for Mnesia
How would you implement Erlang-like send and receive in C++?
What's a good way to write a Cocoa front-end to an Erlang application?
Erlang-style Concurrency for Other Languages
How Do You Determine The PID of the Parent of a Process
Setting up an Erlang development environment