How to output list of floats to a binary file in Python

Question

I have a list of floating-point values in Python:

floats = [3.14, 2.7, 0.0, -1.0, 1.1]

I would like to write these values out to a binary file using IEEE 32-bit encoding. What is the best way to do this in Python? My list actually contains about 200 MB of data, so something "not too slow" would be best.

Since there are 5 values, I just want a 20-byte file as output.

Answer 1

have a look at struct.pack_into

Answer 2

See: Python's struct module

import struct
s = struct.pack('f'*len(floats), *floats)
f = open('file','wb')
f.write(s)
f.close()

Answer 3

struct.pack() looks like what you need.

http://docs.python.org/library/struct.html

Answer 4

The array module in the standard library may be more suitable for this task than the struct module which everybody is suggesting. Performance with 200 MB of data should be substantially better with array.

Answer 5

Alex is absolutely right, it's more efficient to do it this way:

from array import array
output_file = open('file', 'wb')
float_array = array('d', [3.14, 2.7, 0.0, -1.0, 1.1])
float_array.tofile(output_file)
output_file.close()

And then read the array like that:

input_file = open('file', 'r')
float_array = array('d')
float_array.fromstring(input_file.read())

array.array objects also have a .fromfile method which can be used for reading the file, if you know the count of items in advance (e.g. from the file size, or some other mechanism)

Answer 6

I'm not sure how NumPy will compare performance-wise for your application, but it may be worth investigating.

Using NumPy:

from numpy import array
a = array(floats,'float32')
output_file = open('file', 'wb')
a.tofile(output_file)
output_file.close()

results in a 20 byte file as well.