Which Python module is suitable for data manipulation in a list?

Question

I have a sequence of x, y and z -coordinates, which I need to manipulate. They are in one list of three tuples, like {(x1, y1, z1), (x2, y2, z2), ...}.

I need addition, multiplication and logarithm to manipulate my data.

I would like to study a module, which is as powerful as Awk -language.

Answer 1

I'm not sure exactly what you're after. You can do a lot with list comprehensions. For example, if you want to turn a list:

coords = [(x1, y1, z1), (x2, y2, z2), (x3, y3, z3)]  # etc

into a tuple (x1+x2+x3, y1+y2+y3, z1+z2+z3), then you can do:

sums = (sum(a[0] for a in coords), sum(a[1] for a in coords), sum(a[2] for a in coords))

In fact, an experienced python programmer might write that as:

sums = map(sum, zip(*coords))

though that can look a bit like magic to a beginner.

If you want to multiply across coordinates, then the idea is similar. The only problem is python has no builtin multiplication equivalent to sum. We can build our own:

import operator
def prod(lst):
    return reduce(operator.mul, lst)

Then you can multiply your tuples coordinate-wise as:

prods = map(prod, zip(*coords))

If you want to do something more complex with multiplication (inner product?) that will require a little more work (though it won't be very difficult).

I'm not sure what you want to take the logarithm of. But you can find the log function in the math module:

from math import log

Hope this helps.

Answer 2

In Python 3 the reduce function is gone. You can do:

def prod(lst):
    return [x*y*z for x, y, z in list(zip(*lst))]

coords = [(2, 4, 8), (3, 6, 5), (7, 5, 2)]
print(prod(coords))
>>> [42, 120, 80]

Answer 3

If you need many array manipulation, then numpy is the best choice in python

>>> import numpy
>>> data = numpy.array([(2, 4, 8), (3, 6, 5), (7, 5, 2)])
>>> data
array([[2, 4, 8],
       [3, 6, 5],
       [7, 5, 2]])

>>> data.sum()  # product of all elements
42
>>> data.sum(axis=1)   # sum of elements in rows
array([14, 14, 14])
>>> data.sum(axis=0)   # sum of elements in columns
array([12, 15, 15])
>>> numpy.product(data, axis=1)   # product of elements in rows
array([64, 90, 70])
>>> numpy.product(data, axis=0)   # product of elements in columns
array([ 42, 120,  80])
>>> numpy.product(data)      # product of all elements
403200

or element wise operation with arrays

>>> x,y,z = map(numpy.array,[(2, 4, 8), (3, 6, 5), (7, 5, 2)])
>>> x
array([2, 4, 8])
>>> y
array([3, 6, 5])
>>> z
array([7, 5, 2])

>>> x*y
array([ 6, 24, 40])
>>> x*y*z
array([ 42, 120,  80])
>>> x+y+z
array([12, 15, 15])

element wise mathematical operations, e.g.

>>> numpy.log(data)
array([[ 0.69314718,  1.38629436,  2.07944154],
       [ 1.09861229,  1.79175947,  1.60943791],
       [ 1.94591015,  1.60943791,  0.69314718]])
>>> numpy.exp(x)
array([    7.3890561 ,    54.59815003,  2980.95798704])

Answer 4

You don't need a separate library or module to do this. Python has list comprehensions built into the language, which lets you manipulate lists and perform caculations. You could use the numpy module to do the same thing if you want to do lots of scientific calculations, or if you want to do lots of heavy number crunching.