Adding arrays with different number of dimensions

Question

Let's say I have a 2D Numpy array:

>>> a = np.random.random((4,6))

and I want to add a 1D array to each row:

>>> c = np.random.random((6,))
>>> a + c

This works. Now if I try adding a 1D array to each column, I get an error:

>>> b = np.random.random((4,))
>>> a + b
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: shape mismatch: objects cannot be broadcast to a single shape

I can fix this by using np.newaxis:

>>> a + b[:,np.newaxis]

which does work as expected.

What are the shape-matching rules to avoid having to use np.newaxis? Is it that the last element of the numpy shape tuple has to match? Does this rule also apply to higher dimensions? For example, the following works:

>>> a = np.random.random((2,3,4,5))
>>> b = np.random.random((4,5))
>>> a + b

So my question is whether this is documented anywhere, and if it is a behavior that can be relied on, or whether it is best to always use np.newaxis?

Answer 1

This is a distinctive feature of numpy called broadcasting. Its done using four rules which are a bit complicated in formulation but are rather intuitive once understood:

1. All input arrays with ndim smaller than the input array of largest ndim, have 1’s prepended to their shapes.
2. The size in each dimension of the output shape is the maximum of all the input sizes in that dimension.
3. An input can be used in the calculation if its size in a particular dimension either matches the output size in that dimension, or has value exactly 1.
4. If an input has a dimension size of 1 in its shape, the first data entry in that dimension will be used for all calculations along that dimension. In other words, the stepping machinery of the ufunc will simply not step along that dimension (the stride will be 0 for that dimension).

The operation is possible (doesnt result in shape mismatch error you mentioned) in three cases:

1. The arrays all have exactly the same shape.
2. The arrays all have the same number of dimensions and the length of each dimensions is either a common length or 1.
3. The arrays that have too few dimensions can have their shapes prepended with a dimension of length 1 to satisfy property 2.

Examples can be found via the link above.

Answer 2

Let me see if I get it...

>>> from numpy import ones, newaxis
>>> A = ones((4,3))   # 4 rows x 3 cols
>>> A.shape
(4, 3)
>>> A
array([[ 1.,  1.,  1.],
       [ 1.,  1.,  1.],
       [ 1.,  1.,  1.],
       [ 1.,  1.,  1.]])
>>> 
>>> ones((4,1))      # 4 rows x 1 col
array([[ 1.],
       [ 1.],
       [ 1.],
       [ 1.]])
>>> A + ones((4,1))
array([[ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.]])
>>> 
>>> ones((1,3))      # 1 row x 3 cols
array([[ 1.,  1.,  1.]])
>>> A + ones((1,3))  
array([[ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.]])
>>> 
>>> B = ones((3,))   # a 1D array
>>> B
array([ 1.,  1.,  1.])
>>> B.shape
(3,)
>>> A + B
array([[ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.]])
>>> 
>>> C = ones((4,))   # a 1D array
>>> C.shape
(4,)
>>> C
array([ 1.,  1.,  1.,  1.])
>>> A + C
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: shape mismatch: objects cannot be broadcast to a single shape
>>> 
>>> D = C[:,newaxis]
>>> D.shape
(4, 1)
>>> A + D
array([[ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.]])

The broadcast needed to do 4 x 3 vector plus a 1D vector with 3 elements succeeds.

The broadcast needed to do 4 x 3 vector plus a 1D vector with 4 elements fails.

>>> D = C[:,newaxis]

converts C to a 2D vector of a compatible shape.