Python: How to make a completely unshared copy of a complicated list? (Deep copy is not enough)

Question

Have a look at this Python code:

a = [1, 2, 3]
b = [4, 5, 6]
c = [[a, b], [b, a]] # [[[1, 2, 3], [4, 5, 6]], [[4, 5, 6], [1, 2, 3]]]
c[0][0].append(99)   # [[[1, 2, 3, 99], [4, 5, 6]], [[4, 5, 6], [1, 2, 3, 99]]]

Notice how modifying one element of c modifies that everywhere. That is, if 99 is appended to c[0][0], it is also appended to c[1][1]. I am guessing this is because Python is cleverly referring to the same object for c[0][0] and c[1][1]. (That is their id() is the same.)

Question: Is there something that can be done to c so that its list elements can be safely locally modified? Above is just an example, my real problem has a list much more complicated, but having a similar problem.

(Sorry for the poorly formed question above. Python gurus please feel free to modify the question or tags to better express this query.)

Answer 1

Use [:]:

>>> a = [1, 2]
>>> b = a[:]
>>> b.append(9)
>>> a
[1, 2]

Alteratively, use copy or deepcopy:

>>> import copy
>>> a = [1, 2]
>>> b = copy.copy(a)
>>> b.append(9)
>>> a
[1, 2]

copy works on objects other than lists. For lists, it has the same effect as a[:]. deepcopy attempts to recursively copy nested elements, and is thus a more "thorough" operation that copy.

Answer 2

Depending on you situation, you might want to work against a deep copy of this list.

Answer 3

When you want a copy, you explicitly make a copy - the cryptical [:] "slice it all" form is idiomatic, but my favorite is the much-more-readable approach of explicitly calling list.

If c is constructed in the wrong way (with references instead of shallow copies to lists you want to be able to modify independently) the best thing would be to fix the way it's built (why build it wrong and then labor to fix it?!), but if that's outside your control it IS possible to undo the damage -- just loop on c (recursively if needed), with an index, reassigning the relevant sublists to their copies. For example, if you know for sure that c's structure is two-level as you indicate, you can save yourself without recursion:

def fixthewronglymadelist(c):
  for topsublist in c:
    for i, L in enumerate(topsublist):
      topsublist[i] = list(L)

Despite what other answers suggest, copy.deepcopy would be hard to bend to this peculiar purpose, if all you're given is the wrongly made c: doing just copy.deepcopy(c) would carefully replicate whatever c's topology is, including multiple references to the same sublists! :-)

Answer 4

To see Stephan's suggestion at work, compare the two outputs below:

a = [1, 2, 3]
b = [4, 5, 6]
c = [[a, b], [b, a]]
c[0][0].append(99)
print c
print "-------------------"
a = [1, 2, 3]
b = [4, 5, 6]
c = [[a[:], b[:]], [b[:], a[:]]]
c[0][0].append(99)
print c

The output is as follows:

[[[1, 2, 3, 99], [4, 5, 6]], [[4, 5, 6], [1, 2, 3, 99]]]
-------------------
[[[1, 2, 3, 99], [4, 5, 6]], [[4, 5, 6], [1, 2, 3]]]

Answer 5

To convert an existing list of lists to one where nothing is shared, you could recursively copy the list.

deepcopy will not be sufficient, as it will copy the structure as-is, keeping internal references as references, not copies.

def unshared_copy(inList):
    if isinstance(inList, list):
        return list( map(unshared_copy, inList) )
    return inList

alist = unshared_copy(your_function_returning_lists())

Note that this assumes the data is returned as a list of lists (arbitrarily nested). If the containers are of different types (eg. numpy arrays, dicts, or user classes), you may need to alter this.