Most efficient way to remove duplicates from Python list while preserving order and removing the oldest element

Question

I've seen a bunch of solutions on the site to remove duplicates while preserving the oldest element. I'm interested in the opposite: removing duplicates while preserving the newest element, for example:

list = ['1234','2345','3456','1234']
list.append('1234')
>>> ['1234','2345','3456','1234','1234']
list = unique(list)
>>> ['2345','3456','1234']

How would something like this work?

Thanks.

Answer 1

Try to reverse the list with reversed(), then do it like described elsewhere and then reverse it again.

Answer 2

Well, you could reverse the list, do what the other answers said, and reverse it again.

Answer 3

Requires the items (or keys) to be hashable, works in-place on list-likes:

def inplace_unique_latest(L, key=None):
  if key is None:
    def key(x):
      return x
  seen = set()
  n = iter(xrange(len(L) - 1, -2, -1))
  for x in xrange(len(L) - 1, -1, -1):
    item = L[x]
    k = key(item)
    if k not in seen:
      seen.add(k)
      L[next(n)] = item
  L[:next(n) + 1] = []

Answer 4

This is what you want:

>>> from operator import itemgetter
>>> my_list = ['1234','2345','3456','1234','1234']
>>> d = dict((x, i) for i, x in enumerate(my_list))
>>> new_list = [k for k, _ in sorted(d.items(), key=itemgetter(1))]
>>> new_list
['2345', '3456', '1234']

Or:

def uniq_preserving_oldest(my_list):
    from operator import itemgetter
    d = dict((x, i) for i, x in enumerate(my_list))
    return [k for k, _ in sorted(d.items(), key=itemgetter(1))]

my_list = ['1234','2345','3456','1234','1234']
print uniq_preserving_oldest(my_list)