Python: Taking an array and break it into subarrays based on some criteria

Question

I have an array of files. I'd like to be able to break that array down into one array with multiple subarrays, each subarray contains files that were created on the same day. So right now if the array contains files from March 1 - March 31, I'd like to have an array with 31 subarrays (assuming there is at least > 1 file for each day).

In the long run, I'm trying to find the file from each day with the latest creation/modification time. If there is a way to bundle that into the iterations that are required above to save some CPU cycles, that would be even more ideal. Then I'd have one flat array with 31 files, one for each day, for the latest file created on each individual day.

My current data structure is just a flat list of file names.

Answer 1

To get the files with the latest timestamps for each day, use a dict with days as keys and tuples of (filename, timestamp) as the values. Loop through all the files, and update the dict value for that day if the dict timestamp is less than the current file, or if no value for that day exists yet.

Answer 2

If you need to split a list into list of lists by some criteria, have a look at itertools.groupby().

Answer 3

Following up on Messa's answer, if your structure is like:

files=[{'date': datetime(2010, 3, 1, 0, 0, 10), 'file': 'foo'}, 
       {'date': datetime(2010, 3, 1, 12, 0, 10), 'file': 'bar'}, 
       {'date': datetime(2010, 3, 2, 3, 5, 10), 'file': 'baz'}, 
       {'date': datetime(2010, 3, 2, 3, 3, 10), 'file': 'foo'}]

try something like:

from itertools import groupby
map(lambda x: next(x[1]), 
    groupby(sorted(files, 
                   key=lambda x: x['date'],
                   reverse=True), 
         key=lambda x: datetime(x['date'].year, x['date'].month, x['date'].day)))

which will give you:

[{'date': datetime.datetime(2010, 3, 2, 3, 5, 10), 'file': 'baz'}, {'date': datetime.datetime(2010, 3, 1, 12, 0, 10), 'file': 'bar'}]

Basically, it first sorts by date in reverse (sorted), then groups by date (groupby), then takes the first element of every group (next[1]).