views:

25

answers:

1

I'm sending files to my S3 bucket that are basically gzipped database dumps. They keys are a human readable date ("2010-05-04.dump"), and along with that, I'm setting a metadata field to the UNIX time of the dump.

I want to write a script that retrieve the latest dump from the bucket. That is to say I want the the key with the largest unix time metadata value. Is this possible with Amazon S3, or is this not how S3 is meant to work?

I'm using both the command line tool aws, and the python library boto

+1  A: 

Here, this seems to work, but maybe not the most ideal (using boto)

latest_key = None
latest_ts = 0
for key in bucket.get_all_keys():
    # go through all keys and return the one with the higest timestamp
    ts = key.get_metadata('timestamp')

    if ts > latest_ts:
        latest_key = key
        latest_ts = ts
nbv4
Hmm, I don't think you can avoid iterating through all keys. (But does `get_metadata` do another query to S3? If yes, that could be avoided by just using the filenames.)
Jonik
Ah, but do you potentially have several files per day? In that case you'd indeed need to use the timestamp (or add time to filenames).
Jonik