Storing a list of objects in GAE

Question

I need to store some data that looks a little like this:

xyz 123
abc 456
hij 678
rer 838

Now I would just store it as a traditional string and integer model, and put in the datastore. But the data changes regularly, and is ONLY relevant when looked at as a COLLECTION. So it needs to be store as either a list of lists, or a list of objects, both of which can't really be done without pickling as far as I know. Can anyone help? Even storing it as a text file may work :S

Edit: I was a litle vague on details it seems.

I am sampling some data from an external source (scraping via BeautifulSoup/http2lib if it matters). This data then needs to be stored, as a whole, since it will be plotted on a graph. The data changes (but not often - perhaps once a week). Since the dataset is so simple (literally what you see above, 1 string field, and 1 integer) I figured it's easier to store them as a list of lists, then actually store them in a model. I have a feeling I have skipped over an even easier solution by concentrating too much on the fact the data needs to be stored together as one large lump.

I will be storing 500+ of these bits of data as a group, at once.

Answer 1

Your question is lacking some details, but have you tried using a ListProperty (or StringListProperty)? Is there some reason this didn't work for you?

Answer 2

If it's really just a list of tuples / two "columns", could you just use an alternating list and a ListProperty? This would be fine if the data had a consistent dimension, was small, and didn't require indexing.

e.g. To encode the example you gave in a list do:

# i forget if mixed types are allowed, but you get the idea.
["xyz", 123, "abc", 456, "hij", 678, "rer", 838]

Answer 3

You could just store them as two separate lists and only worry about combing them when you actually access them. Something like this:

class MyModel(db.Model):
    my_strings = db.StringListProperty()
    my_ints = db.ListProperty(int)

    def get_data(self):
        return zip(self.my_strings, self.my_ints)

    def set_data(self, data):
        self.my_strings = [element[0] for element in data]
        self.my_ints = [element[1] for element in data]

    data = property(get_data, set_data)

That way, you can just do something like

entity = MyModel()
entity.data = [("xyz", 123), ("abc", 456), ("hij", 678)]
entity.put()

# ...

for string_value, int_value in entity.data:
    # do something