views:

3228

answers:

9

I've been search for quite a while with no success. My project isn't using Django, is there a simple way to serialize App Engine models (google.appengine.ext.db.Model) into JSON or do I need to write my own serializer? My model class is fairly simple. For instance:

class Photo(db.Model):
    filename = db.StringProperty()
    title = db.StringProperty()
    description = db.StringProperty(multiline=True)
    date_taken = db.DateTimeProperty()
    date_uploaded = db.DateTimeProperty(auto_now_add=True)
    album = db.ReferenceProperty(Album, collection_name='photo')

Thanks in advance.

+2  A: 

Even if you are not using django as a framework, those libraries are still available for you to use.

from django.core import serializers
data = serializers.serialize("xml", Photo.objects.all())
Andrew Sledge
Did you mean serializers.serialize("json", ...)? That throws "AttributeError: 'Photo' object has no attribute '_meta'". FYI - serializers.serialize("xml", Photo.objects.all()) throws "AttributeError: type object 'Photo' has no attribute 'objects'". serializers.serialize("xml", Photo.all()) throws "SerializationError: Non-model object (<class 'model.Photo'>) encountered during serialization".
+1  A: 

You don't need to write your own "parser" (a parser would presumably turn JSON into a Python object), but you can still serialize your Python object yourself.

Using simplejson:

import simplejson as json
serialized = json.dumps({
    'filename': self.filename,
    'title': self.title,
    'date_taken': date_taken.isoformat(),
    # etc.
})
Jonathan Feinberg
Yes, but I don't want to have to do this for every model. I'm trying to find a scalable approach.
oh and i'm really surprised that I can't find any best practices on this. I thought app engine model + rpc + json was a given...
+11  A: 

A simple recursive function can be used to convert an entity (and any referents) to a nested dictionary that can be passed to simplejson:

import datetime
import time

SIMPLE_TYPES = (int, long, float, bool, dict, basestring, list)

def to_dict(model):
    output = {}

    for key, prop in model.properties().iteritems():
        value = getattr(model, key)

        if value is None or isinstance(value, SIMPLE_TYPES):
            output[key] = value
        elif isinstance(value, datetime.date):
            # Convert date/datetime to ms-since-epoch ("new Date()").
            ms = time.mktime(value.utctimetuple()) * 1000
            ms += getattr(value, 'microseconds', 0) / 1000
            output[key] = int(ms)
        elif isinstance(value, db.Model):
            output[key] = to_dict(value)
        else:
            raise ValueError('cannot encode ' + repr(prop))

    return output
David Wilson
There is a small mistake in the code:Where you have "output[key] = to_dict(model)" it should be: "output[key] = to_dict(value)".Besides that it's perfect.Thanks!
arikfr
Thanks @arikfr!
David Wilson
This code will fail when it encounters a UserProperty. I worked around it with doing "output[key] = str(value)" in the final else, instead of raising an error.
Boris Terzic
Great stuff. Small improvement is to use iterkeys() instead since you don't use "prop" there.
PEZ
+4  A: 

For simple cases, I like the approach advocated here at the end of the article:

  # after obtaining a list of entities in some way, e.g.:
  user = users.get_current_user().email().lower();
  col = models.Entity.gql('WHERE user=:1',user).fetch(300, 0)

  # ...you can make a json serialization of name/key pairs as follows:
  json = simplejson.dumps(col, default=lambda o: {o.name :str(o.key())})

The article also contains, at the other end of the spectrum, a complex serializer class that enriches django's (and does require _meta -- not sure why you're getting errors about _meta missing, perhaps the bug described here) with the ability to serialize computed properties / methods. Most of the time you serialization needs lay somewhere in between, and for those an introspective approach such as @David Wilson's may be preferable.

Alex Martelli
+1  A: 

If you use app-engine-patch it will automatically declare the _meta attribute for you, and then you can use django.core.serializers as you would normally do on django models (as in sledge's code).

App-engine-patch has some other cool features such has an hybrid authentication (django + google accounts), and the admin part of django works.

mtourne
what's the difference between app-engine-patch vs google-app-engine-django vs the django version shipped with app engine python sdk? From what I understand, app-engine-patch is more complete?
I haven't tried the version of django on app engine, but I think it's integrated as is. google-app-engine-django if I'm not mistaken tries to make django's model work with app-engine (with some limitations). app-engine-patch uses directly app-engine models, they just add some minore stuffs to it. There is a comparison between the two on their website.
mtourne
+2  A: 

This is the simplest solution I found. It requires only 3 lines of codes.

Simply add a method to your model to return a dictionary:

class DictModel(db.Model):
    def to_dict(self):
       return dict([(p, unicode(getattr(self, p))) for p in self.properties()])

SimpleJSON now works properly:

class Photo(DictModel):
   filename = db.StringProperty()
   title = db.StringProperty()
   description = db.StringProperty(multiline=True)
   date_taken = db.DateTimeProperty()
   date_uploaded = db.DateTimeProperty(auto_now_add=True)
   album = db.ReferenceProperty(Album, collection_name='photo')

from django.utils import simplejson
from google.appengine.ext import webapp

class PhotoHandler(webapp.RequestHandler):
   def get(self):
      photos = Photo.all()
      self.response.out.write(simplejson.dumps([p.to_dict() for p in photos]))
Mtgred
hey thanks for the tip. this works great except I can't seem to serialize the date field. I get:TypeError: datetime.datetime(2010, 5, 1, 9, 25, 22, 891937) is not JSON serializable
givp
Hi, thanks for pointing the issue.The solution is to convert the date object to a string. For instance you can wrap the call to "getattr(self, p)" with "unicode()". I edited code to reflect this.
Mtgred
+2  A: 

There's a method, "Model.properties()", defined for all Model classes. It returns the dict you seek.

`from django.utils import simplejson class Photo(db.Model): # ...

my_photo = Photo(...) simplejson.dumps(my_photo.properties()) `

See http://code.google.com/appengine/docs/python/datastore/modelclass.html#Model_properties

Jeff Carollo
A: 

To serialize models, add a custom json encoder as in the following python:

import datetime
from google.appengine.api import users
from google.appengine.ext import db
from django.utils import simplejson

class jsonEncoder(simplejson.JSONEncoder):
  def default(self, obj):
    isa=lambda *xs: any(isinstance(obj, x) for x in xs) # shortcut
    return obj.isoformat() if isa(datetime.datetime) else \
      dict((p, getattr(obj, p)) for p in obj.properties()) if isa(db.Model) else \
      obj.email() if isa(users.User) else \
      simplejson.JSONEncoder.default(self, obj)

# use the encoder as: 
simplejson.dumps(model, cls=jsonEncoder)

This will encode:

  • a date as as isoformat string (per this suggestion),
  • a model as a dict of its properties,
  • a user as his email.

To decode the date you can use this javascript:

function decodeJsonDate(s){
  return new Date(s.slice(0,19).replace('T',' ')+' GMT');
} // Note that this function truncates milliseconds.
dpatru
A: 

Mtgred's answer above worked wonderfully for me -- I slightly modified it so I could also get the key for the entry. Not as few lines of code, but it gives me the unique key:

class DictModel(db.Model):
def to_dict(self):
    tempdict1 = dict([(p, unicode(getattr(self, p))) for p in self.properties()])
    tempdict2 = {'key':unicode(self.key())}
    tempdict1.update(tempdict2)
    return tempdict1
vvanhee