Dynamically loading Python application code from database under Google App Engine

Question

Hi,

I need to store python code in a database and load it in some kind of bootstrap.py application for execution. I cannot use filesystem because I'm using GAE, so this is my only choice.

However I'm not a python experienced user.

I already was able to load 1 line of code and run it using eval, however a piece of code with two lines or more gave me a "invalid syntax" error.

I'm also thinking if it's possible to extend the "import" loader to implement the DB loading.

Thanks!

Answer 1

I somewhat agree with the commentators above, it sounds kind of dangerous. However:

I experimented a little with App Engine Console ( http://con.appspot.com/console/ ), and eval() indeed tended to throw SyntaxError's.

Instead, the exec statement might be your friend ( http://docs.python.org/release/2.5.2/ref/exec.html ).

I managed to run this in App Engine Console:

>>> exec "def f(x):\n    x = x + 1\n    y = 10\n    return x + y"
>>> f(10)
21

So try the exec statement, but remember the many, many (many!) perils of code coming directly from end-users.

Answer 2

I was able to do what I intent after reading more about Python dynamic code loading.

Here is the sample code. I removed headers to be lighter:

Thanks anyway!

=============

class DynCode(db.Model):
    name = db.StringProperty()
    code = db.TextProperty(default=None)

=============

class MainHandler(webapp.RequestHandler):
    def get(self):
        dyn = DynCode()
        dyn = "index"
        dyn.code = """
from google.appengine.ext import webapp
class MainHandler(webapp.RequestHandler):
    def get(self):
        self.response.out.write("Hello World\\n")
        self.response.out.write("Hello World 2\\n")
"""
        dyn.put()
        self.response.out.write("OK.")

def main():
    application = webapp.WSGIApplication([('/update', MainHandler)], debug=True)
    util.run_wsgi_app(application)

if __name__ == '__main__':
    main()

==================================

def main():
    query = DynCode.all()
    dyncodes = query.fetch(1)
    module = imp.new_module('mymodule')
    for dyn in dyncodes:
        exec dyn.code in module.__dict__

    application = webapp.WSGIApplication([('/', module.MainHandler)], debug=True)
    util.run_wsgi_app(application)

if __name__ == '__main__':
    main()

=======================

Answer 3

I was able to do what I intent after reading more about Python dynamic code loading.

Oops, too bad that it is broken by design because you don't understand the operating environment:

App Engine uses multiple web servers to run your application, and automatically adjusts the number of servers it is using to handle requests reliably. A given request may be routed to any server, and it may not be the same server that handled a previous request from the same user.

Answer 4

If you want a more robust mechanism, you probably want to read PEP302, which describes input hooks. You can use these to import code rather than having to eval it.