How to properly add PyTZ to a Google App Engine application?

Question

This is a little embarrassing, but I have not been able to find good resources on this topic.

I'm working on a Google App Engine application that requires sophisticated time zone conversions. Since I am nowhere near the imposed quotas, I have opted to go with PyTZ. However, I must be doing something wrong. What I've done so far is:

1. Downloaded PyTZ as a tarball
2. Installed it and copied the pytz directory into the root of my app (it is a sibling of the webapp directory, where app.yaml is located).

However, if I try to instantiate timezones, PyTZ can never seem to find any. Here is a sample session from GAE's interactive console:

from pytz import timezone

rome = timezone('Europe/Rome')

The output is the following:

Traceback (most recent call last):
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/admin/__init__.py", line 210, in post
    exec(compiled_code, globals())
  File "<string>", line 3, in <module>
  File "/Library/Python/2.5/site-packages/pytz-2009j-py2.5.egg/pytz/__init__.py", line 157, in timezone
UnknownTimeZoneError: 'Europe/Rome'

What is it I am doing wrong? Thank you in advance for your help.

NOTE: If I just use the python interactive shell locally things work as expected:

>>> from pytz import datetime, timezone
>>> rome = timezone('Europe/Rome')
>>> rome.localize(datetime.datetime.now())
datetime.datetime(2009, 11, 12, 0, 4, 52, 990114, tzinfo=<DstTzInfo 'Europe/Rome' CET+1:00:00 STD>)

Edit: I need to clarify I'm not using a zipped version of PyTZ. I have included the whole zoneinfo directory in my project:

pc-morena:pytz lyudmilangelov$ cd zoneinfo/
pc-morena:zoneinfo lyudmilangelov$ ls -l
total 448
drwxr-xr-x@  55 lyudmilangelov  staff   1870 Nov 10 12:48 Africa
drwxr-xr-x@ 135 lyudmilangelov  staff   4590 Nov 10 12:48 America
drwxr-xr-x@  12 lyudmilangelov  staff    408 Nov 10 12:48 Antarctica
drwxr-xr-x@   3 lyudmilangelov  staff    102 Nov 10 12:48 Arctic
drwxr-xr-x@  93 lyudmilangelov  staff   3162 Nov 10 12:48 Asia
...

Answer 1

Looks like you're not uploading the whole zoneinfo subtree of pytz -- 570 files in 22 directories, in the version of pytz I have at hand. Not sure why -- by default if they're in your app dir they should get uploaded. Try appcfg.py --verbose update (or even --noisy) after touching a few to check if it uploads them

Many little files are a bother in app engine, but fixing that requires a little tweak to function open_resource in init.py to make it get the "resource" from inside a zipfile instead of trying to open an actual separate file per "resource". This blog offers more details on how to go about it (esp. useful are the observations in the comments).

Answer 2

Actually that blog post is incorrect and makes a note to look at the new post, which is here: Using the newest zipped pytz on GAE

Answer 3

I figured it out and it was more embarrassing than I anticipated.

The problem was that (as I specified in the question) I had made PyTZ a sibling of webapp. However, in order for GAE to be able to load it, it needs to be a child of webapp. This is not terribly surprising, but I expected import pytz to fail had that been the problem.

Regardless, moving PyTZ under webapp fixes the issue and the module is still accessible from siblings of webapp (e.g. test).

Answer 4

Based on the title of the question, I would like (but don't have the so reputation yet) to vote up jgeewax's answer since it's what I used when I found this question from a search. However, I'll also add that after following the instructions mentioned there, I created a minimal project on github which illustrates using pytz in app engine with the zipped zoneinfo files. Might save someone 30 minutes to get an experiment up and running here: http://github.com/mpstx/appengine_py_pytz_zipimport_zoneinfo

Answer 5

An alternative is to use the packaged version of gae-pytz that includes further optimizations: http://pypi.python.org/pypi/gaepytz

You only need the pytz folder in the zipped archive, Using it in your gae project is as easy as:

from pytz.gae import pytz