Reading files in GAE using python

Question

Hello all!

I created a simple python project that serves up a couple of pages. I'm using the 'webapp' framework and django. What I'm trying to do is use one template file, and load 'content files' that contain the actual page text.

When I try to read the content files using os.open, I get the following error: pageContent = os.open(pageUrl, 'r').read() OSError: [Errno 1] Operation not permitted: 'content_includes/home.inc' error

If I let the django templating system to read the same file for me, everything works fine!

So the question is What am I doing wrong that django isn't??? The same 'pageUrl' is used.

The code below will give me the error, while if I comment out the first pageContent assignment, everything works fine.

Code:

        pageName = "home";

        pageUrl = os.path.join(os.path.normpath('content_includes'), pageName + '.inc')
        pageContent = os.open(pageUrl, 'r').read()
        pageContent=template.render(pageUrl, template_values, debug=True);

        template_values = { 'page': pageContent, 
                           'test': "testing my app" 
                           }

Error:

Traceback (most recent call last):
  File "/opt/apis/google_appengine/google/appengine/ext/webapp/__init__.py", line 511, in __call__
    handler.get(*groups)
  File "/home/odessit/Development/Python/Alpha/main.py", line 19, in get
    pageContent = os.open(pageUrl, 'r').read()
  File "/opt/apis/google_appengine/google/appengine/tools/dev_appserver.py", line 805, in FakeOpen
    raise OSError(errno.EPERM, "Operation not permitted", filename)
OSError: [Errno 1] Operation not permitted: 'content_includes/home.inc'

app.yaml:

handlers:
- url: /javascript
  static_dir: javascript

- url: /images
  static_dir: images

- url: /portfolio
  static_dir: portfolio

- url: /.*
  script: main.py

Answer 1

os.path.normpath() on "content_includes" is a no-op - normpath just removes double slashes and other denormalizations. What you probably want is to build a path relative to the script, in which case you should do something like os.path.join(os.path.dirname(__file__), 'content_includes', pageName + '.inc').

Answer 2

If you dig in the dev_appserver.py source code and related files you see that the server does some incarnate checking to ensure that you open only files from below your applications root (in fact the rules seem even more complex).

For file access troubled I instrumented that "path permission checking" code from the development server to find that I was using absolute paths. We propbably should do a patch to appengine to provide better error reporting on that: IIRC the Appserver does not display the offending path but a mangled version of this which makes debugging difficult.