ansaurus

Question

Pre-interpret Django site at deployment time

Answer 1

+1 A:

python -m compileall /path/to/django/site

Will precompile any .py files under the directory recursively.

How are you running django? If you're using WSGI the interpreter or interpreters are already running and would have already compiled a lot of your django site. What is being dynamically loaded?

MattH 2010-09-23 14:30:38

Cool thanks. Yeah using WSGI w/ Apache. So when I reload Apache much of the python is getting interpreted? I was under the impression that nothing got processed until the first page was requested.

Mike Bannister 2010-09-23 14:45:41

As far as I'm aware, the only way mod_wsgi runs is with a configurable number of python interpreters (>=1) which may or not be threaded. When apache starts these wsgi "worker" processes your django site is loaded.

MattH 2010-09-23 15:19:49

Since they all share a common set of files, the very first time an interpreter starts, .PYC files are built. The rest can then read the .PYC files, saving a tiny bit of time. They do, however, have to read the .PYC files, which does take some time. And, they have to search for template files. And open a database. The compilation time is tiny compared to the real overheads.

S.Lott 2010-09-23 18:36:28

Sure, then if there haven't been requests to the website then the WSGI processes could be swapped out. And when requests are frequent you can get better read performance from templates due to the OS disk cache. There are all sorts of reasons why you might see poorer or better performance over time.

MattH 2010-09-23 19:02:37

@MattH: Agreed. And, these effects (swapping, file system, database) far, far outweigh any benefit from attempting to prevent or avoid "all the python scripts need to be re-interpreted" (Whatever that means.)

S.Lott 2010-09-23 20:04:07

I precompile all of the files for my django projects because it saves me a bunch of selinux warnings. WSGI seems to run under the context of apache which doesn't have rights to write the .pyc files when my project is stored under a home directory with the rest of an application, even if WSGI is running under owning UID.

MattH 2010-09-23 20:23:30

@MattH: We use distutils and a writable egg cache.

S.Lott 2010-09-23 20:37:56

@S.Lott If my statement "all the python scripts need to be re-interpreted" makes no sense please help me (and possibly others) understand what really happens.

Mike Bannister 2010-09-23 21:31:43

@Mike Bannister: Python is generally interpreted. The usual Python configuration means that all Python code is always interpreted at all times. **At All Times**. The "reinterpreted" can't be a problem because it's the normal execution mode. What did **you** mean when **you** used this word?

S.Lott 2010-09-23 22:02:19

ansaurus

tags:

views:

answers:

Pre-interpret Django site at deployment time

related questions