Problems rendering a view with tidy (Django + Apache mod_wsgi)

Question

When running under the in-built development server, all runs fine.

However, when running under mod_wsgi in Apache, the following code (a replacement method for django.shortcuts.render_to_response) causes an internal server error:

# Tidy and render
def render_tidy_response(*args, **kwargs):    
    httpresponse_kwargs = {
                           'mimetype': kwargs.pop('mimetype', None)
                           }
    raw_template = loader.render_to_string(*args, **kwargs)    
    request = kwargs.get('request', None)
    output_options = dict(
                          output_xhtml=1,
                          tidy_mark=0, 
                          numeric_entities=1,
                          output_encoding='utf8',
                          drop_empty_paras=1,
                          escape_cdata=1,
                          clean=1,
                          wrap=0,
                          merge_divs=0,
                          output_bom=0
                          )
    if (request != None):
        request.session['current_user'] = request.user        
    tidy_output = str(tidy.parseString(raw_template, **output_options))        
    return HttpResponse(tidy_output, **httpresponse_kwargs)

The problem appears to be with the call to the tidy library, so the following change works fine:

.. as above
#tidy_output = str(tidy.parseString(raw_template, **output_options))        
return HttpResponse(raw_template, **httpresponse_kwargs)

Does anyone have an idea what the problem might be?

[ADDED] Here are the log entries relating to an erroring request, using debug logging, but I can't seem to find out what is causing the seg fault:

Main log (snippet)

[Tue Oct 26 21:35:14 2010] [info] Shared memory session cache initialised
[Tue Oct 26 21:35:14 2010] [info] Init: Initializing (virtual) servers for SSL
[Tue Oct 26 21:35:14 2010] [info] mod_ssl/2.2.14 compiled against Server: Apache/2.2.14, Library: OpenSSL/0.9.8k
[Tue Oct 26 21:35:14 2010] [info] mod_wsgi (pid=3110): Initializing Python.
[Tue Oct 26 21:35:14 2010] [debug] mod_wsgi.c(8104): mod_wsgi (pid=3110): Socket for 'test' is '/var/run/apache2/wsgi.3110.3.1.sock'.
[Tue Oct 26 21:35:14 2010] [info] mod_wsgi (pid=3948): Starting process 'test' with uid=33, gid=33 and threads=25.
[Tue Oct 26 21:35:14 2010] [notice] Apache/2.2.14 (Ubuntu) mod_ssl/2.2.14 OpenSSL/0.9.8k mod_wsgi/2.8 Python/2.6.5 configured -- resuming normal operations
[Tue Oct 26 21:35:14 2010] [info] Server built: Sep 28 2010 12:54:21
[Tue Oct 26 21:35:14 2010] [debug] worker.c(1757): AcceptMutex: sysvsem (default: sysvsem)
[Tue Oct 26 21:35:14 2010] [info] mod_wsgi (pid=3949): Attach interpreter ''.
[Tue Oct 26 21:35:14 2010] [info] mod_wsgi (pid=3950): Attach interpreter ''.
[Tue Oct 26 21:35:50 2010] [notice] child pid 3948 exit signal Segmentation fault (11)
[Tue Oct 26 21:35:50 2010] [info] mod_wsgi (pid=3948): Process 'test' has died, restarting.
[Tue Oct 26 21:35:50 2010] [info] mod_wsgi (pid=4041): Starting process 'test' with uid=33, gid=33 and threads=25.

Vhost log (snippet)

[Tue Oct 26 21:35:50 2010] [info] mod_wsgi (pid=4041): Attach interpreter ''.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8702): mod_wsgi (pid=4041): Starting 25 threads in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 1 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 2 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 3 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 4 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 5 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 6 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 7 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 8 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 9 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 10 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 11 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 12 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 13 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 14 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 15 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 16 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 17 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 18 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 19 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 20 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 21 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 22 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 23 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 24 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8708): mod_wsgi (pid=4041): Starting thread 25 in daemon process 'test'.
[Tue Oct 26 21:35:50 2010] [info] mod_wsgi (pid=4041): Enable deadlock thread in process 'test'.
[Tue Oct 26 21:35:50 2010] [info] mod_wsgi (pid=4041): Enable monitor thread in process 'test'.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8542): mod_wsgi (pid=4041): Deadlock timeout is 300.
[Tue Oct 26 21:35:50 2010] [debug] mod_wsgi.c(8545): mod_wsgi (pid=4041): Inactivity timeout is 0.

Answer 1

The answer it would seem, is an unfixed bug in µTidylib: Bug #14691 - bindings aren't 64bit safe

The patch in the comments solved the issue:

----------
diff -ruN uTidylib-0.2-orig/tidy/lib.py uTidylib-0.2/tidy/lib.py
--- uTidylib-0.2-orig/tidy/lib.py 2004-02-24 08:12:24.000000000 +0000
+++ uTidylib-0.2/tidy/lib.py 2008-10-16 14:45:56.000000000 +0100
@@ -130,6 +130,8 @@

sinkfactory=SinkFactory()

+_tidy.Create.restype = ctypes.POINTER(ctypes.c_void_p)
+
class _Document(object):
def __init__(self):
self.cdoc = _tidy.Create()
----------

Along with a modification to render_tidy_response:

---     (revision 409)
+++     (working copy)
@@ -15,6 +15,7 @@
                           output_xhtml=1,
                           tidy_mark=0, 
                           numeric_entities=1,
+             input_encoding='utf8',
                           output_encoding='utf8',
                           drop_empty_paras=1,
                           escape_cdata=1,
@@ -25,8 +26,8 @@
                           )
     if (request != None):
         request.session['current_user'] = request.user    
-    #tidy_output = str(tidy.parseString(raw_template, **output_options))    
-    return HttpResponse(raw_template, **httpresponse_kwargs)
+    tidy_output = str(tidy.parseString(raw_template.encode('utf-8'), **output_options))    
+    return HttpResponse(tidy_output, **httpresponse_kwargs)