Memory not released by python cherrypy application on linux

Question

I have a long running process that will fetch 100k rows from the db genrate a web page and then release all the small objets (list, tuples and dicts). On windows, after each request the memory is freed. Howerver, on linux, the memory of the server keeps growing.

The following posts describes what the problem is and one possible solution.

http://pushingtheweb.com/2010/06/python-and-tcmalloc/

Is there any other way to get around this problem without having to compile my own python version which uses tcmalloc. That option is going to be very difficult to do, since python is controlled by the sys admin.

Answer 1

You may be able to compile Python in your own working directory rather than try to have the sysadmin replace the system Python.

First you should confirm that the tcmalloc solution solves your problem and does not impact performance too much for your application