tags:

views:

23

answers:

1

Does paster create a new thread per request?

Can you set the maximum number of threads for paster to use i.e. a thread pool? How can you if this is possible?

+1  A: 

Per the docs, paster supports different server choices, depending on the configuration -- including wsgiutils, "the start of support for twisted.web2 ... patches welcome" (that would be an async server instad), and "SCGI, FastCGI and AJP protocols, for connection an external web server (like Apache) to your application. Both threaded and forking versions are available. This is based on flup."

You can configure maximum number of threads (and/or forked processes) on Apache, for example, and quite independently from paster, by working exclusively on the Apache configuration; clearly that is what you'll want to do if you've picked the flup/Apache/threaded combo.

At (roughly) the other extreme in the simplicity / functionality spectrum, I don't believe wsgiutils, out of the box, can be configured to use a thread pool (i.e., if I'm not mistaken, coding a new server kind around the minimal skeleton that wsgiutil provides would be needed to use a thread pool with it).

Clearly, if you need any kind of advanced configuration options, Apache's enormous power and flexibility are hard to beat:-).

Alex Martelli
i want to use haproxy to multiple paster instaces (each on a seperate port), that is how the big guys seem to be doing it like reddit.
Blankman
@Blankman, I imagine "the big guys" are fully able to write some code to customize paster, or (as can be configured out of the box w/o adding code) run paster on Apache (or nginx or ...). I'm sure no "big guy" (or _sensible_ smaller one) runs the unmodified code of wsgiutils as the underlying server for paster -- that's nice for development or a _tiny_-trafficked site, but (esp. with being so rigid in configuration possibilities) won't scale up much beyond that. You keep saying "paster" and totally ignoring the much more important issue of "what underlying web server"...!
Alex Martelli
@Alex paster responds to http requests, so it IS the server! They use HAProxy to send the requests to there various instances of paster (running on seperate ports ofcourse). see page#10 in the pdf: http://us.pycon.org/2010/conference/schedule/event/135/
Blankman
webserver = paster, n instances running 10 threads each.
Blankman