So, I'm writing a python web application using the twisted web2 framework. There's a library that I need to use (SQLAlchemy, to be specific) that doesn't have asynchronous code. Would it be bad to spawn a thread to handle the request, fetch any data from the DB, and then return a response? I'm afraid that if there was a flood of requests, too many threads would be started and the server would be overwhelmed. Is there something built into twisted that prevents this from happening (eg request throttling)?
A:
See the docs, and specifically the thread pool which lets you control how many threads are active at most. Spawning one new thread per request would definitely be an inferior idea!
Alex Martelli
2010-02-15 05:45:02
Would a thread pool just for using SQLAlchemy be any better?
Psychcf
2010-02-15 05:50:03
@Pschcf, depending on the thread-safety characteristics of the underlying DB API adapter, it may or may not be worth dedicating a small thread pool just to DB work (the fact that it's via SQLAlchemy or other ORM is not as important, in terms of performance tuning, as the nature of the underlying DB engine and adapter).
Alex Martelli
2010-02-15 05:53:46