I'm developing a restful Web service that runs as a servlet (using blocking IO) in Jetty. Figuring out the optimal setting for max threads seems hard.
Is there a researched formula for deciding the max number of threads from some easily measurable characteristics of the rest of the setup?
No there is not. Keep you number of threads limited and under control so you not exceed system resources, Java's limit is usually around 100-200 live threads.
Good way to do it is by using Executors from java.util.concurrent.
The answer depends on the maximum number of simultaneous connections you expect to handle. You should allow as many threads as connections you expect.
andreasmk2 is incorrect about the number of threads. I've run apps with 1000 threads and had no issue with system resources; of course it depends on the specifics on your system. You'd run into a system limitation, not a Java limitation.
My problem is that I don't know how to form a reasonable expectation for the number of simultaneous connections. Presumably at some point it's better to refuse new connections than to let everything slow down because there are too make requests being serviced.
Realistic workloads are hard to simulate, which is why I'm looking for a formula already researched by someone else.
(The obvious upper bound is max heap size divided by the minimum amount of memory required to service a request, but even that is hard to measure in an environment with a garbage collector.)
Very simple and primitive one:
max_number_of_threads = number_of_CPUs * C
Where C depends on other factors of your application :-)
Ask yourself following questions:
Usually I set C rather low, e.g. 2 - 10.
Thanks. I read this as there not being any easy formula. :-(
(My app is an HTML5 validator. Sometimes it is clearly waiting on external servers. However, it's hard to pinpoint when it's actually CPU-bound either on its own or through the garbage collector.)