How to prioritise specific threads in tomcat

I am working on a Java web application for tomcat6 that offers suggest functionality. This means a user types in a free text and get suggestions for completing his input. It is essential that the web application needs to react very fast to make sense.

This web application makes suggestions for data that can be modified at any time. If new data is available the suggest index will be created completely new in the background with the help of a daemon thread. If the data preparation process is finished, the old index is thrown away and the new index comes into play. This offers the advantage of no service gaps.

The background process for data preparation costs a lot of CPU power. This causes the service to hang sometimes for more than a seconds, which makes the service less usable and causes a bad user experience.

My first attempt to solve the problem was to pause all background data preparation threads, when a request has to be processed. This attempt narrows the problem a bit, but the service is still not smooth. It is not that the piece of code for generating the suggests itself gets slower, but it seems as if tomcat does not start the thread for the request immediately all the time because of high load in other threads (I guess)

I have no idea how to face this problem. Can I interact with the thread scheduler of tomcat and tell him to force execution of requests? As expected adjusting the thread priority also did not help. I did not find configuration options for tomcat that offer help. Or is there no way to deal with this and I have to modify the software concept? I am helpless. Do you have any hints for my how to face this problem?

JanP

I would not change the thread priority, By doing that you are slowing down other Threads and will slow down other users. If you have synchronized data then you will run into a priority inversion problem, where your faster threads are waiting on lower priority threads to release locks on data.

Instead I would look at how to optimize the data generation process. What are you doing there ?

EDIT:

You could create an ExecutorService and send messages to it through a Queue like in this example: http://stackoverflow.com/questions/3174221/java-thread-pool-keep-running/3174748#3174748 In order to be able to change the Thread priority of the tasks instead of calling ExecutorService pool = Executors.newFixedThreadPool(3); you would create a ThreadFactory and then have the ThreadFactory lower the priority of the Threads, then call ExecutorService pool = Executors.newSingleThreadExecutor(threadFactory);

The data generation process is independent from requests (no shared objects or data). It just needs a lot of CPU power to build a new suggestion index. As long as it is working the old index is used. It is no problem to pause the data generation thread. When a request comes in, I supend the background working process (using a Semaphore) for the time of the request. But this is to late, because Tomcat itself needs time to forward the request to my web application (I guess), maybe because the underlying thread scheduler do not have any time slices left immediately?

Jan P 2010-08-11 11:18:37

@Jan P Is there one set of data for all users, or is the data different for each user ?

Romain Hippeau 2010-08-11 15:27:18

all users are working on the same suggest index, but this cant be the reason for the slow response times, because the service is slow with just one user too (sequentially accessing the suggest index).

Jan P 2010-08-11 17:46:25

@Jan P How is the index retrieved and calculated ? Is it from info in a DB ? How do you know when to recalculate it ? Could it possibly be recalculated periodically, say every 15 minutes ?

Romain Hippeau 2010-08-11 21:01:12

@jan P What I am getting at is if it possible to have a separate process generate your index periodically and then you simply pick it up from somewhere. This would allow you to scale out your problem or simply play with process priority to get the info.

Romain Hippeau 2010-08-11 21:39:26

The means to restructure the application. This is for sure a solution, but it's a lot of work. If nothing else helps I have to do this. But first I want to know, if there is another solution - especially pausing the background threads early enough. Do you have any hints or ideas refering to tomcat thread scheduler? Maybe there is simply no way to define an own scheduler logik, e.g. serve request-threads always first and immediately pause a set of background threads.

Jan P 2010-08-12 10:56:38

@Jan P I have updated my post to show you how you could put a task queue in your program and send tasks which would then be picked up by lower priority Threads if the index was out of date. You might want to put a flag somewhere to indicate that you were calculating another one.

Romain Hippeau 2010-08-12 12:09:04

ansaurus

tags:

views:

answers:

How to prioritise specific threads in tomcat

related questions