How to increase the performance of a loop which runs for every 'n' minutes.

Question

Hi Giving small background to my requirement and what i had accomplished so far: There are 18 Scheduler tasks run at regular intervals (least being 30 mins) takes input of nearly 5000 eligible employees run into a static method for iteration and generates a mail content for that employee and mails. An average task takes about 9 min multiplied by 18 will be roughly 162 mins meanwhile there would be next tasks which will be in queue (I assume). So my plan is something like the below loop

try {                       
        // Handle Arraylist of alerts eligible employees
        Iterator employee = empIDs.iterator();          
        while (employee.hasNext()) {                

            ScheduledImplementation.getInstance().sendAlertInfoToEmpForGivenAlertType((Long)employee.next(), configType,schedType);                             
        }
    } catch (Exception vEx)
    {
        _log.error("Exception Caught During sending " + configType + " messages:" + configType, vEx);
    }

Since I know how many employees would come to my method I will divide the while loop into two and perform simultaneous operations on two or three employees at a time. Is this possible. Or is there any other ways I can improve the performance. Some of the things I had implemented so far

1.Wherever possible made methods static and variables too

1. Didn't bother to catch exceptions and send back because these are background tasks. (And I assume this improves performance)

2. Get the DB values in one query instead of multiple hits.

If am successful in optimizing the while loop I think i can save couple of mins. Thanks

Answer 1

• Wherever possible make methods static and variables too
• Don't bother to catch exceptions and send back because these are background tasks. (And I assume this improves performance)

That just makes for very bad code style, nothing to do with performance.

• Get the DB values in one query instead of multiple hits.

Yes, this can be quite essential.

But in general, learn how to use a profiler to see where your code actually spends its time so that you can improve those parts, rather than making random "optimizations" based on hearsay, as you seem to be doing now.

Answer 2

Using static & final improves performance only when you are not using a decent JIT-compilers. Since most JREs are already using a good JIT-compiler you can ignore final and static for performance.

Better check your locking and synchronization style. A good indicator for locking problems is the CPU usage of your application. If it is low when it should be hard-working, there may be locks or db-queries blocking your application.

Maybe you can use techniques like Copy-On-Write or ReadWriteLocks.

Also check out the concurrent and atomic packages for some ideas how to improve or eliminate locking.

If there's one CPU-core with high load while others are idling around, try to do things in parallel. An ExecutorService may be helpful here.

Answer 3

Please don't fix anything until you know what needs to be fixed. Michael is right - profile - but I prefer this method.

Get a sense of time scales. When people optimize while loops, they are working at the level of nanoseconds. When you do a DB query, it is at the level of milliseconds. When you send an email and wait for the I/O to complete, you're talking seconds. If you have delays on the order of seconds, doing something that only saves milliseconds or nanoseconds will leave you disappointed in the results. **

So find out (don't guess) what is causing most of the delay, and work from there.

** To give a sense of the time scale, in 1 second a photon can travel most of the way to the moon; in 1 millisecond it can travel across a medium-sized country, and in 1 nanosecond it can travel the length of your foot.

Answer 4

Assuming you have profiled you application first and found that most of the delay is not in your program but somewhere else, e.g. the network/mail server....

You can create a fixed size thread pool e.g. 2-4 threads. and add a task for each mail you would have sent in the current thread.