How can I measure the speed of code written in Java? (AI algorithms)

Answer 1

I usually use

System.currentTimeMillis()

to calculate deltas of time:

long start = System.currentTimeMillis();
/* do your algorithm iteration */
long elapsed = System.currentTimeMillis() - start;

Mind that depending on the operating system you use the precision of the funcion could be greater than 1 millisecond (also tenth of msecs) so you'll have to tweak it to be useful for your analysis.

EDIT: there is also the alternative of doing the same thing with System.nanoTime() but you don't have any guarantee that accuracy will be of nanoseconds.

Answer 2

This is another way (with nanoseconds)

long nanos = System.nanoTime();
// execute your stuff
long duration = System.nanoTime() - nanos;
int seconds = (int) (duration / 1000000000);
int milliseconds = (int) (duration / 1000000) % 1000;
int nanoseconds = (int) (duration % 1000000);
System.out.printf("%d seconds, %d milliseconds en %d nanoseconds\n", seconds, milliseconds, nanoseconds);

The nanos are extra, but nice.

Answer 3

As suggested by others, System.currentTimeMillis() is quite good, but note the following caveats:

• System.currentTimeMillis() measures elapsed physical time ("wall clock time"), not CPU time. If other applications are running on the machine, your code will get less CPU and its speed will decrease. So, bench only on otherwise idle systems.
• Similarly, a multi-threaded application on a multicore system may get extra, hidden CPU. The elapsed time measure does not capture the whole of the complexity of multi-threaded applications.
• Java needs a bit of "warm-up". The VM will first interpret code (which is slow), and, if a given method is used too many times, then the JIT compiler will translate the method to native code. Only at that point will the method reach its top speed. I recommend that you perform a few "empty loops" before calling System.currentTimeMillis().
• Accuracy of System.currentTimeMillis() is rarely of 1 ms. On many systems, accuracy is no better than 10 ms, or even more. Also, the JVM will sometimes run the GC, inducing noticeable pauses. I suggest that you organize your measure in a loop and insist upon running for at least a few seconds.

This yields the following code:

for (int i = 0; i < 10; i ++) {
    runMethod();
}
int count = 10;
for (;;) {
    long begin = System.currentTimeMillis();
    for (int i = 0; i < count; i ++)
        runMethod();
    long end = System.currentTimeMillis();
    if ((end - begin) < 10000) {
        count *= 2;
        continue;
    }
    reportElapsedTime((double)(end - begin) / count);
}

As you see, there is first ten "empty" runs. Then the program runs the method in a loop, as many times as necessary so that the loop takes at least ten seconds. Ten seconds ought to be enough to smooth out GC runs and other system inaccuracies. When I bench hash function implementations, I use two seconds, and even though the function itself triggers no memory allocation at all, I still get variations of up to 3%.

Answer 4

If you are interested in a lot of precision in your measurements, you should measure CPU time, not 'wall clock time'. That way you will not measure the time the OS spends doing something else. In order to measure this time you can perhaps look at Java benchmarking CPU time

Answer 5

Although all the answers here are valid, I would propose that measuring real time may not be an entirely relevant to your goal, which is to compare and contrast varying search algorithms to find the "best". In this case, it's much simpler to count the number of nodes you search. While it's nice to also know the run time, there's a lot of noise involved since each algorithm may hit the CPU/cache/memory/disk in a particular way. By measuring nodes, you're looking at the single most important measure of how good a search algorithm is, since the fewer nodes it searches, the quicker it will find the answer.