How to benchmark functions in Clojure?

Question

I know I can get the time take to evaluate a function can be printed out on the screen/stdout using the time function/macro.

The time macro returns the value of the evaluated function, which makes it great to use it inline. However I want to automatically measure the runtime under specific circumstances.

Is there a function which returns the elapsed time in some library to help with this benchmarking?

Answer 1

If it's just a matter of wanting to capture the string programmatically, you can bind *out* to something else before using time.

user=> (def x (with-out-str (time (+ 2 2))))
#'user/x
user=> x
"\"Elapsed time: 0.119 msecs\"\n"

If you want more control over the format, then you can create your own version of time using Java's System time methods, which is what the time macro uses under the hood anyway:

user => (macroexpand '(time (+ 2 2)))
(let* [start__4197__auto__ (. java.lang.System (clojure.core/nanoTime))  
       ret__4198__auto__ (+ 2 2)] 
     (clojure.core/prn (clojure.core/str "Elapsed time: " (clojure.core//     
          (clojure.core/double 
               (clojure.core/- (. java.lang.System (clojure.core/nanoTime)) 
                                start__4197__auto__)) 1000000.0) " msecs"))
 ret__4198__auto__)

Take that basic structure and replace the call to prn with whatever reporting mechanism you would prefer.

Answer 2

You might want to look into Hugo Duncan's benchmarking library for Clojure -- Criterium.

From the README:

Criterium measures the computation time of an expression. It is designed to address some of the pitfalls of benchmarking, and benchmarking on the JVM in particular.

This includes:

• statistical processing of multiple evaluations
• inclusion of a warm-up period, designed to allow the JIT compiler to optimise its code
• purging of gc before testing, to isolate timings from GC state prior to testing
• a final forced GC after testing to estimate impact of cleanup on the timing results