Python: Time a code segment for testing performance (with timeit)

Question

Hi,

I've a python script which works just as it should but I need to write the time for the execution. I've gooled that I should use timeit but I can't seem to get it to work.

My Python script looks like this:

import sys
import getopt
import timeit
import random
import os
import re
import ibm_db
import time
from string import maketrans
myfile = open("results_update.txt", "a")

for r in range(100):
    rannumber = random.randint(0, 100)

    update = "update TABLE set val = %i where MyCount >= '2010' and MyCount < '2012' and number = '250'" % rannumber
    #print rannumber

    conn = ibm_db.pconnect("dsn=myDB","usrname","secretPWD")

for r in range(5):
    print "Run %s\n" % r        
    ibm_db.execute(query_stmt)
 query_stmt = ibm_db.prepare(conn, update)

myfile.close()
ibm_db.close(conn)

What I need it the time it takes the execution of the query and written to the file "results_update.txt". The purpose is to test an update statement for my database with different indexes and tuning mechanisms.

Sincerely

Mestika

Answer 1

you can use time.time ot time.clock before and after the block you want to time

import time

t0 = time.time()
code_block
t1 = time.time()

total = t1-t0

this method is not as exact as timeit (it does not average several runs) but it is straighforward.

time.time() (in windows and linux) and time.clock(in linux) have not enough resolution for fast functions (you get total = 0). In this case or if you want to average the time elapsed by several runs, you have to manually call the function multiple times (As I think you already do in you example code and timeit does automatically when you set its number argument)

import time

def myfast():
   code

n = 10000
t0 = time.time()
for i in range(n): myfast()
t1 = time.time()

total_n = t1-t0

in windows, as Corey stated in the comment, time.clock() has much higher precision (microsecond instead of second) and must be prefered to time.time().

Answer 2

Quite apart from the timing, this code you show is simply incorrect: you execute 100 connections (completely ignoring all but the last one), and then when you do the first execute call you pass it a local variable query_stmt which you only initialize after the execute call.

First, make your code correct, without worrying about timing yet: i.e. a function that makes or receives a connection and performs 100 or 500 or whatever number of updates on that connection, then closes the connection. Once you have your code working correctly is the correct point at which to think about using timeit on it!

Specifically, if the function you want to time is a parameter-less one called foobar you can use timeit.timeit (2.6 or later -- it's more complicated in 2.5 and before):

timeit.timeit('foobar()', number=1000)

You'd better specify the number of runs because the default, a million, may be high for your use case (leading to spending a lot of time in this code;-).