Cython Speed Boost vs. Usability

Question

I just came across Cython, while I was looking out for ways to optimize Python code. I read various posts on stackoverflow, the python wiki and read the article "General Rules for Optimization".

Cython is something which grasps my interest the most; instead of writing C-code for yourself, you can choose to have other datatypes in your python code itself.

Here is a silly test i tried,

#!/usr/bin/python
# test.pyx
def test(value):
    for i in xrange(value):
    i**2
    if(i==1000000):
        print i

test(10000001)

$ time python test.pyx

real    0m16.774s 
user    0m16.745s
sys     0m0.024s

$ time cython test.pyx

real    0m0.513s 
user    0m0.196s 
sys     0m0.052s

Now, honestly, i`m dumbfounded. The code which I have used here is pure python code, and all I have changed is the interpreter. In this case, if cython is this good, then why do people still use the traditional Python interpretor? Are there any reliability issues for Cython?

Answer 1

Cython is not another interpreter. It generates c-extensions for python, from python(-like) code. cython test.pyx will only generate a 'test.c' file, which (once compiled) can be used by python just like a normal python library.

That means that you are only measuring the time it takes for cython to translate your python code to c, not how fast that version of your code runs.

Answer 2

• cython test.pyx doesn't actually run your program. The cython binary is for processing your Cython code into a Python extension module. You would have to import it in Python to run it.

• #!/usr/bin/python isn't the best shebang line for Python scripts. #!/usr/bin/env python is generally preferred, which runs whatever python would on the command line.

  • Cython pyx files probably shouldn't have a shebang line at all, except in the corner case they are valid Python programs.

• You have an IndentationError in the posted code.

• Using the traditional interpreter is simpler and more portable. Cython is reliable, but has its limitations and quirks. It might be compelling to use it tons more if it magically gave the speedups your timings make it look like it does, but it actually gives smaller ones. You'll have to start using Cython-specific features to use C features to see a lot of speedup.

Answer 3

The other answers have already explained how you were just compiling the Cython code, not executing it. However, I thought that you might want to know how much faster Cython can make your code. When I compiled the code you have (though I ran the function from from a different module) with distutils, I got very marginal speed gains over straight Python - about 1%. However, if I added a few small changes to your code:

def test(long long value):
    cdef long long i
    cdef long long z
    for i in xrange(value):
        z = i**2
        if(i==1000000):
            print i
        if z < i:
            print "yes"

and compiled it, I got the following times:

Pure Python code: 20.4553578737 seconds

Cython code: 0.199339860234 seconds

That's a 100x speed-up. Not too shabby.

Answer 4

A big point that seems to be missing: Cython is not a strict superset of Python. There are some features that Python supports, but Cython does not. Most notably, generators and lambdas (but they are coming).