Ok, the quest for Python optimization continues. A friend of mine suggested that my stock options simulation in Python could be made much faster by using Cython, since it is a pure matematical simulation.
Cython is very easy to work with. Basically, it compiles a Python script into C, which then is compiled natively. Both steps are done automatically by distutils. Each .pyx module becomes a .so module, which can be imported by Python normally. (The main usage of Cython seems to be writing bindings.)
Actually, Cython understands a restricted dialect, but my script compiled almost without changes. No pain, no gain: the performance boost of compiled version was just 10% or so. Adding more and more of Cython 'dialect', like typed parameters, typed functions, increased the performance gain, but not much, not even near Psyco (which gives 4x for absolutely free).
The Activity Monitor's profiling showed that program spent too much time in Python interpreter, which was not right, since we compiled the thing, right? Time to take a look into generated C code. The main problem was usage of math and random modules in my code. Each time Cython code calls a Python module function, it needs to convert parameters from native to Python objects and reconvert the function returns.
Cython documentation tells how to use the C math functions directly. Also, I replaced seemingly innocent functions like max() by cdef'ed functions. Doing that increased the gain to around 1.4-1.5x.
Replacing random.gauss() was more difficult, since it does not have a C counterpart. I found a Java version of it in the Internet and reimplemented in Cython. This time the effort finally paid off: 23x performance gain over vanilla Python, 6x over Psyco.
(@#d00dz note the sacred number, 23!)
The "modular" approach plus the fact that code needs some Cythonization work, means that the programmer should profile the application and try to convert only the critical functions/classes/modules to Cython.
Since the scripts would need extensive rewriting to work under Cython, and I am happy enough with the current performance provided by Psyco, I am probably not going to use Cython in my simulations. But it is good to know that there are alternatives beyond Psyco (and before resorting to C++), should I need more performance.
