Can I improve python runtime by compiling?
I'm writing a small toy simulation in python. Granted, this simulations are slow. To my understanding, the major reason that python codes are slow is the fact that python is in interpreted language. I don't want to give up python since the clear syntax and the available library cut the writing time significantly. So is there a simple way for me to "compile" my python code?
I answer some questions: Yes, I'm using numpy. It greatly simplify the code and I don't think I can improve performance writing the functions on my own. I use numpy for all my lists and and I add all of the beads together. Namely. I invoke
pos += V*dt + forces*0.5*dt**2
where ''pos'', 'V', and 'forces' are all np.array of (2000,3) dimensions. I'm quite certain that the slow part in the forces calculation. This is logical as I have to iterate over all my particles and check their position. For my real project (Ph.D. stuff) I have code of about roughly the same level of complexity, and I know that this is the expensive stuff.
Python is a slightly odd language in that it is both interpreted and compiled. Well sort of. When you run it is compiled to ".pyc" bytecode - so we can quickly get bogged down in semantic details here. Hell I don't even know if what I just said is strictly accurate. But at the end of the day you want to speed things up so...
- First, use the profiler and timeit to work out where all the time is going
- Second, rewrite your pure python code to improve the slow bits you've discovered
- Third, see how it goes when optimised
- Now, depends on your scenario, but seriously think "Can I run it on a bigger CPU/memory"
- Ok, try rewriting those slow sections in C++
- Screw it, write it all in C++
If you get so far as the last option I dare say you're screwed and the savings aren't going to be significant.
If none of the solutions in the comment suffice, you can also take a look at cython. For a quick tutorial & example check:
Used at the correct spots (e.g. around frequently called functions) it can easily speed things up by a factor of 10 - 100.