JIT stands for “Just-In-Time” compilation, which is a technique used by some programming languages and runtime environments to improve the performance of executing code. That would make sense to a developer but for non-developer, I think the definition can be a bit confusing so let me try to explain this in a simple way.
Imagine you have a code blob written in your fav programming language (let’s say python for now). Now Python is an interpreted programming language, which means that the source code is executed line by line by a Python interpreter at runtime. Your machine can only understand binary numbers so python interpreter helps bridge the gap between your python code and the machine.
Due to Python’s dynamic nature and the way it handles data types, pure interpretation can result in relatively slower execution speeds compared to languages that are compiled to machine code directly (For example C, C++) which are signficantly faster to run. For more context on how compilation works, check this blog
Coming back to python(our selected programming language) which is interpreted at run time, our computer has to translate the python code into machine readable code which can be very slow if we have to many recurring function calls or loops (every function call has to be translated to machine code before executing).
Demo code: 🧪💻💥
Let me show the above issue with an example:
Imagine you have a very basic python code file (say sum.py), which runs a loop to add 2 numbers in loop running 1000 times:
def sum(a,b):
return a+b
for i in range(0,1000):
sum(i,i+1)
I can run the above code using time python sum.py to get ideas on the time it took to execute the above code.
For 1000 iterations, it takes 0.03 sec on Mac pro with 32 GB ram and intel processor:
Now, let’s crank the number of iteration to 100 M, the same code takes 14.5 secs to run 🤯 🦥 🐢
Magic of JIT
JIT can significantly help improve python runtime performance. It uses a technique called “tracing JIT” to identify frequently executed code paths and optimize them. JIT can detect any recurring function during the execution and store a compiled version on the fly. During the execution, JIT will start using the compiled function instead of parsing and interpreting the function during the loop.
In our example, JIT will detect that the function sum(a,b) is being called in a loop multiple times. Why not compile it on the fly and use the compiled version for execution which might save some CPU cycles.
JIT in action example:
I am using pypy3 package to run JIT but you can try other packages too (Numba etc). To install pypy3 use the below command in mac:
brew install pypy3
For 100M iterations, it takes 0.16 sec to run the same code with JIT (98.8% faster than python code without JIT 🚀🚀🏃🏃)
We have seen the benefits of JIT in a language like python where it can help speed up the execution times signficantly, but there are some limitations of JIT which I wanted to share here as well:
- JIT has a higher memory overhead than usual runtime. Since we have to store the compiled code in the machine which takes up memory, running JIT on a code base with too many functions would need significantly high memory compared to normal python operations.
- JIT works well with long running programs. For programs with not a lot of loops/functions, the memory overhead of running JIT is not worth the effort.
- It doesn’t do Ahead-Of-Time Compilation (AOT) like C where a program is compiled before running. It compiles the code during runtime but there is no support to pre-compile the code in advance.
Overall, I think JIT can significantly boost the performance of your code base and if you have long running loops, you should conisder it. In Shopify, we use have seen signficant speedups in code compilation using YJIT which is similar to JIT. Check how Ruby 3.2 is YJIT ready in this blog.
For more context on JIT using pypy, check this blog.
For any questions, reach out on LinkedIn