Dropbox builds Python implementation with fast jit compiler

A number of Dropbox developers are working on a new Python implementation that should make the programming language perform as well as code written in, say, C++. For this, the so-called Pyston uses a just-in-time compiler.

The Dropbox developers state that Python is preferred in almost all cases, but that in practice the programming language reaches its limits when software has to be scaled. In particular, the performance would be too low in that situation, making code better written in another programming language, such as the faster C++.

In an effort to make Python faster, the Dropbox team has looked at the just-in-time compilers that have been responsible for significant performance gains in the JavaScript processing of modern browsers in recent years. While some Python implementations based on just-in-time compilers already exist, the Dropbox developers in the Pyston project want to use method-at-a-time jit compilers. These would still perform significantly better than older jit implementations. Furthermore, another method for garbage collection is being looked at.

In order not to completely reinvent the wheel, the Pyston team uses parts of the llvm project that incorporates various open source compiler modules. Furthermore, Pyston is responsible for predicting the type of object, among other things, a mechanism known as type speculation. The techniques inline caches and fast attribute lookups are also applied.

Pyston is far from complete and the developers believe construction will take some time. It would also be too early to test the speed using benchmarks. However, the code has been made available on GitHub, with Dropbox hoping more developers will help with the development.