PyPy on FreeBSD - get the nightlies!
PyPy is a tracing JIT compiler for Python that together with FreeBSD provides an awesome platform for your next-gen stack.
Too long to read?¶
- PyPy is awesome
- FreeBSD is awesome
- PyPy on FreeBSD is awesome2 !
- Tavendo operates a PyPy/FreeBSD buildslave
- Get the latest PyPy nightly for FreeBSD 9.2 amd64
What is PyPy?¶
A tracing JIT will run your program, identify hot loops (code paths that are executed very often) and compile those Python loops into native machine code.
The end result is higher performance - sometimes by a factor of 10 or more. The cost is longer startup time (the JIT needs to "warmup") and higher memory consumption (depends).
The Python language is often identified with it's standard implementation, CPython, but that is just one implementation. PyPy, Jython and IronPython for example are alternative language implementations that strive to be compatible with the Python language and default run-time more or less, and which provide different characteristics.
PyPy on FreeBSD¶
Tavendo is a big fan of FreeBSD. This system is incredible well engineered, feature-rich, robust and consistently integrated.
As it turns out, we are also big fans and users of Python and PyPy. So it's kind of natural that we wanted to give something back to the community (and sure, profit again, since we use this stuff for our own products and projects).
We've been contributing to the PyPy project in the past, e.g. adding support for kqueue() (a fast, scalable network interface to the kernel).
Now we have committed to another ongoing contribution: we have setup and run a FreeBSD buildslave for the PyPy project that compiles and tests PyPy on FreeBSD every night.
The immediate upside for users is: you can get the most current PyPy nightlies for FreeBSD 9.2 amd64 here.
As can be seen from the buildslave test logs, there are some issues left (failing tests). A couple of problems have already been fixed, but there is more to do. Nevertheless, in our experience, PyPy on FreeBSD is ready-to-rock!
Running PyPy on FreeBSD¶
Getting started with PyPy on FreeBSD is totally easy - 1) download, 2) unpack, 3) run:
$ wget http://buildbot.pypy.org/nightly/trunk/pypy-c-jit-latest-freebsd64.tar.bz2 $ tar xvjf pypy-c-jit-latest-freebsd64.tar.bz2 $ ./pypy-c-jit-68318-48ee7b6ba963-freebsd64/bin/pypy Python 2.7.3 (48ee7b6ba963, Nov 25 2013, 23:01:06) [PyPy 2.3.0-alpha0 with GCC 4.2.1 Compatible FreeBSD Clang 3.3 (tags/RELEASE_33/final 183502)] on freebsd9 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``PyPy is a tool to keep otherwise dangerous minds safely occupied.'' >>>>
The PyPy executable is built with the latest Clang 3.3 from the ports collection (though FreeBSD 9 is still using GCC as the default compiler, we build using Clang, since that is what FreeBSD 10 and further will use as default).
The executable is dynamically linked and depends on a couple of libraries from the ports collection:
- libffi (
- libexecinfo (
- gettext (
- ncurses (
- expat2 (
- openssl (
- libiconv (
You'll need to have those installed. If you are unsure, just do a
$ ldd ./pypy-c-jit-68318-48ee7b6ba963-freebsd64/bin/pypy
to check if there are any unresolved dynamic libraries.
Now, does it rock?¶
First tests and measurements indicate that PyPy/FreeBSD is an incredible capable stack to run stuff on - we mostly do networking servers based on Twisted.
We will publish hard numbers soon, and I hopefully find time to write a blog post on my journey into the JITted land.