PyPy on FreeBSD - get the nightlies!

, Tobias Oberstein
Tags: freebsd pypy

PyPy is a tracing JIT compiler for Python that together with FreeBSD provides an awesome platform for your next-gen stack.

Too long to read?

What is PyPy?

I case you've never heard about PyPy (which would be a pity): PyPy is an implementation of the Python language that works with a tracing JIT under the hood.

A tracing JIT will run your program, identify hot loops (code paths that are executed very often) and compile those Python loops into native machine code.

The end result is higher performance - sometimes by a factor of 10 or more. The cost is longer startup time (the JIT needs to "warmup") and higher memory consumption (depends).

The Python language is often identified with it's standard implementation, CPython, but that is just one implementation. PyPy, Jython and IronPython for example are alternative language implementations that strive to be compatible with the Python language and default run-time more or less, and which provide different characteristics.

PyPy on FreeBSD

Tavendo is a big fan of FreeBSD. This system is incredible well engineered, feature-rich, robust and consistently integrated.

As it turns out, we are also big fans and users of Python and PyPy. So it's kind of natural that we wanted to give something back to the community (and sure, profit again, since we use this stuff for our own products and projects).

We've been contributing to the PyPy project in the past, e.g. adding support for kqueue() (a fast, scalable network interface to the kernel).

Now we have committed to another ongoing contribution: we have setup and run a FreeBSD buildslave for the PyPy project that compiles and tests PyPy on FreeBSD every night.

The immediate upside for users is: you can get the most current PyPy nightlies for FreeBSD 9.2 amd64 here.

As can be seen from the buildslave test logs, there are some issues left (failing tests). A couple of problems have already been fixed, but there is more to do. Nevertheless, in our experience, PyPy on FreeBSD is ready-to-rock!

Running PyPy on FreeBSD

Getting started with PyPy on FreeBSD is totally easy - 1) download, 2) unpack, 3) run:

$ wget
$ tar xvjf pypy-c-jit-latest-freebsd64.tar.bz2
$ ./pypy-c-jit-68318-48ee7b6ba963-freebsd64/bin/pypy
Python 2.7.3 (48ee7b6ba963, Nov 25 2013, 23:01:06)
[PyPy 2.3.0-alpha0 with GCC 4.2.1 Compatible FreeBSD Clang 3.3 (tags/RELEASE_33/final 183502)] on freebsd9
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``PyPy is a tool to keep otherwise
dangerous minds safely occupied.''

The PyPy executable is built with the latest Clang 3.3 from the ports collection (though FreeBSD 9 is still using GCC as the default compiler, we build using Clang, since that is what FreeBSD 10 and further will use as default).

The executable is dynamically linked and depends on a couple of libraries from the ports collection:

  • libffi (devel/libffi)
  • libexecinfo (devel/libexecinfo)
  • gettext (devel/gettext)
  • ncurses (devel/ncurses)
  • expat2 (textproc/expat2)
  • openssl (security/openssl)
  • libiconv (converters/libiconv)

You'll need to have those installed. If you are unsure, just do a

$ ldd ./pypy-c-jit-68318-48ee7b6ba963-freebsd64/bin/pypy

to check if there are any unresolved dynamic libraries.

Now, does it rock?

First tests and measurements indicate that PyPy/FreeBSD is an incredible capable stack to run stuff on - we mostly do networking servers based on Twisted.

We will publish hard numbers soon, and I hopefully find time to write a blog post on my journey into the JITted land.

Until then, have fun and may the JIT be with you!

Recent posts

Atom Feed

Search this Site

Community Support Forums