Going asynchronous: from Flask to Twisted Klein

, Tobias Oberstein

Introduction to Twisted Klein, which is like Flask, but allows running asynchronous code.

Preface

This post (part 1/2) is an introduction/tutorial for Twisted Klein and assumes basic knowledge of Python, Flask and Twisted. If you don't know Twisted, here is a good introduction.

Move on to part 2:

Related:

  • Twisted Mixing: A talk by Laurens Van Houtven at PyCon 2014
  • Crochet: A library for running Twisted code from blocking/threaded applications

Introduction

Flask is a Python microframework for the Web. I love it. It's incredible fun to to use and gets out of your way. It's light-weight, and works great when combined with Jinja2 for templating.

However, there is one thing that is limiting:

**I cannot use non-blocking, asynchronous code in my request handlers**
Top

Why is that?

Flask, like Django, Pyramid and others, are based on WSGI, an API standard for connecting Python Web frameworks to Web servers. Which is great. However, WSGI is a (fundamentally) synchronous and blocking API.

WSGI was designed for the old fashioned, threaded/pre-forked Web servers (think Apache). It does not blend well with the new breed of asynchronous networking frameworks like Twisted, asyncio or Tornado.

Why is that limiting?

A (single thread/process) of a WSGI based Web server can only handle one request at a time. The code serving a request will run to its end before another request can be served.

Which means if you want to do e.g. something network related within your request handler, like calling out to a Web service, this will again block the incoming request, and in fact the complete thread/process until the outgoing request comes back.

With more and more requests being blocked, this can quickly lead to a situation where all worker threads/processes are blocked, and no incoming request is served at all anymore. Bad, bad.

Please note that this not only applies to doing Web service requests in your request handlers, but any kind of I/O operation which might take longer, like file or database access.

Can we fix it?

Yes, we can ;) We first show an example using Flask with synchronous code. Then we show how to do the same with asynchronous code using Twisted Klein.

The Synchronous World

We'll first go through a trivial Flask app to demonstrate the issue. In the next section we'll see how to do the same using Twisted Klein, fixing the issue.

To run the following examples, you will need to install:

pip install flask requests

Consider the following Flask app:

from flask import Flask, request

app = Flask(__name__)


@app.route('/square/submit', methods = ['POST'])
def square_submit():
   x = int(request.form.get('x', 0))
   res = x * x
   return "{} squared is {}".format(x, res)


if __name__ == "__main__":
   app.run(port = 8080, debug = True)
   

This app will respond to HTTP/POST requests sent to /square/submit, extract the form variable x from the request, square that number and return a text response.

E.g. you can test it with

Now, what if you want to use a Web service to actually compute the square?

Please note, this is an example - normally you wouldn't call out to a Web service to compute a square ;) In fact, we do a simple HTTP/GET and not a full flavored Ajax whatever Web service request. It's just an example.

Consider this:

import requests
from flask import Flask, request

app = Flask(__name__)


@app.route('/square/submit', methods = ['POST'])
def square_submit():
   x = int(request.form.get('x', 0))

   ## simulate calling out to some Web service:
   ##
   r = requests.get('http://www.tavendo.com/')
   y = len(r.text)

   res = x * x + y
   return "{} squared plus {} is {}".format(x, y, res)


if __name__ == "__main__":
   app.run(port = 8080, debug = True)
   

The requests.get method simulates calling out to some Web service. This call will block until the outgoing HTTP request comes back. And hence, the client's original request to /square/submit will also block. And that means: the whole thread/process will block and cannot do more work like serving other incoming requests while still waiting on outgoing requests.

This can quickly get problematic: when calling out to the Web service takes some time, more and more requests will block until all threads or processes of the WSGI container running the app are blocked.

At that point, no new client requests will be served at all!

Note: It's quite common for WSGI containers to run something like 10 worker threads or processes. That means the app above will stall as soon as 10 incoming requests are served that each still wait on their outgoing request to return. And 10 isn't a large number ;)

Going Asynchronous

Alright, now that we've seen the issue, let's migrate to Twisted Klein and go asynchronous.

To run the following examples, you will need to install:

pip install twisted klein treq

Here is how the first Flask example from above looks when converted to Twisted Klein:

from klein import Klein

app = Klein()


@app.route('/square/submit', methods = ['POST'])
def square_submit(request):
   x = int(request.args.get('x', [0])[0])
   res = x * x
   return "{} squared is {}".format(x, res)


if __name__ == "__main__":
   app.run("localhost", 8080)
   

Alright. Nothing spectacular. It looks almost identical to the Flask version. The most visible difference is that our request handler square_submit() now takes an request arguments provided by Klein, whereas Flask has request as a global object.

Now, the more interesting second Flask example looks like this when converted to Klein (don't worry, I'll go through the details below):

import treq
from klein import Klein
from twisted.internet.defer import inlineCallbacks, returnValue

app = Klein()


@app.route('/square/submit', methods = ['POST'])
@inlineCallbacks
def square_submit(request):
   x = int(request.args.get('x', [0])[0])

   ## simulate calling out to some Web service:
   ##
   r = yield treq.get('http://www.tavendo.com/')
   content = yield r.content()
   y = len(content)

   res = x * x + y
   returnValue("{} squared plus {} is {}".format(x, y, res))


if __name__ == "__main__":
   app.run("localhost", 8080)
   

We are doing an (asynchronous) outgoing HTTP request on line 15. For this, we are using treq, a Twisted port of Requests, a HTTP client library that is easy to use.

The request treq.get() will not return immediately (since we are doing a network request), but it won't block (since we are in asynch land).

It will return a Twisted Deferred (elsewhere known as Futures). The Deferred stands for a result of an asynchronous operation, where the actual return value of the operation will only be available later.

To simplify the look and feel of the code, we are using yield, inlineCallbacks and returnValue following a specific variant of asynchronous programming ("co-routines"). This makes the code "look" synchronous, but nevertheless execute asynchronous under the hood.

Once the HTTP request returns (actually, the HTTP response header), the result of treq.get() becomes available in our variable r, which will hold a response object.

To actually retrieve the response body of our request, we now yield from r.content() (line 16).

There are only two other noteworthy details here: we need to decorate the function with inlineCallbacks to make the co-routine style available (line 9), and we cannot return directly, but use the returnValue wrapper (line 20).

Note: The reason returnValue is necessary goes deep into implementation details of Twisted and Python. In short: co-routines in Python 2 with Twisted are simulated using exceptions. Only Python 3.3+ has gotten native support for co-routines using the new yield from statement.

What have we gained?

Above variant using Twisted Klein will sustain a very high number of concurrent requests. It won't block. It will do so even on one thread, running on one CPU core! We finally are able to use asynchronous code inside Web request handlers. Awesome!

What's the downside?

The downside (obviously) is that you need to write your Web code on top of Twisted Klein, not Flask. At surface, Twisted Klein is very similar to Flask - however it is not a fully compatible drop-in replacement.

Summary

Flask is a great Web framework, and Twisted Klein makes the fun available in fully asynchronous Web applications. Going asynchronous allows you to do powerful things inside your request handlers - without running into performance problems due to blocking.

More

Read more in part 2: Mixing Web and WAMP code with Twisted Klein.

Top

Start experimenting and prototyping for IoT applications with Crossbar.io and Raspberry Pi!

Loaded with all the software you need to make the board part of WAMP applications!



Learn more

Recent posts

Atom Feed

Search this Site

Stay Informed

Sign up for our newsletter to stay informed of new product releases and features:
Community Chat