tornado.gen — Simplify asynchronous code

tornado.gen is a generator-based interface to make it easier to work in an asynchronous environment. Code using the gen module is technically asynchronous, but it is written as a single generator instead of a collection of separate functions.

For example, the following asynchronous handler:

class AsyncHandler(RequestHandler):
    @asynchronous
    def get(self):
        http_client = AsyncHTTPClient()
        http_client.fetch("http://example.com",
                          callback=self.on_fetch)

    def on_fetch(self, response):
        do_something_with_response(response)
        self.render("template.html")

could be written with gen as:

class GenAsyncHandler(RequestHandler):
    @asynchronous
    @gen.engine
    def get(self):
        http_client = AsyncHTTPClient()
        response = yield gen.Task(http_client.fetch, "http://example.com")
        do_something_with_response(response)
        self.render("template.html")

Task works with any function that takes a callback keyword argument. You can also yield a list of Tasks, which will be started at the same time and run in parallel; a list of results will be returned when they are all finished:

def get(self):
    http_client = AsyncHTTPClient()
    response1, response2 = yield [gen.Task(http_client.fetch, url1),
                                  gen.Task(http_client.fetch, url2)]

For more complicated interfaces, Task can be split into two parts: Callback and Wait:

class GenAsyncHandler2(RequestHandler):
    @asynchronous
    @gen.engine
    def get(self):
        http_client = AsyncHTTPClient()
        http_client.fetch("http://example.com",
                          callback=(yield gen.Callback("key"))
        response = yield gen.Wait("key")
        do_something_with_response(response)
        self.render("template.html")

The key argument to Callback and Wait allows for multiple asynchronous operations to be started at different times and proceed in parallel: yield several callbacks with different keys, then wait for them once all the async operations have started.

The result of a Wait or Task yield expression depends on how the callback was run. If it was called with no arguments, the result is None. If it was called with one argument, the result is that argument. If it was called with more than one argument or any keyword arguments, the result is an Arguments object, which is a named tuple (args, kwargs).

Decorator

tornado.gen.engine(func)[source]

Decorator for asynchronous generators.

Any generator that yields objects from this module must be wrapped in this decorator. The decorator only works on functions that are already asynchronous. For RequestHandler get/post/etc methods, this means that both the tornado.web.asynchronous and tornado.gen.engine decorators must be used (for proper exception handling, asynchronous should come before gen.engine). In most other cases, it means that it doesn’t make sense to use gen.engine on functions that don’t already take a callback argument.

Yield points

Instances of the following classes may be used in yield expressions in the generator.

class tornado.gen.Task(func, *args, **kwargs)[source]

Runs a single asynchronous operation.

Takes a function (and optional additional arguments) and runs it with those arguments plus a callback keyword argument. The argument passed to the callback is returned as the result of the yield expression.

A Task is equivalent to a Callback/Wait pair (with a unique key generated automatically):

result = yield gen.Task(func, args)

func(args, callback=(yield gen.Callback(key)))
result = yield gen.Wait(key)
class tornado.gen.Callback(key)[source]

Returns a callable object that will allow a matching Wait to proceed.

The key may be any value suitable for use as a dictionary key, and is used to match Callbacks to their corresponding Waits. The key must be unique among outstanding callbacks within a single run of the generator function, but may be reused across different runs of the same function (so constants generally work fine).

The callback may be called with zero or one arguments; if an argument is given it will be returned by Wait.

class tornado.gen.Wait(key)[source]

Returns the argument passed to the result of a previous Callback.

class tornado.gen.WaitAll(keys)[source]

Returns the results of multiple previous Callbacks.

The argument is a sequence of Callback keys, and the result is a list of results in the same order.

WaitAll is equivalent to yielding a list of Wait objects.

Other classes

class tornado.gen.Arguments[source]

The result of a yield expression whose callback had more than one argument (or keyword arguments).

The Arguments object can be used as a tuple (args, kwargs) or an object with attributes args and kwargs.