tornado.gen
— Generator-based coroutines¶
tornado.gen
implements generator-based coroutines.
Note
The “decorator and generator” approach in this module is a
precursor to native coroutines (using async def
and await
)
which were introduced in Python 3.5. Applications that do not
require compatibility with older versions of Python should use
native coroutines instead. Some parts of this module are still
useful with native coroutines, notably multi
, sleep
,
WaitIterator
, and with_timeout
. Some of these functions have
counterparts in the asyncio
module which may be used as well,
although the two may not necessarily be 100% compatible.
Coroutines provide an easier way to work in an asynchronous environment than chaining callbacks. Code using coroutines is technically asynchronous, but it is written as a single generator instead of a collection of separate functions.
For example, here’s a coroutine-based handler:
class GenAsyncHandler(RequestHandler):
@gen.coroutine
def get(self):
http_client = AsyncHTTPClient()
response = yield http_client.fetch("http://example.com")
do_something_with_response(response)
self.render("template.html")
Asynchronous functions in Tornado return an Awaitable
or Future
;
yielding this object returns its result.
You can also yield a list or dict of other yieldable objects, which will be started at the same time and run in parallel; a list or dict of results will be returned when they are all finished:
@gen.coroutine
def get(self):
http_client = AsyncHTTPClient()
response1, response2 = yield [http_client.fetch(url1),
http_client.fetch(url2)]
response_dict = yield dict(response3=http_client.fetch(url3),
response4=http_client.fetch(url4))
response3 = response_dict['response3']
response4 = response_dict['response4']
If tornado.platform.twisted
is imported, it is also possible to
yield Twisted’s Deferred
objects. See the convert_yielded
function to extend this mechanism.
Changed in version 3.2: Dict support added.
Changed in version 4.1: Support added for yielding asyncio
Futures and Twisted Deferreds
via singledispatch
.
Decorators¶
- tornado.gen.coroutine(func: Callable[[...], Generator[Any, Any, _T]]) Callable[[...], Future[_T]] [source]¶
- tornado.gen.coroutine(func: Callable[[...], _T]) Callable[[...], Future[_T]]
Decorator for asynchronous generators.
For compatibility with older versions of Python, coroutines may also “return” by raising the special exception
Return(value)
.Functions with this decorator return a
Future
.Warning
When exceptions occur inside a coroutine, the exception information will be stored in the
Future
object. You must examine the result of theFuture
object, or the exception may go unnoticed by your code. This means yielding the function if called from another coroutine, using something likeIOLoop.run_sync
for top-level calls, or passing theFuture
toIOLoop.add_future
.Changed in version 6.0: The
callback
argument was removed. Use the returned awaitable object instead.
- exception tornado.gen.Return(value: Optional[Any] = None)[source]¶
Special exception to return a value from a
coroutine
.If this exception is raised, its value argument is used as the result of the coroutine:
@gen.coroutine def fetch_json(url): response = yield AsyncHTTPClient().fetch(url) raise gen.Return(json_decode(response.body))
In Python 3.3, this exception is no longer necessary: the
return
statement can be used directly to return a value (previouslyyield
andreturn
with a value could not be combined in the same function).By analogy with the return statement, the value argument is optional, but it is never necessary to
raise gen.Return()
. Thereturn
statement can be used with no arguments instead.
Utility functions¶
- tornado.gen.with_timeout(timeout: Union[float, datetime.timedelta], future: Yieldable, quiet_exceptions: Union[Type[Exception], Tuple[Type[Exception], ...]] = ())[source]¶
Wraps a
Future
(or other yieldable object) in a timeout.Raises
tornado.util.TimeoutError
if the input future does not complete beforetimeout
, which may be specified in any form allowed byIOLoop.add_timeout
(i.e. adatetime.timedelta
or an absolute time relative toIOLoop.time
)If the wrapped
Future
fails after it has timed out, the exception will be logged unless it is either of a type contained inquiet_exceptions
(which may be an exception type or a sequence of types), or anasyncio.CancelledError
.The wrapped
Future
is not canceled when the timeout expires, permitting it to be reused.asyncio.wait_for
is similar to this function but it does cancel the wrappedFuture
on timeout.New in version 4.0.
Changed in version 4.1: Added the
quiet_exceptions
argument and the logging of unhandled exceptions.Changed in version 4.4: Added support for yieldable objects other than
Future
.Changed in version 6.0.3:
asyncio.CancelledError
is now always considered “quiet”.Changed in version 6.2:
tornado.util.TimeoutError
is now an alias toasyncio.TimeoutError
.
- tornado.gen.sleep(duration: float) Future[None] [source]¶
Return a
Future
that resolves after the given number of seconds.When used with
yield
in a coroutine, this is a non-blocking analogue totime.sleep
(which should not be used in coroutines because it is blocking):yield gen.sleep(0.5)
Note that calling this function on its own does nothing; you must wait on the
Future
it returns (usually by yielding it).New in version 4.1.
- class tornado.gen.WaitIterator(*args: Future, **kwargs: Future)[source]¶
Provides an iterator to yield the results of awaitables as they finish.
Yielding a set of awaitables like this:
results = yield [awaitable1, awaitable2]
pauses the coroutine until both
awaitable1
andawaitable2
return, and then restarts the coroutine with the results of both awaitables. If either awaitable raises an exception, the expression will raise that exception and all the results will be lost.If you need to get the result of each awaitable as soon as possible, or if you need the result of some awaitables even if others produce errors, you can use
WaitIterator
:wait_iterator = gen.WaitIterator(awaitable1, awaitable2) while not wait_iterator.done(): try: result = yield wait_iterator.next() except Exception as e: print("Error {} from {}".format(e, wait_iterator.current_future)) else: print("Result {} received from {} at {}".format( result, wait_iterator.current_future, wait_iterator.current_index))
Because results are returned as soon as they are available the output from the iterator will not be in the same order as the input arguments. If you need to know which future produced the current result, you can use the attributes
WaitIterator.current_future
, orWaitIterator.current_index
to get the index of the awaitable from the input list. (if keyword arguments were used in the construction of theWaitIterator
,current_index
will use the corresponding keyword).On Python 3.5,
WaitIterator
implements the async iterator protocol, so it can be used with theasync for
statement (note that in this version the entire iteration is aborted if any value raises an exception, while the previous example can continue past individual errors):async for result in gen.WaitIterator(future1, future2): print("Result {} received from {} at {}".format( result, wait_iterator.current_future, wait_iterator.current_index))
New in version 4.1.
Changed in version 4.3: Added
async for
support in Python 3.5.
- tornado.gen.multi(Union[List[Yieldable], Dict[Any, Yieldable]], quiet_exceptions: Union[Type[Exception], Tuple[Type[Exception], ...]] = ())[source]¶
Runs multiple asynchronous operations in parallel.
children
may either be a list or a dict whose values are yieldable objects.multi()
returns a new yieldable object that resolves to a parallel structure containing their results. Ifchildren
is a list, the result is a list of results in the same order; if it is a dict, the result is a dict with the same keys.That is,
results = yield multi(list_of_futures)
is equivalent to:results = [] for future in list_of_futures: results.append(yield future)
If any children raise exceptions,
multi()
will raise the first one. All others will be logged, unless they are of types contained in thequiet_exceptions
argument.In a
yield
-based coroutine, it is not normally necessary to call this function directly, since the coroutine runner will do it automatically when a list or dict is yielded. However, it is necessary inawait
-based coroutines, or to pass thequiet_exceptions
argument.This function is available under the names
multi()
andMulti()
for historical reasons.Cancelling a
Future
returned bymulti()
does not cancel its children.asyncio.gather
is similar tomulti()
, but it does cancel its children.Changed in version 4.2: If multiple yieldables fail, any exceptions after the first (which is raised) will be logged. Added the
quiet_exceptions
argument to suppress this logging for selected exception types.Changed in version 4.3: Replaced the class
Multi
and the functionmulti_future
with a unified functionmulti
. Added support for yieldables other thanYieldPoint
andFuture
.
- tornado.gen.multi_future(Union[List[Yieldable], Dict[Any, Yieldable]], quiet_exceptions: Union[Type[Exception], Tuple[Type[Exception], ...]] = ())[source]¶
Wait for multiple asynchronous futures in parallel.
Since Tornado 6.0, this function is exactly the same as
multi
.New in version 4.0.
Changed in version 4.2: If multiple
Futures
fail, any exceptions after the first (which is raised) will be logged. Added thequiet_exceptions
argument to suppress this logging for selected exception types.Deprecated since version 4.3: Use
multi
instead.
- tornado.gen.convert_yielded(yielded: Union[None, Awaitable, List[Awaitable], Dict[Any, Awaitable], Future]) Future [source]¶
Convert a yielded object into a
Future
.The default implementation accepts lists, dictionaries, and Futures. This has the side effect of starting any coroutines that did not start themselves, similar to
asyncio.ensure_future
.If the
singledispatch
library is available, this function may be extended to support additional types. For example:@convert_yielded.register(asyncio.Future) def _(asyncio_future): return tornado.platform.asyncio.to_tornado_future(asyncio_future)
New in version 4.1.
- tornado.gen.maybe_future(x: Any) Future [source]¶
Converts
x
into aFuture
.If
x
is already aFuture
, it is simply returned; otherwise it is wrapped in a newFuture
. This is suitable for use asresult = yield gen.maybe_future(f())
when you don’t know whetherf()
returns aFuture
or not.Deprecated since version 4.3: This function only handles
Futures
, not other yieldable objects. Instead ofmaybe_future
, check for the non-future result types you expect (often justNone
), andyield
anything unknown.
- tornado.gen.is_coroutine_function(func: Any) bool [source]¶
Return whether func is a coroutine function, i.e. a function wrapped with
coroutine
.New in version 4.5.
- tornado.gen.moment¶
A special object which may be yielded to allow the IOLoop to run for one iteration.
This is not needed in normal use but it can be helpful in long-running coroutines that are likely to yield Futures that are ready instantly.
Usage:
yield gen.moment
In native coroutines, the equivalent of
yield gen.moment
isawait asyncio.sleep(0)
.New in version 4.0.
Deprecated since version 4.5:
yield None
(oryield
with no argument) is now equivalent toyield gen.moment
.