What’s new in Tornado 5.0

Mar 5, 2018

Highlights

• The focus of this release is improving integration with asyncio. On Python 3, the IOLoop is always a wrapper around the asyncio event loop, and asyncio.Future and asyncio.Task are used instead of their Tornado counterparts. This means that libraries based on asyncio can be mixed relatively seamlessly with those using Tornado. While care has been taken to minimize the disruption from this change, code changes may be required for compatibility with Tornado 5.0, as detailed in the following section.

• Tornado 5.0 supports Python 2.7.9+ and 3.4+. Python 2.7 and 3.4 are deprecated and support for them will be removed in Tornado 6.0, which will require Python 3.5+.

Backwards-compatibility notes

• Python 3.3 is no longer supported.

• Versions of Python 2.7 that predate the ssl module update are no longer supported. (The ssl module was updated in version 2.7.9, although in some distributions the updates are present in builds with a lower version number. Tornado requires ssl.SSLContext, ssl.create_default_context, and ssl.match_hostname)

• Versions of Python 3.5 prior to 3.5.2 are no longer supported due to a change in the async iterator protocol in that version.

• The trollius project (asyncio backported to Python 2) is no longer supported.

• tornado.concurrent.Future is now an alias for asyncio.Future when running on Python 3. This results in a number of minor behavioral changes:

  • Future objects can only be created while there is a current IOLoop

  • The timing of callbacks scheduled with Future.add_done_callback has changed. tornado.concurrent.future_add_done_callback can be used to make the behavior more like older versions of Tornado (but not identical). Some of these changes are also present in the Python 2 version of tornado.concurrent.Future to minimize the difference between Python 2 and 3.

  • Cancellation is now partially supported, via asyncio.Future.cancel. A canceled Future can no longer have its result set. Applications that handle Future objects directly may want to use tornado.concurrent.future_set_result_unless_cancelled. In native coroutines, cancellation will cause an exception to be raised in the coroutine.

  • The exc_info and set_exc_info methods are no longer present. Use tornado.concurrent.future_set_exc_info to replace the latter, and raise the exception with result to replace the former.

• io_loop arguments to many Tornado functions have been removed. Use IOLoop.current() instead of passing IOLoop objects explicitly.

• On Python 3, IOLoop is always a wrapper around the asyncio event loop. IOLoop.configure is effectively removed on Python 3 (for compatibility, it may be called to redundantly specify the asyncio-backed IOLoop)

• IOLoop.instance is now a deprecated alias for IOLoop.current. Applications that need the cross-thread communication behavior facilitated by IOLoop.instance should use their own global variable instead.

Other notes

• The futures (concurrent.futures backport) package is now required on Python 2.7.

• The certifi and backports.ssl-match-hostname packages are no longer required on Python 2.7.

• Python 3.6 or higher is recommended, because it features more efficient garbage collection of asyncio.Future objects.

tornado.auth

• GoogleOAuth2Mixin now uses a newer set of URLs.

tornado.autoreload

• On Python 3, uses __main__.__spec to more reliably reconstruct the original command line and avoid modifying PYTHONPATH.

• The io_loop argument to tornado.autoreload.start has been removed.

tornado.concurrent

• tornado.concurrent.Future is now an alias for asyncio.Future when running on Python 3. See “Backwards-compatibility notes” for more.

• Setting the result of a Future no longer blocks while callbacks are being run. Instead, the callbacks are scheduled on the next IOLoop iteration.

• The deprecated alias tornado.concurrent.TracebackFuture has been removed.

• tornado.concurrent.chain_future now works with all three kinds of Futures (Tornado, asyncio, and concurrent.futures)

• The io_loop argument to tornado.concurrent.run_on_executor has been removed.

• New functions future_set_result_unless_cancelled, future_set_exc_info, and future_add_done_callback help mask the difference between asyncio.Future and Tornado’s previous Future implementation.

tornado.curl_httpclient

• Improved debug logging on Python 3.

• The time_info response attribute now includes appconnect in addition to other measurements.

• Closing a CurlAsyncHTTPClient now breaks circular references that could delay garbage collection.

• The io_loop argument to the CurlAsyncHTTPClient constructor has been removed.

tornado.gen

• tornado.gen.TimeoutError is now an alias for tornado.util.TimeoutError.

• Leak detection for Futures created by this module now attributes them to their proper caller instead of the coroutine machinery.

• Several circular references that could delay garbage collection have been broken up.

• On Python 3, asyncio.Task is used instead of the Tornado coroutine runner. This improves compatibility with some asyncio libraries and adds support for cancellation.

• The io_loop arguments to YieldFuture and with_timeout have been removed.

tornado.httpclient

• The io_loop argument to all AsyncHTTPClient constructors has been removed.

tornado.httpserver

• It is now possible for a client to reuse a connection after sending a chunked request.

• If a client sends a malformed request, the server now responds with a 400 error instead of simply closing the connection.

• Content-Length and Transfer-Encoding headers are no longer sent with 1xx or 204 responses (this was already true of 304 responses).

• When closing a connection to a HTTP/1.1 client, the Connection: close header is sent with the response.

• The io_loop argument to the HTTPServer constructor has been removed.

• If more than one X-Scheme or X-Forwarded-Proto header is present, only the last is used.

tornado.httputil

• The string representation of HTTPServerRequest objects (which are sometimes used in log messages) no longer includes the request headers.

• New function qs_to_qsl converts the result of urllib.parse.parse_qs to name-value pairs.

tornado.ioloop

• tornado.ioloop.TimeoutError is now an alias for tornado.util.TimeoutError.

• IOLoop.instance is now a deprecated alias for IOLoop.current.

• IOLoop.install and IOLoop.clear_instance are deprecated.

• The IOLoop.initialized method has been removed.

• On Python 3, the asyncio-backed IOLoop is always used and alternative IOLoop implementations cannot be configured. IOLoop.current and related methods pass through to asyncio.get_event_loop.

• run_sync cancels its argument on a timeout. This results in better stack traces (and avoids log messages about leaks) in native coroutines.

• New methods IOLoop.run_in_executor and IOLoop.set_default_executor make it easier to run functions in other threads from native coroutines (since concurrent.futures.Future does not support await).

• PollIOLoop (the default on Python 2) attempts to detect misuse of IOLoop instances across os.fork.

• The io_loop argument to PeriodicCallback has been removed.

• It is now possible to create a PeriodicCallback in one thread and start it in another without passing an explicit event loop.

• The IOLoop.set_blocking_signal_threshold and IOLoop.set_blocking_log_threshold methods are deprecated because they are not implemented for the asyncio event loop`. Use the PYTHONASYNCIODEBUG=1 environment variable instead.

• IOLoop.clear_current now works if it is called before any current loop is established.

tornado.iostream

• The io_loop argument to the IOStream constructor has been removed.

• New method BaseIOStream.read_into provides a minimal-copy alternative to BaseIOStream.read_bytes.

• BaseIOStream.write is now much more efficient for very large amounts of data.

• Fixed some cases in which IOStream.error could be inaccurate.

• Writing a memoryview can no longer result in “BufferError: Existing exports of data: object cannot be re-sized”.

tornado.locks

• As a side effect of the Future changes, waiters are always notified asynchronously with respect to Condition.notify.

tornado.netutil

• The default Resolver now uses IOLoop.run_in_executor. ExecutorResolver, BlockingResolver, and ThreadedResolver are deprecated.

• The io_loop arguments to add_accept_handler, ExecutorResolver, and ThreadedResolver have been removed.

• add_accept_handler returns a callable which can be used to remove all handlers that were added.

• OverrideResolver now accepts per-family overrides.

tornado.options

• Duplicate option names are now detected properly whether they use hyphens or underscores.

tornado.platform.asyncio

• AsyncIOLoop and AsyncIOMainLoop are now used automatically when appropriate; referencing them explicitly is no longer recommended.

• Starting an IOLoop or making it current now also sets the asyncio event loop for the current thread. Closing an IOLoop closes the corresponding asyncio event loop.

• to_tornado_future and to_asyncio_future are deprecated since they are now no-ops.

• AnyThreadEventLoopPolicy can now be used to easily allow the creation of event loops on any thread (similar to Tornado’s prior policy).

tornado.platform.caresresolver

• The io_loop argument to CaresResolver has been removed.

tornado.platform.twisted

• The io_loop arguments to TornadoReactor, TwistedResolver, and tornado.platform.twisted.install have been removed.

tornado.process

• The io_loop argument to the Subprocess constructor and Subprocess.initialize has been removed.

tornado.routing

• A default 404 response is now generated if no delegate is found for a request.

tornado.simple_httpclient

• The io_loop argument to SimpleAsyncHTTPClient has been removed.

• TLS is now configured according to ssl.create_default_context by default.

tornado.tcpclient

• The io_loop argument to the TCPClient constructor has been removed.

• TCPClient.connect has a new timeout argument.

tornado.tcpserver

• The io_loop argument to the TCPServer constructor has been removed.

• TCPServer no longer logs EBADF errors during shutdown.

tornado.testing

• The deprecated tornado.testing.get_unused_port and tornado.testing.LogTrapTestCase have been removed.

• AsyncHTTPTestCase.fetch now supports absolute URLs.

• AsyncHTTPTestCase.fetch now connects to 127.0.0.1 instead of localhost to be more robust against faulty ipv6 configurations.

tornado.util

• tornado.util.TimeoutError replaces tornado.gen.TimeoutError and tornado.ioloop.TimeoutError.

• Configurable now supports configuration at multiple levels of an inheritance hierarchy.

tornado.web

• RequestHandler.set_status no longer requires that the given status code appear in http.client.responses.

• It is no longer allowed to send a body with 1xx or 204 responses.

• Exception handling now breaks up reference cycles that could delay garbage collection.

• RedirectHandler now copies any query arguments from the request to the redirect location.

• If both If-None-Match and If-Modified-Since headers are present in a request to StaticFileHandler, the latter is now ignored.

tornado.websocket

• The C accelerator now operates on multiple bytes at a time to improve performance.

• Requests with invalid websocket headers now get a response with status code 400 instead of a closed connection.

• WebSocketHandler.write_message now raises WebSocketClosedError if the connection closes while the write is in progress.

• The io_loop argument to websocket_connect has been removed.