Twisted Documentation Release 14.0.2 Twisted Matrix Labs Aug 12, 2017 Contents 1 Twisted Core 3 1.1 Developer Guides............................................3 1.2 Examples................................................. 289 1.3 Specifications............................................... 291 1.4 Development of Twisted......................................... 294 2 Twisted Conch 321 2.1 Developer Guides............................................ 321 2.2 Examples................................................. 329 3 Twisted Lore 331 3.1 Developer Guides............................................ 331 3.2 Examples................................................. 340 4 Twisted Mail 341 4.1 Examples................................................. 341 4.2 Twisted Mail Tutorial: Building an SMTP Client from Scratch..................... 341 5 Twisted Names 353 5.1 Developer Guides............................................ 353 5.2 Examples................................................. 363 6 Twisted Pair 365 6.1 Developer Guides............................................ 365 6.2 Examples................................................. 367 7 Twisted Web 369 7.1 Developer Guides............................................ 369 7.2 Examples................................................. 436 8 Twisted Words 439 8.1 Developer Guides............................................ 439 8.2 Examples................................................. 442 9 Historical Documents 443 9.1 2003................................................... 443 9.2 Previously................................................ 444 i ii Twisted Documentation, Release 14.0.2 Contents: Contents 1 Twisted Documentation, Release 14.0.2 2 Contents CHAPTER 1 Twisted Core Developer Guides The Vision For Twisted Many other documents in this repository are dedicated to defining what Twisted is. Here, I will attempt to explain not what Twisted is, but what it should be, once I’ve met my goals with it. First, Twisted should be fun. It began as a game, it is being used commercially in games, and it will be, I hope, an interactive and entertaining experience for the end-user. Twisted is a platform for developing internet applications. While Python by itself is a very powerful language, there are many facilities it lacks which other languages have spent great attention to adding. It can do this now; Twisted is a good (if somewhat idiosyncratic) pure-python framework or library, depending on how you treat it, and it continues to improve. As a platform, Twisted should be focused on integration. Ideally, all functionality will be accessible through all proto- cols. Failing that, all functionality should be configurable through at least one protocol, with a seamless and consistent user-interface. The next phase of development will be focusing strongly on a configuration system which will unify many disparate pieces of the current infrastructure, and allow them to be tacked together by a non-programmer. Writing Servers Overview This document explains how you can use Twisted to implement network protocol parsing and handling for TCP servers (the same code can be reused for SSL and Unix socket servers). There is a separate document covering UDP. Your protocol handling class will usually subclass twisted.internet.protocol.Protocol. Most protocol handlers inherit either from this class or from one of its convenience children. An instance of the protocol class is instantiated per- connection, on demand, and will go away when the connection is finished. This means that persistent configuration is not saved in the Protocol. 3 Twisted Documentation, Release 14.0.2 The persistent configuration is kept in a Factory class, which usually inherits from twisted.internet.protocol.Factory. The buildProtocol method of the Factory is used to create a Protocol for each new connection. It is usually useful to be able to offer the same service on multiple ports or network addresses. This is why the Factory does not listen to connections, and in fact does not know anything about the network. See the endpoints documentation for more information, or IReactorTCP.listenTCP and the other IReactor*.listen* APIs for the lower level APIs that endpoints are based on. This document will explain each step of the way. Protocols As mentioned above, this, along with auxiliary classes and functions, is where most of the code is. A Twisted protocol handles data in an asynchronous manner. The protocol responds to events as they arrive from the network and the events arrive as calls to methods on the protocol. Here is a simple example: from twisted.internet.protocol import Protocol class Echo(Protocol): def dataReceived(self, data): self.transport.write(data) This is one of the simplest protocols. It simply writes back whatever is written to it, and does not respond to all events. Here is an example of a Protocol responding to another event: from twisted.internet.protocol import Protocol class QOTD(Protocol): def connectionMade(self): self.transport.write("An apple a day keeps the doctor away\r\n") self.transport.loseConnection() This protocol responds to the initial connection with a well known quote, and then terminates the connection. The connectionMade event is usually where setup of the connection object happens, as well as any initial greetings (as in the QOTD protocol above, which is actually based on RFC 865). The connectionLost event is where tearing down of any connection-specific objects is done. Here is an example: from twisted.internet.protocol import Protocol class Echo(Protocol): def __init__(self, factory): self.factory= factory def connectionMade(self): self.factory.numProtocols= self.factory.numProtocols+1 self.transport.write( "Welcome! There are currently %d open connections.\n"% (self.factory.numProtocols,)) def connectionLost(self, reason): self.factory.numProtocols= self.factory.numProtocols-1 4 Chapter 1. Twisted Core Twisted Documentation, Release 14.0.2 def dataReceived(self, data): self.transport.write(data) Here connectionMade and connectionLost cooperate to keep a count of the active protocols in a shared object, the factory. The factory must be passed to Echo.__init__ when creating a new instance. The factory is used to share state that exists beyond the lifetime of any given connection. You will see why this object is called a “factory” in the next section. loseConnection() and abortConnection() In the code above, loseConnection is called immediately after writing to the transport. The loseConnection call will close the connection only when all the data has been written by Twisted out to the operating system, so it is safe to use in this case without worrying about transport writes being lost. If a producer is being used with the transport, loseConnection will only close the connection once the producer is unregistered. In some cases, waiting until all the data is written out is not what we want. Due to network failures, or bugs or maliciousness in the other side of the connection, data written to the transport may not be deliverable, and so even though loseConnection was called the connection will not be lost. In these cases, abortConnection can be used: it closes the connection immediately, regardless of buffered data that is still unwritten in the transport, or producers that are still registered. Note that abortConnection is only available in Twisted 11.1 and newer. Using the Protocol In this section, you will learn how to run a server which uses your Protocol. Here is code that will run the QOTD server discussed earlier: from twisted.internet.protocol import Factory from twisted.internet.endpoints import TCP4ServerEndpoint from twisted.internet import reactor class QOTDFactory(Factory): def buildProtocol(self, addr): return QOTD() # 8007 is the port you want to run under. Choose something >1024 endpoint= TCP4ServerEndpoint(reactor, 8007) endpoint.listen(QOTDFactory()) reactor.run() In this example, I create a protocol Factory. I want to tell this factory that its job is to build QOTD protocol instances, so I set its buildProtocol method to return instances of the QOTD class. Then, I want to listen on a TCP port, so I make a TCP4ServerEndpoint to identify the port that I want to bind to, and then pass the factory I just created to its listen method. endpoint.listen() tells the reactor to handle connections to the endpoint’s address using a particular protocol, but the reactor needs to be running in order for it to do anything. reactor.run() starts the reactor and then waits forever for connections to arrive on the port you’ve specified. You can stop the reactor by hitting Control-C in a terminal or calling reactor.stop(). For more information on different ways you can listen for incoming connections, see the documentation for the end- points API. For more information on using the reactor, see the reactor overview. 1.1. Developer Guides 5 Twisted Documentation, Release 14.0.2 Helper Protocols Many protocols build upon similar lower-level abstractions. For example, many popular internet protocols are line-based, containing text data terminated by line breaks (commonly CR-LF), rather than containing straight raw data. However, quite a few protocols are mixed - they have line-based sections and then raw data sections. Examples include HTTP/1.1 and the Freenet protocol. For those cases, there is the LineReceiver>‘ protocol. This protocol dispatches to two different event handlers – lineReceived and rawDataReceived. By default, only lineReceived will be called, once for each line. However,