Txtorcon Documentation Release 1.0.0

txtorcon Documentation Release 1.0.0

meejah

January 25, 2017

Contents

1 Documentation 3 1.1 Introduction...... 3 1.1.1 Features Overview...... 3 1.1.2 Shell-cast Overview...... 4 1.1.3 Known Users...... 4 1.2 Installing txtorcon...... 4 1.2.1 Latest Release...... 4 1.2.2 Compatibility...... 5 1.2.3 Tor Configuration...... 5 1.2.4 Source Code...... 5 1.2.5 Development Environment...... 6 1.2.6 Integration Tests...... 6 1.2.7 Dependencies / Requirements...... 6 1.3 Programming Guide...... 7 1.3.1 API Stability...... 7 1.3.2 A Tor Instance...... 8 1.3.3 A Note On Style...... 8 1.3.4 Tracking and Changing Tor’s Configuration...... 8 1.3.5 Monitor and Change Tor’s State...... 9 1.3.6 Making Connections Over Tor...... 10 1.3.7 Onion (Hidden) Services...... 11 1.3.8 Custom Circuits...... 14 1.3.9 Attaching Streams to Circuits...... 15 1.4 Examples...... 15 1.4.1 Web: clients...... 15 1.4.2 Web: servers (services)...... 17 1.4.3 Starting Tor...... 21 1.4.4 Circuits and Streams...... 24 1.4.5 Configuration...... 28 1.4.6 Events...... 29 1.4.7 Miscellaneous...... 29 1.5 Contributions...... 31 1.5.1 Contact Information...... 31 1.5.2 Public Key...... 31 1.5.3 Pull Requests...... 32 1.5.4 Making a Release...... 32

2 Ofﬁcial Releases: 35

i 2.1 Releases...... 35 2.1.1 upcoming: 1.0.0...... 35 2.1.2 unreleased...... 36 2.1.3 v0.18.0...... 36 2.1.4 v0.17.0...... 36 2.1.5 v0.16.1...... 36 2.1.6 v0.16.0...... 36 2.1.7 v0.15.1...... 36 2.1.8 v0.15.0...... 37 2.1.9 v0.14.1...... 37 2.1.10 v0.14.0...... 37 2.1.11 v0.13.0...... 38 2.1.12 v0.12.0...... 38 2.1.13 v0.11.0...... 38 2.1.14 v0.10.1...... 39 2.1.15 v0.10.0...... 39 2.1.16 v0.9.2...... 39 2.1.17 v0.9.1...... 40 2.1.18 v0.8.2...... 40 2.1.19 v0.8.1...... 40 2.1.20 v0.8.0...... 41 2.1.21 v0.7...... 41 2.1.22 v0.6...... 41 2.1.23 v0.5...... 42 2.1.24 v0.4...... 42 2.1.25 v0.3...... 42 2.1.26 v0.2...... 42 2.1.27 v0.1...... 43

3 API Documentation 45 3.1 API Documentation...... 45 3.1.1 High Level API...... 45 3.1.2 Tracking and Changing Live Tor State...... 47 3.1.3 Reading and Writing Live Tor Conﬁguration...... 54 3.1.4 Endpoints and Related Classes...... 57 3.1.5 Low-Level Protocol Classes...... 61 3.1.6 txtorcon.interface Module...... 66 3.1.7 txtorcon.util Module...... 69

4 Indices and tables 71

ii txtorcon Documentation, Release 1.0.0

• docs: https://txtorcon.readthedocs.org or http://timaq4ygg2iegci7.onion • code: https://github.com/meejah/txtorcon • torsocks git clone git://timaq4ygg2iegci7.onion/txtorcon.git • If this is your ﬁrst time exploring txtorcon, please look at the Introduction ﬁrst. Supported and tested platforms: Python 2.7+, Python 3.5+, PyPy 5.0.0+ using Twisted 15.5.0+, 16.3.0+ or later.

Contents 1 txtorcon Documentation, Release 1.0.0

2 Contents CHAPTER 1

Documentation

1.1 Introduction txtorcon is an implementation of the control-spec for Tor using the Twisted networking library for Python. With txtorcon you can launch tor; connect to already-running tor instances; use tor as a client (via SOCKS5); set up services over tor; change all aspects of conﬁguration; track live state (active circuits and streams, etc); do DNS via Tor; and query other information from the tor daemon. txtorcon would be of interest to anyone wishing to write event-based software in Python that uses the Tor network as a client or a service (or just wants to display information about a locally running tor). Twisted already provides many robust protocol implementations, deployment, logging and integration with GTK, Qt and other graphics frameworks – so txtorcon can be used for command-line or GUI applications or integrate with long-lived daemons easily. In fact, due to support for endpoints (adding the tor: and onion: plugins), many Twisted applications can now integrate with Tor with no code changes. For example, you can use the existing Twisted webserver via twistd to serve your ~/public_html directory over an onion service: $ sudo apt-get install python-txtorcon $ twistd web --port "onion:80" --path ~/public_html txtorcon strives to provide sane and safe defaults.

1.1.1 Features Overview

Currently, txtorcon is capable of: • making arbitrary client connections to other services over Tor; • configuring twisted.web.client.Agent instances to do Web requests over Tor; • doing both of the above over specific circuits; • listening as an Onion service; • maintaining up-to-date (live) state information about Tor: Circuits, Streams and Routers (relays); • maintaining current (live) configuration information; • maintaining representation of Tor’s address mappings (with expiry); • interrogating initial state of all three of the above; • listening for and altering stream -> circuit mappings; • building custom circuits;

3 txtorcon Documentation, Release 1.0.0

• Circuit and Stream state listeners; • listening for any Tor EVENT; • launching and/or controlling a Tor instance (including Tor Browser Bundle); • complete Twisted endpoint support (both “onion”/server side and client-side). This means you may be able to use existing Twisted software via Tor with no code changes. It also is the preferred way to connect (or listen) in Twisted. Comments (positive or negative) appreciated. Even better if they come with patches

1.1.2 Shell-cast Overview

A text-only screencast-type overview of some of txtorcon’s features, from asciinema.org:

1.1.3 Known Users

• magic-wormhole “get things from one computer to another, safely” • Tahoe-LAFS a Free and Open encrypted distributed storage system • txtorcon received a brief mention at 29C3 starting at 12:20 (or via youtube). • carml command-line utilities for Tor • foolscap RPC system inspired by Twisted’s built-in “Perspective Broker” package. • APAF anonymous Python application framework • OONI the Open Observatory of Network Interference • exitaddr scan Tor exit addresses • txtorhttpproxy simple HTTP proxy in Twisted • bulb Web-based Tor status monitor • onionvpn “ipv6 to onion service virtual public network adapter” • torperf2 new Tor node network performance measurement service • torweb web-based Tor controller/monitor • potator “A Tor-based Decentralized Virtual Private Network Application”

1.2 Installing txtorcon

1.2.1 Latest Release txtorcon is on PyPI and in Debian since jessie (thanks to Lunar and now irl!). So, one of these should work: • install latest release: pip install txtorcon • Debian or Ubuntu: apt-get install python-txtorcon • Watch an asciinema demo for an overview.

4 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

Rendered documentation for the latest release is at txtorcon.readthedocs.org. What exists for release-notes are in “Releases”. If you’re still using wheezy, python-txtorcon is also in wheezy-backports. To install, do this as root: # echo "deb http://ftp.ca.debian.org/debian/ wheezy-backports main" >> /etc/apt/sources.list # apt-get update # apt-get install python-txtorcon

It also appears txtorcon is in Gentoo but I don’t use Gentoo (if anyone has a shell-snippet that installs it, send a pull-request). I am told this package also needs a maintainer; see XXX. Installing the wheel ﬁles requires a recent pip and setuptools. At least on Debian, it is important to upgrade setuptools before pip. This procedure appears to work ﬁne: virtualenv foo . foo/bin/activate pip install --upgrade setuptools pip install --upgrade pip pip install path/to/txtorcon-*.whl

1.2.2 Compatibility txtorcon runs all tests cleanly under Python2, Python3 and PyPy on: • Debian: “squeeze”, “wheezy” and “jessie” • OS X: 10.4 (naif), 10.8 (lukas lueg), 10.9 (kurt neufeld) • Fedora 18 (lukas lueg) • FreeBSD 10 (enrique fynn) (needed to install “lsof”) • RHEL6 • Reports from other OSes appreciated.

1.2.3 Tor Conﬁguration

Using Tor’s cookie authentication is the most convenient way to connect; this proves that your user can read a cookie ﬁle written by Tor. To enable this, you’ll want to have the following options on in your torrc: CookieAuthentication 1 CookieAuthFileGroupReadable 1

Note that “Tor BrowserBundle” is conﬁgured this way by default, on port 9151. If you want to use unix sockets to speak to tor (highly recommended) add this to your conﬁg (Debian is already set up like this): ControlSocketsGroupWritable 1 ControlSocket /var/run/tor/control

1.2.4 Source Code

Most people will use the code from https://github.com/meejah/txtorcon The canonical URI is http://timaq4ygg2iegci7.onion I sign tags with my public key (meejah.asc) • git clone https://github.com/meejah/txtorcon.git • torsocks git clone git://timaq4ygg2iegci7.onion/meejah/txtorcon.git

1.2. Installing txtorcon 5 txtorcon Documentation, Release 1.0.0

Rendered documentation for the latest release is at txtorcon.readthedocs.org. See Contributions if you wish to contribute back to txtorcon :)

1.2.5 Development Environment

I like to set up my Python development like this: $ git clone https://github.com/meejah/txtorcon.git $ echo "if you later clone it on github, do this:" $ git remote add -f github git+ssh://[email protected]//txtorcon.git $ cd txtorcon $ virtualenv venv $ source venv/bin/activate (venv)$ pip install --editable .[dev] # "dev" adds more deps, like Sphinx (venv)$ make doc (venv)$ make test (venv)$ tox # run all tests, in all supported configs

You can now edit code in the repository as normal. To submit a patch, the easiest way is to “clone” the txtorcon project, then “fork” on github and add a remote called “github” with your copy of the code to which you can push (git remote add -f github git+ssh://[email protected]//txtorcon.git). The -f is so you don’t have to run git fetch right after. Now, you can push a new branch you’ve made to GitHub with git push github branch-name and then examine it and open a pull-request. This will trigger Travis to run the tests, after which coverage will be produced (and a bot comments on the pull-request). If you require any more changes, the easiest thing to do is just commit them and push them. (If you know how, re-basing/re-arranging/squashing etc is nice to do too). See Contributions for more.

1.2.6 Integration Tests

There are a couple of simple integration tests using Docker in the integration/ directory; these make a debootstrap-built base image and then do the test inside containers cloned from this – no trusting https://docker.io required. See integration/README for more information. If you’re on Debian, there’s a decent chance running make txtorcon-tester followed by make integration from the root of the checkout will work (the ﬁrst commands ultimately runs debootstrap and some apt commands besides docker things).

1.2.7 Dependencies / Requirements

These should have been installed by whichever method you chose above, but are listed here for completeness. You can get all the development requirements with e.g. pip install txtorcon[dev]. • twisted: txtorcon should work with any Twisted 11.1.0 or newer. Twisted 15.4.0+ works with Python3, and so does txtorcon (if you ﬁnd something broken on Py3 please ﬁle a bug). • ipaddress: a standard module in Python3, but requires installing the backported package on Python2. • dev only: Sphinx if you want to build the documentation. In that case you’ll also need something called python-repoze.sphinx.autointerface (at least in Debian) to build the Interface-derived docs properly. • dev only: coverage to run the code-coverage metrics. • dev only: Tox to run different library revisions.

6 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

• dev optional: GraphViz is used in the tests (and to generate state-machine diagrams, if you like) but those tests are skipped if “dot” isn’t in your path In any case, on a Debian wheezy, squeeze or Ubuntu system, this should work (as root): # apt-get install -y python-setuptools python-twisted python-ipaddress graphviz tor # echo "for development:" # apt-get install -y python-sphinx python-repoze.sphinx.autointerface python-coverage libgeoip-dev

Using pip this would be: $ pip install --user Twisted ipaddress pygeoip $ echo "for development:" $ pip install --user GeoIP Sphinx repoze.sphinx.autointerface coverage

or: $ pip install -r requirements.txt $ pip install -r dev-requirements.txt

1.3 Programming Guide

• API Stability • A Tor Instance – Connecting to a Running Tor – Launching a New Tor • A Note On Style • Tracking and Changing Tor’s Conﬁguration • Monitor and Change Tor’s State • Making Connections Over Tor – SOCKS5 • Onion (Hidden) Services – Onion Services Endpoints API – Creating Onion Endpoints – Non-Authenticated Services – Authenticated Services – Onion Service Conﬁguration • Custom Circuits – Building a Single Circuit – Building Many Circuits • Attaching Streams to Circuits

1.3.1 API Stability

In general, any method or class preﬁxed with an underscore (like _method or _ClassName) is private, and the API may change at any time. You SHOULD NOT use these. Any method in an interface class (which all begin with I, like IAnInterface) are stable, public APIs and will maintain backwards-compatibility between releases. There is one exception to this at the moment: the hidden- / onion- services APIs are NOT yet considered stable, and may still change somewhat. Any APIs that will go away will ﬁrst be deprecated for at least one major release before being removed.

1.3. Programming Guide 7 txtorcon Documentation, Release 1.0.0

1.3.2 A Tor Instance

You will need a connection to a Tor instance for txtorcon to control. This can be either an already-running tor that you’re authorized to connect to, or a tor instance that has been freshly launched by txtorcon. We abstract “a tor instance” behind the txtorcon.Tor class, which provides a very high-level API for all the other things you might want to do with that Tor: • make client-type connections over tor (see “Making Connections Over Tor”); • change its conﬁguration (see “Tracking and Changing Tor’s Conﬁguration”); • monitor its state (see “Monitor and Change Tor’s State”); • offer hidden-/onion- services via tor (see ”:ref:‘‘”); • issue low-level commands (see “Low-Level Protocol Classes”) The actual control-protocol connection to tor is abstracted behind txtorcon.TorControlProtocol. This can usually be ignored by most users, but can be useful to issue protocol commands directly, listen to raw events, etc.

Connecting to a Running Tor

Tor can listen for control connections on TCP ports, or UNIX sockets. See “Tor Configuration” for information on how to configure tor to work with txtorcon. By default, “COOKIE” authentication is used; only if that is not available do we try password authentication. To connect, use txtorcon.connect() which returns a Deferred that will fire with a txtorcon.Tor instance. If you need access to the txtorcon.TorControlProtocol instance, it’s available via the .protocol property (there is always exactly one of these per txtorcon.Tor instance).

Launching a New Tor

It’s also possible to launch your own Tor instance. txtorcon keeps a “global” tor available for use by e.g. the .global_tor endpoint factory functions (like txtorcon.TCPHiddenServiceEndpoint.global_tor()). You can access it via txtorcon.get_global_tor(). There is exactly zero or one of these per Python process that uses txtorcon. To explicitly launch your own Tor instance, use txtorcon.launch(). You can pass a couple of minimal options (data_directory being recommended). If you need to set other Tor options, use .config to retrieve the txtorcon.TorConfig instance associated with this tor and change conﬁguration afterwards.

1.3.3 A Note On Style

Most of txtorcon tends towards “attribute-style access”. The guiding principle is that “mere data” that is immediately available will be an attribute, whereas things that “take work” or are async (and thus return Deferred s) will be functions. For example, txtorcon.Router.get_location() is a method because it potentially has to ask Tor for the country, whereas txtorcon.Router.hex_id is a plain attribute because it’s always available.

1.3.4 Tracking and Changing Tor’s Conﬁguration

Instances of the txtorcon.TorConfig class represent the current, live state of a running Tor. There is a bit of attribute-magic to make it possible to simply get and set things easily:

8 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

tor= launch(..) print("SOCKS ports: {}".format(tor.config.SOCKSPort)) tor.config.ControlPort.append(4321) tor.config.save()

Only when .save() is called are any SETCONF commands issued – and then, all changed configuration values are sent in a single command. All TorConfig instances subscribe to configuration updates from Tor, so “live state” includes actions by any other controllers that may be connected. For some configuration items, the order they’re sent to Tor matters. Sometimes, if you change one config item, you have to set a series of related items. TorConfig handles these cases for you – you just manipulate the configuration, and wait for .save()‘s Deferred to fire and the running tor’s configuration is updated. Note there is a tiny window during which the state may appear slightly inconsistent if you have multiple TorConfig instances: after Tor has acknowledged a SETCONF command, but before a separate TorConfig instance has gotten all the CONF_CHANGED events (because they’re hung up in the networking stack for some reason). This shouldn’t concern most users. (I’m not even 100% sure this is possible; it may be that Tor doesn’t send the OK until after all the CONF_CHANGED events) Since txtorcon.TorConfig conforms to the Iterator protocol, you can easily find all the config-options that Tor supports: tor= launch(..) for config_key in tor.config: print("{} has value: {}".format(config_key, getattr(tor.config.config_key)))

These come from interrogating tor using GETINFO config/names and so represent the conﬁguration options of the current connected Tor process. If the value “isn’t set” (i.e. is the default), the value from Tor will be txtorcon.DEFAULT_VALUE. When you set values into TorConfig, they are parsed according to control-spec for the different types given to the values, via information from GETINFO config/names. So, for example, setting .SOCKSPort to a "quux" won’t work. Of course, it would also fail the whole SETCONF command if txtorcon happens to allow some values that Tor doesn’t. Unfortunately, for any item that’s a list, Tor doesn’t tell us anything about each element so they’re all strings.

1.3.5 Monitor and Change Tor’s State

Instances of txtorcon.TorState prepresent a live, interactive version of all the relays/routers (txtorcon.Router instances), all circuits (txtorcon.Circuit instances) and streams (txtorcon.Stream instances) active in the underlying Tor instance. As the TorState instance has subscribed to various events from Tor, the “live” state represents an “as up-to-date as possible” view.

Note: If you need to be absolutely sure there’s nothing stuck in networking buffers, you can issue a do-nothing command to Tor via txtorcon.TorControlProtocol.queue_command() (e.g. yield queue_command("GETINFO version")). Most users shouldn’t have to worry about this edge-case.

You can modify the state of these things in a few simple ways. For example, you can call txtorcon.Stream.close() or txtorcon.Circuit.close() to cause a stream or circuit to be closed. You can wait for a circuit to become usable with txtorcon.Circuit.when_built(). For a lot of the read-only state, you can simply access interesting attributes. The relays through which a circuit traverses are in Circuit.path (a list of txtorcon.Router instances), Circuit.streams contains a list

1.3. Programming Guide 9 txtorcon Documentation, Release 1.0.0

of txtorcon.Stream instances, .state and .purpose are strings. .time_created returns a datetime instance. There are also some convenience functions like txtorcon.Circuit.age(). For sending streams over a particular circuit, txtorcon.Circuit.stream_to() returns an IStreamClientEnd- point implementation that will cause a subsequent .connect() on it to go via the given circuit in Tor. Combined with a txtorcon.CircuitBuilder this gives the power to do many things. Listening for certain events to happen can be done by implementing the interfaces txtorcon.interface.IStreamListener and txtorcon.interface.ICircuitListener. You can request notiﬁcations on a tor-wide basis with txtorcon.TorState.add_circuit_listener() or txtorcon.TorState.add_stream_listener(). If you are just interested in a single circuit, you can call txtorcon.Circuit.listen() directly on a Circuit instance. (XXX think about the composible-style API; e.g. circuit.on(’extend’, call_back) and/or state.on(’circuit_extend’, call_back)) The Tor relays are abstracted with txtorcon.Router instances. Again, these have read-only attributes for interesting information, e.g.: id_hex, ip, flags (a list of strings), bandwidth, policy, etc. Note that all information in these objects is from “microdescriptors”. If you’re doing a long-running iteration over relays, it may be important to remember that the collection of routers can change every hour (when a new “consensus” from the Directory Authori- ties is published) which may change the underlying collection (e.g. txtorcon.TorState.routers_by_hash) over which you’re iterating. Here’s a simple sketch that traverses all circuits printing their router IDs, and closing each stream and circuit afterwards: (XXX FIXME test this for realz; can we put it in a “listing”-type ﬁle?) @inlineCallbacks def main(reactor): tor= yield connect(reactor, UNIXClientEndpoint('/var/run/tor/control')) state= yield tor.get_state() for circuit in state.circuits.values(): path='->'.join(map(lambdar:r.id_hex, circuit.streams)) print("Circuit {} through {}".format(circuit.id, path)) for stream in circuit.streams: print(" Stream {} to {}".format(stream.id, stream.target_host)) yield stream.close() yield circuit.close()

1.3.6 Making Connections Over Tor

SOCKS5

Tor exposes a SOCKS5 interface to make client-type connections over the network. There are also a couple of custom extensions tor provides to do DNS resolution over a Tor circuit (txtorcon supports these, too). All client-side interactions are via instances that implement IStreamClientEndpoint. There are several factory functions used to create suitable instances. The recommended API is to acquire a txtorcon.Tor instance (see “A Tor Instance”) and then call txtorcon.Tor.create_client_endpoint(). To do DNS lookups (or reverse lookups) via a Tor circuit, use txtorcon.Tor.dns_resolve() and txtorcon.Tor.dns_resolve_ptr(). A common use-case is to download a Web resource; you can do so via Twisted’s built-in twisted.web.client package, or using the friendlier treq library. In both cases, you need a twisted.web.client.Agent instance which you can acquire with txtorcon.Tor.web_agent() or txtorcon.Circuit.web_agent(). The latter is used to

10 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

make the request over a speciﬁc circuit. Usually, txtorcon will simply use one of the available SOCKS ports conﬁgured in the Tor it is connected to – if you care which one, you can specify it as the optional socks_endpoint= argument.

Note: Tor supports SOCKS over Unix sockets. So does txtorcon. To take advantage of this, simply pass a valid SocksPort value for unix sockets (e.g. unix:/tmp/foo/socks) as the socks_config argument to either web_agent() call. If this doesn’t already exist in the underlying Tor, it will be added. Tor has particular requirements for the directory in which the socket ﬁle is (0700).

If you need a stream to go over a speciﬁc circuit, see “Building Many Circuits”. (notes to self): • CircuitBuilder (for the the open ticket making a higher-level Attacher) - a factory/builder that creates Circuit instances • Circuit.create_client_endpoint() ? (i.e. makes an endpoint whose streams all go over this circuit) - hence can use via TorState or via CircuitBuilder You can also use Twisted’s clientFromString API as txtorcon registers a tor: plugin. This also implies that any Twisted-using program that supports conﬁguring endpoint strings gets Tor support “for free”. For example, passing a string like tor:timaq4ygg2iegci7.onion:80 to clientFromString will return an endpoint that will connect to txtorcon’s hidden-service website. Note that these endpoints will use the “global to txtorcon” tor instance (available from txtorcon.get_global_tor()). Thus, if you want to control which tor instance your circuit goes over, this is not a suitable API. There are also lower-level APIs to create txtorcon.TorClientEndpoint instances directly if you have a txtorcon.TorConfig instance. These very APIs are used by the Tor object mentioned above. If you have a use-case that requires using this API, I’d be curious to learn why the txtorcon.Tor methods are un-suitable (as those are the suggested API).

1.3.7 Onion (Hidden) Services

An “Onion Service” (also called a “Hidden Service”) refers to a feature of Tor allowing servers (e.g. a Web site) to get additional security properties such as: hiding their network location; providing end-to-end encryption; self-certifying domain-names; or offering authentication. For details of how this works, please read Tor’s documentation on Hidden Services. For more background, the RiseUp Onion service best-practices guide is a good read as well. From an API perspective, here are the parts we care about: • each service has a secret, private key (with a corresponding public part): – these keys can be on disk (in the “hidden service directory”); – or, they can be “ephemeral” (only in memory); • the “host name” is a hash of the public-key (e.g. timaq4ygg2iegci7.onion); • a “Descriptor” (which tells clients how to connect) must be published; • a service has a list of port-mappings (public -> local) – e.g. "80 127.0.0.1:5432" says you can contact the service publically on port 80, which Tor will redirect to a daemon running locally on port 5432; – note that “Descriptors” don’t show this information • services can be “authenticated”, which means they have a list of client names for which Tor creates associated keys (.auth_token).

1.3. Programming Guide 11 txtorcon Documentation, Release 1.0.0

• Tor has two ﬂavours of service authentication: basic and stealth – there’s no API-level difference, but the .hostname is unique for each client in the stealth case. • See Creating Onion Endpoints for details on how to choose which (if any) authentication method you’d like To summarize the above in a table format, here are the possible types of Onion Service interfaces classes you may interact with (ephemeral services don’t yet support any authentication). Keys on disk Keys in memory no authentication IFilesystemOnionService IOnionService basic/stealth authentication IOnionClients Note that it’s up to you to save the private keys of ephemeral services if you want to re-launch them later; the “ephemeral” refers to the fact that Tor doesn’t persist the private keys – when Tor shuts down, they’re gone and there will never be a service at the same URI again.

Onion Services Endpoints API

No matter which kind of service you need, you interact via Twisted’s IStreamServerEndpoint interface. There are various txtorcon methods (see “Creating Onion Endpoints”) which return some instance implementing that interface. These instances will also implement txtorcon.IProgressProvider – which is a hook to register listeners which get updates about Tor’s launching progress (if we started a new Tor) and Descriptor uploading. Fundamentally, “authenticated” services are different from non-authenticated services because they have a list of clients. Therefore, there are two different endpoint types: • txtorcon.TCPHiddenServiceEndpoint • txtorcon.TCPAuthenticatedHiddenServiceEndpoint In either case, the listen method will return an instance implementing IListeningPort. In addition to IListeningPort, these instances will implement one of: • txtorcon.IOnionService or; • txtorcon.IOnionClients The ﬁrst one corresponds to a non-authenticated service, while the latter is authenticated. The latter manages a collection of instances by (arbitrary) client names, where each of these instances implements txtorcon.IOnionClient (and therefore also txtorcon.IOnionService). Note that the .auth_token member is secret, private data which you need to give to one client; this information goes in the client’s Tor conﬁguration as HidServAuth onion-address auth-cookie [service-name]. See the Tor manual for more information. Also note that Tor’s API for adding “ephemeral” services doesn’t yet support any type of authentication (however, it may in the future).

Creating Onion Endpoints

The easiest to use API are methods of txtorcon.Tor, which allow you to create IStreamServerEndpoint instances for the various Onion Service types. Both the main endpoint types have several factory-methods to return instances – so you ﬁrst must decide whether to use an “authenticated” service or not. • if you want anyone with e.g. the URL http://timaq4ygg2iegci7.onion to be able to put it in Tor Browser Bundle and see a Web site, you do not want authentication; • if you want only people with the URL and a secret authentication token to see the Web site, you want basic authentication (these support many more clients than stealth auth);

12 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

• if you don’t even want anyone to be able to decrypt the descriptor without a unique URL and a secret authentication token, you want stealth authentication (a lot less scalable; for only “a few” clients).

Non-Authenticated Services

For non-authenticated services, you want to create a txtorcon.TCPHiddenServiceEndpoint instance. You can do this via the txtorcon.create_onion_service() factory function or with txtorcon.Tor.create_onion_service(). It’s also possible to use Twisted’s serverFromString API with the onion: prefix. (Thus, any program supporting endpoint strings for configuration can use Tor Onion Services with no code changes). If you don’t want to manage launching or connecting to Tor yourself, you can use one of the three @classmethods on the class, which all return a new endpoint instance: • txtorcon.TCPHiddenSeviceEndpoint.global_tor(): uses a Tor instance launched at most once in this Python process (the underlying txtorcon.Tor instance for this is available via txtorcon.get_global_tor() if you need to make manual configuration adjustments); • txtorcon.TCPHiddenSeviceEndpoint.system_tor(): connects to the control-protocol endpoint you provide (a good choice on Debian would be UNIXClientEndpoint(’/var/run/tor/control’)); • txtorcon.TCPHiddenSeviceEndpoint.private_tor(): causes a fresh, private instance of Tor to be launched for this service alone. This uses a tempdir (honoring $TMP) which is deleted upon reactor shutdown or loss of the control connection. Note that nothing actually “happens” until you call .listen() on the IStreamServerEndpoint at which point Tor will possibly be launched, the Onion Service created, and the descriptor published.

Authenticated Services

To use authenticated services, you want to create a txtorcon.TCPAuthenticatedHiddenServiceEndpoint instance. This provides the very same factory methods as for non-authenticatd instances, but adds arguments for a list of clients (strings) and an authentication method ("basic" or "stealth"). For completeness, the methods to create authenticated endpoints are: • txtorcon.Tor.create_authenticated_onion_service(); • txtorcon.create_authenticated_onion_service(); • txtorcon.TCPAuthenticatedHiddenSeviceEndpoint.global_tor() • txtorcon.TCPAuthenticatedHiddenSeviceEndpoint.system_tor() • txtorcon.TCPAuthenticatedHiddenSeviceEndpoint.private_tor()

Onion Service Conﬁguration

If you just want to “look at” the conﬁguration of existing onion services, they are avaialble via txtorcon.TorConfig and the .HiddenServices attribute. This presents a “ﬂattened” version of any authenticated services, so that each element in the list of .HiddenServices is itself at least a txtorcon.IOnionService (it may also implement other interfaces, but every one will implement IOnionService).

1.3. Programming Guide 13 txtorcon Documentation, Release 1.0.0

You can still set any settable attributes on these objects, and Tor’s conﬁguration for them will be updated when you call txtorcon.TorConfig.save() with an important exception: “ephemeral” services cannot be updated after they’re created. Note that it’s possible for other controllers to create ephemeral services that your controller can’t enumerate.

1.3.8 Custom Circuits txtorcon provides a low-level interface over top of Tor’s circuit-attachment API, which allows you to specify which circuit any new streams use. Often, though, you also want to create custom circuits for streams – and so we also provide a more convenient higher-level API (see “Building Many Circuits”). For one-shot connections, use txtorcon.Circuit.create_client_endpoint() to acquire an IStreamClientEndpoint instance. Calling connect() on this endpoint instance causes the resulting stream to go via the particular txtorcon.Circuit instance. (If the circuit has closed by the time you call connect(), the connection will fail). See web_client_custom_circuit.py. Note that Tor doesn’t currently allow controllers to attach circuits destined for hidden-services (even over an otherwise suitable circuit).

Building a Single Circuit

If your use-case needs just a single circuit, it is probably easiest to call txtorcon.TorState.build_circuit(). This methods takes a list of txtorcon.Router instances, which you can get from the txtorcon.TorState instance by using one of the attributes: • .all_routers • .routers • .routers_by_name or • .routers_by_hash The last three are all hash-tables. For relays that have the Guard ﬂag, you can access the hash-tables .guards (for all of them) or .entry_guards (for just the entry guards conﬁgured on this Tor client). If you don’t actually care which relays are used, but simply want a fresh circuit, you can call txtorcon.TorState.build_circuit() without any arguments (or, set routers=None).

Building Many Circuits

If you would like to build many circuits, you’ll want an instance that implements txtorcon.ICircuitBuilder (which is usually simply an instance of txtorcon.CircuitBuilder). Instances of this class can be created by calling one of the factory functions like txtorcon.circuit_builder_fixed_exit(). XXX what about a “conﬁg object” idea, e.g. could have keys: • guard_selection: one of entry_only (use one of the current entry guards) or random_guard (use any relay with the Guard ﬂag, selected by XXX). • middle_selection: one of uniform (selected randomly from all relays), weighted (selected randomly, but weighted by consensus weight – basically same way as Tor would select).

14 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

1.3.9 Attaching Streams to Circuits

Tor allows the controller to decide how to attach new streams to circuits. This doesn’t work for hidden- service bound streams. The lower-level API is to implement an txtorcon.IStreamAttacher and call txtorcon.TorState.set_stream_attacher() on your TorState instance. Often, however, making low-level per-stream decisions isn’t what you want – you just want to create a stream that goes over a particular circuit. For this use-case, you use txtorcon.Circuit().

1.4 Examples

The examples are grouped by functionality and serve as mini-HOWTOs – if you have a use-case that is missing, it may be useful to add an example, so please ﬁle a bug. All ﬁles are in the examples/ sub-directory and are ready to run, usually with defaults designed to work with Tor Browser Bundle (localhost:9151). XXX maybe honour TOR_CONTROL env-var for just examples?

• Web: clients – web_client.py – web_client_custom_circuit.py • Web: servers (services) – web_onion_service.py – web_onion_service_endpoints.py – web_onion_service_klein.py • Starting Tor – launch_tor.py – launch_tor_endpoint.py • Circuits and Streams – disallow_streams_by_port.py – stream_circuit_logger.py – attach_streams_by_country.py – schedule_bandwidth.py • Conﬁguration – dump_config.py • Events – monitor.py • Miscellaneous – stem_relay_descriptor.py – circuit_failure_rates.py – txtorcon.tac

1.4.1 Web: clients web_client.py

Download the example. Uses twisted.web.client to download a Web page using a twisted.web.client.Agent, via any circuit Tor chooses.

1.4. Examples 15 txtorcon Documentation, Release 1.0.0

# this example shows how to use Twisted's web client with Tor via # txtorcon from __future__ import print_function from twisted.internet.defer import inlineCallbacks from twisted.internet.task import react from twisted.internet.endpoints import TCP4ClientEndpoint from twisted.web.client import readBody import txtorcon from txtorcon.util import default_control_port

@inlineCallbacks def main(reactor): # use port 9051 for system tor instances, or: # ep = UNIXClientEndpoint(reactor, '/var/run/tor/control') # ep = UNIXClientEndpoint(reactor, '/var/run/tor/control') ep= TCP4ClientEndpoint(reactor,'127.0.0.1', default_control_port()) tor= yield txtorcon.connect(reactor, ep) print("Connected to {tor} via localhost:{port}".format( tor=tor, port=default_control_port(), ))

# create a web.Agent that will talk via Tor. If the socks port # given isn't yet configured, this will do so. It may also be # None, which means "the first configured SOCKSPort" # agent = tor.web_agent(u'9999') agent= tor.web_agent() uri=b'http://surely-this-has-not-been-registered-and-is-invalid.com' uri=b'https://www.torproject.org' uri=b'http://timaq4ygg2iegci7.onion/' # txtorcon documentation print("Downloading {}".format(uri)) resp= yield agent.request(b'GET', uri)

print("Response has {} bytes".format(resp.length)) body= yield readBody(resp) print("received body ({} bytes)".format(len(body))) print("{}\n[...]\n{}\n".format(body[:200], body[-200:])) if __name__ =='__main__': react(main) web_client_custom_circuit.py

Download the example. Builds a custom circuit, and then uses twisted.web.client to download a Web page using the circuit created. # this example shows how to use Twisted's web client with Tor via # txtorcon from __future__ import print_function

16 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

from twisted.internet.defer import inlineCallbacks from twisted.internet.task import react from twisted.internet.endpoints import TCP4ClientEndpoint from twisted.web.client import readBody import txtorcon from txtorcon.util import default_control_port

@inlineCallbacks def main(reactor): # use port 9051 for system tor instances, or: # ep = UNIXClientEndpoint(reactor, '/var/run/tor/control') ep= TCP4ClientEndpoint(reactor,'127.0.0.1', default_control_port()) tor= yield txtorcon.connect(reactor, ep) print("Connected:", tor)

state= yield tor.create_state() socks= tor.config.socks_endpoint(reactor)

# create a custom circuit; in this case we're just letting Tor # decide the path -- but this could be done several other ways # e.g. with txtorcon.CircuitBuilder circ= yield state.build_circuit() print("Building a circuit:", circ)

# at this point, the circuit will be "under way" but may not yet # be in BUILT state -- and hence usable. So, we wait. yield circ.when_built() print("Circuit is ready:", circ)

# create a web.Agent that will use this circuit (or fail) agent= circ.web_agent(reactor, socks)

uri='https://www.torproject.org' print("Downloading {}".format(uri)) resp= yield agent.request('GET', uri)

print("Response has {} bytes".format(resp.length)) body= yield readBody(resp) print("received body ({} bytes)".format(len(body))) print("{}\n[...]\n{}\n".format(body[:200], body[-200:])) react(main)

1.4.2 Web: servers (services) web_onion_service.py

Download the example. Set up a twisted.web.server listening as a onion service. This uses the ADD_ONION API from Tor. If you don’t know what that means, see the spec. If you know you want to keep the private key for your onion service on disk somewhere, see the next example.

1.4. Examples 17 txtorcon Documentation, Release 1.0.0

#!/usr/bin/env python

# This shows how to leverage the endpoints API to get a new hidden # service up and running quickly. You can pass along this API to your # users by accepting endpoint strings as per Twisted recommendations. # # http://twistedmatrix.com/documents/current/core/howto/endpoints.html#maximizing-the-return-on-your-endpoint-investment # # note that only the progress-updates needs the "import txtorcon" -- # you do still need it installed so that Twisted finds the endpoint # parser plugin but code without knowledge of txtorcon can still # launch a Tor instance using it. cool! from __future__ import print_function from twisted.internet import defer, task, endpoints from twisted.web import server, static, resource import txtorcon from txtorcon.util import default_control_port

@defer.inlineCallbacks def main(reactor): tor= yield txtorcon.connect( reactor, endpoints.TCP4ClientEndpoint(reactor,'127.0.0.1', default_control_port()), # endpoints.UNIXClientEndpoint(reactor, "/var/run/tor/control"), ) ep= tor.create_onion_endpoint(80) res= resource.Resource() res.putChild( b'', static.Data("Hello, onion-service world!",'text/html'), )

def on_progress(percent, tag, msg): print('%03d: %s'%(percent, msg)) txtorcon.IProgressProvider(ep).add_progress_listener(on_progress)

port= yield ep.listen(server.Site(res)) print("Site listening: {}".format(port.getHost())) print("Private key:\n{}".format(port.getHost().onion_key)) yield defer.Deferred() # wait forever if __name__ =='__main__': task.react(main) web_onion_service_endpoints.py

Download the example. This uses a server endpoint string via the serverFromString API in Twisted to do “whatever it takes” to set up a new Onion (location-hidden) service. If a Twisted application lets you conﬁgure server endpoint strings to listen on, you may get hidden-service support without having to change any code. If, instead, you’re writing Python code and wish to have more control over the endpoint used (and e.g. whether a new

18 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

Tor instance is launched or not) use one of the following examples. #!/usr/bin/env python

# This shows how to leverage the endpoints API to get a new hidden # service up and running quickly. You can pass along this API to your # users by accepting endpoint strings as per Twisted recommendations. # # http://twistedmatrix.com/documents/current/core/howto/endpoints.html#maximizing-the-return-on-your-endpoint-investment # # note that only the progress-updates needs the "import txtorcon" -- # you do still need it installed so that Twisted finds the endpoint # parser plugin but code without knowledge of txtorcon can still # launch a Tor instance using it. cool! from __future__ import print_function from twisted.internet import defer, task, endpoints from twisted.web import server, resource import txtorcon from txtorcon.util import default_control_port class Simple(resource.Resource): """ A really simple Web site. """ isLeaf= True

def render_GET(self, request): return"Hello, world! I'm a hidden service!"

@defer.inlineCallbacks def main(reactor): # "onion:" is for Tor Onion Services, and the only required # argument is the public port we advertise. You can pass # "controlPort=9051" for example, to connect to a system Tor # (accepts paths, too, e.g. "controlPort=/var/run/tor/control") # ep = endpoints.serverFromString(reactor, "onion:80:controlPort=9151")

# to re-create a previous hidden-service, pass the private key # blob you retrieved earler via port.getHost().onion_key like so: # ep = endpoints.serverFromString(reactor, "onion:80:privateKey=") # Beware that you have to escape the ":" after RSA1024 because of # reasons (the endpoint syntax uses them). For this reason, you # can omit the "RSA1024:" part if you wish. # ep = endpoints.serverFromString(reactor, "onion:80")

# this one should be "nyfnesplt3f5nvn3.onion"

ep= endpoints.serverFromString(reactor,"onion:80:controlPort={port}:privateKey=RSA1024\\:MIICWwIBAAKBgQDml0L1Btxe1QIs88mvKvcgAEd19bUorzMndfXXBbPt2y1lTjm+vGldJRCXb/RArfCb9F2q7IWL4ScuJBiUCqpKVG2aGK8yOxw4c5WKvnLW8MRf5+jAPlR3h7idBdrVGCY/9gXf9JzWfpIhMfFidM4Xq6VpzMvignss6FB6i9zhOwIDAQABAoGAXuWjVamUKabp9UwDFYbOGypiPmZ3Pp4TpErEeNBNAzdvUEDIPPnXNtEZKemWEMREwDnqDny2XSG0+SU7xDk7aQGTFxipo+NAl18QMW2XcBjWrIG5P0L9E+j58k5Nq6EEaMQ8G8X3hsnX7EwRqnJYOwUWUQ4emi6TvNScSMS251kCQQD6KJXltkSfwU3d5hOh37x3pOp4ZcpI6eKwwfgqP+1pVfOwjvXfLqLgLRf9+NtmG+cU5HRDwmf9rbJNCOE++11HAkEA6/mz/L+54NRk9tPN4vYfn969v7fz9CQndQUsTTrtArqtjg7baKts3ndagj+/itJfY6qV/OonN9XdntQXTWWGbQJAY244TmrJEfqieZ2WlhO49JFPRPWolpyoJvuiKSDpu6GXT8ky/zepM5OY4rDEe+yBR/OaJsihztn4cdgit4bvxwJAFsnZiOoXEFBSo8eWlXmBWlYPawlfxM8NBG8IdTjglKfkhNiIddZAQEe0dOmlHMnuLljV/UO7n9fGfEUtLutEDQJAC3c9gSe2of41TaZhQ+aHzQ8E9cs7fg3gXXUgWlocQK6fYq+tC0CF7dDmydShF8vI8oEcgJGhgtUAXQDwH9eD0A==".format(port=default_control_port()))

def on_progress(percent, tag, msg): print('%03d: %s'%(percent, msg)) txtorcon.IProgressProvider(ep).add_progress_listener(on_progress) print("Note: descriptor upload can take several minutes")

port= yield ep.listen(server.Site(Simple()))

1.4. Examples 19 txtorcon Documentation, Release 1.0.0

print("Site listening: {}".format(port.getHost())) print("Private key:\n{}".format(port.getHost().onion_key)) yield defer.Deferred() # wait forever task.react(main) web_onion_service_klein.py

Download the example. Set up a ‘twisted.web.server <>‘_ listening as a onion service. This uses the ADD_ONION API from Tor. If you don’t know what that means, see the spec. If you know you want to keep the private key for your onion service on disk somewhere, see the next example. #!/usr/bin/env python

# This shows how to leverage the endpoints API to get a new hidden # service up and running quickly. You can pass along this API to your # users by accepting endpoint strings as per Twisted recommendations. # # http://twistedmatrix.com/documents/current/core/howto/endpoints.html#maximizing-the-return-on-your-endpoint-investment # # note that only the progress-updates needs the "import txtorcon" -- # you do still need it installed so that Twisted finds the endpoint # parser plugin but code without knowledge of txtorcon can still # launch a Tor instance using it. cool! from __future__ import print_function import sys from twisted.internet import defer, task, endpoints from twisted.web import server import txtorcon from txtorcon.util import default_control_port try: from klein import Klein except ImportError: print("To use this example, you must install Klein:") print(" pip install klein") sys.exit(1) app= Klein()

@app.route('/') def home(request): return'Hello from Klein, Onion World'

@defer.inlineCallbacks def main(reactor): ep= endpoints.serverFromString( reactor,

20 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

"onion:80:controlPort={port}".format(port=default_control_port()) )

def on_progress(percent, tag, msg): print('%03d: %s'%(percent, msg)) txtorcon.IProgressProvider(ep).add_progress_listener(on_progress) print("Note: descriptor upload can take several minutes")

port= yield ep.listen(server.Site(app.resource())) print("Site listening: {}".format(port.getHost())) print("Private key:\n{}".format(port.getHost().onion_key)) print("\nVisit using Tor Browser: http://{}\n".format(port.getHost().onion_uri)) yield defer.Deferred() # wait forever if __name__ =='__main__': task.react(main)

1.4.3 Starting Tor launch_tor.py

Download the example. Launch a new Tor instance. This takes care of setting Tor’s notion ownership so that when the control connection goes away the running Tor exits. from __future__ import print_function

""" Launch a private Tor instance. """ import sys import txtorcon from twisted.web.client import readBody from twisted.internet.task import react from twisted.internet.defer import inlineCallbacks

@inlineCallbacks def main(reactor): # note that you can pass a few options as kwargs # (e.g. data_directory=, or socks_port= ). For other torrc # changes, see below. tor= yield txtorcon.launch( reactor, data_directory="./tordata", stdout=sys.stdout, socks_port='unix:/tmp/tor2/socks', ) # tor = yield txtorcon.connect( # reactor, # clientFromString(reactor, "unix:/var/run/tor/control"), #) print("Connected to Tor version'{}'".format(tor.protocol.version))

state= yield tor.create_state()

1.4. Examples 21 txtorcon Documentation, Release 1.0.0

# or state = yield txtorcon.TorState.from_protocol(tor.protocol)

print("This Tor has PID {}".format(state.tor_pid)) print("This Tor has the following {} Circuits:".format(len(state.circuits))) forc in state.circuits.values(): print("{}".format(c))

agent= tor.web_agent(u'unix:/tmp/tor2/socks') uri='https://www.torproject.org' print("Downloading {}".format(uri)) resp= yield agent.request('GET', uri) print("Response has {} bytes".format(resp.length)) body= yield readBody(resp) print("received body ({} bytes)".format(len(body))) print("{}\n[...]\n{}\n".format(body[:200], body[-200:])) if __name__ =='__main__': react(main) launch_tor_endpoint.py

Download the example. Using the txtorcon.TCP4HiddenServiceEndpoint class to start up a Tor with a hidden service pointed to an IStreamServerEndpoint. from __future__ import print_function

# Here we set up a Twisted Web server and then launch our own tor with # a configured hidden service directed at the Web server we set # up. This uses serverFromString to translate the "onion" endpoint # descriptor into a TCPHiddenServiceEndpoint object... from twisted.web import server, resource from twisted.internet.defer import inlineCallbacks from twisted.internet.task import react, deferLater from twisted.internet.endpoints import serverFromString import txtorcon class Simple(resource.Resource): """ A really simple Web site. """ isLeaf= True

def render_GET(self, request): return"Hello, world! I'm a hidden service!"

@inlineCallbacks def main(reactor): # several ways to proceed here and what they mean: # # "onion:80": # launch a new Tor instance, configure a hidden service on some # port and pubish descriptor for port 80

22 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

# # "onion:80:controlPort=9051:localPort=8080:socksPort=9089:hiddenServiceDir=/home/human/src/txtorcon/hidserv": # connect to existing Tor via control-port 9051, configure a hidden # service listening locally on 8080, publish a descriptor for port # 80 and use an explicit hiddenServiceDir (where "hostname" and # "private_key" files are put by Tor). We set SOCKS port # explicitly, too. # # "onion:80:localPort=8080:socksPort=9089:hiddenServiceDir=/home/human/src/txtorcon/hidserv": # all the same as above, except we launch a new Tor (because no # "controlPort=9051")

ep="onion:80:controlPort=9051:localPort=8080:socksPort=9089:hiddenServiceDir=/home/human/src/txtorcon/hidserv" ep="onion:80:localPort=8080:socksPort=9089:hiddenServiceDir=/home/human/src/txtorcon/hidserv" ep="onion:80" hs_endpoint= serverFromString(reactor, ep)

def progress(percent, tag, message): bar= int(percent/ 10) print("[{}{}] {}".format("#" * bar,"." * (10- bar), message)) txtorcon.IProgressProvider(hs_endpoint).add_progress_listener(progress)

# create our Web server and listen on the endpoint; this does the # actual launching of (or connecting to) tor. site= server.Site(Simple()) port= yield hs_endpoint.listen(site) hs= port.onion_service

# "port" is an IAddress implementor, in this case TorOnionAddress # so you can get most useful information from it -- but you can # also access .onion_service (see below) print( "I have set up a hidden service, advertised at:\n" "http://{host}:{port}\n" "locally listening on {local_address}\n" "Will stop in 60 seconds...".format( host=port.getHost().onion_uri, # or hs.hostname port=port.public_port, # port.local_address will be a twisted.internet.tcp.Port # or a twisted.internet.unix.Port -- both have .getHost() local_address=port.local_address.getHost(), ) )

# if you prefer, hs (port.onion_service) is an instance providing # IOnionService (there's no way to do authenticated services via # endpoints yet, but if there was then this would implement # IOnionClients instead) print("private key:\n{}".format(hs.private_key))

def sleep(s): return deferLater(reactor,s, lambda: None)

yield sleep(50) fori in range(10): print("Stopping in {}...".format(10-i)) yield sleep(1)

1.4. Examples 23 txtorcon Documentation, Release 1.0.0

react(main)

1.4.4 Circuits and Streams disallow_streams_by_port.py

Download the example. An example using IStreamAttacher which is very simple and does just what it sounds like: never attaches Streams exiting to a port in the “disallowed” list (it also explicitly closes them). Note that Tor already has this feature; this is just to illustrate how to use IStreamAttacher and that you may close streams. XXX keep this one? from __future__ import print_function # # This uses a very simple custom txtorcon.IStreamAttacher to disallow # certain streams based solely on their port; by default it closes # all streams on port 80 or 25 without ever attaching them to a # circuit. # # For a more complex IStreamAttacher example, see # attach_streams_by_country.py # from twisted.python import log from twisted.internet.task import react from twisted.internet.defer import inlineCallbacks, Deferred from twisted.internet.endpoints import clientFromString from zope.interface import implementer import txtorcon

@implementer(txtorcon.IStreamAttacher) class PortFilterAttacher:

def __init__(self, state): self.state= state self.disallow_ports=[80, 25] print( "Disallowing all streams to ports: {ports}".format( ports=",".join(map(str, self.disallow_ports)), ) )

def attach_stream(self, stream, circuits): """ IStreamAttacher API """

def stream_closed(x): print("Stream closed:",x)

if stream.target_port in self.disallow_ports: print( "Disallowing {stream} to port {stream.target_port}".format( stream=stream,

24 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

) ) d= self.state.close_stream(stream) d.addCallback(stream_closed) d.addErrback(log.err) return txtorcon.TorState.DO_NOT_ATTACH

# Ask Tor to assign stream to a circuit by itself return None

@inlineCallbacks def main(reactor): control_ep= clientFromString(reactor,"tcp:localhost:9051") tor= yield txtorcon.connect(reactor, control_ep) print("Connected to a Tor version={version}".format( version=tor.protocol.version, )) state= yield tor.create_state() yield state.add_attacher(PortFilterAttacher(state), reactor)

print("Existing streams:") fors in state.streams.values(): print("",s) yield Deferred() react(main) stream_circuit_logger.py

Download the example. For listening to changes in the Circuit and State objects, this example is the easiest to understand as it just prints out (some of) the events that happen. Run this, then visit some Web sites via Tor to see what’s going on. #!/usr/bin/env python

# This uses an IStreamListener and an ICircuitListener to log all # built circuits and all streams that succeed. import sys from twisted.python import log from twisted.internet import reactor import txtorcon def logCircuit(circuit): path='->'.join(map(lambdax: str(x.location.countrycode), circuit.path)) log.msg('Circuit %d(%s) is %s for purpose"%s"'% (circuit.id, path, circuit.state, circuit.purpose)) def logStream(stream, state): circ='' if stream.circuit: path='->'.join(map(lambdax:x.location.countrycode, stream.circuit.path)) circ=' via circuit %d(%s)'%(stream.circuit.id, path)

1.4. Examples 25 txtorcon Documentation, Release 1.0.0

proc= txtorcon.util.process_from_address( stream.source_addr, stream.source_port, state ) if proc: proc=' from process"%s"'%(proc,)

elif stream.source_addr =='(Tor_internal)': proc=' for Tor internal use'

else: proc=' from remote"%s:%s"'%(str(stream.source_addr), str(stream.source_port)) log.msg('Stream %d to %s:%d attached%s%s'% (stream.id, stream.target_host, stream.target_port, circ, proc)) class StreamCircuitLogger(txtorcon.StreamListenerMixin, txtorcon.CircuitListenerMixin):

def __init__(self, state): self.state= state

def stream_attach(self, stream, circuit): logStream(stream, self.state)

def stream_failed(self, stream, reason='', remote_reason='', **kw): print'Stream %d failed because"%s"'%(stream.id, remote_reason)

def circuit_built(self, circuit): logCircuit(circuit)

def circuit_failed(self, circuit, **kw): log.msg('Circuit %d failed"%s"'%(circuit.id, kw['REASON'])) def setup(state): log.msg('Connected to a Tor version %s'% state.protocol.version)

listener= StreamCircuitLogger(state) state.add_circuit_listener(listener) state.add_stream_listener(listener)

state.protocol.add_event_listener('STATUS_GENERAL', log.msg) state.protocol.add_event_listener('STATUS_SERVER', log.msg) state.protocol.add_event_listener('STATUS_CLIENT', log.msg)

log.msg('Existing state when we connected:') fors in state.streams.values(): logStream(s, state)

log.msg('Existing circuits:') forc in state.circuits.values(): logCircuit(c) def setup_failed(arg):

26 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

print"SETUP FAILED", arg log.err(arg) reactor.stop() log.startLogging(sys.stdout) d= txtorcon.build_local_tor_connection(reactor) d.addCallback(setup).addErrback(setup_failed) reactor.run() attach_streams_by_country.py

Download the example. This is one of the more complicated examples. It uses a custom Stream attacher (via IStreamAttacher) to only attach Streams to a Circuit with an exit node in the same country as the server to which the Stream is going (as determined by GeoIP). Caveat: the DNS lookups go via a Tor-assigned stream, so for sites which use DNS trickery to get you to a “close” server, this won’t be as interesting. For bonus points, if there is no Circuit exiting in the correct country, one is created before the Stream is attached. schedule_bandwidth.py

Download the example. This is pretty similar to a feature Tor already has and is basically useless as-is since what it does is toggle the amount of relay bandwidth you’re willing to carry from 0 to 20KiB/s every 20 minutes. A slightly-more-entertaining way to illustate conﬁg changes. (This is useless because your relay takes at least an hour to appear in the consensus). #!/usr/bin/env python

# Here, we do something possible-useful and schedule changes to the # "BandWidthRate" and optionally "BandWidthBurst" settings in Tor. import datetime from twisted.internet import reactor from twisted.internet.interfaces import IReactorTime from txtorcon import build_local_tor_connection, TorConfig class BandwidthUpdater:

def __init__(self, config, scheduler): self.bandwidth=0 self.config= config self.scheduler= IReactorTime(scheduler) self.generator= self.next_update()

def next_update(self): """ Generator that gives out the next time to do a bandwidth update, as well as what the new bandwidth value should be. Here, we toggle the bandwidth every 20 minutes. """

while True: if self.bandwidth: self.bandwidth=0

1.4. Examples 27 txtorcon Documentation, Release 1.0.0

self.burst=0 else: self.bandwidth= 20 * 1024 * 1024 self.burst= self.bandwidth yield(datetime.datetime.now()+ datetime.timedelta(minutes=20), self.bandwidth, self.burst)

def do_update(self): x= self.generator.next() future=x[0] self.new_bandwidth=x[1] self.new_burst=x[2]

tm=(future- datetime.datetime.now()).seconds self.scheduler.callLater(tm, self.really_update) print"waiting", tm,"seconds to adjust bandwidth"

def really_update(self): print"setting bandwidth + burst to", self.new_bandwidth, self.new_burst self.config.set_config('BandWidthBurst', self.new_burst, 'BandWidthRate', self.new_bandwidth) self.doUpdate() def setup_complete(conf): print"Connected." bwup= BandwidthUpdater(conf, reactor) bwup.do_update() def setup_failed(arg): print"SETUP FAILED", arg reactor.stop() def bootstrap(proto): config= TorConfig(proto) config.post_bootstrap.addCallback(setup_complete).addErrback(setup_failed) print"Connection is live, bootstrapping config..." d= build_local_tor_connection(reactor, build_state=False, wait_for_proto=False) d.addCallback(bootstrap).addErrback(setup_failed) reactor.run()

1.4.5 Conﬁguration dump_config.py

Download the example. Very simple read-only use of txtorcon.TorConfig

28 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

1.4.6 Events monitor.py

Download the example. Use a plain txtorcon.TorControlProtocol instance to listen for some simple events – in this case marginally useful, as it listens for logging at level INFO, NOTICE, WARN and ERR. #!/usr/bin/env python

# Just listens for a few EVENTs from Tor (INFO NOTICE WARN ERR) and # prints out the contents, so functions like a log monitor. from twisted.internet import task, defer from twisted.internet.endpoints import UNIXEndpoint import txtorcon

@defer.inlineCallbacks def main(reactor): ep= UNIXEndpoint(reactor,'/var/run/tor/control') tor= yield txtorcon.connect(reactor, ep)

def log(msg): print msg print"Connected to a Tor version", tor.protocol.version for event in['INFO','NOTICE','WARN','ERR']: tor.protocol.add_event_listener(event, log) is_current= yield tor.protocol.get_info('status/version/current') version= yield tor.protocol.get_info('version') print("Version'{}', is_current={}".format(version, is_current)) task.react(main)

1.4.7 Miscellaneous stem_relay_descriptor.py

Download the example. Get information about a relay descriptor with the help of Stem’s Relay Descriptor class. We need to specify the nickname or the ﬁngerprint to get back the details. #!/usr/bin/env python

# This shows how to get the detailed information about a # relay descriptor and parse it into Stem's Relay Descriptor # class. More about the class can be read from # # https://stem.torproject.org/api/descriptor/server_descriptor.html#stem.descriptor.server_descriptor.RelayDescriptor # # We need to pass the nickname or the fingerprint of the onion # router for which we need the the descriptor information, from twisted.internet.task import react

1.4. Examples 29 txtorcon Documentation, Release 1.0.0

from twisted.internet.defer import inlineCallbacks import txtorcon

@inlineCallbacks def main(reactor): proto= yield txtorcon.build_local_tor_connection(reactor, build_state=False)

or_nickname="moria1" print"Trying to get decriptor information about", or_nickname # If the fingerprint is used in place of nickname then, desc/id/ # should be used. descriptor_info= yield proto.get_info('desc/name/'+ or_nickname)

descriptor_info= descriptor_info['desc/name/'+ or_nickname] try: from stem.descriptor.server_descriptor import RelayDescriptor relay_info= RelayDescriptor(descriptor_info) print"The relay's fingerprint is:", relay_info.fingerprint print"Time in UTC when the descriptor was made:", relay_info.published except ImportError ase: print"Error:",e if __name__ =='__main__': react(main) circuit_failure_rates.py

Download the example. txtorcon.tac

Download the example Create your own twisted Service for deploying using twistd. import functools from os.path import dirname import sys from tempfile import mkdtemp import txtorcon from twisted.application import service, internet from twisted.internet import reactor from twisted.internet.endpoints import TCP4ClientEndpoint from twisted.python import log from twisted.web import static, server from zope.interface import implements class TorService(service.Service): implements(service.IService) directory= dirname(__file__) port= 8080

30 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

def __init__(self): self.torfactory= txtorcon.TorProtocolFactory() self.connection= TCP4ClientEndpoint(reactor,'localhost', 9052) self.resource= server.Site(static.File(self.directory))

def startService(self): service.Service.startService(self)

reactor.listenTCP(self.port, self.resource) self._bootstrap().addCallback(self._complete)

def _bootstrap(self): self.config= txtorcon.TorConfig() self.config.HiddenServices=[ txtorcon.HiddenService(self.config, mkdtemp(), ['%d 127.0.0.1:%d'%(80, self.port)]) ] self.config.save() return txtorcon.launch_tor(self.config, reactor, progress_updates=self._updates, tor_binary='tor')

def _updates(self, prog, tag, summary): log.msg('%d%%: %s'%(prog, summary))

def _complete(self, proto): log.msg(self.config.HiddenServices[0].hostname) application= service.Application("Txtorcon Application") torservice= TorService() torservice.setServiceParent(application)

1.5 Contributions

You can help contribute to txtorcon by reporting bugs, sending success stories, by adding a feature or ﬁxing a bug. Even asking that “silly” question helps me with documentation writing.

1.5.1 Contact Information

Discussing txtorcon is welcome in the following places: • IRC: #tor-dev on OFTC (please preﬁx lines with meejah: to get my attention, and be patient for replies). • email: preferably on the tor-dev list, or see meejah.ca for other ways to contact me. • bugs: use txtorcon’s issues tracker on GitHub. • @txtorcon on Twitter (announcements only)

1.5.2 Public Key

You can download my key from a keyserver (0xC2602803128069A7) or see meejah.asc in the repository. The ﬁngerprint is 9D5A 2BD5 688E CB88 9DEB CD3F C260 2803 1280 69A7.

1.5. Contributions 31 txtorcon Documentation, Release 1.0.0

Also available at https://meejah.ca/meejah.asc. For convenience: curl https://meejah.ca/meejah.asc | gpg --import

1.5.3 Pull Requests

Yes, please! If you have a new feature or a bug fix, the very best way is to submit a pull-request on GitHub. Since we have 100% coverage, all new lines of code should at least be covered by unit-tests. You can also include a note about the change in docs/releases.rst if you like (or I can make one up after the merge). I prefer if you rebase/squash commits into logical chunks. Discussion of any implementation details can simply occur on the pull-request itself. Force-pushing to the same branch/PR is fine by me if you want to re-order commits etcetera (but, it’s also fine if you just want to push new “fix issues” commits instead). Some example pull-requests: • good discussion + more commits: PR #150; • a simple one that was “ready-to-go”: PR #51.

1.5.4 Making a Release

Mostly a note-to-self, but here is my release checklist.

Release Checklist

• ensure local copy is on master, up-to-date: – git checkout master – git pull • double-check version updated, sadly in a few places: – Makefile – txtorcon/_metadata.py • run all tests, on all configurations – “tox” • “make pep8” should run cleanly (ideally) • update docs/releases.rst to reflect upcoming reality – blindly make links to the signatures – update heading, date • on both signing-machine and build-machine shells: – export VERSION=0.18.0 • (if on signing machine) “make dist” and “make dist-sigs” – creates: dist/txtorcon-${VERSION}.tar.gz.asc dist/txtorcon-${VERSION}-py2-none-any.whl.asc – add the signatures to “signatues/” – add ALL FOUR files to dist/ (OR fix twine commands)

32 Chapter 1. Documentation txtorcon Documentation, Release 1.0.0

• (if not on signing machine) do “make dist” * scp dist/txtorcon-${VERSION}.tar.gz dist/txtorcon-${VERSION}- py2-none-any.whl signingmachine: * sign both, with .asc detached signatures – gpg –no-version –detach-sign –armor –local-user [email protected] txtorcon-${VERSION}-py2-none- any.whl – gpg –no-version –detach-sign –armor –local-user [email protected] txtorcon-${VERSION}.tar.gz – copy signatures back to build machine, in dist/ – double-check that they validate

* gpg –verify dist/txtorcon-${VERSION}-py2-none-any.whl.asc * gpg –verify dist/txtorcon-${VERSION}.tar.gz.asc • generate sha256sum for each: sha256sum dist/txtorcon-${VERSION}.tar.gz dist/txtorcon-${VERSION}- py2-none-any.whl • copy signature ﬁles to /signatures and commit them along with the above changes for versions, etc. • draft email to tor-dev (and probably twisted-python): – example: https://lists.torproject.org/pipermail/tor-dev/2014-January/006111.html – example: https://lists.torproject.org/pipermail/tor-dev/2014-June/007006.html – copy-paste release notes, un-rst-format them – include above sha256sums – clear-sign the announcement – gpg –armor –clearsign -u [email protected] release-announce-${VERSION} – Example boilerplate: I’m [adjective] to announce txtorcon 0.10.0. This adds several amazing features, including levitation. Full list of improvements:

* take from releases.rst * ...but un-rST them You can download the release from PyPI or GitHub (or of course “pip install txtorcon”): https://pypi.python.org/pypi/txtorcon/0.10.0 https://github.com/meejah/txtorcon/releases/tag/v0.10.0 Releases are also available from the hidden service: http://timaq4ygg2iegci7.onion/txtorcon-0.12.0.tar.gz http://timaq4ygg2iegci7.onion/txtorcon- 0.12.0.tar.gz.asc You can verify the sha256sum of both by running the following 4 lines in a shell wherever you have the ﬁles downloaded: cat <

1.5. Contributions 33 txtorcon Documentation, Release 1.0.0

– git pull • create signed tag – git tag -s -u [email protected] -F release-announce-${VERSION} v${VERSION} • copy dist/* ﬁles + signatures to hidden-service machine • copy them to the HTML build directory! (docs/_build/html/) • git pull and build docs there – FIXME: why aren’t all the dist ﬁles copied as part of doc build (only .tar.gz) • download both distributions + signatures from hidden-service – verify sigs – verify sha256sums versus announcement text – verify tag (git tag –verify v${VERSION}) on machine other than signing-machine – run: ./scripts/download-release-onion.sh • upload release – to PyPI: “make release” (which uses twine so this isn’t the same step as “sign the release”)

* make sure BOTH the .tar.gz and .tar.gz.asc (ditto for .whl) are in the dist/ directory first!!) * ls dist/txtorcon-${VERSION}* * note this depends on a ~/.pypirc file with [server-login] section containing “username:” and “password:” – git push origin master – git push origin v${VERSION} – to github: use web-upload interface to upload the 4 files (both dists, both signature) • make announcement – post to tor-dev@ the clear-signed release announcement – post to twisted-python@ the clear-signed release announcement – tweet as @txtorcon – tell #tor-dev??

34 Chapter 1. Documentation CHAPTER 2

Ofﬁcial Releases:

All ofﬁcial releases are tagged in Git, and signed by my key. All ofﬁcial releases on PyPI have a corresponding GPG signature of the build. Please be aware that pip does not check GPG signatures by default; please see this ticket if you care. The most reliable way to verify you got what I intended is to clone the Git repository, git checkout a tag and verify its signature. The second-best would be to download a release + tag from PyPI and verify that.

2.1 Releases

There isn’t a “release schedule” in any sense. If there is something in master your project depends upon, let me know and I’ll do a release.

2.1.1 upcoming: 1.0.0 branch release-1.x will become release 1.0.0 in the future. Some APIs will break, but wherever possible backwards- compatiblity with 0.x will be kept. Rendered docs are available already, at txtorcon.readthedocs. • full Python3 support • re-work (re-do?) most documentation, adding a Programming Guide and extensive re-organization • a bunch of API fixups, improvements and better-ments. • new high-level API txtorcon.Tor abstracts “a single running Tor process” (that we either connected to or launched). • twisted.web.client support, via txtorcon.Tor.web_agent() and txtorcon.Circuit.web_agent() • txtorcon.Circuit.stream_via(): easily attach client-type streams to specific Circuits • txtorcon.Tor.stream_via(): create client-type streams that go via Tor • SOCKS5: built-in support, including for Tor custom extensions (see txtorcon.Tor.resolve() and txtorcon.Tor.resolve_ptr()) • SOCKS5: see also txtorcon.TorConfig.socks_endpoint() • txtorcon.TorControlProtocol.get_info() now returns a list of values, in the same order as the keys. (Used to return a dict) (XXX is this true yet?) • many things which “should” have been “private” all along grew an underscore prefix.

35 txtorcon Documentation, Release 1.0.0

• txtorcon.Router no longer has a .location attribute. Instead, use txtorcon.Router.get_location() (which returns a Deferred) • dropped GeoIP dependency entirely (no GeoIP support).

2.1.2 unreleased

git master will likely become v0.19.0 • Full Python3 support • Drop txsocksx and use a custom implementation (this also implements the custom Tor SOCKS5 methods RE- SOLVE and RESOLVE_PTR • Drop support for older Twisted releases (12, 13 and 14 are no longer supported).

2.1.3 v0.18.0

January 11, 2017 • txtorcon-0.18.0.tar.gz(PyPI( local-sig or github-sig)(source) • issue 200: better feedback if the cookie data can’t be read

2.1.4 v0.17.0

October 4, 2016 • txtorcon-0.17.0.tar.gz(PyPI( local-sig or github-sig)(source) • issue 187: ﬁx unix-socket control endpoints • sometimes mapping streams to hostnames wasn’t working properly • backwards-compatibility API for socks_hostname was incorrectly named

2.1.5 v0.16.1

August 31, 2016 • txtorcon-0.16.1.tar.gz(PyPI( local-sig or github-sig)(source) • issue 172: give TorProcessProtocol a .quit method • issue 181: enable SOCKS5-over-unix-sockets for TorClientEndpoint (thanks to david415

2.1.6 v0.16.0

• there wasn’t one, because reasons.

2.1.7 v0.15.1

• txtorcon-0.15.1.tar.gz(PyPI( local-sig or github-sig)(source) • ﬁx issue 179 with Circuit.age.

36 Chapter 2. Ofﬁcial Releases: txtorcon Documentation, Release 1.0.0

2.1.8 v0.15.0

July 26, 2016 • txtorcon-0.15.0.tar.gz(PyPI( local-sig or github-sig)(source) • added support for NULL control-port-authentication which is often appropriate when used with a UNIX domain socket • switched to ipaddress instead of Google’s ipaddr; the API should be the same from a user perspective but packagers and tutorials will want to change their instructions slightly (pip install ipaddress or apt-get install python-ipaddress are the new ways). • support the new ADD_ONION and DEL_ONION “ephemeral hidden services” commands in TorConfig • a first stealth-authentication implementation (for “normal” hidden services, not ephemeral) • bug-fix from david415 to raise ConnectionRefusedError instead of StopIteration when running out of SOCKS ports. • new feature from david415 adding a build_timeout_circuit method which provides a Deferred that callbacks only when the circuit is completely built and errbacks if the provided timeout expires. This is useful because txtorcon.TorState.build_circuit() callbacks as soon as a Circuit instance can be provided (and then you’d use txtorcon.Circuit.when_built() to find out when it’s done building). • new feature from coffeemakr falling back to password authentication if cookie authentication isn’t available (or fails, e.g. because the file isn’t readable). • both TorState and TorConfig now have a .from_protocol class-method. • spec-compliant string-un-escaping from coffeemakr • a proposed new API: txtorcon.connect() • fix issue 176

2.1.9 v0.14.1

October 25, 2015 • subtle bug with .is_built on Circuit; changing the API (but with backwards-compatibility until 0.15.0 at least)

2.1.10 v0.14.0

September 26, 2015 • txtorcon-0.14.0.tar.gz(PyPI( local-sig or github-sig)(source) • txtorcon.interface.IStreamAttacher handling was missing None and DO_NOT_ATTACH cases if a Deferred was returned. • add .is_built Deferred to txtorcon.Circuit that gets ‘callback()‘d when the circuit becomes BUILT • david415 ported his tor: endpoint parser so now both client and server endpoints are supported. This means any Twisted program using endpoints can use Tor as a client. For example, to connect to txtorcon’s Web site: ep = clientFromString("tor:timaq4ygg2iegci7.onion:80"). (In the future, I’d like to automatically launch Tor if required, too). • Python3 ﬁxes from isis (note: needs Twisted 15.4.0+)

2.1. Releases 37 txtorcon Documentation, Release 1.0.0

2.1.11 v0.13.0

May 10, 2015 • txtorcon-0.13.0.tar.gz(PyPI( local-sig or github-sig)(source) • support basic and stealth hidden service authorization, and parse client_keys ﬁles. • 2x speedup for TorState parsing (mostly by lazy-parsing timestamps) • can now parse ~75000 microdescriptors/second per core of 3.4GHz Xeon E3 • launch_tor now doesn’t use a temporary torrc (command-line options instead) • tons of pep8 cleanups • several improvements to hidden-service conﬁguration from sambuddhabasu1. • populated valid signals from GETINFO signals/names from sambuddhabasu1.

2.1.12 v0.12.0

February 3, 2015 • txtorcon-0.12.0.tar.gz(PyPI( local-sig or github-sig)(source) • doc, code and import cleanups from Kali Kaneko • HiddenServiceDirGroupReadable support • Issue #80: honour ControlPort 0 in incoming TorConfig instance. The caller owns both pieces: you have to figure out when it’s bootstraped, and are responsible for killing it off. • Issue #88: clarify documentation and fix appending to some config lists • If GeoIP data isn’t loaded in Tor, it sends protocol errors; if txtorcon also hasn’t got GeoIP data, the queries for country-code fail; this error is now ignored. • 100% unit-test coverage! (line coverage) • PyPy support (well, at least all tests pass) • TCP4HiddenServiceEndpoint now waits for descriptor upload before the listen() call does its callback (this means when using onion: endpoint strings, or any of the endpoints APIs your hidden service is 100% ready for action when you receive the callback) • TimeIntervalCommaList from Tor config supported • TorControlProtocol now has a .all_routers member (a set() of all Routers) • documentation fix from sammyshj

2.1.13 v0.11.0

August 16, 2014 • September 6, 2015. bugﬁx release: txtorcon-0.11.1.tar.gz(PyPI( local-sig or github-sig)(source) • ﬁxed Debian bug 797261 causing 3 tests to fail • txtorcon-0.11.0.tar.gz(PyPI( local-sig or github-sig)(source) • More control for launch_tor: access stdout, stderr in real-time and control whether we kill Tor on and stderr output. See issue #79.

38 Chapter 2. Ofﬁcial Releases: txtorcon Documentation, Release 1.0.0

• Warning about build_circuit being called without a guard ﬁrst is now optional (default is still warn) (from arlolra) • available_tcp_port() now in util (from arlolra) • TorState now has a .routers_by_hash member (from arlolra)

2.1.14 v0.10.1

July 20, 2014 • txtorcon-0.10.1.tar.gz(PyPI( local-sig or github-sig)(source) • ﬁx bug incorrectly issuing RuntimeError in brief window of time on event-listeners • issue #78: Add tox tests and ﬁx for Twisted 12.0.0 (and prior), as this is what Debian squeeze ships • issue #77: properly expand relative and tilde paths for hiddenServiceDir via endpoints

2.1.15 v0.10.0

June 15, 2014 • txtorcon-0.10.0.tar.gz(PyPI( local-sig or github-sig)(source) • In collaboration with David Stainton after a pull-request, we have endpoint parser plugins for Twisted! This means code like serverFromString("onion:80").listen(...) is enough to start a service. • The above also means that any endpoint-using Twisted program can immediately offer its TCP services via Hidden Service with no code changes. For example, using Twisted Web to serve a WSGI web application would be simply: twistd web --port onion:80 --wsgi web.app • switch to a slightly-modiﬁed Alabaster Sphinx theme • added howtos to documentation

2.1.16 v0.9.2

April 23, 2014 • txtorcon-0.9.2.tar.gz( local-sig or github-sig)(source) • add on_disconnect callback for TorControlProtocol (no more monkey-patching Protocol API) • add age() method to Circuit • add time_created property to Circuit • don’t incorrectly listen for NEWDESC events in TorState • add .flags dict to track ﬂags in Circuit, Stream • build_circuit() can now take hex IDs (as well as Router instances) • add unique_name property to Router (returns the hex id, unless Named then return name) • add location property to Router • TorState.close_circuit now takes either a Circuit ID or Circuit instance • TorState.close_stream now takes either a Stream ID or Stream instance • support both GeoIP API versions

2.1. Releases 39 txtorcon Documentation, Release 1.0.0

• more test-coverage • small patch from enriquefynn improving tor binary locating • strip OK lines in TorControlProtocol (see issue #8) • use TERM not KILL when Tor launch times out (see issue #68) from hellais

2.1.17 v0.9.1

January 20, 2014 • txtorcon-0.9.1.tar.gz( local-sig or github-sig)(source) • put test/ directory at the top level • using “coverage” tool instead of custom script • using coveralls.io and travis-ci for test coverage and continuous integration • issue #56: added Circuit.close() and Stream.close() starting from aagbsn’s patch • parsing issues with multi-line keyword discovered and resolved • preserve router nicks from long-names if consensus lacks an entry (e.g. bridges) • using Twine for releases • Wheel release now also available • issue #57: “python setup.py develop” now supported • issue #59: if tor_launch() times out, Tor is properly killed (starting with pull-request from Ryman) • experimental docker.io-based tests (for HS listening, and tor_launch() timeouts) • issue #55: pubkey link on readthedocs • issue #63 • clean up GeoIP handling, and support pygeoip both pre and post 0.3 • slightly improve unit-test coverage (now at 97%, 61 lines of 2031 missing) • added a Walkthrough to the documentation

2.1.18 v0.8.2

November 22, 2013 • txtorcon-0.8.2.tar.gz( local-sig or github-sig)(source) • ensure hidden service server-side endpoints listen only on 127.0.0.1

2.1.19 v0.8.1

May 13, 2013 • txtorcon-0.8.1.tar.gz( local-sign or github-sig)(source) • ﬁxed improper import in setup.py preventing 0.8.0 from installing • signatures with proper subkey this time

40 Chapter 2. Ofﬁcial Releases: txtorcon Documentation, Release 1.0.0

• Proper file-flushing in tests and PyPy fixes from Lukas Lueg • docs build issue from isis

2.1.20 v0.8.0

April 11, 2013 (actually uploaded May 11) • Please use 0.8.1; this won’t install due to import problem in setup.py (unless you have pypissh). • following semantic versioning; • slight API change ICircuitListener.circuit_failed(), circuit_closed() and IStreamListener.stream_failed(), stream_closed() and stream_detach() all now include any keywords in the notification method (some of these lacked flags, or only included some) (issue #18); • launch_tor() can take a timeout (starting with a patch from hellais); • cleanup from aagbsn; • more test coverage; • run tests cleanly without graphviz (from lukaslueg); • issue #26 fix from lukaslueg; • pep8 and whitespace targets plus massive cleanup (now pep8 clean, from lukaslueg); • issue #30 fix reported by webmeister making ipaddr actually-optional; • example using synchronous web server (built-in SimpleHTTPServer) with txtorcon (from lukaslueg); • TorState can now create circuits without an explicit path; • passwords for non-cookie authenticated sessions use a password callback (that may return a Deferred) instead of a string (issue #44); • fixes for AddrMap in case #8596 is implemented;

2.1.21 v0.7

November 21, 2012 • txtorcon-0.7.tar.gz( local-sig or github-sig)(source) • issue #20 conﬁg object now hooked up correctly after launch_tor(); • patch from hellais for properly handling data_dir given to TCPHiddenServiceEndpoint; • .tac example from mmaker; • allow TorConﬁg().hiddenservices.append(hs) to work properly with no attached protocol

2.1.22 v0.6

October 10, 2012 • txtorcon-0.6.tar.gz( local-sig or github-sig)(source) • debian packaging (mmaker); • psutil fully gone;

2.1. Releases 41 txtorcon Documentation, Release 1.0.0

• changed API for launch_tor() to use TorConfig instead of args; • TorConfig.save() works properly with no connected Tor; • fix incorrect handling of 650 immediately after connect; • pep8 compliance; • use assertEqual in tests; • messages with embdedded keywords work properly; • fix bug with setup.py + pip; • issue #15 reported along with patch by Isis Lovecruft; • consolidate requirements (from aagbsn); • increased test coverage and various minor fixes; • https URIs for ReadTheDocs;

2.1.23 v0.5

June 20, 2012 • txtorcon-0.5.tar.gz (txtorcon-0.5.tar.gz.sig) (source) • remove psutil as a dependency, including from util.process_from_address

2.1.24 v0.4

June 6, 2012 • txtorcon-0.4.tar.gz (txtorcon-0.4.tar.gz.sig) • remove built documentation from distribution; • ﬁx PyPI problems (“pip install txtorcon” now works)

2.1.25 v0.3

• 0.3 was broken when released (docs couldn’t build).

2.1.26 v0.2

June 1, 2012 • txtorcon-0.2.tar.gz (txtorcon-0.2.tar.gz.sig) • incremental parsing; • faster TorState startup; • SAFECOOKIE support; • several bug ﬁxes; • options to circuit_failure_rates.py example to make it actually-useful; • include built documentation + sources in tarball;

42 Chapter 2. Ofﬁcial Releases: txtorcon Documentation, Release 1.0.0

• include tests in tarball; • improved logging; • patches from mmaker and kneufeld;

2.1.27 v0.1 march, 2012 • txtorcon-0.1.tar.gz (txtorcon-0.1.tar.gz.sig)

2.1. Releases 43 txtorcon Documentation, Release 1.0.0

44 Chapter 2. Ofﬁcial Releases: CHAPTER 3

API Documentation

These are the lowest-level documents, directly from the doc-strings in the code with some minimal organization; if you’re just getting started with txtorcon the “Programming Guide” is a better place to start.

3.1 API Documentation

3.1.1 High Level API

This is the recommended API. See the Programming Guide for “prose” documentation of these (and other) APIs.

Tor class txtorcon.Tor(reactor, tor_conﬁg, _process_proto=None) Bases: object I represent a single instance of Tor and act as a Builder/Factory for several useful objects you will probably want. There are two ways to create a Tor instance: •txtorcon.connect‘() to connect to a tor that is already running (e.g. Tor Browser Bundle, a system Tor, ...). •txtorcon.launch‘() to launch a fresh tor instance If you desire more control, there are “lower level” APIs which are the very ones used by this class. However, this “highest level” API should cover many use-cases: import txtorcon

@inlineCallbacks def main(reactor): # tor = yield txtorcon.connect(UNIXClientEndpoint(reactor, "/var/run/tor/control")) tor= yield txtorcon.launch(reactor)

onion_ep= tor.create_onion_endpoint(port=80) port= yield onion_ep.listen(Site()) print(port.getHost())

45 txtorcon Documentation, Release 1.0.0

don’t instantiate this class yourself – instead use the factory methods txtorcon.launch() or txtorcon.connect() quit(*args, **kwargs) Closes the control connection, and if we launched this Tor instance we’ll send it a TERM and wait until it exits. process protocol The TorControlProtocol instance that is communicating with this Tor instance. config The TorConfig instance associated with the tor instance we launched. This instance represents up-to-date configuration of the tor instance (even if another controller is connected). web_agent(socks_config=None, pool=None) Parameters • socks_config – If None (the default), a suitable SOCKS port is chosen from our config (or added). If supplied, should be either a string which is a valid option for Tor’s SocksPort option or a Deferred which fires an IStreamClientEndpoint (e.g. the return- value from txtorcon.TorConfig.socks_endpoint()) • pool – passed on to the Agent (as pool=) dns_resolve(*args, **kwargs) Parameters hostname – a string Returns a Deferred that calbacks with the hostname as looked-up via Tor (or errback). This uses Tor’s custom extension to the SOCKS5 protocol. dns_resolve_ptr(*args, **kwargs) Parameters ip – a string, like “127.0.0.1” Returns a Deferred that calbacks with the IP address as looked-up via Tor (or errback). This uses Tor’s custom extension to the SOCKS5 protocol. stream_via(host, port, tls=False, socks_port=None) This returns an IStreamClientEndpoint instance that will use this Tor (via SOCKS) to visit the (host, port) indicated. Parameters • host – The host to connect to. You MUST pass host-names to this. If you absolutely know that you’ve not leaked DNS (e.g. you save IPs in your app’s configuration or similar) then you can pass an IP. • port – Port to connect to. • tls – If True, it will wrap the return endpoint in one that does TLS (default: False). • socks_port – Normally not needed (default: None) but you can pass any valid Tor “SocksPort” option here, and one will be configured into this Tor instance (when you call .connect on the endpoint) create_onion_endpoint(port, private_key=None) Returns an object that implements IStreamServerEndpoint, which will create an “ephemeral” Onion service when .listen() is called. This uses the ADD_ONION tor control-protocol command. Parameters private_key – if not None (the default), this should be the same blob of key material that you received from a previous call to this method.

46 Chapter 3. API Documentation txtorcon Documentation, Release 1.0.0

“Retrieved” here means by accessing the .onion_private_key attribute of the object returned from .listen() (see txtorcon.IHiddenService and txtorcon.TCPHiddenServiceEndpoint.listen()) which will be a txtorcon.TorOnionListeningPort – and therefore implments txtorcon.IOnionService (XXX FIXME it implements IHiddenService). create_onion_disk_endpoint(port, hs_dir=None, group_readable=False) create_state(*args, **kwargs) returns a Deferred that ﬁres with a ready-to-go txtorcon.TorState instance. connect

# FIXME why doesn’t “txtorcon.connect” work here with automethod?? txtorcon.connect() launch txtorcon.launch()

3.1.2 Tracking and Changing Live Tor State

TorState class txtorcon.TorState(protocol, bootstrap=True) Bases: object This tracks the current state of Tor using a TorControlProtocol. On setup it ﬁrst queries the initial state of streams and circuits. It then asks for updates via the listeners. It requires an ITorControlProtocol instance. The control protocol doesn’t need to be bootstrapped yet. The Deferred .post_boostrap is driggered when the TorState instance is fully ready to go. The easiest way is to use the helper method txtorcon.build_tor_connection(). For details, see the implementation of that. You may add an txtorcon.interface.IStreamAttacher to provide a custom mapping for Strams to Circuits (by default Tor picks by itself). This is also a good example of the various listeners, and acts as an txtorcon.interface.ICircuitContainer and txtorcon.interface.IRouterContainer. Variables DO_NOT_ATTACH – Constant to return from an IAttacher indicating you don’t want to attach this stream at all. classmethod from_protocol(protocol, **kw) Create a new, boot-strapped TorState from a TorControlProtocol instance. Returns a Deferred that ﬁres with a TorState instance circuits = None keys on id (integer) streams = None keys on id (integer) all_routers = None list of unique routers

3.1. API Documentation 47 txtorcon Documentation, Release 1.0.0

routers = None keys by hexid (string) and by unique names routers_by_name = None keys on name, value always list (many duplicate “Unnamed” routers, for example) routers_by_hash = None keys by hexid (string) guards = None potentially-usable as entry guards, I think? (any router with ‘Guard’ flag) entry_guards = None from GETINFO entry-guards, our current entry guards unusable_entry_guards = None list of entry guards we didn’t parse out authorities = None keys by name add_attacher(attacher, myreactor) Provide an txtorcon.interface.IStreamAttacher to associate streams to circuits. This won’t get turned on until after bootstrapping is completed. (‘__LeaveStreamsUnattached’ needs to be set to ‘1’ and the existing circuits list needs to be populated). attachers are called in the order they’re added until one of them returns non-None. If all attachers return None, we just ask Tor to attach the stream. If you don’t want to attach the stream at all (e.g. reject it) then call txtorcon.Stream.close() on the stream. remove_attacher(attacher, myreactor) Returns a Deferred that fires when we’ve successfully removed the give attacher. Note: this must be async becase we may have to issue cleanup commands to Tor if this is the last attacher. XXX probably just put reactor arg into TorState ctor? stream_close_reasons = {‘REASON_EXITPOLICY’: 4, ‘REASON_TORPROTOCOL’: 13, ‘REASON_DONE’: 6, ‘REASON_CONNRESET’: 12, ‘REASON_RESOLVEFAILED’: 2, ‘REASON_RESOURCELIMIT’: 11, ‘REASON_NOROUTE’: 8, ‘REASON_NOTDIRECTORY’: 14, ‘REASON_HIBERNATING’: 9, ‘REASON_MISC’: 1, ‘REASON_TIMEOUT’: 7, ‘REASON_DESTROY’: 5, ‘REASON_INTERNAL’: 10, ‘REASON_CONNECTREFUSED’: 3} close_stream(stream, reason=’REASON_MISC’, **kwargs) This sends a STREAMCLOSE command, using the specified reason (either an int or one of the 14 strings in section 6.3 of tor-spec.txt if the argument is a string). Any kwards are passed through as flags if they evaluated to true (e.g. “SomeFlag=True”). Currently there are none that Tor accepts. close_circuit(circid, **kwargs) This sends a CLOSECIRCUIT command, using any keyword arguments passed as the Flags (currently, that is just ‘IfUnused’ which means to only close the circuit when it is no longer used by any streams). Parameters circid – Either a circuit-id (int) or a Circuit instance Returns a Deferred which callbacks with the result of queuing the command to Tor (usually “OK”). If you want to instead know when the circuit is actually-gone, see Circuit.close on_circuit_new(callback) experimental Callback should take same args as txtorcon.interface.ICircuitListener.circuit_new() See also txtorcon.TorState.add_circuit_listener() on_circuit_launched(callback) experimental Callback should take same args as txtorcon.interface.ICircuitListener.circuit_launched()

48 Chapter 3. API Documentation txtorcon Documentation, Release 1.0.0

See also txtorcon.TorState.add_circuit_listener() on_circuit_extend(callback) experimental Callback should take same args as txtorcon.interface.ICircuitListener.circuit_extend() See also txtorcon.TorState.add_circuit_listener() on_circuit_built(callback) experimental Callback should take same args as txtorcon.interface.ICircuitListener.circuit_built() See also txtorcon.TorState.add_circuit_listener() on_circuit_closed(callback) experimental Callback should take same args as txtorcon.interface.ICircuitListener.circuit_closed() See also txtorcon.TorState.add_circuit_listener() on_circuit_failed(callback) experimental Callback should take same args as txtorcon.interface.ICircuitListener.circuit_failed() See also txtorcon.TorState.add_circuit_listener() add_circuit_listener(icircuitlistener) experimental Adds a new instance of txtorcon.interface.ICircuitListener which will receive updates for all existing and new circuits. add_stream_listener(istreamlistener) experimental Adds a new instance of txtorcon.interface.IStreamListener which will receive updates for all existing and new streams. build_circuit(routers=None, using_guards=True) Builds a circuit consisting of exactly the routers specified, in order. This issues an EXTENDCIRCUIT call to Tor with all the routers specified. Parameters • routers – a list of Router instances which is the path desired. To allow Tor to choose the routers itself, pass None (the default) for routers. • using_guards – A warning is issued if the first router isn’t in self.entry_guards. Returns A Deferred that will callback with a Circuit instance (with the .id member being valid, and probably nothing else). DO_NOT_ATTACH =