Txtorcon Documentation Release 20.0.0

txtorcon Documentation Release 20.0.0

meejah

Jun 09, 2020

Contents

1 Documentation 3 1.1 Introduction...... 3 1.1.1 Features Overview...... 3 1.1.2 Shell-cast Overview...... 4 1.1.3 Example Code...... 4 1.1.4 Known Users...... 5 1.2 Installing txtorcon...... 6 1.2.1 Latest Release...... 6 1.2.2 Compatibility...... 6 1.2.3 Tor Conﬁguration...... 7 1.2.4 Source Code...... 7 1.2.5 Development Environment...... 7 1.2.6 Integration Tests...... 8 1.2.7 Dependencies / Requirements...... 8 1.3 Programming Guide...... 8 1.3.1 API Stability...... 9 1.3.2 High Level Overview...... 9 1.3.3 A Tor Instance...... 10 1.3.4 A Note On Style...... 11 1.3.5 Tracking and Changing Tor’s Conﬁguration...... 11 1.3.6 Monitor and Change Tor’s State...... 12 1.3.7 Making Connections Over Tor...... 13 1.3.8 Onion (Hidden) Services...... 14 1.3.9 Custom Circuits...... 17 1.3.10 Building Your Own Circuits...... 18 1.4 Examples...... 19 1.4.1 Web: clients...... 19 1.4.2 Starting Tor...... 23 1.4.3 Circuits and Streams...... 26 1.4.4 Events...... 29 1.4.5 Miscellaneous...... 30 1.5 Using Asyncio Libraries with txtorcon...... 32 1.5.1 web_onion_service_aiohttp.py ...... 32 1.6 Contributions...... 34 1.6.1 Contact Information...... 34 1.6.2 Public Key...... 35

i 1.6.3 Pull Requests...... 35 1.6.4 Making a Release...... 35

2 Ofﬁcial Releases: 39 2.1 Releases...... 39 2.1.1 v20.0.0...... 39 2.1.2 v19.1.0...... 40 2.1.3 v19.0.0...... 40 2.1.4 v18.3.0...... 40 2.1.5 v18.2.0...... 40 2.1.6 v18.1.0...... 40 2.1.7 v18.0.2...... 41 2.1.8 v18.0.1...... 41 2.1.9 v18.0.0...... 41 2.1.10 v0.20.0...... 42 2.1.11 v0.19.3...... 42 2.1.12 v0.19.2...... 43 2.1.13 v0.19.1...... 43 2.1.14 v0.19.0...... 43 2.1.15 v0.18.0...... 44 2.1.16 v0.17.0...... 44 2.1.17 v0.16.1...... 44 2.1.18 v0.16.0...... 44 2.1.19 v0.15.1...... 44 2.1.20 v0.15.0...... 44 2.1.21 v0.14.2...... 45 2.1.22 v0.14.1...... 45 2.1.23 v0.14.0...... 45 2.1.24 v0.13.0...... 46 2.1.25 v0.12.0...... 46 2.1.26 v0.11.0...... 46 2.1.27 v0.10.1...... 47 2.1.28 v0.10.0...... 47 2.1.29 v0.9.2...... 47 2.1.30 v0.9.1...... 48 2.1.31 v0.8.2...... 48 2.1.32 v0.8.1...... 48 2.1.33 v0.8.0...... 49 2.1.34 v0.7...... 49 2.1.35 v0.6...... 49 2.1.36 v0.5...... 50 2.1.37 v0.4...... 50 2.1.38 v0.3...... 50 2.1.39 v0.2...... 50 2.1.40 v0.1...... 51

3 API Documentation 53 3.1 API Documentation...... 53 3.1.1 High Level API...... 53 3.1.2 Tracking and Changing Live Tor State...... 60 3.1.3 Reading and Writing Live Tor Conﬁguration...... 68 3.1.4 Endpoints and Related Classes...... 70 3.1.5 Onion APIs...... 75 3.1.6 Low-Level Protocol Classes...... 81 ii 3.1.7 txtorcon.socks Module...... 87 3.1.8 txtorcon.interface Module...... 89 3.1.9 txtorcon.util Module...... 93

4 Indices and tables 95

Index 97

iii iv txtorcon Documentation, Release 20.0.0

• docs: – v3 onion: http://fjblvrw2jrxnhtg67qpbzi45r7ofojaoo3orzykesly2j3c2m3htapid.onion/ – v2 onion: http://timaq4ygg2iegci7.onion – clearnet: https://txtorcon.readthedocs.org • code: https://github.com/meejah/txtorcon • torsocks git clone git://timaq4ygg2iegci7.onion/txtorcon.git

•

If this is your ﬁrst time exploring txtorcon, please look at the Introduction ﬁrst. These docs are for version 20.0.0. Supported and tested platforms: Python 3.5+, PyPy 5.0.0+, Python 2.7+ (deprecated) using Twisted 15.5.0+, 16.3.0+, or 17.1.0+ (see travis). Asycnio inter-operation is now possible, see Using Asyncio Libraries with txtorcon

Contents 1 txtorcon Documentation, Release 20.0.0

2 Contents CHAPTER 1

Documentation

1.1 Introduction

txtorcon is an implementation of the control-spec for Tor using the Twisted networking library for Python( supports Py2, PyPy and Py3). txtorcon gives you a live view of all Tor state and the ability to control most aspects of Tor’s operation. With txtorcon you can launch tor; connect to already-running tor instances; use tor as a client (via SOCKS5); set up (onion) services over tor; change all aspects of conﬁguration; track live state (active circuits and streams, etc); do DNS via Tor; and query other information from the tor daemon. txtorcon is the library to use if you want to write event-based software in Python that uses the Tor network as a client or a service (or integrate Tor support for existing Twisted-using applications, or display information about a locally running tor). Twisted already provides many robust protocol implementations, deployment, logging and integration with GTK, Qt and other graphics frameworks – so txtorcon can be used for command-line or GUI applications or integrate with long-lived daemons easily. In fact, due to support for endpoints (adding the tor: and onion: plugins), many Twisted applications can now integrate with Tor with no code changes. For example, you can use the existing Twisted webserver via twistd to serve your ~/public_html directory over an onion service:

$ sudo apt-get install --install-suggests python-txtorcon $ twistd web --port "onion:80" --path ~/public_html

(You should ideally enable the Tor project repositories ﬁrst). txtorcon strives to provide sane and safe defaults. txtorcon is a Tor project. The applications Tahoe-LAFS and Crossbar.io have successfully integrated Tor support using txtorcon.

1.1.1 Features Overview

Currently, txtorcon is capable of: • making arbitrary client connections to other services over Tor;

3 txtorcon Documentation, Release 20.0.0

• configuring twisted.web.client.Agent instances to do Web requests over Tor; • doing both of the above over specific circuits; • listening as an Onion service; • maintaining up-to-date (live) state information about Tor: Circuits, Streams and Routers (relays); • maintaining current (live) configuration information; • maintaining representation of Tor’s address mappings (with expiry); • interrogating initial state of all three of the above; • listening for and altering stream -> circuit mappings; • building custom circuits; • Circuit and Stream state listeners; • listening for any Tor EVENT; • launching and/or controlling a Tor instance (including Tor Browser Bundle); • complete Twisted endpoint support (both “onion”/server side and client-side). This means you may be able to use existing Twisted software via Tor with no code changes. It also is the preferred way to connect (or listen) in Twisted. Comments (positive or negative) appreciated. Even better if they come with patches

1.1.2 Shell-cast Overview

A text-only screencast-type overview of some of txtorcon’s features, from asciinema.org:

1.1.3 Example Code download (also python3 style) from twisted.internet.task import react from twisted.internet.defer import ensureDeferred from twisted.internet.endpoints import UNIXClientEndpoint import treq import txtorcon async def main(reactor): tor= await txtorcon.connect( reactor, UNIXClientEndpoint(reactor,"/var/run/tor/control") )

print("Connected to Tor version{}".format(tor.version))

url=u'https://www.torproject.org:443' print(u"Downloading{}".format(repr(url))) resp= await treq.get(url, agent=tor.web_agent())

print(u"{} bytes".format(resp.length)) data= await resp.text() (continues on next page)

4 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

(continued from previous page) print(u"Got{} bytes:\n{}\n[...]{}".format( len(data), data[:120], data[-120:], ))

print(u"Creating a circuit") state= await tor.create_state() circ= await state.build_circuit() await circ.when_built() print(u" path:{}".format(" ->".join([r.ip forr in circ.path])))

print(u"Downloading meejah's public key via above circuit...") config= await tor.get_config() resp= await treq.get( u'https://meejah.ca/meejah.asc', agent=circ.web_agent(reactor, config.socks_endpoint(reactor)), ) data= await resp.text() print(data)

@react def _main(reactor): return ensureDeferred(main(reactor))

1.1.4 Known Users

• magic-wormhole “get things from one computer to another, safely” • Tahoe-LAFS a Free and Open encrypted distributed storage system • Crossbar.io a Free and Open distributed-systems (RPC and PubSub) protocol (called WAMP) router. Supports e2e-encrypted payloads. • txtorcon received a brief mention at 29C3 starting at 12:20 (or via youtube). • carml command-line utilities for Tor • foolscap RPC system inspired by Twisted’s built-in “Perspective Broker” package. • bwscanner next-gen bandwidth scanner for Tor network • unmessage Privacy enhanced instant messenger • APAF anonymous Python application framework • OONI the Open Observatory of Network Interference • exitaddr scan Tor exit addresses • txtorhttpproxy simple HTTP proxy in Twisted • bulb Web-based Tor status monitor • onionvpn “ipv6 to onion service virtual public network adapter” • torperf2 new Tor node network performance measurement service • torweb web-based Tor controller/monitor

1.1. Introduction 5 txtorcon Documentation, Release 20.0.0

• potator “A Tor-based Decentralized Virtual Private Network Application”

1.2 Installing txtorcon

1.2.1 Latest Release txtorcon is on PyPI and in Debian since jessie (thanks to Lunar and now irl!). So, one of these should work: • install latest release: pip install txtorcon • Debian or Ubuntu: apt-get install python-txtorcon • Watch an asciinema demo for an overview. Rendered documentation for the latest release is at txtorcon.readthedocs.org. What exists for release-notes are in “Releases”. If you’re still using wheezy, python-txtorcon is also in wheezy-backports. To install, do this as root:

# echo "deb http://ftp.ca.debian.org/debian/ wheezy-backports main" >> /etc/apt/

˓→sources.list # apt-get update # apt-get install python-txtorcon

It also appears txtorcon is in Gentoo but I don’t use Gentoo (if anyone has a shell-snippet that installs it, send a pull-request). I am told this package also needs a maintainer; see XXX. Installing the wheel ﬁles requires a recent pip and setuptools. At least on Debian, it is important to upgrade setuptools before pip. This procedure appears to work ﬁne: virtualenv foo . foo/bin/activate pip install--upgrade setuptools pip install--upgrade pip pip install path/to/txtorcon-*.whl

If you get an error like SyntaxError: invalid syntax on txtorcon/test/py3_torstate.py or similar, you have an out-of-date pip or setuptools; see above.

1.2.2 Compatibility txtorcon runs all tests cleanly under Python2, Python3 and PyPy on: • Debian: “squeeze”, “wheezy” and “jessie” • OS X: 10.4 (naif), 10.8 (lukas lueg), 10.9 (kurt neufeld) • Fedora 18 (lukas lueg) • FreeBSD 10 (enrique fynn) (needed to install “lsof”) • RHEL6 • Reports from other OSes appreciated.

6 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

1.2.3 Tor Conﬁguration

Using Tor’s cookie authentication is the most convenient way to connect; this proves that your user can read a cookie ﬁle written by Tor. To enable this, you’ll want to have the following options on in your torrc:

CookieAuthentication1 CookieAuthFileGroupReadable1

Note that “Tor BrowserBundle” is conﬁgured this way by default, on port 9151. If you want to use unix sockets to speak to tor (highly recommended) add this to your conﬁg (Debian is already set up like this):

ControlSocketsGroupWritable1 ControlSocket/var/run/tor/control

1.2.4 Source Code

Most people will use the code from https://github.com/meejah/txtorcon The canonical URI is http://timaq4ygg2iegci7. onion I sign tags with my public key (meejah.asc) • git clone https://github.com/meejah/txtorcon.git • torsocks git clone git://timaq4ygg2iegci7.onion/meejah/txtorcon.git Rendered documentation for the latest release is at txtorcon.readthedocs.org. See Contributions if you wish to contribute back to txtorcon :)

1.2.5 Development Environment

I like to set up my Python development like this:

$ git clone https://github.com/meejah/txtorcon.git $ echo "if you later fork it on github, do this:" $ git remote add -f github git+ssh://[email protected]//txtorcon.git $ cd txtorcon $ virtualenv venv $ source venv/bin/activate (venv)$ pip install --editable .[dev] # "dev" adds more deps, like Sphinx (venv)$ make doc (venv)$ make test (venv)$ tox # run all tests, in all supported configs

You can now edit code in the repository as normal. To submit a patch, the easiest way is to “clone” the txtorcon project, then “fork” on github and add a remote called “github” with your copy of the code to which you can push (git remote add -f github git+ssh://[email protected]// txtorcon.git). The -f is so you don’t have to run git fetch right after. Now, you can push a new branch you’ve made to GitHub with git push github branch-name and then examine it and open a pull-request. This will trigger Travis to run the tests, after which coverage will be produced (and a bot comments on the pull-request). If you require any more changes, the easiest thing to do is just commit them and push them. (If you know how, re-basing/re-arranging/squashing etc is nice to do too). See Contributions for more.

1.2. Installing txtorcon 7 txtorcon Documentation, Release 20.0.0

1.2.6 Integration Tests

There are a couple of simple integration tests using Docker in the integration/ directory; these make a debootstrap-built base image and then do the test inside containers cloned from this – no trusting https:// docker.io required. See integration/README for more information. If you’re on Debian, there’s a decent chance running make txtorcon-tester followed by make integration from the root of the checkout will work (the ﬁrst commands ultimately runs debootstrap and some apt commands besides docker things).

1.2.7 Dependencies / Requirements

These should have been installed by whichever method you chose above, but are listed here for completeness. You can get all the development requirements with e.g. pip install txtorcon[dev]. • twisted: txtorcon should work with any Twisted 11.1.0 or newer. Twisted 15.4.0+ works with Python3, and so does txtorcon (if you find something broken on Py3 please file a bug). • automat: “a library for concise, idiomatic Python expression of finite-state automata (particularly deterministic finite-state transducers).” • ipaddress: a standard module in Python3, but requires installing the backported package on Python2. • dev only: Sphinx if you want to build the documentation. In that case you’ll also need something called python-repoze.sphinx.autointerface (at least in Debian) to build the Interface-derived docs properly. • dev only: coverage to run the code-coverage metrics. • dev only cuv’ner for coverage visualization • dev only: Tox to run different library revisions. • dev optional: GraphViz is used in the tests (and to generate state-machine diagrams, if you like) but those tests are skipped if “dot” isn’t in your path In any case, on a Debian wheezy, squeeze or Ubuntu system, this should work (as root):

# apt-get install -y python-setuptools python-twisted python-ipaddress graphviz tor # echo "for development:" # apt-get install -y python-sphinx python-repoze.sphinx.autointerface python-coverage

˓→libgeoip-dev

Using pip this would be:

$ pip install --user Twisted ipaddress pygeoip $ echo "for development:" $ pip install --user GeoIP Sphinx repoze.sphinx.autointerface coverage

or:

$ pip install -r requirements.txt $ pip install -r dev-requirements.txt

1.3 Programming Guide

8 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

• API Stability • High Level Overview • A Tor Instance – Connecting to a Running Tor – Launching a New Tor • A Note On Style • Tracking and Changing Tor’s Conﬁguration • Monitor and Change Tor’s State • Making Connections Over Tor – SOCKS5 • Onion (Hidden) Services – Onion Services Endpoints API – Creating Onion Endpoints – Non-Authenticated Services – Authenticated Services – Onion Service Conﬁguration • Custom Circuits – High Level – Low Level • Building Your Own Circuits – Building a Single Circuit – Building Many Circuits

1.3.1 API Stability

In general, any method or class preﬁxed with an underscore (like _method or _ClassName) is private, and the API may change at any time. You SHOULD NOT use these. Any method in an interface class (which all begin with I, like IAnInterface) are stable, public APIs and will maintain backwards-compatibility between releases. There is one exception to this at the moment: the hidden- / onion- services APIs are NOT yet considered stable, and may still change somewhat. Any APIs that will go away will ﬁrst be deprecated for at least one major release before being removed. There are also some attributes which don’t have underscores but really should; these will get “deprecated” via an @property decorator so your code will still work.

1.3.2 High Level Overview

Interacting with Tor via txtorcon should involve only calling methods of the Tor class. You get an instance of Tor in one of two ways:

1.3. Programming Guide 9 txtorcon Documentation, Release 20.0.0

• call txtorcon.connect() or; • call txtorcon.launch() Once you’ve got a Tor instance you can use it to gain access to (or create) instances of the other interesting classes; see “A Tor Instance” below for various use-cases. Note that for historical reasons (namely: Tor is a relatively new class) there are many other functions and classes exported from txtorcon but you shouldn’t need to instantiate these directly. If something is missing from this top-level class, please get in touch (ﬁle a bug, chat on IRC, etc) because it’s probably a missing feature.

1.3.3 A Tor Instance

You will need a connection to a Tor instance for txtorcon to control. This can be either an already-running Tor that you’re authorized to connect to, or a Tor instance that has been freshly launched by txtorcon. We abstract “a Tor instance” behind the Tor class, which provides a very high-level API for all the other things you might want to do: • make client-type connections over tor (see “Making Connections Over Tor”); • change its configuration (see “Tracking and Changing Tor’s Configuration”); • monitor its state (see “Monitor and Change Tor’s State”); • offer hidden-/onion- services via Tor (see “Onion (Hidden) Services”); • create and use custom circuits (see “Custom Circuits”); • issue low-level commands (see “Low-Level Protocol Classes”) The actual control-protocol connection to tor is abstracted behind TorControlProtocol. This can usually be ignored by most users, but can be useful to issue protocol commands directly, listen to raw events, etc. In general, txtorcon tries to never look at Tor’s version and instead queries required information directly via the control- protocol (there is only one exception to this). So the names of configuration values and events may change (or, more typically, expand) depending on what version of Tor you’re connected to.

Connecting to a Running Tor

Tor can listen for control connections on TCP ports or UNIX sockets. See “Tor Configuration” for information on how to configure Tor to work with txtorcon. By default, “COOKIE” authentication is used; only if that is not available do we try password authentication. To connect, use txtorcon.connect() which returns a Deferred that will fire with a Tor instance. If you need access to the TorControlProtocol instance, it’s available via the .protocol property (there is always exactly one of these per Tor instance). Similarly, the current configuration is available via .get_config (which returns a Deferred firing a TorConfig). You can change the configuration by updating attributes on this class but it won’t take effect until you call TorConfig.save().

Launching a New Tor

It’s also possible to launch your own Tor instance. txtorcon keeps a “global” tor available for use by e.g. the . global_tor endpoint factory functions (like TCPHiddenServiceEndpoint.global_tor()). You can access it via get_global_tor_instance(). There is exactly zero or one of these per Python process that uses txtorcon.

10 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

To explicitly launch your own Tor instance, use launch(). You can pass a couple of minimal options (data_directory being recommended). If you need to set other Tor options, use .config to retrieve the TorConfig instance associated with this tor and change conﬁguration afterwards. Setting data_directory gives your Tor instance a place to cache its state information which includes the current “consensus” document. If you don’t set it, txtorcon creates a temporary directory (which is deleted when this Tor instance exits). Startup time is drammatically improved if Tor already has a recent consensus, so when integrating with Tor by launching your own client it’s highly recommended to specify a data_directory somewhere sen- sible (e.g. ~/.config/your_program_name/ is a popular choice on Linux). See the Tor manual under the DataDirectory option for more information. Tor itself will create a missing data_directory with the correct permissions and Tor will also chdir into its DataDirectory when running. For these reasons, txtorcon doesn’t try to create the data_directory nor do any chdir-ing, and neither should you.

1.3.4 A Note On Style

Most of txtorcon tends towards “attribute-style access”. The guiding principle is that “mere data” that is immediately available will be an attribute, whereas things that “take work” or are async (and thus return Deferred s) will be functions. For example, Router.get_location() is a method because it potentially has to ask Tor for the country, whereas Router.hex_id is a plain attribute because it’s always available.

1.3.5 Tracking and Changing Tor’s Conﬁguration

Instances of the TorConfig class represent the current, live state of a running Tor. There is a bit of attribute-magic to make it possible to simply get and set things easily:

tor= launch(..) print("SOCKS ports: {}".format(tor.config.SOCKSPort)) tor.config.ControlPort.append(4321) tor.config.save()

Only when .save() is called are any SETCONF commands issued – and then, all configuration values are sent in a single command. All TorConfig instances subscribe to configuration updates from Tor, so “live state” includes actions by any other controllers that may be connected. For some configuration items, the order they’re sent to Tor matters. Sometimes, if you change one config item, you have to set a series of related items. TorConfig handles these cases for you – you just manipulate the configuration, and wait for .save() ‘s Deferred to fire and the running Tor’s configuration is updated. Note there is a tiny window during which the state may appear slightly inconsistent if you have multiple TorConfig instances: after Tor has acknowledged a SETCONF command, but before a separate TorConfig instance has gotten all the CONF_CHANGED events (because they’re hung up in the networking stack for some reason). This shouldn’t concern most users. (I’m not even 100% sure this is possible; it may be that Tor doesn’t send the OK until after all the CONF_CHANGED events). In normal use, there should only be a single TorConfig instance for every Tor instance so this shouldn’t affect you unless you’ve created your own TorConfig. Since TorConfig conforms to the Iterator protocol, you can easily find all the config-options that Tor supports:

tor= launch(..) for config_key in tor.config: print("{} has value: {}".format(config_key, getattr(tor.config.config_key)))

These come from interrogating Tor using GETINFO config/names and so represent the conﬁguration options of the current connected Tor process. If the value “isn’t set” (i.e. is the default), the value from Tor will be . DEFAULT_VALUE.

1.3. Programming Guide 11 txtorcon Documentation, Release 20.0.0

When you set values into TorConfig, they are parsed according to control-spec for the different types given to the values, via information from GETINFO config/names. So, for example, setting .SOCKSPort to a "quux" won’t work. Of course, it would also fail the whole SETCONF command if txtorcon happens to allow some values that Tor doesn’t. Unfortunately, for any item that’s a list, Tor doesn’t tell us anything about each element so they’re all strings. This means we can’t pre-validate them and so some things may not fail until you call .save().

1.3.6 Monitor and Change Tor’s State

Instances of TorState prepresent a live, interactive version of all the relays/routers (Router instances), all circuits (Circuit instances) and streams (Stream instances) active in the underlying Tor instance. As the TorState instance has subscribed to various events from Tor, the “live” state represents an “as up-to-date as possible” view. This includes all other controlers, Tor Browser, etcetera that might be interacting with your Tor client. A Tor instance doesn’t have a TorState instance by default (it can take a few hundred milliseconds to set up) and so one is created via the asynchronous method Tor.get_state().

Note: If you need to be absolutely sure there’s nothing stuck in networking buffers and that your instance is “deﬁ- nitely up-to-date” you can issue a do-nothing command to Tor via TorControlProtocol.queue_command() (e.g. yield queue_command("GETINFO version")). Most users shouldn’t have to worry about this edge- case. In any case, there could be a new update that Tor decides to issue at any moment.

You can modify the state of Tor in a few simple ways. For example, you can call Stream.close() or Circuit. close() to cause a stream or circuit to be closed. You can wait for a circuit to become usable with Circuit. when_built(). For a lot of the read-only state, you can simply access interesting attributes. The relays through which a circuit traverses are in Circuit.path (a list of Router instances), Circuit.streams contains a list of Stream instances, .state and .purpose are strings. .time_created returns a datetime instance. There are also some convenience functions like Circuit.age(). For sending streams over a particular circuit, Circuit.stream_via() returns an IStreamClientEndpoint implementation that will cause a subsequent .connect() on it to go via the given circuit in Tor. A similar method (Circuit.web_agent()) exists for Web requests. Listening for certain events to happen can be done by implementing the interfaces interface. IStreamListener and interface.ICircuitListener. You can request notiﬁcations on a Tor-wide basis with TorState.add_circuit_listener() or TorState.add_stream_listener(). If you are just interested in a single circuit, you can call Circuit.listen() directly on a Circuit instance. You can instead use methods (which also function as decorators) such as TorState.on_circuit_launched() or TorState.on_stream_closed() to add listeners for single events. The Tor relays are abstracted with Router instances. Again, these have read-only attributes for interesting information, e.g.: id_hex, ip, flags (a list of strings), bandwidth, policy, etc. Note that all information in these objects is from “microdescriptors”. If you’re doing a long-running iteration over relays, it may be important to remember that the collection of routers can change every hour (when a new “consensus” from the Directory Authorities is published) which may change the underlying collection (e.g. TorState.routers_by_hash) over which you’re iterating. Here’s a simple sketch that traverses all circuits printing their router IDs, and closing each stream and circuit afterwards:

from twisted.internet.task import react from twisted.internet.defer import inlineCallbacks from twisted.internet.endpoints import UNIXClientEndpoint (continues on next page)

12 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

(continued from previous page)

import txtorcon

@react @inlineCallbacks def main(reactor): """ Close all open streams and circuits in the Tor we connect to """ control_ep= UNIXClientEndpoint(reactor,'/var/run/tor/control') tor= yield txtorcon.connect(reactor, control_ep) state= yield tor.create_state() print("Closing all circuits:") for circuit in list(state.circuits.values()): path='->'.join(map(lambdar:r.id_hex, circuit.path)) print("Circuit{} through{}".format(circuit.id, path)) for stream in circuit.streams: print(" Stream{} to{}".format(stream.id, stream.target_host)) yield stream.close() print(" closed") yield circuit.close() print("closed") yield tor.quit()

1.3.7 Making Connections Over Tor

SOCKS5

Tor exposes a SOCKS5 interface to make client-type connections over the network. There are also a couple of custom extensions Tor provides to do DNS resolution over a Tor circuit (txtorcon supports these, too). All client-side interactions are via instances that implement IStreamClientEndpoint. There are several factory functions used to create suitable instances. The recommended API is to acquire a Tor instance (see “A Tor Instance”) and then call Tor. create_client_endpoint(). To do DNS lookups (or reverse lookups) via a Tor circuit, use Tor. dns_resolve() and Tor.dns_resolve_ptr(). A common use-case is to download a Web resource; you can do so via Twisted’s built-in twisted.web.client package, or using the friendlier treq library. In both cases, you need a twisted.web.client.Agent instance which you can acquire with Tor.web_agent() or Circuit.web_agent(). The latter is used to make the request over a speciﬁc circuit. Usually, txtorcon will simply use one of the available SOCKS ports conﬁgured in the Tor it is connected to – if you care which one, you can specify it as the optional _socks_endpoint= argument (this starts with an underscore on purpose as it’s not recommended for “public” use and its semantics might change in the future).

Note: Tor supports SOCKS over Unix sockets. So does txtorcon. To take advantage of this, simply pass a valid SocksPort value for unix sockets (e.g. unix:/tmp/foo/socks) as the _socks_endpoint argument to either web_agent() call. If this doesn’t already exist in the underlying Tor, it will be added. Tor has particular requirements for the directory in which the socket ﬁle is (0700). We don’t have a way (yet?) to auto-discover if the Tor we’re connected to can support Unix sockets so the default is to use TCP.

You can also use Twisted’s clientFromString API as txtorcon registers a tor: plugin. This also implies that any Twisted-using program that supports conﬁguring endpoint strings gets Tor support “for free”. For example, passing a

1.3. Programming Guide 13 txtorcon Documentation, Release 20.0.0 string like tor:timaq4ygg2iegci7.onion:80 to clientFromString will return an endpoint that will connect to txtorcon’s onion-service website. Note that these endpoints will use the “global to txtorcon” Tor instance (available from get_global_tor()). Thus, if you want to control which tor instance your circuit goes over, this is not a suitable API. There are also lower-level APIs to create TorClientEndpoint instances directly if you have a TorConfig instance. These very APIs are used by the Tor object mentioned above. If you have a use-case that requires using this API, I’d be curious to learn why the Tor methods are un-suitable (as those are the suggested API). You should expect these APIs to raise SOCKS5 errors, which can all be handled by catching the socks. SocksError class. If you need to work with each speciﬁc error (corresponding to the RFC-speciﬁed SOCKS5 replies), see the “txtorcon.socks Module” for a list of them.

1.3.8 Onion (Hidden) Services

An “Onion Service” (also called a “Hidden Service”) refers to a feature of Tor allowing servers (e.g. a Web site) to be availble via Tor. These bring additional security properties such as: • hiding the server’s network location; • providing end-to-end encryption; • self-certifying domain-names; • NAT penetration (connections to Tor network are client-like); • or offering authentication. For details of how this works, please read Tor’s documentation on Hidden Services. For more background, the RiseUp Onion service best-practices guide is a good read as well. In the newest Tor versions, Onion services have been upgraded (“Proposition 279”) and these are known as “version 3” services. The prior / legacy is “version 2”. In txtorcon, the default version is usually 3 but in some older APIs the default has to remain “2” if unspeciﬁed (for backwards-compatibility). You should thus usually pass version=3 (and if you expect to connect to an older Tor release, pass version=2).

Note: In some places there will be “Hidden” in a classname; these are typically for backwards-compatilibity reasons. “Onion service” is the preferred name.

From an API perspective, here are the parts we care about: • each service has a secret, private key (with a corresponding public part); – these keys can be on disk (in the “hidden service directory”); – or, they can be “ephemeral” (only in memory); • the “host name” is a hash of the public-key (e.g. timaq4ygg2iegci7.onion); • a “Descriptor” (which tells clients how to connect) must be published (to a “Hidden Service Directory”, or HSDir); • a service has a list of port-mappings (public -> local): – e.g. "80 127.0.0.1:5432" says you can contact the service publically on port 80, which Tor will redirect to a daemon running locally on port 5432; – note that “Descriptors” only show the public port • services can be “authenticated”, which means they have a list of client names for which Tor creates associated keys (.auth_token).

14 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

• Tor has two ﬂavours of service authentication: basic and stealth – there’s no API-level difference, but the .hostname is unique for each client in the stealth case. To summarize the above in a table format, here are the possible types of Onion Service interfaces classes you may interact with.

Keys on disk Keys in memory no authentication txtorcon. txtorcon.IOnionService IFilesystemOnionService basic/stealth authenti- txtorcon. txtorcon. cation IAuthenticatedOnionClients IAuthenticatedOnionClients

txtorcon.IFilesystemOnionService is a subclass of txtorcon.IOnionService and the concrete objects will be different for on-disk versus in-memory keys; depend on the methods in the interfaces (only). Note that it’s up to you to save the private keys of ephemeral services if you want to re-launch them later; the “ephemeral” refers to the fact that Tor doesn’t persist the private keys – when Tor shuts down, they’re gone and there will never be a service at the same URI again (unless you saved the key).

Onion Services Endpoints API

No matter which kind of service you need, you interact via Twisted’s IStreamServerEndpoint interface. There are various txtorcon methods (see “Creating Onion Endpoints”) which return some instance implementing that interface. These instances will also implement txtorcon.IProgressProvider – which is a hook to register listeners which get updates about Tor’s launching progress (if we started a new Tor) and Descriptor uploading. Fundamentally, “authenticated” services are different from non-authenticated services because they have a list of clients. Services on-disk are “slightly” different because the user may need to know the “hidden service dir” that contains the private keys. However, there is a single endpoint which takes enough options to produce any kind of onion service. The service instance you retrieve after the .listen() call will, however, be different and implement one of the interfaces in the table above. Those are: • txtorcon.IOnionService • txtorcon.IFilesystemOnionService (also includes all of IOnionService) • txtorcon.IAuthenticatedOnionClients (for authenticated services) The .listen() method of the endpoint will return an instance implementing IListeningPort. This will have a .onion_service property that gives you an instance implementing one of the above interfaces. txtorcon.IOnionService and its subclass txtorcon.IFilesystemOnionService correspond to a non-authenticated services, while txtorcon.IAuthenticatedOnionClients is authenticated. The latter manages a collection of instances by (arbitrary) client names, where each of these instances implements txtorcon. IOnionClient. Note that the .auth_token member is secret, private data which you need to give to one client; this information goes in the client’s Tor conﬁguration as HidServAuth onion-address auth-cookie [service-name]. See the Tor manual for more information.

Creating Onion Endpoints

The easiest-to-use API are methods of Tor, which allow you to create IStreamServerEndpoint instances for the various Onion Service types. For all service types there is a single endpoint that you create: TCPHiddenServiceEndpoint. Thus, you are advised to use a factory-method to create the instance It’s also possible to use Twisted’s serverFromString API with the onion: preﬁx. (Thus, any program support- ing endpoint strings for conﬁguration can use Tor Onion Services with no code changes).

1.3. Programming Guide 15 txtorcon Documentation, Release 20.0.0

Each of the four main classes of onion service has a corresponding factory method (while these get nearly to Java lengths, these are at least explicit): • Tor.create_onion_endpoint(): ephemeral service • Tor.create_authenticated_onion_endpoint(): ephemeral service with authentication • Tor.create_filesystem_onion_endpoint(): on-disk service • Tor.create_authenticated_filesystem_onion_endpoint(): on-disk service with authentication Factors to consider when deciding whether to use “authenticated” service or not: • if you want anyone with e.g. the URL http://timaq4ygg2iegci7.onion to be able to put it in Tor Browser Bundle and see a Web site, you do not want authentication; • if you want only people with the URL and a secret authentication token to see the Web site, you want basic authentication (these support many more clients than stealth auth); • if you don’t even want anyone to be able to decrypt the descriptor without a unique URL and a secret authentication token, you want stealth authentication (a lot less scalable; for only “a few” clients – less than 16 in latest Tor).

Non-Authenticated Services

You can create non-authenticated with Tor.create_onion_service() (for an ephemeral service) or Tor. create_filesystem_onion_service() (for an on-disk service). If you don’t want to manage launching or connecting to Tor yourself (and thus a Tor instance), you can use one of the three factory-functions in TCPHiddenServiceEndpoint, which all return a new endpoint instance: • TCPHiddenServiceEndpoint.global_tor(): uses a Tor instance launched at most once in this Python process (the underlying Tor instance for this is available via get_global_tor() if you need to make manual conﬁguration adjustments); • TCPHiddenServiceEndpoint.system_tor(): connects to the control-protocol endpoint you provide (a good choice on Debian would be UNIXClientEndpoint('/var/run/tor/control')); • TCPHiddenServiceEndpoint.private_tor(): causes a fresh, private instance of Tor to be launched for this service alone. This uses a tempdir (honoring $TMP) which is deleted upon reactor shutdown or loss of the control connection. Note that nothing actually “happens” until you call .listen() on the IStreamServerEndpoint at which point Tor will possibly be launched, the Onion Service created, and the descriptor published.

Authenticated Services

Authenticated services take an instance of AuthBasic or AuthStealth. You may use the factory methods on Tor: Tor.create_authenticated_onion_service() (for an ephemeral service) or Tor. create_authenticated_filesystem_onion_service() (for an on-disk service). You may also use one of the three @classmethod-s on TCPHiddenServiceEndpoint (and passing an auth= kwarg): • TCPHiddenServiceEndpoint.global_tor() • TCPHiddenServiceEndpoint.system_tor() • TCPHiddenServiceEndpoint.private_tor()

16 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

Onion Service Conﬁguration

If you just want to “look at” the configuration of existing onion services, they are avaialble via TorConfig and the .HiddenServices or .EphemeralHiddenServices attributes. These presents a “flattened” version of any authenticated services, so that each element in the list of . HiddenServices is itself at least a txtorcon.IOnionService (it may also implement other interfaces, but every one will implement IOnionService). You can still set any settable attributes on these objects, and Tor’s configuration for them will be updated when you call TorConfig.save() with an important exception: “ephemeral” services cannot be updated after they’re created. Note that it’s possible for other controllers to create ephemeral services that Tor doesn’t allow your controller to enumerate.

1.3.9 Custom Circuits

Tor provides a way to let controllers like txtorcon decide which streams go on which circuits. Since your Tor client will then be acting differently from a “normal” Tor client, it may become easier to de-anonymize you.

High Level

With that in mind, you may still decide to attach streams to circuits. Most often, this means you simply want to make a client connection over a particluar circuit. The recommended API uses Circuit.stream_via() for arbitrary protocols or Circuit.web_agent() as a convenience for Web connections. The latter can be used via Twisted’s Web client or via treq (a “requests”-like library for Twisted). See the following examples: • web_client.py • web_client_treq.py • web_client_custom_circuit.py Note that these APIs mimic Tor.stream_via() and Tor.web_agent() except they use a particular Circuit.

Low Level

Under the hood of these calls, txtorcon provides a low-level interface directly over top of Tor’s circuit-attachment API. This works by: • setting __LeaveStreamsUnattached 1 in the Tor’s conﬁguration • listening for STREAM events • telling Tor (via ATTACHSTREAM) what circuit to put each new stream on • (we can also choose to tell Tor “attach this one however you normally would”) This is an asynchronous API (i.e. Tor isn’t “asking us” for each stream) so arbitrary work can be done on a per-stream basis before telling Tor which circuit to use. There are two limitations though: • Tor doesn’t play nicely with multiple controllers playing the role of attaching circuits. Generally, there’s not a good way to know if there’s another controller trying to attach streams, but basically the ﬁrst one to answer “wins”. • Tor doesn’t currently allow controllers to attach circuits destined for onion-services (even if the circuit is actually suitable and goes to the correct Introduction Point).

1.3. Programming Guide 17 txtorcon Documentation, Release 20.0.0

In order to do custom stream -> circuit mapping, you call TorState.set_attacher() with an object implementing interface.IStreamAttacher. Then every time a new stream is detected, txtorcon will call interface. IStreamAttacher.attach_stream() with the Stream instance and a list of all available circuits. You make an appropriate return. There can be either no attacher at all or a single attacher object. You can “un-set” an attacher by calling set_attacher(None) (in which case __LeaveStreamsUnattached will be set back to 0). If you really do need multiple attachers, you can use the utility class attacher.PriorityAttacher which acts as the “top level” one (so you add your multiple attachers to it). Be aware that txtorcon internally uses this API itself if you’ve ever called the “high level” API (Circuit. stream_via() or Circuit.web_agent()) and so it is an error to set a new attacher if there is already an existing attacher.

1.3.10 Building Your Own Circuits

To re-iterate the warning above, making your own circuits differently from how Tor normally does runs a high risk of de-anonymizing you. That said, you can build custom circuits using txtorcon.

Building a Single Circuit

If your use-case needs just a single circuit, it is probably easiest to call TorState.build_circuit(). This methods takes a list of Router instances, which you can get from the TorState instance by using one of the attributes: • .all_routers • .routers • .routers_by_name or • .routers_by_hash The last three are all dicts. For relays that have the Guard ﬂag, you can access the dicts .guards (for all of them) or .entry_guards (for just the entry guards conﬁgured on this Tor client). If you don’t actually care which relays are used, but simply want a fresh circuit, you can call TorState. build_circuit() without any arguments at all which asks Tor to build a new circuit in the way it normally would (i.e. respecting your guard nodes etc). There is also build_timeout_circuit() as a convenience method if you wish the attempt to time out after a while.

Building Many Circuits

Caution: This API doesn’t exist yet; this is documenting what may become a new API in a future version of txtorcon. Please get in touch if you want this now.

If you would like to build many circuits, you’ll want an instance that implements txtorcon.ICircuitBuilder (which is usually simply an instance of CircuitBuilder). Instances of this class can be created by calling one of the factory functions like circuit_builder_fixed_exit(). XXX what about a “conﬁg object” idea, e.g. could have keys:

18 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

• guard_selection: one of entry_only (use one of the current entry guards) or random_guard (use any relay with the Guard ﬂag, selected by XXX). • middle_selection: one of uniform (selected randomly from all relays), weighted (selected randomly, but weighted by consensus weight – basically same way as Tor would select).

1.4 Examples

The examples are grouped by functionality and serve as mini-HOWTOs – if you have a use-case that is missing, it may be useful to add an example, so please ﬁle a bug. All ﬁles are in the examples/ sub-directory and are ready to run, usually with defaults designed to work with Tor Browser Bundle (localhost:9151). The examples use default_control_port() to determine how to connect which you can override with an environment variable: TX_CONTROL_PORT. So e.g. export TX_CONTROL_PORT=9050 to run the examples again a system-wide Tor daemon.

• Web: clients – web_client.py – web_client_treq.py – web_client_custom_circuit.py • Starting Tor – launch_tor.py – launch_tor_endpoint.py • Circuits and Streams – disallow_streams_by_port.py – stream_circuit_logger.py • Events – monitor.py • Miscellaneous – stem_relay_descriptor.py – txtorcon.tac

1.4.1 Web: clients web_client.py

Download the example. Uses twisted.web.client to download a Web page using a twisted.web.client.Agent, via any circuit Tor chooses.

1.4. Examples 19 txtorcon Documentation, Release 20.0.0

# this example shows how to use Twisted's web client with Tor via # txtorcon from __future__ import print_function from twisted.internet.defer import inlineCallbacks from twisted.internet.task import react from twisted.internet.endpoints import TCP4ClientEndpoint from twisted.web.client import readBody import txtorcon from txtorcon.util import default_control_port

@react @inlineCallbacks def main(reactor): # use port 9051 for system tor instances, or: # ep = UNIXClientEndpoint(reactor, '/var/run/tor/control') # ep = UNIXClientEndpoint(reactor, '/var/run/tor/control') ep= TCP4ClientEndpoint(reactor,'127.0.0.1', default_control_port()) tor= yield txtorcon.connect(reactor, ep) print("Connected to {tor} via localhost:{port}".format( tor=tor, port=default_control_port(), ))

# create a web.Agent that will talk via Tor. If the socks port # given isn't yet configured, this will do so. It may also be # None, which means "the first configured SOCKSPort" # agent = tor.web_agent(u'9999') agent= tor.web_agent() uri=b'http://surely-this-has-not-been-registered-and-is-invalid.com' uri=b'https://www.torproject.org' uri=b'http://timaq4ygg2iegci7.onion/' # txtorcon documentation print("Downloading{}".format(uri)) resp= yield agent.request(b'GET', uri)

print("Response has{} bytes".format(resp.length)) body= yield readBody(resp) print("received body ({} bytes)".format(len(body))) print("{}\n[...]\n{}\n".format(body[:200], body[-200:])) web_client_treq.py

Download the example. Uses treq to download a Web page via Tor.

# just copying over most of "carml checkpypi" because it's a good # example of "I want a stream over *this* circuit". from __future__ import print_function from twisted.internet.defer import inlineCallbacks from twisted.internet.task import react (continues on next page)

20 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

(continued from previous page) from twisted.internet.endpoints import TCP4ClientEndpoint import txtorcon from txtorcon.util import default_control_port try: import treq except ImportError: print("To use this example, please install'treq':") print("pip install treq") raise SystemExit(1)

@react @inlineCallbacks def main(reactor): ep= TCP4ClientEndpoint(reactor,'127.0.0.1', default_control_port()) # ep = UNIXClientEndpoint(reactor, '/var/run/tor/control') tor= yield txtorcon.connect(reactor, ep) print("Connected:", tor)

resp= yield treq.get( 'https://www.torproject.org:443', agent=tor.web_agent(), )

print("Retrieving{} bytes".format(resp.length)) data= yield resp.text() print("Got{} bytes:\n{}\n[...]{}".format( len(data), data[:120], data[-120:], )) web_client_custom_circuit.py

Download the example. Builds a custom circuit, and then uses twisted.web.client to download a Web page using the circuit created.

# this example shows how to use specific circuits over Tor (with # Twisted's web client or with a custom protocol) # # NOTE WELL: this functionality is for advanced use-cases and if you # do anything "special" to select your circuit hops you risk making it # easy to de-anonymize this (and all other) Tor circuits. from __future__ import print_function from twisted.internet.protocol import Protocol, Factory from twisted.internet.defer import inlineCallbacks, Deferred from twisted.internet.task import react from twisted.internet.endpoints import TCP4ClientEndpoint from twisted.web.client import readBody

(continues on next page)

1.4. Examples 21 txtorcon Documentation, Release 20.0.0

(continued from previous page) import txtorcon from txtorcon.util import default_control_port

@react @inlineCallbacks def main(reactor): # use port 9051 for system tor instances, or: # ep = UNIXClientEndpoint(reactor, '/var/run/tor/control') ep= TCP4ClientEndpoint(reactor,'127.0.0.1', default_control_port()) tor= yield txtorcon.connect(reactor, ep) print("Connected:", tor)

config= yield tor.get_config() state= yield tor.create_state() socks= config.socks_endpoint(reactor)

# create a custom circuit; in this case we're just letting Tor # decide the path but you *can* select a path (again: for advanced # use cases that will probably de-anonymize you) circ= yield state.build_circuit() print("Building a circuit:", circ)

# at this point, the circuit will be "under way" but may not yet # be in BUILT state -- and hence usable. So, we wait. (Just for # demo purposes: the underlying connect will wait too) yield circ.when_built() print("Circuit is ready:", circ)

if True: # create a web.Agent that will use this circuit (or fail) agent= circ.web_agent(reactor, socks)

uri='https://www.torproject.org' print("Downloading{}".format(uri)) resp= yield agent.request('GET', uri)

print("Response has{} bytes".format(resp.length)) body= yield readBody(resp) print("received body ({} bytes)".format(len(body))) print("{}\n[...]\n{}\n".format(body[:200], body[-200:]))

if True: # make a plain TCP connection to a thing ep= circ.stream_via(reactor,'torproject.org', 80, config.socks_

˓→endpoint(reactor))

d= Deferred()

class ToyWebRequestProtocol(Protocol):

def connectionMade(self): print("Connected via{}".format(self.transport.getHost())) self.transport.write( 'GET http://torproject.org/ HTTP/1.1\r\n' 'Host: torproject.org\r\n' '\r\n' (continues on next page)

22 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

(continued from previous page) )

def dataReceived(self,d): print(" received{} bytes".format(len(d)))

def connectionLost(self, reason): print("disconnected:{}".format(reason.value)) d.callback(None)

yield ep.connect(Factory.forProtocol(ToyWebRequestProtocol)) # returns "proto

˓→" yieldd print("All done, closing the circuit") yield circ.close()

1.4.2 Starting Tor launch_tor.py

Download the example. Launch a new Tor instance. This takes care of setting Tor’s notion ownership so that when the control connection goes away the running Tor exits. from __future__ import print_function

""" Launch a private Tor instance. """ import sys import txtorcon from twisted.web.client import readBody from twisted.internet.task import react from twisted.internet.defer import inlineCallbacks

@react @inlineCallbacks def main(reactor): # note that you can pass a few options as kwargs # (e.g. data_directory=, or socks_port= ). For other torrc # changes, see below. tor= yield txtorcon.launch( reactor, data_directory="./tordata", stdout=sys.stdout, socks_port='unix:/tmp/tor2/socks', ) # tor = yield txtorcon.connect( # reactor, # clientFromString(reactor, "unix:/var/run/tor/control"), #) print("Connected to Tor version'{}'".format(tor.protocol.version))

config= yield tor.get_config() (continues on next page)

1.4. Examples 23 txtorcon Documentation, Release 20.0.0

(continued from previous page) state= yield tor.create_state() # or state = yield txtorcon.TorState.from_protocol(tor.protocol)

print("This Tor has PID{}".format(state.tor_pid)) print("This Tor has the following{} Circuits:".format(len(state.circuits))) forc in state.circuits.values(): print("{}".format(c))

endpoint_d= config.socks_endpoint(reactor,u'unix:/tmp/tor2/socks') agent= tor.web_agent(socks_endpoint=endpoint_d) uri=b'https://www.torproject.org' print("Downloading{}".format(uri)) resp= yield agent.request(b'GET', uri) print("Response has{} bytes".format(resp.length)) body= yield readBody(resp) print("received body ({} bytes)".format(len(body))) print("{}\n[...]\n{}\n".format(body[:200], body[-200:]))

# SOCKSPort is 'really' a list of SOCKS ports in Tor now, so we # have to set it to a list ... :/ print("Changing our config (SOCKSPort=9876)") # config.SOCKSPort = ['unix:/tmp/foo/bar'] config.SOCKSPort=['9876'] yield config.save()

print("Querying to see it changed:") socksport= yield tor.protocol.get_conf("SOCKSPort") print("SOCKSPort", socksport) launch_tor_endpoint.py

Download the example. Using the txtorcon.TCP4HiddenServiceEndpoint class to start up a Tor with a hidden service pointed to an IStreamServerEndpoint. from __future__ import print_function

# Here we set up a Twisted Web server and then launch our own tor with # a configured hidden service directed at the Web server we set # up. This uses serverFromString to translate the "onion" endpoint # descriptor into a TCPHiddenServiceEndpoint object... from twisted.web import server, resource from twisted.internet.defer import inlineCallbacks from twisted.internet.task import react, deferLater from twisted.internet.endpoints import serverFromString import txtorcon class Simple(resource.Resource): """ A really simple Web site. """ isLeaf= True

(continues on next page)

24 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

(continued from previous page) def render_GET(self, request): return"Hello, world! I'm a hidden service!"

@react @inlineCallbacks def main(reactor): # several ways to proceed here and what they mean: # # "onion:80": # launch a new Tor instance, configure a hidden service on some # port and pubish descriptor for port 80 # # "onion:80:controlPort=9051:localPort=8080:socksPort=9089:hiddenServiceDir=/home/

˓→human/src/txtorcon/hidserv": # connect to existing Tor via control-port 9051, configure a hidden # service listening locally on 8080, publish a descriptor for port # 80 and use an explicit hiddenServiceDir (where "hostname" and # "private_key" files are put by Tor). We set SOCKS port # explicitly, too. # # "onion:80:localPort=8080:socksPort=9089:hiddenServiceDir=/home/human/src/

˓→txtorcon/hidserv": # all the same as above, except we launch a new Tor (because no # "controlPort=9051")

ep="onion:80:controlPort=9051:localPort=8080:socksPort=9089:hiddenServiceDir=/

˓→home/human/src/txtorcon/hidserv" ep="onion:80:localPort=8080:socksPort=9089:hiddenServiceDir=/home/human/src/

˓→txtorcon/hidserv" ep="onion:80" hs_endpoint= serverFromString(reactor, ep)

def progress(percent, tag, message): bar= int(percent/ 10) print("[{}{}]{}".format("#" * bar,"." * (10- bar), message)) txtorcon.IProgressProvider(hs_endpoint).add_progress_listener(progress)

# create our Web server and listen on the endpoint; this does the # actual launching of (or connecting to) tor. site= server.Site(Simple()) port= yield hs_endpoint.listen(site) # XXX new accessor in newer API hs= port.onion_service

# "port" is an IAddress implementor, in this case TorOnionAddress # so you can get most useful information from it -- but you can # also access .onion_service (see below) print( "I have set up a hidden service, advertised at:\n" "http://{host}:{port}\n" "locally listening on {local_address}\n" "Will stop in 60 seconds...".format( host=port.getHost().onion_uri, # or hs.hostname port=port.public_port, # port.local_address will be a twisted.internet.tcp.Port # or a twisted.internet.unix.Port -- both have .getHost() (continues on next page)

1.4. Examples 25 txtorcon Documentation, Release 20.0.0

(continued from previous page) local_address=port.local_address.getHost(), ) )

# if you prefer, hs (port.onion_service) is an instance providing # IOnionService (there's no way to do authenticated services via # endpoints yet, but if there was then this would implement # IOnionClients instead) print("private key:\n{}".format(hs.private_key))

def sleep(s): return deferLater(reactor,s, lambda: None)

yield sleep(50) fori in range(10): print("Stopping in{}...".format(10-i)) yield sleep(1)

1.4.3 Circuits and Streams disallow_streams_by_port.py

Download the example. An example using IStreamAttacher which is very simple and does just what it sounds like: never attaches Streams exiting to a port in the “disallowed” list (it also explicitly closes them). Note that Tor already has this feature; this is just to illustrate how to use IStreamAttacher and that you may close streams. XXX keep this one? from __future__ import print_function # # This uses a very simple custom txtorcon.IStreamAttacher to disallow # certain streams based solely on their port; by default it closes # all streams on port 80 or 25 without ever attaching them to a # circuit. # # For a more complex IStreamAttacher example, see # attach_streams_by_country.py # from twisted.python import log from twisted.internet.task import react from twisted.internet.defer import inlineCallbacks, Deferred from twisted.internet.endpoints import clientFromString from zope.interface import implementer import txtorcon

@implementer(txtorcon.IStreamAttacher) class PortFilterAttacher:

def __init__(self, state): self.state= state self.disallow_ports=[80, 25] print( (continues on next page)

26 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

(continued from previous page) "Disallowing all streams to ports: {ports}".format( ports=",".join(map(str, self.disallow_ports)), ) )

def attach_stream(self, stream, circuits): """ IStreamAttacher API """

def stream_closed(x): print("Stream closed:",x)

if stream.target_port in self.disallow_ports: print( "Disallowing {stream} to port {stream.target_port}".format( stream=stream, ) ) d= self.state.close_stream(stream) d.addCallback(stream_closed) d.addErrback(log.err) return txtorcon.TorState.DO_NOT_ATTACH

# Ask Tor to assign stream to a circuit by itself return None

@react @inlineCallbacks def main(reactor): control_ep= clientFromString(reactor,"tcp:localhost:9051") tor= yield txtorcon.connect(reactor, control_ep) print("Connected to a Tor version={version}".format( version=tor.protocol.version, )) state= yield tor.create_state() yield state.set_attacher(PortFilterAttacher(state), reactor)

print("Existing streams:") fors in state.streams.values(): print("",s) yield Deferred() stream_circuit_logger.py

Download the example. For listening to changes in the Circuit and State objects, this example is the easiest to understand as it just prints out (some of) the events that happen. Run this, then visit some Web sites via Tor to see what’s going on.

#!/usr/bin/env python

# This uses an IStreamListener and an ICircuitListener to log all # built circuits and all streams that succeed.

(continues on next page)

1.4. Examples 27 txtorcon Documentation, Release 20.0.0

(continued from previous page) from __future__ import print_function import sys from twisted.python import log from twisted.internet.task import react from twisted.internet.defer import inlineCallbacks, Deferred import txtorcon def log_circuit(circuit): path='->'.join(map(lambdax: str(x.location.countrycode), circuit.path)) log.msg('Circuit %d(%s) is %s for purpose"%s"'% (circuit.id, path, circuit.state, circuit.purpose)) def log_stream(stream): circ='' if stream.circuit: path='->'.join(map(lambdax: str(x.location.countrycode), stream.circuit.

˓→path)) circ=' via circuit %d(%s)'%(stream.circuit.id, path) proc= txtorcon.util.process_from_address( stream.source_addr, stream.source_port, ) if proc: proc=' from process"%s"'%(proc,)

elif stream.source_addr =='(Tor_internal)': proc=' for Tor internal use'

else: proc=' from remote"%s:%s"'%(str(stream.source_addr), str(stream.source_port)) log.msg('Stream %d to %s:%d attached%s%s'% (stream.id, stream.target_host, stream.target_port, circ, proc)) class StreamCircuitLogger(txtorcon.StreamListenerMixin, txtorcon.CircuitListenerMixin):

def stream_attach(self, stream, circuit): log_stream(stream)

def stream_failed(self, stream, reason='', remote_reason='', **kw): print('Stream %d failed because"%s"'%(stream.id, remote_reason))

def circuit_built(self, circuit): log_circuit(circuit)

def circuit_failed(self, circuit, **kw): log.msg('Circuit %d failed"%s"'%(circuit.id, kw['REASON']))

@react @inlineCallbacks def main(reactor): (continues on next page)

28 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

(continued from previous page) log.startLogging(sys.stdout)

tor= yield txtorcon.connect(reactor) log.msg('Connected to a Tor version %s'% tor.protocol.version) state= yield tor.create_state()

listener= StreamCircuitLogger() state.add_circuit_listener(listener) state.add_stream_listener(listener)

tor.protocol.add_event_listener('STATUS_GENERAL', log.msg) tor.protocol.add_event_listener('STATUS_SERVER', log.msg) tor.protocol.add_event_listener('STATUS_CLIENT', log.msg)

log.msg('Existing state when we connected:') fors in state.streams.values(): log_stream(s)

log.msg('Existing circuits:') forc in state.circuits.values(): log_circuit(c) yield Deferred()

1.4.4 Events monitor.py

Download the example. Use a plain txtorcon.TorControlProtocol instance to listen for some simple events – in this case marginally useful, as it listens for logging at level INFO, NOTICE, WARN and ERR.

#!/usr/bin/env python

# Just listens for a few EVENTs from Tor (INFO NOTICE WARN ERR) and # prints out the contents, so functions like a log monitor. from __future__ import print_function from twisted.internet import task, defer from twisted.internet.endpoints import UNIXClientEndpoint import txtorcon

@task.react @defer.inlineCallbacks def main(reactor): ep= UNIXClientEndpoint(reactor,'/var/run/tor/control') tor= yield txtorcon.connect(reactor, ep)

def log(msg): print(msg) print("Connected to a Tor version", tor.protocol.version) for event in['INFO','NOTICE','WARN','ERR']: tor.protocol.add_event_listener(event, log) (continues on next page)

1.4. Examples 29 txtorcon Documentation, Release 20.0.0

(continued from previous page) is_current= yield tor.protocol.get_info('status/version/current') version= yield tor.protocol.get_info('version') print("Version'{}', is_current={}".format(version, is_current['status/version/

˓→current'])) yield defer.Deferred()

1.4.5 Miscellaneous stem_relay_descriptor.py

Download the example. Get information about a relay descriptor with the help of Stem’s Relay Descriptor class. We need to specify the nickname or the ﬁngerprint to get back the details.

#!/usr/bin/env python

# This shows how to get the detailed information about a # relay descriptor and parse it into Stem's RelayDescriptor # class. More about the class can be read from # # https://stem.torproject.org/api/descriptor/server_descriptor.html#stem.descriptor.

˓→server_descriptor.RelayDescriptor # # We need to pass the nickname or the fingerprint of the onion router # for which we need the the descriptor information. # # Also you need to configure Tor to actually download these # descriptors -- by default Tor only downloads "microdescriptors" # (whose information is already available live via txtorcon.Router # instances). Set "UseMicrodescriptors 0" to download "full" descriptors from __future__ import print_function from twisted.internet.task import react from twisted.internet.defer import inlineCallbacks import txtorcon try: from stem.descriptor.server_descriptor import RelayDescriptor except ImportError: print("You must install'stem' to use this example:") print(" pip install stem") raise SystemExit(1)

@react @inlineCallbacks def main(reactor): tor= yield txtorcon.connect(reactor)

or_nickname="moria1" print("Trying to get decriptor information about'{}'".format(or_nickname)) # If the fingerprint is used in place of nickname then, desc/id/ # should be used. try: (continues on next page)

30 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

(continued from previous page) descriptor_info= yield tor.protocol.get_info('desc/name/'+ or_nickname) except txtorcon.TorProtocolError: print("No information found. Enable descriptor downloading by setting:") print(" UseMicrodescritors 0") print("In your torrc") raise SystemExit(1)

descriptor_info= descriptor_info.values()[0] relay_info= RelayDescriptor(descriptor_info) print("The relay's fingerprint is:{}".format(relay_info.fingerprint)) print("Time in UTC when the descriptor was made:{}".format(relay_info.published)) txtorcon.tac

Download the example Create your own twisted Service for deploying using twistd. import functools from os.path import dirname import sys from tempfile import mkdtemp import txtorcon from twisted.application import service, internet from twisted.internet import reactor from twisted.internet.endpoints import TCP4ClientEndpoint from twisted.python import log from twisted.web import static, server from zope.interface import implements class TorService(service.Service): implements(service.IService) directory= dirname(__file__) port= 8080

def __init__(self): self.torfactory= txtorcon.TorProtocolFactory() self.connection= TCP4ClientEndpoint(reactor,'localhost', 9052) self.resource= server.Site(static.File(self.directory))

def startService(self): service.Service.startService(self)

reactor.listenTCP(self.port, self.resource) self._bootstrap().addCallback(self._complete)

def _bootstrap(self): self.config= txtorcon.TorConfig() self.config.HiddenServices=[ txtorcon.HiddenService(self.config, mkdtemp(), ['%d 127.0.0.1:%d'%(80, self.port)]) ] (continues on next page)

1.4. Examples 31 txtorcon Documentation, Release 20.0.0

(continued from previous page) self.config.save() return txtorcon.launch_tor(self.config, reactor, progress_updates=self._updates, tor_binary='tor')

def _updates(self, prog, tag, summary): log.msg('%d%%: %s'%(prog, summary))

def _complete(self, proto): log.msg(self.config.HiddenServices[0].hostname) application= service.Application("Txtorcon Application") torservice= TorService() torservice.setServiceParent(application)

1.5 Using Asyncio Libraries with txtorcon

It is possible to use Twisted’s asyncioreactor in order to use Twisted together with asyncio libraries. This comes with a couple caveats: • You need to install Twisted • Twisted “owns” the event-loop (i.e. you call reactor.run()); • You need to convert Futures/co-routines to Deferred sometimes (Twisted provides the required machinery) Here is an example using the aiohttp library as a Web server behind an Onion service that txtorcon has set up (in a newly-launched Tor process): wanted: I can’t get this example to work properly with a Unix socket.

1.5.1 web_onion_service_aiohttp.py

Download the example.

# This launches Tor and starts an Onion service using Twisted and # txtorcon, and then starts a Web server using the aiohttp library. # # This style of interop between asyncio and Twisted requires twisted # to use the "asyncioreactor" and for code to convert Futures/Tasks to # Deferreds (most of which is already in Deferred) # # Thanks to Mark Williams for the inspiration, and this code: # https://gist.github.com/markrwilliams/bffb9c293194d105169ea06f03484ba1 # # note: if run in Python2, there are SyntaxErrors before we can tell # the user nicely import asyncio from twisted.internet import asyncioreactor

# get our reactor installed as early as possible, in case other # imports decide to import a reactor and we get the default asyncioreactor.install(asyncio.get_event_loop()) (continues on next page)

32 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

(continued from previous page) from twisted.internet.task import react from twisted.internet.defer import ensureDeferred, Deferred from twisted.internet.endpoints import UNIXClientEndpoint import txtorcon try: from aiohttp import web except ImportError: raise Exception( "You need aiohttp to run this example:\n pip install aiohttp" ) def as_future(d): returnd.asFuture(asyncio.get_event_loop()) def as_deferred(f): return Deferred.fromFuture(asyncio.ensure_future(f)) def get_slash(request): return web.Response( text="I am an aiohttp Onion service\n", ) def create_aio_application(): app= web.Application() app.add_routes([ web.get('/', get_slash) ]) return app async def _main(reactor): if False: print("launching tor") tor= await txtorcon.launch(reactor, progress_updates=print) else: tor= await txtorcon.connect( reactor, UNIXClientEndpoint(reactor,"/var/run/tor/control"), ) print("Connected to tor{}".format(tor.version))

# here, we've just chosen 1234 as the port. We have three other # options: # - select a random, unused one ourselves # - put "ports=[80]" below, and find out which port txtorcon # selected after # - use a Unix-domain socket

# we create a Tor onion service on a specific local TCP port print("Creating onion service") onion= await tor.create_onion_service( (continues on next page)

1.5. Using Asyncio Libraries with txtorcon 33 txtorcon Documentation, Release 20.0.0

(continued from previous page) ports=[ (80, 1234) # 80 is the 'public' port, 1234 is local ], private_key='RSA1024:MIICWwIBAAKBgQCmHEH1y7/

˓→RUUeeaSTgB3iQFfWMep38JDlAbDoEPltRxzgEh8bXMsNbemdiCuZmJVni96KrRh2/

˓→I2NwWi6C81xfcA8BjVzdCmEbL1B+KOeqZlrjoEMQl56NpbXIIzFZdyILaQtv3EZMoShNHSkta6e66oWUu2B2fkluwYyPxRAdvQIDAQABAoGAYkObHX2PlpK/

˓→jE1k3AZvYsUqwhSTOuJu39ZmJ7Z/

˓→rQvt7ngnv4wvFwF9APmzvD9iQir+FtXeqQCVRZSDqUGvpW0WgA+8aDA3BGWCZwKhWRWj18RLjsMX+wKP6OBpSIlNjELU8zc5PWWsCmT7AqAdVD7vqp2895LiP4M8vwwZB30CQQDb/

˓→fjoG1VWpFWXgjRHEYOoPj7d7J5FcRrbSgc57lvMv/2+4OVl2aRaGEjigfBnR7Pjbyxv/

˓→5K1h078PBWNumjPAkEAwUyN3SLJOMBM74LS2jh9AB/sNitLT7/

˓→O1f8zT0siC58TmTbeZsj3VqSsmrUiVSptQcOm+5F0UPvYxsI+B2UbswJAdV9dq8jZkS6AlCNd7QUFL4B2XkVedEJSR+mJTXlE9UsCARNQkTS7oW4PhPo633+8FH4+QUskZUHZ/

˓→G26OjHYtQJAIAKyd418LzbBRuSuUE8MfEnND0dqKGHGOfASKi5yC+SjFTtd5z2eoC2TG+elMN9eyoZBD+YNkh+yzW97YDQhOwJAKFKLmdlJve1lJah1ZllZfk2ipNeYVX+q1Mv7TE6IXGqU/

˓→Xt3HS8h9Zd8ml/Yms1z9X7hFIjQ/XcSiJhqcin8Vg==', version=2, # FIXME use v3; using old tor for now progress=print, )

# we're now listening on some onion service URL and re-directing # public port 80 requests to local TCP port 1234. app= create_aio_application() runner= web.AppRunner(app) await as_deferred(runner.setup()) site= web.TCPSite(runner,'localhost', 1234) await as_deferred(site.start())

# now we're completely set up print("Onion site on http://{}".format(onion.hostname)) await Deferred() def main(): return react( lambda reactor: ensureDeferred( _main(reactor) ) ) if __name__ =='__main__': main()

1.6 Contributions

You can help contribute to txtorcon by reporting bugs, sending success stories, by adding a feature or ﬁxing a bug. Even asking that “silly” question helps me with documentation writing.

1.6.1 Contact Information

Discussing txtorcon is welcome in the following places: • IRC: #tor-dev on OFTC (please preﬁx lines with meejah: to get my attention, and be patient for replies). • email: preferably on the tor-dev list, or see meejah.ca for other ways to contact me. • bugs: use txtorcon’s issues tracker on GitHub.

34 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

• @txtorcon on Twitter (announcements only)

1.6.2 Public Key

You can download my key from a keyserver (0xC2602803128069A7) or see meejah.asc in the repository. The ﬁngerprint is 9D5A 2BD5 688E CB88 9DEB CD3F C260 2803 1280 69A7. Also available at https://meejah.ca/meejah.asc. For convenience: curl https://meejah.ca/meejah.asc | gpg --import

1.6.3 Pull Requests

Yes, please! If you have a new feature or a bug fix, the very best way is to submit a pull-request on GitHub. Since we have 100% coverage, all new lines of code should at least be covered by unit-tests. You can also include a note about the change in docs/releases.rst if you like (or I can make one up after the merge). I prefer if you rebase/squash commits into logical chunks. Discussion of any implementation details can simply occur on the pull-request itself. Force-pushing to the same branch/PR is fine by me if you want to re-order commits etcetera (but, it’s also fine if you just want to push new “fix issues” commits instead). Some example pull-requests: • good discussion + more commits: PR #150; • a simple one that was “ready-to-go”: PR #51. If you want an easy thing to start with, here are all issues tagged “easy”

1.6.4 Making a Release

Mostly a note-to-self, but here is my release checklist.

Release Checklist

• ensure local copy is on main, up-to-date: – git checkout main – git pull • double-check version updated, sadly in a few places: – Makeﬁle – txtorcon/_metadata.py • run all tests, on all conﬁgurations – “detox” • ensure long_description will render properly: – python setup.py check -r -s – tox -e readme_render • “make pep8” should run cleanly (ideally)

1.6. Contributions 35 txtorcon Documentation, Release 20.0.0

• update docs/releases.rst to reflect upcoming reality – blindly make links to the signatures – update heading, date • on both signing-machine and build-machine shells: – export VERSION=20.0.0 • (if on signing machine) “make dist” and “make dist-sigs” – creates: dist/txtorcon-${VERSION}.tar.gz.asc dist/txtorcon-${VERSION}-py2.py3-none- any.whl.asc – add the signatures to “signatures/” cp dist/txtorcon-${VERSION}.tar.gz.asc dist/txtorcon- ${VERSION}-py2.py3-none-any.whl.asc signatures/ – add ALL FOUR files to dist/ (OR fix twine commands) • (if not on signing machine) do “make dist” * scp dist/txtorcon-${VERSION}.tar.gz dist/txtorcon-${VERSION}- py2-none-any.whl signingmachine: * sign both, with .asc detached signatures – gpg –no-version –detach-sign –armor –local-user [email protected] txtorcon-${VERSION}-py2-none- any.whl – gpg –no-version –detach-sign –armor –local-user [email protected] txtorcon-${VERSION}.tar.gz – copy signatures back to build machine, in dist/ – double-check that they validate:: gpg –verify dist/txtorcon-${VERSION}-py2.py3-none-any.whl.asc gpg –verify dist/txtorcon-${VERSION}.tar.gz.asc • generate sha256sum for each:: sha256sum dist/txtorcon-${VERSION}.tar.gz dist/txtorcon-${VERSION}- py2.py3-none-any.whl • copy signature files to /signatures and commit them along with the above changes for versions, etc. • draft email to tor-dev (and probably twisted-python): – example: https://lists.torproject.org/pipermail/tor-dev/2014-January/006111.html – example: https://lists.torproject.org/pipermail/tor-dev/2014-June/007006.html – copy-paste release notes, un-rst-format them – include above sha256sums – clear-sign the announcement – gpg –armor –clearsign -u [email protected] release-announce-${VERSION} – Example boilerplate: I’m [adjective] to announce txtorcon 0.10.0. This adds several amazing features, including levitation. Full list of improvements:

* take from releases.rst * . . . but un-rST them You can download the release from PyPI or GitHub (or of course “pip install txtorcon”): https://pypi.python.org/pypi/txtorcon/0.10.0 https://github.com/meejah/txtorcon/ releases/tag/v0.10.0 Releases are also available from the hidden service:

36 Chapter 1. Documentation txtorcon Documentation, Release 20.0.0

http://timaq4ygg2iegci7.onion/txtorcon-0.12.0.tar.gz http://timaq4ygg2iegci7.onion/ txtorcon-0.12.0.tar.gz.asc Or via a “version 3” service: http://fjblvrw2jrxnhtg67qpbzi45r7ofojaoo3orzykesly2j3c2m3htapid.onion/txtorcon-18. 0.0.tar.gz http://fjblvrw2jrxnhtg67qpbzi45r7ofojaoo3orzykesly2j3c2m3htapid.onion/ txtorcon-18.0.0.tar.gz.asc You can verify the sha256sum of both by running the following 4 lines in a shell wherever you have the ﬁles downloaded: cat <

* make sure BOTH the .tar.gz and .tar.gz.asc (ditto for .whl) are in the dist/ directory first!!) * ls dist/txtorcon-${VERSION}* * note this depends on a ~/.pypirc file with [server-login] section containing “username:” and “password:” – git push origin main – git push origin v${VERSION} – to github: use web-upload interface to upload the 4 files (both dists, both signature) • make announcement – post to tor-dev@ the clear-signed release announcement

1.6. Contributions 37 txtorcon Documentation, Release 20.0.0

– post to twisted-python@ the clear-signed release announcement – tweet as @txtorcon – tell #tor-dev??

38 Chapter 1. Documentation CHAPTER 2

Ofﬁcial Releases:

All ofﬁcial releases are tagged in Git, and signed by my key. All ofﬁcial releases on PyPI have a corresponding GPG signature of the build. Please be aware that pip does not check GPG signatures by default; please see this ticket if you care. The most reliable way to verify you got what I intended is to clone the Git repository, git checkout a tag and verify its signature. The second-best would be to download a release + tag from PyPI and verify that.

2.1 Releases

There isn’t a “release schedule” in any sense. If there is something in main your project depends upon, let me know and I’ll do a release. txtorcon follows calendar versioning with the major version being the 2-digit year. The second digit will be “non- trivial” releases and the third will be for bugﬁx releases. So the second release in 2019 would be “19.2.0” and a bug-ﬁx release of that will be “19.2.1”. See also API Stability. git main will likely become v20.1.0

2.1.1 v20.0.0

April 1, 2020 • Use real GeoIP database or nothing (#250) • Change abstract base classes import in preperation for Python 3.8 (thanks @glowatsk • Python 3.4 is no longer supported • Python 2 is deprecated; all new code should be Python 3. Support for Python 2 will be removed in a future release.

39 txtorcon Documentation, Release 20.0.0

2.1.2 v19.1.0

September 10, 2019 • txtorcon-19.1.0.tar.gz(PyPI( local-sig or github-sig)(source) • TorControlProtocol.on_disconnect is deprecated in favour of TorControlProtocol. when_disconnected() • introduce non_anonymous_mode= kwarg in txtorcon.launch() enabling Tor options making Onion Ser- vices non-anonymous for the server (but they use a single hop instead of three to the Introduction Point so they’re slightly faster). • add an API to listen to individual circuit and stream events (without subclassing anything). Can be used as decorators too. See e.g. TorState.on_circuit_new() • ﬁxes to the CI setup to properly test Twisted versions

2.1.3 v19.0.0

January 15, 2019 • txtorcon-19.0.0.tar.gz(PyPI( local-sig or github-sig)(source) • add TorControlProtocol.when_disconnected() (will replace .on_disconnect) • add detach= kwarg to Tor.create_onion_service() • add purpose= kwarg to TorState.build_circuit()

2.1.4 v18.3.0

• txtorcon-18.3.0.tar.gz(PyPI( local-sig or github-sig)(source) • add singleHop={true,false} for endpoint-strings as well

2.1.5 v18.2.0

• txtorcon-18.2.0.tar.gz(PyPI( local-sig or github-sig)(source) • add privateKeyFile= option to endpoint parser (ticket 313) • use privateKey= option properly in endpoint parser • support NonAnonymous mode for ADD_ONION via single_hop= kwarg

2.1.6 v18.1.0

September 26, 2018 • txtorcon-18.1.0.tar.gz(PyPI( local-sig or github-sig)(source) • better error-reporting (include REASON and REMOTE_REASON if available) when circuit-builds fail (thanks David Stainton) • more-robust detection of “do we have Python3” (thanks Balint Reczey) • ﬁx parsing of Unix-sockets for SOCKS

40 Chapter 2. Ofﬁcial Releases: txtorcon Documentation, Release 20.0.0

• better handling of concurrent Web agent requests before SOCKS ports are known • allow fowarding to ip:port pairs for Onion services when using the “list of 2-tuples” method of specifying the remote vs local connections.

2.1.7 v18.0.2

July 2, 2018 • txtorcon-18.0.2.tar.gz(PyPI( local-sig or github-sig)(source) • Python3.4 doesn’t support async-def or await

2.1.8 v18.0.1

June 30, 2018 • txtorcon-18.0.1.tar.gz(PyPI( local-sig or github-sig)(source) • ﬁx a Python2/3 regression when parsing onion services

2.1.9 v18.0.0

June 21, 2018 • txtorcon-18.0.0.tar.gz(PyPI( local-sig or github-sig)(source) • await_all_uploads options when creating Onions • properly re-map progress percentages (including descriptor uploads) • properly wait for all event-listeners during startup • re-work how TorState.event_map works, hopefully reducing reproducible-builds issues • txtorcon.TorControlProtocol.add_event_listener() and txtorcon. TorControlProtocol.remove_event_listener() are now async methods returning Deferred – they always should have been; new code can now be assured that the event-listener change is known to Tor by awaiting this Deferred. • txtorcon.TorControlProtocol.get_conf_single() method added, which gets and returns (asynchronously) a single GETCONF key (instead of a dict) • also txtorcon.TorControlProtocol.get_info_single() similar to above • if Tor disconnects while a command is in-progress or pending, the .errback() for the corresponding Deferred is now correctly ﬁred (with a txtorcon.TorDisconnectError • tired: get_global_tor() (now deprecated) wired: txtorcon.get_global_tor_instance() • Adds a comprehensive set of Onion Services APIs (for all six variations). For non-authenticated services, instances of txtorcon.IOnionService represent services; for authenticated services, instances of txtorcon.IAuthenticatedOnionClients encapsulated named lists of clients (each client is an instance implementing IOnionService). • Version 3 (“Proposition 279”) Onion service support (same APIs) as released in latest Tor • Four new methods to handle creating endpoints for Onion services (either ephemeral or not and authenticated or not): ** :method:‘txtorcon.Tor.create_authenticated_onion_endpoint‘

2.1. Releases 41 txtorcon Documentation, Release 20.0.0

** :method:‘txtorcon.Tor.create_authenticated_filesystem_onion_endpoint‘ ** :method:‘txtorcon.Tor.create_onion_endpoint‘ ** :method:‘txtorcon.Tor.create_filesystem_onion_endpoint‘ • see Creating Onion Endpoints for information on how to choose an appropriate type of Onion Service. • :method:‘txtorcon.Tor.create_onion_service‘ to add a new ephemeral Onion service to Tor. This uses the ADD_ONION command under the hood and can be version 2 or version 3. Note that there is an endpoint-style API as well so you don’t have to worry about mapping ports yourself (see below). • :method:‘txtorcon.Tor.create_filesystem_onion_service‘ to add a new Onion service to Tor with configuration (private keys) stored in a provided directory. These can be version 2 or version 3 services. Note that there is an endpoint-style API as well so you don’t have to worry about mapping ports yourself (see below). • Additional APIs to make visiting authenticated Onion services as a client easier: • :method:‘txtorcon.Tor.add_onion_authentication‘ will add a client-side Onion service authentication token. If you add a token for a service which already has a token, it is an error if they don’t match. This corresponds to HidServAuth lines in torrc. • :method:‘txtorcon.Tor.remove_onion_authentication‘ will remove a previously added client-side Onion service authentication token. Fires with True if such a token existed and was removed or False if no existing token was found. • :method:‘txtorcon.Tor.onion_authentication‘ (Python3 only) an async context-manager that adds and removes an Onion authentication token (i.e. adds in on __aenter__ and removes it on __aexit__). • onion services support listening on Unix paths. • make sure README renders on Warehouse/PyPI

2.1.10 v0.20.0

February 22, 2018 • txtorcon-0.20.0.tar.gz(PyPI( local-sig or github-sig)(source) • doc fixes from hotelzululima • fix endpoints so .connect on them works properly more than once from Brian Warner • allow a CertificateOptions to be passed as tls= to endpoints • add method txtorcon.Tor.is_ready() • add method txtorcon.Tor.become_ready() • fix handling of certain defaults (*PortLines and friends) • fix last router (usually) missing with (new) MicroDescriptorParser • use OnionOO via Onion service tgel7v4rpcllsrk2.onion for txtorcon.Router. get_onionoo_details() • fix parsing of Router started-times • Issue 255 removed routers now deleted following NEWCONSENSUS • Issue 279 remember proxy endpoint

2.1.11 v0.19.3

May 24, 2017

42 Chapter 2. Ofﬁcial Releases: txtorcon Documentation, Release 20.0.0

• txtorcon-0.19.3.tar.gz(PyPI( local-sig or github-sig)(source) • Incorrect parsing of SocksPort options (see Issue 237)

2.1.12 v0.19.2

May 11, 2017 • txtorcon-0.19.2.tar.gz(PyPI( local-sig or github-sig)(source) • Work around a bug in incremental (see Issue 233) • Fix for Issue 190 from Felipe Dau. • add txtorcon.Circuit.when_built().

2.1.13 v0.19.1

April 26, 2017 • txtorcon-0.19.1.tar.gz(PyPI( local-sig or github-sig)(source) • Fix a regression in launch_tor, see Issue 227

2.1.14 v0.19.0

April 20, 2017 • txtorcon-0.19.0.tar.gz(PyPI( local-sig or github-sig)(source) • Full Python3 support • Drop txsocksx and use a custom implementation (this also implements the custom Tor SOCKS5 methods RE- SOLVE and RESOLVE_PTR • Drop support for older Twisted releases (12, 13 and 14 are no longer supported). • Add a top-level API object, txtorcon.Tor that abstracts a running Tor. Instances of this class are created with txtorcon.connect() or txtorcon.launch(). These instances are intended to be “the” high- level API and most users shouldn’t need anything else. • Integrated support for twisted.web.client.Agent, baked into txtorcon.Tor. This allows simple, straightfor- ward use of treq or “raw” twisted.web.client for making client-type Web requests via Tor. Automatically handles configuration of SOCKS ports. See txtorcon.Tor.web_agent() • new high-level API for putting streams on specific Circuits. This adds txtorcon.Circuit. stream_via() and txtorcon.Circuit.web_agent() methods that work the same as the “Tor” equivalent methods except they use a specific circuit. This makes txtorcon.TorState. set_attacher() the “low-level” / “expert” interface. Most users should only need the new API. • big revamp / re-write of the documentation, including the new Programming Guide • Issue 203 • new helper: :meth:‘txtorcon.Router.get_onionoo_details‘_ • new helper: :func:‘txtorcon.util.create_tbb_web_headers‘_ • Issue 72 • Felipe Dau added specific SocksError subclasses for all the available SOCKS5 errors.

2.1. Releases 43 txtorcon Documentation, Release 20.0.0

• (more) Python3 ﬁxes from rodrigc

2.1.15 v0.18.0

January 11, 2017 • txtorcon-0.18.0.tar.gz(PyPI( local-sig or github-sig)(source) • issue 200: better feedback if the cookie data can’t be read

2.1.16 v0.17.0

October 4, 2016 • txtorcon-0.17.0.tar.gz(PyPI( local-sig or github-sig)(source) • issue 187: ﬁx unix-socket control endpoints • sometimes mapping streams to hostnames wasn’t working properly • backwards-compatibility API for socks_hostname was incorrectly named

2.1.17 v0.16.1

August 31, 2016 • txtorcon-0.16.1.tar.gz(PyPI( local-sig or github-sig)(source) • issue 172: give TorProcessProtocol a .quit method • issue 181: enable SOCKS5-over-unix-sockets for TorClientEndpoint (thanks to david415

2.1.18 v0.16.0

• there wasn’t one, because reasons.

2.1.19 v0.15.1

• txtorcon-0.15.1.tar.gz(PyPI( local-sig or github-sig)(source) • ﬁx issue 179 with Circuit.age.

2.1.20 v0.15.0

July 26, 2016 • txtorcon-0.15.0.tar.gz(PyPI( local-sig or github-sig)(source) • added support for NULL control-port-authentication which is often appropriate when used with a UNIX domain socket • switched to ipaddress instead of Google’s ipaddr; the API should be the same from a user perspective but packagers and tutorials will want to change their instructions slightly (pip install ipaddress or apt-get install python-ipaddress are the new ways). • support the new ADD_ONION and DEL_ONION “ephemeral hidden services” commands in TorConﬁg

44 Chapter 2. Ofﬁcial Releases: txtorcon Documentation, Release 20.0.0

• a first stealth-authentication implementation (for “normal” hidden services, not ephemeral) • bug-fix from david415 to raise ConnectionRefusedError instead of StopIteration when running out of SOCKS ports. • new feature from david415 adding a build_timeout_circuit method which provides a Deferred that callbacks only when the circuit is completely built and errbacks if the provided timeout expires. This is useful because txtorcon.TorState.build_circuit() callbacks as soon as a Circuit instance can be provided (and then you’d use txtorcon.Circuit.when_built() to find out when it’s done building). • new feature from coffeemakr falling back to password authentication if cookie authentication isn’t available (or fails, e.g. because the file isn’t readable). • both TorState and TorConfig now have a .from_protocol class-method. • spec-compliant string-un-escaping from coffeemakr • a proposed new API: txtorcon.connect() • fix issue 176

2.1.21 v0.14.2

December 2, 2015 • txtorcon-0.14.2.tar.gz(PyPI( local-sig or github-sig)(source) • compatibility for Twisted 15.5.0 (released on 0.14.x for OONI)

2.1.22 v0.14.1

October 25, 2015 • subtle bug with .is_built on Circuit; changing the API (but with backwards-compatibility until 0.15.0 at least)

2.1.23 v0.14.0

September 26, 2015 • txtorcon-0.14.0.tar.gz(PyPI( local-sig or github-sig)(source) • txtorcon.interface.IStreamAttacher handling was missing None and DO_NOT_ATTACH cases if a Deferred was returned. • add .is_built Deferred to txtorcon.Circuit that gets ‘callback()‘d when the circuit becomes BUILT • david415 ported his tor: endpoint parser so now both client and server endpoints are supported. This means any Twisted program using endpoints can use Tor as a client. For example, to connect to txtorcon’s Web site: ep = clientFromString("tor:timaq4ygg2iegci7.onion:80"). (In the future, I’d like to automatically launch Tor if required, too). • Python3 ﬁxes from isis (note: needs Twisted 15.4.0+)

2.1. Releases 45 txtorcon Documentation, Release 20.0.0

2.1.24 v0.13.0

May 10, 2015 • txtorcon-0.13.0.tar.gz(PyPI( local-sig or github-sig)(source) • support basic and stealth hidden service authorization, and parse client_keys ﬁles. • 2x speedup for TorState parsing (mostly by lazy-parsing timestamps) • can now parse ~75000 microdescriptors/second per core of 3.4GHz Xeon E3 • launch_tor now doesn’t use a temporary torrc (command-line options instead) • tons of pep8 cleanups • several improvements to hidden-service conﬁguration from sambuddhabasu1. • populated valid signals from GETINFO signals/names from sambuddhabasu1.

2.1.25 v0.12.0

February 3, 2015 • txtorcon-0.12.0.tar.gz(PyPI( local-sig or github-sig)(source) • doc, code and import cleanups from Kali Kaneko • HiddenServiceDirGroupReadable support • Issue #80: honour ControlPort 0 in incoming TorConfig instance. The caller owns both pieces: you have to figure out when it’s bootstraped, and are responsible for killing it off. • Issue #88: clarify documentation and fix appending to some config lists • If GeoIP data isn’t loaded in Tor, it sends protocol errors; if txtorcon also hasn’t got GeoIP data, the queries for country-code fail; this error is now ignored. • 100% unit-test coverage! (line coverage) • PyPy support (well, at least all tests pass) • TCP4HiddenServiceEndpoint now waits for descriptor upload before the listen() call does its callback (this means when using onion: endpoint strings, or any of the endpoints APIs your hidden service is 100% ready for action when you receive the callback) • TimeIntervalCommaList from Tor config supported • TorControlProtocol now has a .all_routers member (a set() of all Routers) • documentation fix from sammyshj

2.1.26 v0.11.0

August 16, 2014 • September 6, 2015. bugﬁx release: txtorcon-0.11.1.tar.gz(PyPI( local-sig or github-sig)(source) • ﬁxed Debian bug 797261 causing 3 tests to fail • txtorcon-0.11.0.tar.gz(PyPI( local-sig or github-sig)(source) • More control for launch_tor: access stdout, stderr in real-time and control whether we kill Tor on and stderr output. See issue #79.

46 Chapter 2. Ofﬁcial Releases: txtorcon Documentation, Release 20.0.0

• Warning about build_circuit being called without a guard ﬁrst is now optional (default is still warn) (from arlolra) • available_tcp_port() now in util (from arlolra) • TorState now has a .routers_by_hash member (from arlolra)

2.1.27 v0.10.1

July 20, 2014 • txtorcon-0.10.1.tar.gz(PyPI( local-sig or github-sig)(source) • ﬁx bug incorrectly issuing RuntimeError in brief window of time on event-listeners • issue #78: Add tox tests and ﬁx for Twisted 12.0.0 (and prior), as this is what Debian squeeze ships • issue #77: properly expand relative and tilde paths for hiddenServiceDir via endpoints

2.1.28 v0.10.0

June 15, 2014 • txtorcon-0.10.0.tar.gz(PyPI( local-sig or github-sig)(source) • In collaboration with David Stainton after a pull-request, we have endpoint parser plugins for Twisted! This means code like serverFromString("onion:80").listen(...) is enough to start a service. • The above also means that any endpoint-using Twisted program can immediately offer its TCP services via Hidden Service with no code changes. For example, using Twisted Web to serve a WSGI web application would be simply: twistd web --port onion:80 --wsgi web.app • switch to a slightly-modiﬁed Alabaster Sphinx theme • added howtos to documentation

2.1.29 v0.9.2

April 23, 2014 • txtorcon-0.9.2.tar.gz( local-sig or github-sig)(source) • add on_disconnect callback for TorControlProtocol (no more monkey-patching Protocol API) • add age() method to Circuit • add time_created property to Circuit • don’t incorrectly listen for NEWDESC events in TorState • add .flags dict to track ﬂags in Circuit, Stream • build_circuit() can now take hex IDs (as well as Router instances) • add unique_name property to Router (returns the hex id, unless Named then return name) • add location property to Router • TorState.close_circuit now takes either a Circuit ID or Circuit instance • TorState.close_stream now takes either a Stream ID or Stream instance • support both GeoIP API versions

2.1. Releases 47 txtorcon Documentation, Release 20.0.0

• more test-coverage • small patch from enriquefynn improving tor binary locating • strip OK lines in TorControlProtocol (see issue #8) • use TERM not KILL when Tor launch times out (see issue #68) from hellais

2.1.30 v0.9.1

January 20, 2014 • txtorcon-0.9.1.tar.gz( local-sig or github-sig)(source) • put test/ directory at the top level • using “coverage” tool instead of custom script • using coveralls.io and travis-ci for test coverage and continuous integration • issue #56: added Circuit.close() and Stream.close() starting from aagbsn’s patch • parsing issues with multi-line keyword discovered and resolved • preserve router nicks from long-names if consensus lacks an entry (e.g. bridges) • using Twine for releases • Wheel release now also available • issue #57: “python setup.py develop” now supported • issue #59: if tor_launch() times out, Tor is properly killed (starting with pull-request from Ryman) • experimental docker.io-based tests (for HS listening, and tor_launch() timeouts) • issue #55: pubkey link on readthedocs • issue #63 • clean up GeoIP handling, and support pygeoip both pre and post 0.3 • slightly improve unit-test coverage (now at 97%, 61 lines of 2031 missing) • added a Walkthrough to the documentation

2.1.31 v0.8.2

November 22, 2013 • txtorcon-0.8.2.tar.gz( local-sig or github-sig)(source) • ensure hidden service server-side endpoints listen only on 127.0.0.1

2.1.32 v0.8.1

May 13, 2013 • txtorcon-0.8.1.tar.gz( local-sign or github-sig)(source) • ﬁxed improper import in setup.py preventing 0.8.0 from installing • signatures with proper subkey this time

48 Chapter 2. Ofﬁcial Releases: txtorcon Documentation, Release 20.0.0

• Proper file-flushing in tests and PyPy fixes from Lukas Lueg • docs build issue from isis

2.1.33 v0.8.0

April 11, 2013 (actually uploaded May 11) • Please use 0.8.1; this won’t install due to import problem in setup.py (unless you have pypissh). • following semantic versioning; • slight API change ICircuitListener.circuit_failed(), circuit_closed() and IStreamListener.stream_failed(), stream_closed() and stream_detach() all now include any keywords in the notification method (some of these lacked flags, or only included some) (issue #18); • launch_tor() can take a timeout (starting with a patch from hellais); • cleanup from aagbsn; • more test coverage; • run tests cleanly without graphviz (from lukaslueg); • issue #26 fix from lukaslueg; • pep8 and whitespace targets plus massive cleanup (now pep8 clean, from lukaslueg); • issue #30 fix reported by webmeister making ipaddr actually-optional; • example using synchronous web server (built-in SimpleHTTPServer) with txtorcon (from lukaslueg); • TorState can now create circuits without an explicit path; • passwords for non-cookie authenticated sessions use a password callback (that may return a Deferred) instead of a string (issue #44); • fixes for AddrMap in case #8596 is implemented;

2.1.34 v0.7

November 21, 2012 • txtorcon-0.7.tar.gz( local-sig or github-sig)(source) • issue #20 conﬁg object now hooked up correctly after launch_tor(); • patch from hellais for properly handling data_dir given to TCPHiddenServiceEndpoint; • .tac example from mmaker; • allow TorConﬁg().hiddenservices.append(hs) to work properly with no attached protocol

2.1.35 v0.6

October 10, 2012 • txtorcon-0.6.tar.gz( local-sig or github-sig)(source) • debian packaging (mmaker); • psutil fully gone;

2.1. Releases 49 txtorcon Documentation, Release 20.0.0

• changed API for launch_tor() to use TorConfig instead of args; • TorConfig.save() works properly with no connected Tor; • fix incorrect handling of 650 immediately after connect; • pep8 compliance; • use assertEqual in tests; • messages with embdedded keywords work properly; • fix bug with setup.py + pip; • issue #15 reported along with patch by Isis Lovecruft; • consolidate requirements (from aagbsn); • increased test coverage and various minor fixes; • https URIs for ReadTheDocs;

2.1.36 v0.5

June 20, 2012 • txtorcon-0.5.tar.gz(txtorcon-0.5.tar.gz.sig)(source) • remove psutil as a dependency, including from util.process_from_address

2.1.37 v0.4

June 6, 2012 • txtorcon-0.4.tar.gz(txtorcon-0.4.tar.gz.sig) • remove built documentation from distribution; • ﬁx PyPI problems (“pip install txtorcon” now works)

2.1.38 v0.3

• 0.3 was broken when released (docs couldn’t build).

2.1.39 v0.2

June 1, 2012 • txtorcon-0.2.tar.gz(txtorcon-0.2.tar.gz.sig) • incremental parsing; • faster TorState startup; • SAFECOOKIE support; • several bug ﬁxes; • options to circuit_failure_rates.py example to make it actually-useful; • include built documentation + sources in tarball;

50 Chapter 2. Ofﬁcial Releases: txtorcon Documentation, Release 20.0.0

• include tests in tarball; • improved logging; • patches from mmaker and kneufeld;

2.1.40 v0.1 march, 2012 • txtorcon-0.1.tar.gz(txtorcon-0.1.tar.gz.sig)

2.1. Releases 51 txtorcon Documentation, Release 20.0.0

52 Chapter 2. Ofﬁcial Releases: CHAPTER 3

API Documentation

These are the lowest-level documents, directly from the doc-strings in the code with some minimal organization; if you’re just getting started with txtorcon the “Programming Guide” is a better place to start.

3.1 API Documentation

3.1.1 High Level API

This is the recommended API. See the Programming Guide for “prose” documentation of these (and other) APIs.

Tor class txtorcon.Tor(reactor, control_protocol, _tor_conﬁg=None, _process_proto=None, _non_anonymous=None) Bases: object I represent a single instance of Tor and act as a Builder/Factory for several useful objects you will probably want. There are two ways to create a Tor instance: • txtorcon.connect() to connect to a Tor that is already running (e.g. Tor Browser Bundle, a system Tor, . . . ). • txtorcon.launch() to launch a fresh Tor instance The stable API provided by this class is txtorcon.interface.ITor If you desire more control, there are “lower level” APIs which are the very ones used by this class. However, this “highest level” API should cover many use-cases:

53 txtorcon Documentation, Release 20.0.0

import txtorcon

@inlineCallbacks def main(reactor): # tor = yield txtorcon.connect(UNIXClientEndpoint(reactor, "/var/run/tor/

˓→control")) tor= yield txtorcon.launch(reactor)

onion_ep= tor.create_onion_endpoint(port=80) port= yield onion_ep.listen(Site()) print(port.getHost())

don’t instantiate this class yourself – instead use the factory methods txtorcon.launch() or txtorcon. connect() quit(**kwargs) Closes the control connection, and if we launched this Tor instance we’ll send it a TERM and wait until it exits. process An object implementing twisted.internet.interfaces.IProcessProtocol if this Tor instance was launched, or None. protocol The TorControlProtocol instance that is communicating with this Tor instance. version get_config(**kwargs) Returns a Deferred that fires with a TorConfig instance. This instance represents up-to-date configuration of the tor instance (even if another controller is connected). If you call this more than once you’ll get the same TorConfig back. web_agent(pool=None, socks_endpoint=None) Parameters • socks_endpoint – If None (the default), a suitable SOCKS port is chosen from our config (or added). If supplied, should be a Deferred which fires an IStreamClientEnd- point (e.g. the return-value from txtorcon.TorConfig.socks_endpoint()) or an immediate IStreamClientEndpoint You probably don’t need to mess with this. • pool – passed on to the Agent (as pool=) dns_resolve(**kwargs) Parameters hostname – a string Returns a Deferred that calbacks with the hostname as looked-up via Tor (or errback). This uses Tor’s custom extension to the SOCKS5 protocol. dns_resolve_ptr(**kwargs) Parameters ip – a string, like “127.0.0.1” Returns a Deferred that calbacks with the IP address as looked-up via Tor (or errback). This uses Tor’s custom extension to the SOCKS5 protocol. add_onion_authentication(**kwargs) Add a client-side authentication token for a particular Onion service.

54 Chapter 3. API Documentation txtorcon Documentation, Release 20.0.0

remove_onion_authentication(**kwargs) Remove a token for an onion host Returns True if successful, False if there wasn’t a token for that host. onion_authentication(onion_host, token) (Python3 only!) This returns an async context-manager that will add and remove onion authentication. For example, inside an async def method that’s had ensureDeferred called on it:

async with tor.onion_authentication("timaq4ygg2iegci7.onion","seekrit token

˓→"): agent= tor.web_agent() resp= await agent.request(b'GET',"http://timaq4ygg2iegci7.onion/") body= await readBody(resp) # after the "async with" the token will be removed from Tor's configuration

Under the hood, this just uses the add_onion_authentication and remove_onion_authentication methods so on Python2 you can use those together with try/finally to get the same effect. stream_via(host, port, tls=False, socks_endpoint=None) This returns an IStreamClientEndpoint instance that will use this Tor (via SOCKS) to visit the (host, port) indicated. Parameters • host – The host to connect to. You MUST pass host-names to this. If you absolutely know that you’ve not leaked DNS (e.g. you save IPs in your app’s configuration or similar) then you can pass an IP. • port – Port to connect to. • tls – If True, it will wrap the return endpoint in one that does TLS (default: False). • socks_endpoint – Normally not needed (default: None) but you can pass an IStream- ClientEndpoint directed at one of the local Tor’s SOCKS5 ports (e.g. created with txtorcon.TorConfig.create_socks_endpoint()). Can be a Deferred. create_authenticated_onion_endpoint(port, auth, private_key=None, version=None) WARNING: API subject to change When creating an authenticated Onion service a token is created for each user. For ‘stealth’ authentication, the hostname is also different for each user. The difference between this method and txtorcon.Tor. create_onion_endpoint() is in this case the “onion_service” instance implements txtorcon. IAuthenticatedOnionClients. Returns an object that implements IStreamServerEndpoint, which will create an “ephemeral” Onion service when .listen() is called. This uses the ADD_ONION Tor control-protocol command. The object returned from .listen() will be a :class:TorOnionListeningPort‘‘; its .onion_service attribute will be a txtorcon. IAuthenticatedOnionClients instance. Parameters • port – the port to listen publically on the Tor network on (e.g. 80 for a Web server) • private_key – if not None (the default), this should be the same blob of key material that you received from the txtorcon.IOnionService object during a previous run (i.e. from the .provate_key attribute). • version – if not None, a specific version of service to use; version=3 is Proposition 224 and version=2 is the older 1024-bit key based implementation. • auth – a AuthBasic or AuthStealth instance

3.1. API Documentation 55 txtorcon Documentation, Release 20.0.0

create_onion_endpoint(port, private_key=None, version=None, single_hop=None) WARNING: API subject to change Returns an object that implements IStreamServerEndpoint, which will create an “ephemeral” Onion service when .listen() is called. This uses the ADD_ONION tor control-protocol command. The object returned from .listen() will be a :class:TorOnionListeningPort‘‘; its .onion_service attribute will be a txtorcon.IOnionService instance. Parameters • port – the port to listen publically on the Tor network on (e.g. 80 for a Web server) • private_key – if not None (the default), this should be the same blob of key material that you received from the txtorcon.IOnionService object during a previous run (i.e. from the .private_key attribute). • version – if not None, a specific version of service to use; version=3 is Proposition 224 and version=2 is the older 1024-bit key based implementation. • single_hop – if True, pass the NonAnonymous flag. Note that Tor options HiddenSer- viceSingleHopMode, HiddenServiceNonAnonymousMode must be set to 1 and there must be no SOCKSPort configured for this to actually work. create_filesystem_onion_endpoint(port, hs_dir, group_readable=False, version=None) WARNING: API subject to change Returns an object that implements IStreamServerEndpoint. When the .listen() method is called, the endpoint will create an Onion service whose keys are on disk when .listen() is called. The object returned from .listen() will be a :class:TorOnionListeningPort‘‘; its .onion_service attribute will be a txtorcon.IOnionService instance. Parameters • port – the port to listen publically on the Tor network on (e.g. 80 for a Web server) • hs_dir – the directory in which keys are stored for this service. • group_readable – controls the Tor HiddenServiceDirGroupReadable which will either set (or not) group read-permissions on the hs_dir. • version – if not None, a specific version of service to use; version=3 is Proposition 224 and version=2 is the older 1024-bit key based implementation. The default is version 3. create_filesystem_authenticated_onion_endpoint(port, hs_dir, auth, group_readable=False, version=None) WARNING: API subject to change Returns an object that implements IStreamServerEndpoint. When the .listen() method is called, the endpoint will create an Onion service whose keys are on disk when .listen() is called. The object returned from .listen() will be a :class:TorOnionListeningPort‘‘; its .onion_service attribute will be a txtorcon.IOnionService instance. Parameters • port – the port to listen publically on the Tor network on (e.g. 80 for a Web server) • hs_dir – the directory in which keys are stored for this service. • auth – instance of txtorcon.AuthBasic or txtorcon.AuthStealth controlling the type of authentication to use. • group_readable – controls the Tor HiddenServiceDirGroupReadable which will either set (or not) group read-permissions on the hs_dir.

56 Chapter 3. API Documentation txtorcon Documentation, Release 20.0.0

• version – if not None, a specific version of service to use; version=3 is Proposition 224 and version=2 is the older 1024-bit key based implementation. The default is version 3. create_state(**kwargs) returns a Deferred that fires with a ready-to-go txtorcon.TorState instance. is_ready(**kwargs) Returns a Deferred that fires with True if this Tor is non-dormant and ready to go. This will return True if GETINFO dormant is false or if GETINFO status/enough-dir-info is true or if GETINFO status/circuit-established true. become_ready(**kwargs) Make sure Tor is no longer dormant. If Tor is currently dormant, it is woken up by doing a DNS request for torproject.org create_onion_service(**kwargs) Create a new Onion service This method will create a new Onion service, returning (via Deferred) an instance that implements IOn- ionService. (To create authenticated onion services, see XXX). This method awaits at least one upload of the Onion service’s ‘descriptor’ to the Tor network – this can take from 30s to a couple minutes. Parameters • private_key – None, txtorcon.DISCARD or a key-blob retained from a prior run Passing None means a new one will be created. It can be retrieved from the . private_key property of the returned object. You must retain this key yourself (and pass it in to this method in the future) if you wish to keep the same .onion domain when re-starting your program. Passing txtorcon.DISCARD means txtorcon will never learn the private key from Tor and so there will be no way to re-create an Onion Service on the same address after Tor exits. • version – The latest Tor releases support ‘Proposition 224’ (version 3) services. These are the default. • progress – if provided, a function that takes 3 arguments: (percent_done, tag, description) which may be called any number of times to indicate some progress has been made. • await_all_uploads – if False (the default) then we wait until at least one upload of our Descriptor to a Directory Authority has completed; if True we wait until all have completed. • single_hop – if True, pass the NonAnonymous flag. Note that Tor options HiddenSer- viceSingleHopMode, HiddenServiceNonAnonymousMode must be set to 1 and there must be no SOCKSPort configured for this to actually work. • detach – if True, the created service won’t be tied to this control connection and will still be active when this control-connection goes away (this means the service will appear in GETINFO onions/detached to all other controllers) create_filesystem_onion_service(**kwargs) Create a new Onion service stored on disk This method will create a new Onion service, returning (via Deferred) an instance that implements IOn- ionService. (To create authenticated onion services, see XXX). This method awaits at least one upload of the Onion service’s ‘descriptor’ to the Tor network – this can take from 30s to a couple minutes.

3.1. API Documentation 57 txtorcon Documentation, Release 20.0.0

Parameters • ports – a collection of ports to advertise; these are forwarded locally on a random port. Each entry may instead be a 2-tuple, which chooses an explicit local port. • onion_service_dir – a path to an Onion Service directory. Tor will write a hostname ﬁle in this directory along with the private keys for the service (if they do not already exist). You do not need to retain the private key yourself. • version – which kind of Onion Service to create. The default is 3 which are the Propo- sition 224 services. Version 2 are the previous services. There are no other valid versions currently. • group_readable – if True, Tor creates the directory with group read permissions. The default is False. • progress – if provided, a function that takes 3 arguments: (percent_done, tag, description) which may be called any number of times to indicate some progress has been made. connect controller.connect(**kwargs) Creates a txtorcon.Tor instance by connecting to an already-running tor’s control port. For example, a common default tor uses is UNIXClientEndpoint(reactor, ‘/var/run/tor/control’) or TCP4ClientEndpoint(reactor, ‘localhost’, 9051) If only password authentication is available in the tor we connect to, the password_function is called (if supplied) to retrieve a valid password. This function can return a Deferred. For example:

import txtorcon from twisted.internet.task import react from twisted.internet.defer import inlineCallbacks

@inlineCallbacks def main(reactor): tor= yield txtorcon.connect( TCP4ClientEndpoint(reactor,"localhost", 9051) ) state= yield tor.create_state() for circuit in state.circuits: print(circuit)

Parameters • control_endpoint – None, an IStreamClientEndpoint to connect to, or a Sequence of IStreamClientEndpoint instances to connect to. If None, a list of defaults are tried. • password_function – See txtorcon.TorControlProtocol Returns a Deferred that ﬁres with a txtorcon.Tor instance launch controller.launch(**kwargs) launches a new Tor process, and returns a Deferred that ﬁres with a new txtorcon.Tor instance. From this

58 Chapter 3. API Documentation txtorcon Documentation, Release 20.0.0

instance, you can create or get any “interesting” instances you need: the txtorcon.TorConfig instance, create endpoints, create txtorcon.TorState instance(s), etc. Note that there is NO way to pass in a conﬁg; we only expost a couple of basic Tor options. If you need anything beyond these, you can access the TorConfig instance (via .config) and make any changes there, reﬂecting them in tor with .config.save(). You can igore all the options and safe defaults will be provided. However, it is recommended to pass data_directory especially if you will be starting up Tor frequently, as it saves a bunch of time (and bandwidth for the directory authorities). “Safe defaults” means: • a tempdir for a DataDirectory is used (respecting TMP) and is deleted when this tor is shut down (you therefore probably want to supply the data_directory= kwarg); • a random, currently-unused local TCP port is used as the SocksPort (specify socks_port= if you want your own). If you want no SOCKS listener at all, pass socks_port=0 • we set __OwningControllerProcess and call TAKEOWNERSHIP so that if our control connection goes away, tor shuts down (see control-spec 3.23). • the launched Tor will use COOKIE authentication.

Parameters • reactor – a Twisted IReactorCore implementation (usually twisted.internet.reactor) • progress_updates – a callback which gets progress updates; gets 3 args: percent, tag, summary (FIXME make an interface for this). • data_directory – set as the DataDirectory option to Tor, this is where tor keeps its state information (cached relays, etc); starting with an already-populated state directory is a lot faster. If None (the default), we create a tempdir for this and delete it on exit. It is recommended you pass something here. • non_anonymous_mode – sets the Tor options HiddenServiceSingleHopMode and Hid- denServiceNonAnonymousMode to 1 and un-sets any SOCKSPort config, thus putting this Tor client into “non-anonymous mode” which allows starting so-called Single Onion services – which use single-hop circuits to rendezvous points. See WARNINGs in Tor manual! Also you need Tor 0.3.4.1 or later (e.g. any 0.3.5.* or newer) for this to work properly. • stdout – a file-like object to which we write anything that Tor prints on stdout (just needs to support write()). • stderr – a file-like object to which we write anything that Tor prints on stderr (just needs .write()). Note that we kill Tor off by default if anything appears on stderr; pass “kill_on_stderr=False” if you don’t want this behavior. • tor_binary – path to the Tor binary to run. If None (the default), we try to find the tor binary. • kill_on_stderr – When True (the default), if Tor prints anything on stderr we kill off the process, close the TorControlProtocol and raise an exception. • connection_creator – is mostly available to ease testing, so you probably don’t want to supply this. If supplied, it is a callable that should return a Deferred that delivers an IProtocol or ConnectError. See twisted.internet.interfaces.IStreamClientEndpoint.connect Note that this parameter is ignored if config.ControlPort == 0 Returns a Deferred which callbacks with txtorcon.Tor instance, from which you can retrieve the TorControlProtocol instance via the .protocol property.

HACKS:

3.1. API Documentation 59 txtorcon Documentation, Release 20.0.0

1. It’s hard to know when Tor has both (completely!) written its authentication cookie file AND is listening on the control port. It seems that waiting for the first ‘bootstrap’ message on stdout is sufficient. Seems fragile. . . and doesn’t work 100% of the time, so FIXME look at Tor source. XXX this “User” thing was, IIRC, a feature for root-using scripts (!!) that were going to launch tor, but where tor would drop to a different user. Do we still want to support this? Probably relevant to Docker (where everything is root! yay!) User: if this exists, we attempt to set ownership of the tempdir to this user (but only if our effective UID is 0).

3.1.2 Tracking and Changing Live Tor State

TorState class txtorcon.TorState(protocol, bootstrap=True) Bases: object This tracks the current state of Tor using a TorControlProtocol. On setup it first queries the initial state of streams and circuits. It then asks for updates via the listeners. It requires an ITorControlProtocol instance. The control protocol doesn’t need to be bootstrapped yet. The Deferred .post_boostrap is driggered when the TorState instance is fully ready to go. The easiest way is to use the helper method txtorcon.build_tor_connection(). For details, see the implementation of that. You may add an txtorcon.interface.IStreamAttacher to provide a custom mapping for Strams to Circuits (by default Tor picks by itself). This is also a good example of the various listeners, and acts as an txtorcon.interface. ICircuitContainer and txtorcon.interface.IRouterContainer. Variables DO_NOT_ATTACH – Constant to return from an IAttacher indicating you don’t want to attach this stream at all. classmethod from_protocol(protocol, **kw) Create a new, boot-strapped TorState from a TorControlProtocol instance. Returns a Deferred that fires with a TorState instance circuits = None keys on id (integer) streams = None keys on id (integer) all_routers = None list of unique routers routers = None keys by hexid (string) and by unique names routers_by_name = None keys on name, value always list (many duplicate “Unnamed” routers, for example) routers_by_hash = None keys by hexid (string) guards = None potentially-usable as entry guards, I think? (any router with ‘Guard’ flag) entry_guards = None from GETINFO entry-guards, our current entry guards

60 Chapter 3. API Documentation txtorcon Documentation, Release 20.0.0

unusable_entry_guards = None list of entry guards we didn’t parse out authorities = None keys by name undo_attacher() Shouldn’t Tor handle this by turning this back to 0 if the controller that twiddled it disconnects? set_attacher(attacher, myreactor) Provide an txtorcon.interface.IStreamAttacher to associate streams to circuits. You are Strongly Encouraged to not use this API directly, and instead use txtorcon.Circuit. stream_via() or txtorcon.Circuit.web_agent() instead. If you do need to use this API, it’s an error if you call either of the other two methods. This won’t get turned on until after bootstrapping is completed. (‘__LeaveStreamsUnattached’ needs to be set to ‘1’ and the existing circuits list needs to be populated). stream_close_reasons = {'REASON_CONNECTREFUSED': 3, 'REASON_CONNRESET': 12, 'REASON_DESTROY': 5, 'REASON_DONE': 6, 'REASON_EXITPOLICY': 4, 'REASON_HIBERNATING': 9, 'REASON_INTERNAL': 10, 'REASON_MISC': 1, 'REASON_NOROUTE': 8, 'REASON_NOTDIRECTORY': 14, 'REASON_RESOLVEFAILED': 2, 'REASON_RESOURCELIMIT': 11, 'REASON_TIMEOUT': 7, 'REASON_TORPROTOCOL': 13} close_stream(stream, reason=’REASON_MISC’, **kwargs) This sends a STREAMCLOSE command, using the speciﬁed reason (either an int or one of the 14 strings in section 6.3 of tor-spec.txt if the argument is a string). Any kwards are passed through as ﬂags if they evaluated to true (e.g. “SomeFlag=True”). Currently there are none that Tor accepts. close_circuit(circid, **kwargs) This sends a CLOSECIRCUIT command, using any keyword arguments passed as the Flags (currently, that is just ‘IfUnused’ which means to only close the circuit when it is no longer used by any streams). Parameters circid – Either a circuit-id (int) or a Circuit instance Returns a Deferred which callbacks with the result of queuing the command to Tor (usually “OK”). If you want to instead know when the circuit is actually-gone, see Circuit.close on_circuit_new(callback) Parameters callback – will be called (with ‘circuit’ instance) when a CIRC NEW event happens on_circuit_launched(callback) Parameters callback – will be called (with ‘circuit’ instance) when a CIRC LAUNCHED event happens on_circuit_extend(callback) Parameters callback – will be called (with ‘circuit’ and ‘router’ instances) when a CIRC EXTENDED event happens on_circuit_built(callback) Parameters callback – will be called (with ‘circuit’ instance) when a CIRC BUILT event happens on_circuit_closed(callback) Parameters callback – will be called (with ‘circuit’ instance, and arbitrary kwargs) when a CIRC CLOSED event happens on_circuit_failed(callback) Parameters callback – will be called (with ‘circuit’ instance, and arbitrary kwargs) when a CIRC FAILED event happens

3.1. API Documentation 61 txtorcon Documentation, Release 20.0.0

add_circuit_listener(icircuitlistener) Adds a new instance of txtorcon.interface.ICircuitListener which will receive updates for all existing and new circuits. on_stream_new(callback) Parameters callback – will be called (with ‘stream’ instance) when a STREAM NEW event happens. on_stream_succeeded(callback) Parameters callback – will be called (with ‘stream’ instance) when a STREAM SUC- CEEDED event happens. on_stream_attach(callback) Parameters callback – will be called (with ‘stream’ and ‘circuit’ instances) when a stream is attached to a circuit on_stream_detach(callback) Parameters callback – will be called (with ‘stream’ instance and arbitrary kwargs) when a STREAM DETACHED happens on_stream_closed(callback) Parameters callback – will be called (with ‘stream’ instance and arbitrary kwargs) when a STREAM CLOSED event happens. on_stream_failed(callback) Parameters callback – will be called (with ‘stream’ instance and arbitrary kwargs) when a STREAM FAILED event happens. add_stream_listener(istreamlistener) Adds a new instance of txtorcon.interface.IStreamListener which will receive updates for all existing and new streams. build_circuit(routers=None, using_guards=True, purpose=None) Builds a circuit consisting of exactly the routers specified, in order. This issues an EXTENDCIRCUIT call to Tor with all the routers specified. Parameters • routers – a list of Router instances which is the path desired. To allow Tor to choose the routers itself, pass None (the default) for routers. • using_guards – A warning is issued if the first router isn’t in self.entry_guards. Returns A Deferred that will callback with a Circuit instance (with the .id member being valid, and probably nothing else). DO_NOT_ATTACH =