1 GossipSub: A Secure PubSub Protocol for Unstructured, 59 2 60 3 Decentralised P2P Overlays 61 4 62 5 Dimitris Vyzovitis Yiannis Psaras 63 6 64 7 Protocol Labs UCL, UK & Protocol Labs 65 8 [email protected] [email protected] 66 9 ABSTRACT 67 10 68 11 This report is discussing the design choices behind gossipsub, the 69 12 pubsub protocol in use today in the IPFS ecosystem and in partic- 70 13 ular as a message mechanism protocol for IPNS records. We are 71 14 discussing the requirements of the protocol, related works in the 72 15 area, as well as the specific parameters that influence its behaviour. 73 16 74 17 CCS CONCEPTS 75 18 • Computer systems organization → Embedded systems; Re- Figure 1: Topics & Peers 76 19 dundancy; Robotics; • Networks → Network reliability. 77 20 78 21 KEYWORDS 79 22 IPFS, libp2p, pubsub, gossipsub 80 23 81 ACM Reference Format: 24 Dimitris Vyzovitis and Yiannis Psaras. 2019. GossipSub: A Secure PubSub 82 25 Protocol for Unstructured, Decentralised P2P Overlays. In Proceedings of 83 26 Protocol Labs TechRep (PL-TechRep-gossipsub-v0.1-Dec19). Protocol Labs, 84 27 8 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn 85 28 Figure 2: Message Delivered to Subscribers 86 29 1 INTRODUCTION 87 30 Publish/Subscribe systems have traditionally been used to facilitate 88 31 distribution of messages in an asynchronous manner between a set of pub/sub messaging and other operations related to the system 89 32 of publishers and subscribers. Senders (publishers) and receivers on a stable and continuous fashion. 90 33 (subscribers) are not in direct communication, but instead com- Unstructured P2P overlay networks, on the other hand, do not 91 34 municate through the pub/sub system. Subscribers declare their assume any connectivity properties for any of its nodes. That is, 92 35 topics of interest; publishers publish in one of the system’s topics. in unstructured P2P networks, nodes can be of any type (i.e., from 93 36 The pub/sub system then matches the two and delivers new mes- always-on rack servers, to ephemerally connected laptops and mo- 94 37 sages (or more commonly called events) to all subscribers under bile devices) and thus, connect and disconnect at random times. 95 38 a topic. Pub/Sub systems have been extensively used by Internet These random connectivity patterns make it impossible to assign 96 39 applications (see Twitter, RSS Feeds, Facebook), but also by general extra event routing or message caching responsibilities to any node 97 40 purpose Peer-to-Peer (P2P) systems. Not for distribution.in unstructured P2P networks. In turn, designing message propaga- 98 41 By and large, peer-to-peerUnpublished (P2P) networks can be split in two workingtion and guaranteeing draft. reliability of message delivery (that is, that a 99 42 categories: i) structured P2P overlays and ii) unstructured P2P over- message will reach all nodes in the network within a given amount 100 43 lays. In structured P2P overlays (or networks), the network has of time) is very difficult. 101 44 some structure, e.g., it is based on some topological or node hier- For these reasons, unstructured P2P networks very often use 102 45 archy. In such cases, some nodes (often called Super Nodes) can pub/sub protocols that are closer to flooding, or random walks 103 46 be assigned more responsibilities than others, such as for example, on the overlay to propagate event and membership information. 104 47 relay published events to subscribed nodes. Those nodes are also However, naïve flooding introduces a lot of extra traffic inthe 105 48 assumed to be dedicated servers, hence, they can support routing network, while random walks may take extended amounts of time 106 49 before reaching all nodes. 107 50 Permission to make digital or hard copies of all or part of this work for personal or 108 classroomUnpublished use is granted working without fee draft. provided Not that copies for distribution.are not made or distributed While previous work has addressed many different aspects and 51 for profit or commercial advantage and that copies bear this notice and the full citation requirements of pub/sub design for structured P2P networks, little 109 52 on the first page. Copyrights for components of this work owned by others than ACM has been done for unstructured P2P networks. “Gossip" has been 110 must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, 53 111 to post on servers or to redistribute to lists, requires prior specific permission and/or a introduced as a way to limit the number of messages propagated be- 54 fee. Request permissions from [email protected]. tween peers in pub/sub systems, as compared to flooding, where all 112 55 PL-TechRep-gossipsub-v0.1-Dec19, Dec 2019, AoE published messages are forwarded to all subscribed peers. In gossip- 113 © 2019 Association for Computing Machinery. 56 114 ACM ISBN 978-x-xxxx-xxxx-x/YY/MM...$15.00 based approaches, peers forward metadata related to messages they 57 https://doi.org/10.1145/nnnnnnn.nnnnnnn have “seen" without forwarding the messages themselves. There 115 58 2019-12-30 05:20. Page 1 of 1–8. 116 PL-TechRep-gossipsub-v0.1-Dec19, Dec 2019, AoE Vyzovitis and Psaras 117 have been numerous studies that explored gossip-based systems, paper intends to pursue. In its current version, we are starting by 175 118 but the scalability of large-scale, unstructured pub/sub systems has illustrating the protocol’s main design choices, together with a 176 119 not been addressed thoroughly. conceptual comparison against related works in the area. 177 120 In recent years, there has been significant momentum for design Gossipsub Design Summary: The pubsub protocol proposed 178 121 and deployment of decentralised Internet services and applications. here falls in the category of gossip-based pubsub protocols. Its main 179 122 Among others, such services include distributed and decentralised design features are a connected mesh, complemented by gossip 180 123 storage [8], [7], [12], [34], [2] but also computation systems [3]. functionality and lazy push. Lazy push is a technique used in pub- 181 124 The aim is to replace, or complement traditional centrally man- sub systems, according to which only metadata and not the full 182 125 aged and operated cloud services. End-users are contributing part message, is forwarded to subscribed nodes. In turn, if nodes are in- 183 126 of their resources to the network and get rewarded according to terested in the actual message whose metadata they have received, 184 127 contribution. These emerging systems are distributed in the sense they request the message explicitly. As is the case with many un- 185 128 of geographical spread and decentralised from the point of view of structured P2P networks, the first version of the protocol was closer 186 129 ownership, management and operation. to flood-based pubsub. However, as the IPFS network grew over 187 130 We are, therefore, witnessing a trend towards building P2P over- time, it was clear that flooding is not an efficient approach. Scal- 188 131 lays, where, in most cases, unreliable and non-dedicated end-user ability requirements quickly surfaced and resulted in the design 189 132 devices are active contributors to the network. In the absence of of meshsub, a connected mesh which is complemented by gossip 190 133 central control, messaging in those systems is of utmost impor- and lazy push as core features of the protocol. The combination of 191 134 tance in order to communicate operational processes (e.g., find file those techniques, along with the ability to plug in custom routing 192 135 or execute function), but also propagate management events. heuristics, constitute the main novelty of gossipsub. Simplicity of 193 136 Pub/Sub has seen a surge in usage from distributed applications implementation, as well as being reactive to dynamic network con- 194 137 in the area of decentralised services, such as decentralised chat ditions have been the driving philosophies behind the design of 195 138 and social networks [12], [10], collaborative editing tools without a gossipsub. 196 139 backend server [11], hosting of dynamic website content in unman- 197 140 aged P2P networks [8], storage and synchronisation of evolving 2 BACKGROUND & RELATED WORK 198 141 datasets, to name a few. 2.1 Tradeoffs in PubSub 199 142 In distributed storage systems and in the case of the InterPlan- 200 143 etary File System (IPFS) ecosystem in particular, pub/sub can be General purpose pub/sub messaging systems can prove very useful 201 144 used for several purposes, including content routing, i.e., one of the from several different aspects (from management and operation 202 145 most central and vital functions of the system. IPFS is a content- to performance) in P2P networks and as such come with many 203 146 addressable, distributed P2P storage network with hundreds of tradeoffs [19], [25]. Due to the wide variety of applications building 204 147 thousands of daily users. Users can participate in the network on top of pubsub systems, not all tradeoffs apply to all systems and 205 148 as unreliable nodes, e.g., using laptop devices and with frequent many of them are contradictory to each other. Below, we discuss 206 149 disconnections. The gossip-based pubsub protocol proposed here some of them and, where relevant, note how they influenced the 207 150 (acronymed “gossipsub") was developed with those system and design of gossipsub. 208 151 environment requirements in mind (i.e., unmanaged network and (1) Reliable Delivery. In case of no node downtime, all pub- 209 152 unreliable nodes) and is currently in use to push naming record lished messages should be delivered to all subscriber nodes.