Blockchain Layer Zero: Characterizing the Bitcoin Network through Measurements, Models, and Simulations Elias Rohrer Florian Tschorsch Distributed Security Infrastructures Distributed Security Infrastructures Technical University of Berlin Technical University of Berlin [email protected] fl[email protected] Abstract—In recent years, research has shown the networking In this work, we methodically characterize blockchain net- layer’s significant influence on the scalability, security, and works in reference to the paradigmatic Bitcoin peer-to-peer privacy of blockchain systems. Such large-scale networks how- network, explore the state-of-the-art protocols, and highlight ever exhibit a degree of complexity that demands model-based simulations as real-world experiments are often not possible. this vital design space. Our main contributions comprise a In this work, we methodically characterize blockchain networks comprehensive measurement study, designing a blockchain by reference to the paradigmatic Bitcoin peer-to-peer network, network simulation model, and a simulation study validating explore the state-of-the-art protocols, and emphasize this key the model as well as providing additional insights. design space. To this end, we conducted a longitudinal measure- In a first step, we study the Bitcoin network empirically ment study on the Bitcoin network, from which we extract a comprehensive network model and implement it as part of the through an extended measurement study. To this end, we bns network simulation framework. We validate the model in deployed measurement nodes in seven geographically dis- comparison to real-world measurements as well as to results from tributed locations all over the world and conducted long-term related work. Moreover, we experimentally show how network measurements on the provisioned bandwidths and latencies of utilization and miners’ geographical location impact the block Bitcoin peers. We moreover recorded the regional distributions propagation characteristics. of peers and miners in the network, as well as block sizes and the corresponding validation delays incurred. This comprehen- I. INTRODUCTION sive measurement study confirms and extends prior work on Even though they have become increasingly relevant infras- blockchain networks. tructures, open blockchain networks, such as Bitcoin [1] and Second, we extract parameters from this data set that induce Ethereum [2], are still struggling to get ready for global mass a geographically clustered network model, which builds the adoption. Research on blockchain systems in recent years often foundation for realistic network simulations. We implement aimed at addressing the bottlenecks of the consensus layer by this model in bns, a modular simulation framework for scaling-out and adding additional layers of abstraction, such blockchain networks that allows for configurable networking as transaction off-chaining. However, prior empirical work stacks. It is based on ns-3 and offers a layered design closely has shown the properties of the peer-to-peer network layer to resembling the internet architecture. To the best of our knowl- have a significant influence on the security and performance edge, bns is the first networking-centric simulation framework of blockchain systems [3, 4]. It even has been identified as allowing to capture complex network effects of blockchains. the impeding factor for transaction scalability [5]. Moreover, Lastly, we validate the introduced simulation model by com- analytical works focusing on the consensus layer proved paring the results of simulated network experiments to real- that the consistency properties of Nakamoto-style blockchain world measurements. We show that the simulations using bns protocols hold in the partially synchronous network model, framework in comparison to other blockchain simulators [4, that is, when all blocks reach all participants in bounded 9] resemble the real-world ground truth more accurately. time [6, 7, 8]. Consequently, consistency, security, and chain- Moreover, we show the capabilities of bns by evaluating quality properties suffer when network-induced propagation the influence of different block propagation mechanisms and delay increases. block sizes on the network utilization. We confirm that high Given the complexity of large-scale peer-to-peer networks, network utilization or even congestion significantly increases network-layer behavior is often studied in controlled environ- the rate of stale blocks, which has been shown to negatively ments such as network simulations. While relying on empirical impact the security of the consensus layer [4]. Furthermore, datasets and simulation models is generally inevitable, prior we study the impact of miners’ geographic locations on the entries studying blockchain systems based on network simu- block propagation process and show some geographic regions lation often revert to assumptions that are of simplistic nature. to be disadvantaged in the block race. The remainder of this paper is structured as follows. Sec- protocol, blocks were announced via immediately forwarded tion II gives an overview of the Bitcoin network protocol. In inv messages, and receiving nodes would request blocks Section III, we present the results of our measurement study and headers independently with getheaders and getdata on the Bitcoin peer-to-peer network. Section IV introduces the requests. These would then again answered by corresponding bns network simulator enabling the simulation of blockchain headers and block messages. networks in accordance to a geographic topology model. In However, since protocol version 70012 was introduced with Section V, we validate the underlying network model and Bitcoin Core v0.12.0, the default block propagation scheme study the impact of the miners’ geographic location on the changed [12]: blocks are announced directly by sending block propagation. Section VI discusses related work before headers messages, which reduce the propagation delay. the paper concludes with Section VII. Yet, when more than one block has to be announced, the client falls back to the inv-based announcement scheme. II. STATE OF THE BITCOIN NETWORK PROTOCOL Nodes enable this new protocol by sending a sendheaders The backbone of the Bitcoin network is comprised of message after the initial handshake. In addition, a scheme for nodes that form an open and unstructured peer-to-peer overlay compact block relay was introduced [13], which allows to send network: everyone who wants to participate in the network block announcements in a more bandwidth efficient way. In can setup a so-called full node, i.e., run a software which particular, it allows nodes to only retrieve the transaction data implements the Bitcoin protocol and replicates the entire they are missing from an announced block, which can severely blockchain. We base our account of the protocol on the reduce the bandwidth overhead of block propagation, but is behavior of the reference client, Bitcoin Core [10]. prone to induce an additional latency overhead. The Bitcoin peer protocol is an unencrypted TCP-based While other blockchain networks may exhibit their own network protocol in which nodes pick their neighbors in idiosyncrasies, they often coincide with Bitcoin’s general a randomized fashion: every node establishes 8 outgoing networking paradigm, i.e., block and transaction propagation connections and, if reachable and configured, accepts up to through broadcast in an unstructured overlay topology. For 117 incoming connections, resulting in a maximum connection example, even though Ethereum’s [2] peer discovery is based count of 125. By convention, nodes that accept incoming on a Kademlia [14] overlay construction, the actual eth connections from other peers are called servers, and clients, if propagation protocol follows a schematism similar to that of they only establish outgoing connections. The Bitcoin software Bitcoin [15, 16]. However, at the time of writing, it chooses keeps a local database of known peer addresses from which it to forgo the request-response scheme over unsolicited block randomly draws candidates for outgoing connections. If this propagation and also does not implement discussed protocol database is empty, e.g., when the software is started for the first extensions, such as a compact block relay mechanism [17]. time, it is bootstrapped by querying a number of community- As there are still many similarities, the Bitcoin approach to run DNS servers whose addresses are hard-coded in the client networking might rightfully be classified as paradigmatic for software. They return peer IP addresses as the contents of SRV blockchain networks overall. resource records. Peer addresses are also gossiped to network III. BITCOIN MEASUREMENT STUDY neighbors and can likewise be requested by and from each network participant. In order to characterize the network conditions currently After a TCP connection is established, the nodes exchange found in the Bitcoin network, we conducted a network mea- version/verack messages which transport crucial peer surement study that allows us to categorize the network peers data and additionally serve as basic handshake messages. along the lines of seven regional clusters: North America (NA), Moreover, the exchange of peer address data, transaction South America (SA), Europe (EU), Oceania (OC), Asia (AS), 1 forwarding, and block propagation is initiated. Every network Africa (AF), and China (CN) . In the following, we present participant may insert new