Mastering Bitcoin Andreas M. Antonopoulos Mastering Bitcoin by Andreas M. Antonopoulos Copyright © 2010 Andreas M. Antonopoulos LLC.. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/ institutional sales department: 800-998-9938 or [email protected]. Editors: Mike Loukides and Allyson MacDonald Indexer: FIXME FIXME Production Editor: Melanie Yarbrough Cover Designer: Karen Montgomery Copyeditor: FIXME FIXME Interior Designer: David Futato Proofreader: FIX ME! Illustrator: Rebecca Demarest December 2014: First Edition Revision History for the First Edition: 2014-04-07: Early release revision 1 2014-06-02: Early release revision 2 2014-06-27: Early release revision 3 2014-07-30: Early release revision 4 2014-09-16: Early release revision 5 2014-10-07: Early release revision 6 See http://oreilly.com/catalog/errata.csp?isbn=9781449374044 for release details. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc. !!FILL THIS IN!! and related trade dress are trademarks of O’Reilly Media, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly Media, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. While every precaution has been taken in the preparation of this book, the publisher and authors assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. ISBN: 978-1-449-37404-4 [?] Table of Contents Preface. ix 1. Introduction. 1 What is Bitcoin? 1 History of Bitcoin 3 Bitcoin Uses, Users and Their Stories 4 Getting Started 6 Quick Start 7 Getting your first bitcoins 9 Sending and receiving bitcoins 10 2. How Bitcoin Works. 15 Transactions, Blocks, Mining, and the Blockchain 15 Bitcoin Overview 16 Buying a cup of coffee 16 Bitcoin Transactions 18 Common Transaction Forms 20 Constructing a Transaction 22 Getting the right inputs 22 Creating the outputs 24 Adding the transaction to the ledger 25 Bitcoin Mining 26 Mining transactions in blocks 28 Spending the transaction 29 3. The Bitcoin Client. 31 Bitcoin Core - The reference implementation 31 Running Bitcoin Core for the first time 32 Compiling Bitcoin Core from the source code 33 iii Using Bitcoin Core’s JSON-RPC API from the command line 39 Getting information on the Bitcoin Core client status 41 Wallet setup and encryption 41 Wallet backup, plain-text dump and restore 42 Wallet addresses and receiving transactions 43 Exploring and decoding transactions 45 Exploring blocks 48 Creating, signing and submitting transactions based on unspent outputs 50 Alternative clients, libraries and toolkits 56 Libbitcoin and sx tools 56 pycoin 57 btcd 58 4. Keys, Addresses, Wallets. 61 Introduction 61 Public key cryptography and crypto-currency 62 Private and Public Keys 63 Private Keys 63 Public Keys 65 Elliptic Curve Cryptography Explained 65 Generating a public key 68 Bitcoin Addresses 70 Base58 and Base58Check Encoding 72 Key Formats 76 Implementing Keys and Addresses in Python 81 Wallets 84 Non-Deterministic (Random) Wallets 85 Deterministic (Seeded) Wallets 86 Mnemonic Code Words 86 Hierarchical Deterministic Wallets (BIP0032/BIP0044) 88 Advanced Keys and Addresses 98 Encrypted Private Keys (BIP0038) 98 Pay To Script Hash (P2SH) and Multi-Sig Addresses 99 Vanity Addresses 100 Paper Wallets 105 5. Transactions. 111 Introduction 111 Transaction Lifecycle 111 Creating Transactions 112 Broadcasting Transactions to the Bitcoin Network 112 Propagating Transactions on the Bitcoin Network 113 iv | Table of Contents Transaction Structure 113 Transaction Outputs and Inputs 114 Transaction Outputs 115 Transaction Inputs 117 Transaction Fees 120 Adding Fees to Transactions 121 Transaction Chaining and Orphan Transactions 122 Transaction Scripts and Script Language 123 Script Construction (Lock + Unlock) 123 Scripting Language 125 Turing Incompleteness 127 Stateless Verification 128 Standard Transactions 128 Pay to Public Key Hash (P2PKH) 128 Pay-to-Public-Key 131 Multi-Signature 132 Data Output (OP_RETURN) 133 Pay to Script Hash (P2SH) 134 6. The Bitcoin Network. 139 Peer-to-Peer Network Architecture 139 Nodes Types and Roles 140 The Extended Bitcoin Network 142 Network Discovery 144 Full Nodes 147 Exchanging “Inventory” 148 Simplified Payment Verification (SPV) Nodes 150 Bloom Filters 154 Bloom Filters and Inventory Updates 159 Transaction Pools 160 Alert Messages 161 7. The Blockchain. 163 Introduction 163 Structure of a Block 164 Block Header 164 Block Identifiers - Block Header Hash and Block Height 165 The Genesis Block 166 Linking Blocks in the Blockchain 167 Merkle Trees 170 Merkle Trees and Simplified Payment Verification (SPV) 175 Table of Contents | v 8. Mining and Consensus. 177 Introduction 177 Bitcoin Economics and Currency Creation 178 De-centralized Consensus 181 Independent Verification of Transactions 182 Mining Nodes 183 Aggregating Transactions into Blocks 184 Transaction Age, Fees, and Priority 184 The Generation Transaction 186 Coinbase Reward and Fees 187 Structure of the Generation Transaction 188 Coinbase Data 189 Constructing the Block Header 191 Mining the Block 192 Proof-of-Work Algorithm 193 Difficulty Representation 198 Difficulty Target and Re-Targeting 199 Successfully Mining the Block 201 Validating a New Block 201 Assembling and Selecting Chains of Blocks 202 Blockchain Forks 204 Mining and the Hashing Race 209 The Extra Nonce Solution 210 Mining Pools 211 Consensus Attacks 214 9. Alternative Chains, Currencies, and Applications. 219 A taxonomy of alternative currencies and chains 220 Meta-Coin Platforms 220 Colored Coins 221 Mastercoin 222 Counterparty 222 Alt-coins 222 Evaluating an alt-coin 224 Monetary Parameter Alternatives: Litecoin, Dogecoin, Freicoin 224 Consensus Innovation: Peercoin, Myriad, Blackcoin, Vericoin, NXT 225 Dual-Purpose Mining Innovation: Primecoin, Curecoin, Gridcoin 227 Anonymity-Focused Alt-Coins: CryptoNote, Bytecoin, Monero, Zerocash/ Zerocoin, Darkcoin 228 Non-currency alt-chains 230 Namecoin 230 Bitmessage 232 vi | Table of Contents Ethereum 232 Future of Currencies 233 10. Bitcoin Security. 235 Security principles 235 Developing Bitcoin Systems Securely 236 The Root of Trust 237 User Security Best Practices 238 Physical Bitcoin Storage 239 Hardware Wallets 239 Balancing Risk (loss vs. theft) 239 Diversifying Risk 239 Multi-sig and Governance 240 Survivability 240 Conclusion 240 A. Appendix: Available commands with sx tools. 241 B. Appendix - pycoin, ku and tx.. 247 C. Appendix: Transaction Script Language Operators, Constants and Symbols. 255 D. Appendix - Bitcoin Improvement Proposals. 259 Table of Contents | vii Preface Writing the Bitcoin Book I first stumbled upon bitcoin in mid-2011. My immediate reaction was more or less “Pfft! Nerd money!” and I ignored it for another 6 months, failing to grasp its impor‐ tance. This is a reaction which I have seen repeated among many of the smartest people I know, which gives me some consolation. The second time I came across bitcoin in a mailing list discussion, I decided to read the white paper written by Satoshi Nakamoto, to study the authoritative source and see what it was all about. I still remember the moment I finished reading those 9 pages, when I realized that bitcoin was not simply a digital currency, but a network of trust that could also provide the basis for so much more than just currencies. That realization: “This isn’t money, it’s a de-centralized trust network,” started me on a four month journey to devour every scrap of information about bitcoin I could find. I became obsessed and enthralled, spending twelve or more hours each day glued to a screen, reading, writing, coding and learning as much as I could. I emerged from this state of fugue, more than 20 lbs lighter from lack of consistent meals, determined to dedicate myself to working on bitcoin. Two years later, after creating a number of small startups to explore various bitcoin- related services and products, I decided that it was time to write my first book. Bitcoin was the topic that had driven me into a frenzy of creativity, consumed my thoughts and was the most exciting technology I had encountered since the Internet. It was now time to share my passion about this amazing technology with a broader audience. Intended Audience This book is mostly intended for coders. If you can use a programming language, this book will teach you how cryptographic currencies work, how to use them and how to develop software that works with them. The first few chapters are also suitable as an in- depth introduction to bitcoin for non-coders - those trying to understand the inner workings of bitcoin and crypto-currencies. ix Why Are There Bugs On The Cover? The Leafcutter Ant is a species that exhibits highly complex behavior in a colony super- organism, but each individual ant operates on a set of simple rules driven by social interaction and the exchange of chemical scents (pheromones). Per Wikipedia: “Next to humans, leafcutter ants form the largest and most complex animal societies on Earth.” Leafcutter ants don’t actually eat leaves, but rather use them to farm a fungus, which is the central food source for the colony. Get that? These ants are farming! While ants form a caste-based society and have a queen for producing offspring, there is no central authority or leader in an ant colony. The highly intelligent and sophisticated behavior exhibited by a multi-million member colony is an emergent property from the interaction of the individuals in a social network. Nature demonstrates that de-centralized systems can be resilient and can produce emergent complexity and incredible sophistication without the need for a central au‐ thority, hierarchy or complex parts.