Non-Commutative Cryptography and Complexity of Group-Theoretic Problems
Total Page:16
File Type:pdf, Size:1020Kb
Mathematical Surveys and Monographs Volume 177 Non-commutative Cryptography and Complexity of Group-theoretic Problems Alexei Myasnikov Vladimir Shpilrain Alexander Ushakov With an appendix by Natalia Mosina American Mathematical Society http://dx.doi.org/10.1090/surv/177 Mathematical Surveys and Monographs Volume 177 Non-commutative Cryptography and Complexity of Group-theoretic Problems Alexei Myasnikov Vladimir Shpilrain Alexander Ushakov With an appendix by Natalia Mosina American Mathematical Society Providence, Rhode Island EDITORIAL COMMITTEE Ralph L. Cohen, Chair MichaelA.Singer Jordan S. Ellenberg Benjamin Sudakov MichaelI.Weinstein 2010 Mathematics Subject Classification. Primary 94A60, 20F10, 68Q25, 94A62, 11T71. For additional information and updates on this book, visit www.ams.org/bookpages/surv-177 Library of Congress Cataloging-in-Publication Data Myasnikov, Alexei G., 1955– Non-commutative cryptography and complexity of group-theoretic problems / Alexei Myas- nikov, Vladimir Shpilrain, Alexander Ushakov ; with an appendix by Natalia Mosina. p. cm. – (Mathematical surveys and monographs ; v. 177) Includes bibliographical references and index. ISBN 978-0-8218-5360-3 (alk. paper) 1. Combinatorial group theory. 2. Cryptography. 3. Computer algorithms. 4. Number theory. I. Shpilrain, Vladimir, 1960– II. Ushakov, Alexander. III. Title. QA182.5.M934 2011 005.82–dc23 2011020554 Copying and reprinting. Individual readers of this publication, and nonprofit libraries acting for them, are permitted to make fair use of the material, such as to copy a chapter for use in teaching or research. Permission is granted to quote brief passages from this publication in reviews, provided the customary acknowledgment of the source is given. Republication, systematic copying, or multiple reproduction of any material in this publication is permitted only under license from the American Mathematical Society. Requests for such permission should be addressed to the Acquisitions Department, American Mathematical Society, 201 Charles Street, Providence, Rhode Island 02904-2294 USA. Requests can also be made by e-mail to [email protected]. c 2011 by the American Mathematical Society. All rights reserved. The American Mathematical Society retains all rights except those granted to the United States Government. Printed in the United States of America. ∞ The paper used in this book is acid-free and falls within the guidelines established to ensure permanence and durability. Visit the AMS home page at http://www.ams.org/ 10987654321 161514131211 To our children: Nikita, Olga, Marie, . Contents Preface xiii Introduction 1 Part 1. Background on Groups, Complexity, and Cryptography Chapter 1. Background on Public-Key Cryptography 7 1.1. From key establishment to encryption 8 1.2. The Diffie-Hellman key establishment 8 1.3. The ElGamal cryptosystem 9 1.4. The RSA cryptosystem 10 1.5. Rabin’s cryptosystem 11 1.6. Authentication 11 1.6.1. The Feige-Fiat-Shamir scheme 12 Chapter 2. Background on Combinatorial Group Theory 13 2.1. Basic definitions and notation 13 2.2. Presentations of groups by generators and relators 14 2.3. Algorithmic problems of group theory: decision, witness, search 15 2.3.1. The word problem 15 2.3.2. The conjugacy problem 16 2.3.3. The decomposition and factorization problems 16 2.3.4. The membership problem 17 2.3.5. The isomorphism problem 17 2.3.6. More on search/witness problems 18 2.4. Nielsen’s and Schreier’s methods 20 2.5. Tietze’s method 21 2.6. Normal forms 22 Chapter 3. Background on Computational Complexity 25 3.1. Algorithms 25 3.1.1. Deterministic Turing machines 25 3.1.2. Non-deterministic Turing machines 26 3.1.3. Probabilistic Turing machines 26 3.2. Computational problems 26 3.2.1. Decision and search computational problems 27 3.2.2. Size functions 28 v vi CONTENTS 3.2.3. Stratification 30 3.2.4. Reductions and complete problems 31 3.2.5. Many-to-one reductions 31 3.2.6. Turing reductions 31 3.3. The worst case complexity 32 3.3.1. Complexity classes 32 3.3.2. Class NP 33 3.3.3. Polynomial-time many-to-one reductions and class NP 34 3.3.4. NP-complete problems 35 3.3.5. Deficiency of the worst case complexity 37 Part 2. Non-commutative Cryptography Chapter 4. Canonical Non-commutative Cryptography 41 4.1. Protocols based on the conjugacy search problem 41 4.2. Protocols based on the decomposition problem 43 4.2.1. “Twisted” protocol 44 4.2.2. Hiding one of the subgroups 44 4.2.3. Commutative subgroups 45 4.2.4. Using matrices 45 4.2.5. Using the triple decomposition problem 45 4.3. A protocol based on the factorization search problem 46 4.4. Stickel’s key exchange protocol 47 4.4.1. Linear algebra attack 48 4.5. The Anshel-Anshel-Goldfeld protocol 50 4.6. Relations between different problems 51 Chapter 5. Platform Groups 55 5.1. Braid groups 55 5.1.1. A group of braids and its presentation 56 5.1.2. Dehornoy handle free form 58 5.1.3. Garside normal form 59 5.2. Thompson’s group 60 5.3. Groups of matrices 63 5.4. Small cancellation groups 65 5.4.1. Dehn’s algorithm 65 5.5. Solvable groups 65 5.5.1. Normal forms in free metabelian groups 66 5.6. Artin groups 68 5.7. Grigorchuk’s group 69 Chapter 6. More Protocols 73 6.1. Using the subgroup membership search problem 73 6.1.1. Groups of the form F/[R, R]76 6.2. The MOR cryptosystem 76 CONTENTS vii Chapter 7. Using Decision Problems in Public-Key Cryptography 79 7.1. The Shpilrain-Zapata scheme 79 7.1.1. The protocol 80 7.1.2. Pool of group presentations 82 7.1.3. Generating random elements in finitely presented groups 83 7.2. Public-key encryption and encryption emulation attacks 85 Chapter 8. Authentication 91 8.1. A Diffie-Hellman-like scheme 91 8.2. A Feige-Fiat-Shamir-like scheme 92 8.3. Authentication based on the twisted conjugacy problem 93 8.4. Authentication from matrix conjugation 94 8.4.1. The protocol, beta version 94 8.4.2. The protocol, full version 95 8.4.3. Cryptanalysis 96 8.5. Authentication from actions on graphs, groups, or rings 97 8.5.1. When a composition of functions is hard-to- invert 97 8.5.2. Three protocols 98 8.5.3. Subgraph isomorphism 100 8.5.4. Graph homomorphism 101 8.5.5. Graph colorability 102 8.5.6. Endomorphisms of groups or rings 103 8.5.7. Platform: free metabelian group of rank 2 104 Z∗ 8.5.8. Platform: p 104 8.6. No-leak authentication by the Sherlock Holmes method 104 8.6.1. No-leak vs. zero-knowledge 105 8.6.2. The meta-protocol for authentication 106 8.6.3. Correctness of the protocol 107 8.6.4. Questions and answers 108 8.6.5. Why is the Graph ISO proof system not “no- leak”? 109 8.6.6. A particular realization: subset sum 109 8.6.7. A particular realization: polynomial equations 111 Part 3. Generic Complexity and Cryptanalysis Chapter 9. Distributional Problems and the Average Case Complexity 117 9.1. Distributional computational problems 117 9.1.1. Distributions and computational problems 117 9.1.2. Stratified problems with ensembles of distributions 119 9.1.3. Randomized many-to-one reductions 119 9.2. Average case complexity 120 9.2.1. Polynomial on average functions 121 9.2.2. Average case behavior of functions 125 9.2.3. Average case complexity of algorithms 125 viii CONTENTS 9.2.4. Average case vs. worst case 126 9.2.5. Average case behavior as a trade-off 126 9.2.6. Deficiency of the average case complexity 130 Chapter 10. Generic Case Complexity 131 10.1. Generic Complexity 131 10.1.1. Generic sets 131 10.1.2. Asymptotic density 132 10.1.3. Convergence rates 133 10.1.4. Generic complexity of algorithms and algorithmic problems 134 10.1.5. Deficiency of the generic complexity 135 10.2. Generic versus average case complexity 136 10.2.1. Comparing generic and average case complexities 136 10.2.2. When average polynomial time implies generic 136 10.2.3. When generically easy implies easy on average 138 Chapter 11. Generic Complexity of NP-complete Problems 141 11.1. The linear generic time complexity of subset sum problem 141 11.2. A practical algorithm for the subset sum problem 142 11.3. 3-Satisfiability 143 Chapter 12. Generic Complexity of Undecidable Problems 147 12.1. The halting problem 147 12.2. The Post correspondence problem 150 12.3. Finitely presented semigroups with undecidable word problem 150 Chapter 13. Strongly, Super, and Absolutely Undecidable Problems 155 13.1. Halting problem for Turing machines 157 13.2. Strongly undecidable problems 158 13.2.1. The halting problem is strongly undecidable 158 13.2.2. Strongly undecidable analog of Rice’s theorem 159 13.3. Generic amplification of undecidable problems 160 13.4. Semigroups with super-undecidable word problems 163 13.5. Absolutely undecidable problems 165 Part 4. Asymptotically Dominant Properties and Cryptanalysis Chapter 14. Asymptotically Dominant Properties 171 14.1. A brief description 171 14.2. Random subgroups and generating tuples 172 14.3. Asymptotic properties of subgroups 173 14.4. Groups with generic free basis property 174 14.5. Quasi-isometrically embedded subgroups 176 Chapter 15. Length Based and Quotient Attacks 179 CONTENTS ix 15.1. Anshel-Anshel-Goldfeld scheme 179 15.1.1. Description of the Anshel-Anshel-Goldfeld scheme 179 15.1.2. Security assumptions of the AAG scheme 179 15.2. Length based attacks 181 15.2.1.