Malicious Cryptography Exposing Cryptovirology Adam Young Moti Yung Wiley Publishing, Inc. Malicious Cryptography Malicious Cryptography Exposing Cryptovirology Adam Young Moti Yung Wiley Publishing, Inc. Executive Publisher: Robert Ipsen Executive Editor: Carol A. Long Developmental Editor: Eileen Bien Calabro Editorial Manager: Kathryn A. Malm Production Manager: Fred Bernardi This book is printed on acid-free paper. Copyright c 2004 by Adam Young and Moti Yung. All rights reserved. Published by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or trans- mitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clear- ance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470. Requests to the Publisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4447, E-mail: [email protected]. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specif- ically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commer- cial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Trademarks: Wiley, the Wiley Publishing logo and related trade dress are trademarks or registered trademarks of Wiley Publishing, Inc. and/or its affiliates in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated with any product or vendor mentioned in this book. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. ISBN: 0-7645-4975-8 Printed in the United States of America 10 9 8 7 6 5 4 3 2 1 Dedicated to Elisa (A. Y.) and to Maya (M. Y.) Contents Foreword xiii Acknowledgments xix Introduction xxi 1 Through Hacker’s Eyes 1 2 Cryptovirology 33 3 Tools for Security and Insecurity 51 3.1 Sources of Entropy . 53 3.2 Entropy Extraction via Hashing . 54 3.3 Unbiasing a Biased Coin . 57 3.3.1 Von Neumann’s Coin Flipping Algorithm . 57 3.3.2 Iterating Neumann’s Algorithm . 59 3.3.3 Heuristic Bias Matching . 60 3.4 Combining Weak Sources of Entropy . 62 3.5 Pseudorandom Number Generators . 66 3.5.1 Heuristic Pseudorandom Number Generation . 66 3.5.2 PRNGs Based on Reduction Arguments . 67 3.6 Uniform Sampling . 68 3.7 Random Permutation Generation . 71 3.7.1 Shuffling Cards by Repeated Sampling . 71 3.7.2 Shuffling Cards Using Trotter-Johnson . 73 3.8 Sound Approach to Random Number Generation and Use 76 3.9 RNGs Are the Beating Heart of System Security . 77 3.10 Cryptovirology Benefits from General Advances . 78 3.10.1 Strong Crypto Yields Strong Cryptoviruses . 78 3.10.2 Mix Networks and Cryptovirus Extortion . 80 vii viii Contents 3.11 Anonymizing Program Propagation . 85 4 The Two Faces of Anonymity 89 4.1 Anonymity in a Digital Age . 89 4.1.1 From Free Elections to the Unabomber . 90 4.1.2 Electronic Money and Anonymous Payments . 90 4.1.3 Anonymous Assassination Lotteries . 92 4.1.4 Kidnapping and Perfect Crimes . 93 4.1.5 Conducting Criminal Operations with Mixes . 94 4.2 Deniable Password Snatching . 97 4.2.1 Password Snatching and Security by Obscurity . 97 4.2.2 Solving the Problem Using Cryptovirology . 98 4.2.3 Zero-Knowledge Proofs to the Rescue . 100 4.2.4 Improving the Attack Using ElGamal . 101 5 Cryptocounters 103 5.1 Overview of Cryptocounters . 104 5.2 Implementing Cryptocounters . 105 5.2.1 A Simple Counter Based on ElGamal . 105 5.2.2 Drawback to the ElGamal Solution . 106 5.2.3 Cryptocounter Based on Squaring . 107 5.2.4 The Paillier Encryption Algorithm . 108 5.2.5 A Simple Counter Based on Paillier . 111 5.3 Other Approaches to Cryptocounters . 111 6 Computationally Secure Information Stealing 113 6.1 Using Viruses to Steal Information . 114 6.2 Private Information Retrieval . 115 6.2.1 PIR Based on the Phi-Hiding Problem . 117 6.2.2 Security of the Phi-Hiding PIR . 120 6.2.3 Application of the Phi-Hiding Technique . 122 6.3 A Variant of the Phi-Hiding Scheme . 122 6.4 Tagged Private Information Retrieval . 126 6.5 Secure Information Stealing Malware . 131 6.6 Deniable Password Snatching Based on Phi-Hiding . 132 6.6.1 Improved Password-Snatching Algorithm . 133 6.6.2 Questionable Encryptions . 134 6.6.3 Deniable Encryptions . 139 6.7 Malware Loaders . 140 6.8 Cryptographic Computing . 141 Contents ix 7 Non-Zero Sum Games and Survivable Malware 147 7.1 Survivable Malware . 148 7.2 Elements of Game Theory . 150 7.3 Attacking a Brokerage Firm . 151 7.3.1 Assumptions for the Attack . 152 7.3.2 The Distributed Cryptoviral Attack . 153 7.3.3 Security of the Attack . 158 7.3.4 Utility of the Attack . 159 7.4 Other Two-Player Game Attacks . 161 7.4.1 Key Search via Facehuggers . 161 7.4.2 Catalyzing Conflict Among Hosts . 167 7.5 Future Possibilities . 167 8 Coping with Malicious Software 171 8.1 Undecidability of Virus Detection . 171 8.2 Virus Identification and Obfuscation . 172 8.2.1 Virus String Matching . 173 8.2.2 Polymorphic Viruses . 176 8.3 Heuristic Virus Detection . 182 8.3.1 Detecting Code Abnormalities . 182 8.3.2 Detecting Abnormal Program Behavior . 183 8.3.3 Detecting Cryptographic Code . 191 8.4 Change Detection . 197 8.4.1 Integrity Self-Checks . 197 8.4.2 Program Inoculation . 198 8.4.3 Kernel Based Signature Verification . 199 9 The Nature of Trojan Horses 201 9.1 Text Editor Trojan Horse . 202 9.2 Salami Slicing Attacks . 202 9.3 Thompson’s Password Snatcher . 203 9.4 The Subtle Nature of Trojan Horses . 206 9.4.1 Bugs May In Fact Be Trojans . 208 9.4.2 RNG Biasing Trojan Horse . 208 10 Subliminal Channels 211 10.1 Brief History of Subliminal Channels . 212 10.2 The Difference Between a Subliminal and a Covert Channel 214 10.3 The Prisoner’s Problem of Gustavus Simmons . 215 10.4 Subliminal Channels New and Old . 216 x Contents 10.4.1 The Legendre Channel of Gus Simmons . 217 10.4.2 The Oracle Channel . 220 10.4.3 Subliminal Card Marking . 222 10.4.4 The Newton Channel . 223 10.4.5 Subliminal Channel in Composites . 224 10.5 The Impact of Subliminal Channels on Key Escrow . 226 11 SETUP Attack on Factoring Based Key Generation 229 11.1 Honest Composite Key Generation . 231 11.2 Weak Backdoor Attacks on Composite Key Generation . 232 11.2.1 Using a Fixed Prime . 233 11.2.2 Using a Pseudorandom Function . 234 11.2.3 Using a Pseudorandom Generator . 236 11.3 Probabilistic Bias Removal Method . 239 11.4 Secretly Embedded Trapdoors . 241 11.5 Key Generation SETUP Attack . 244 11.6 Security of the SETUP Attack . 249 11.6.1 Indistinguishability of Outputs . 249 11.6.2 Confidentiality of Outputs . 252 11.7 Detecting the Attack in Code Reviews . 256 11.8 Countering the SETUP Attack . 259 11.9 Thinking Outside the Box . 261 11.10 The Isaac Newton Institute Lecture . 262 12 SETUP Attacks on Discrete-Log Cryptosystems 265 12.1 The Discrete-Log SETUP Primitive . 266 12.2 Diffie-Hellman SETUP Attack . 268 12.3 Security of the Diffie-Hellman SETUP Attack . 270 12.3.1 Indistinguishability of Outputs . 270 12.3.2 Confidentiality of Outputs . 271 12.4 Intuition Behind the Attack . 275 12.5 Kleptogram Attack Methodology . 276 12.6 PKCS SETUP Attacks . 277 12.6.1 ElGamal PKCS SETUP Attack . 277 12.6.2 Cramer-Shoup PKCS SETUP Attack . 279 12.7 SETUP Attacks on Digital Signature Algorithms . 280 12.7.1 SETUP in the ElGamal Signature Algorithm . 281 12.7.2 SETUP in the Pointcheval-Stern Algorithm . 282 12.7.3 SETUP in DSA . 283 Contents xi 12.7.4 SETUP in the Schnorr Signature Algorithm . 284 12.8 Rogue Use of DSA for Encryption . 285 12.9 Other Work in Kleptography . 286 12.10 Should You Trust Your Smart Card? . 288 Appendix A: Computer Virus Basics 295 A.1 Origins of Malicious Software . 295 A.2 Trojans, Viruses, and Worms: What Is the Difference? . 297 A.3 A Simple DOS COM Infector . 299 A.4 Viruses Don’t Have to Gain Control Before the Host . 303 Appendix B: Notation and Other Background Information 307 B.1 Notation Used Throughout the Book .