Adding Public Key Security to SSH
Total Page:16
File Type:pdf, Size:1020Kb
Adding Public Key Security to SSH A Thesis Submitted to the Faculty in partial fulfillment of the requirements for the degree of Master of Science in Computer Science by Yasir Ali DARTMOUTH COLLEGE Hanover, New Hampshire Feb, 20th, 2003 Examining Committee: ____________________________ Sean Smith (chair) ____________________________ Edward Feustel ____________________________ Christopher Hawblitzel !!!!!!!!!____________________________ !!!!!!!!!Carol Folt !!!!!!!!! Dean of Graduate Studies 1 2 Abstract SSH, the Secure Shell, is a popular software-based approach to network security. It is a protocol that allows user to log into another computer over a network, to execute commands in a remote machine, and to move files from one machine to another. It provides authentication and encrypted communications over unsecured channels. However, SSH protocol has an inherent security flaw. It is vulnerable to the “man-in-the- middle Attack”, when a user establishes his first SSH connection from a particular client to a remote machine. My thesis entails designing, evaluating and prototyping a public key infrastructure which can be used with the SSH2 protocol, in an academic setting, thus eliminating this vulnerability due to the man in the middle attack. The approach presented is different from the one that is based on the deployment of a Certificate Authority. My scheme does not necessarily require third party verification using a Certificate Authority; it is decentralized in nature and is relatively easy to set up. Keywords used: SSH, PKI, digital certificates, Certificate Authority, certification path, LDAP servers, Certificate Revocation List, X509v3 Certificate, OpenSSL, mutual authentication, and tunneled authentication. 3 Acknowledgments I want to thank Professor Sean Smith for his guidance, assistance and unremitting support over the last two years. He has always been always there to guide me and provide his insight on all the technical and non-technical aspects of this thesis work. His intellectual creativity, and insightful suggestions have been invaluable for my thesis research. I could never thank him enough for being an excellent mentor and a wonderful person. Sometimes words are just not enough to express one’s heartfelt gratitude. My thanks also go to the other members of my thesis committee – Professor Chris Hawblitzel, who assisted me in setting up and using Java Cryptographic API for my project; Professor Edward Feustel for the discussions during the course of the project, which helped me clarify my concepts. I would like to thank Professor Fillia Makedon for guiding and assisting me in every possible way throughout my course of study. I would also like to thank Carl Ellison for his discerning suggestions that were incorporated in developing a solution that is proposed in this paper. This work was supported in part by the Mellon Foundation, Internet2/AT&T, and by the U.S. Department of Justice, contract 2000-DT-CX-K001. Therefore, special thanks to them for making this project possible. 4 TABLE OF CONTENTS 1. INTRODUCTION............................................................................................................. 7 1A. This Paper: ...................................................................................................................... 8 2. BACKGROUND............................................................................................................... 8 2A. The Man in the middle Attack ....................................................................................... 14 2B. Spoofing Attack ............................................................................................................. 15 2C. A lesser significant Knownhosts2 file Attack................................................................ 16 2D. SSH Security Vulnerability............................................................................................ 16 3. POSSIBLE SOLUTIONS .............................................................................................. 17 3A. Solution 1: TLS (Transport Layer Security using User Certificates)........................... 17 3B. Solution 2: Smart Cards/Tokens (using X509 Certificates)......................................... 21 3C. Solution 3: TTLS (Tunneled Transport Layer Security using Passwords) ................. 22 3D. Solution 4: LEAP (Lightweight Extensible Authentication Protocol) ........................... 23 3E. Our Solution:.................................................................................................................. 26 i. HMAC...................................................................................................................... 28 ii. Centralized Minimal PKI Approach:........................................................................ 29 iii. How it works:............................................................................................................ 31 iv. A Decentralized Non-CA Approach:....................................................................... 32 v. Is it really secure? ................................................................................................... 34 4. DESIGN CHOICES FOR DARTMOUTH’S ACADEMIC SETTING............................ 40 4A. Suitable and scalable Certificate deployment methodology ....................................... 40 5. IMPLEMENTATION ...................................................................................................... 45 5A. Application Interface...................................................................................................... 46 5 6. SCOPE OF FUTURE WORK ....................................................................................... 50 7. RELATED WORK AND CONCLUSION....................................................................... 52 7A. F-SSH: ........................................................................................................................... 52 7B. SSH Client 3.2.0, SSH Communications Security Inc:................................................ 52 8. REFERENCES.............................................................................................................. 55 9. Appendix - Code Snippets Used to Extend TTSSH. .................................................. 58 6 LIST OF FIGURES Figure 1 Transport Layer Key Exchange ........................................................... 11 Figure 2 User Authentication Layer................................................................... 12 Figure 3 Semi-centralized minimal PKI Scheme................................................ 39 Figure 4 Decentralized Scheme ......................................................................... 39 Figure 5 Extended SSH2 ................................................................................... 46 Figure 6 Authentication Setup ........................................................................... 48 Figure 7 Retrieve Hash Store............................................................................. 49 Figure 8 Verify server public key received ........................................................ 49 Figure 9 Enter Pass Phrase ................................................................................ 49 Figure 10 Certificate Selection Window ............................................................ 50 LIST OF TABLES Table 1 Authentication Schemes........................................................................................... 25 7 1. INTRODUCTION This research thesis develops a minimal and yet scalable solution for the “Man-in-the- Middle-Attack” problem in the SSH protocol. This paper offers guidelines to develop a practicable and a viable Infrastructure for certificate usage for SSH so that the user can be certain that he has mutually1 authenticated with the server, in an academic setting such as the one at Dartmouth. Design constraints on the certificate authority, such as certificate generation, distribution, verification, revocation, and management have also been evaluated so that the user has to make a minimal effort to set up an ssh client and use it securely. The deployment constraints, design considerations and scalability issues with in a Dartmouth scenario are also evaluated in this paper. Dartmouth College has a decentralized network with several LANS each independent of the other and connected to a central backbone. Each building has its own LAN that is independently managed. The network at Dartmouth has roughly 8,000 ports. The design and code that was written to develop the prototype is open source and modifies the SSH protocol. It is based on the original open source code for TeraTerm SSH client that was written by Robert O'Callahan and OpenSSL libraries2. There are two ssh protocols, the SSH1 protocol and the SSH2 protocol. The SSH1 and the SSH2 protocols encrypt at different parts of the packets. The SSH2 protocol is a complete rewrite of the SSH1 protocol, and it does not use the same networking implementation that the SSH1 does [14.ii]. They have slightly different Key Exchange Algorithms. 1 The user is authenticated at the server and the server key is verified on the client machine. 2 The source code written and modified for this project is placed at www.cs.dartmouth.edu/~yasir/pki 8 Generally, the SSH2 protocol is considered more secure, as it avoids known vulnerabilities in the SSH1 implementation. Amongst the SSH products, there are OpenSSH and SSH clients and servers. Both the products implement