Flexible and Scalable Public Key Security for SSH
Total Page:16
File Type:pdf, Size:1020Kb
Dartmouth College Dartmouth Digital Commons Dartmouth College Master’s Theses Theses and Dissertations 2-20-2003 Flexible and Scalable Public Key Security for SSH Yasir Ali Dartmouth College Follow this and additional works at: https://digitalcommons.dartmouth.edu/masters_theses Part of the Computer Sciences Commons Recommended Citation Ali, Yasir, "Flexible and Scalable Public Key Security for SSH" (2003). Dartmouth College Master’s Theses. 31. https://digitalcommons.dartmouth.edu/masters_theses/31 This Thesis (Master's) is brought to you for free and open access by the Theses and Dissertations at Dartmouth Digital Commons. It has been accepted for inclusion in Dartmouth College Master’s Theses by an authorized administrator of Dartmouth Digital Commons. For more information, please contact [email protected]. Adding Public Key Security to SSH A Thesis Submitted to the Faculty in partial fulfillment of the requirements for the degree of Master of Science in Computer Science by Yasir Ali DARTMOUTH COLLEGE Hanover, New Hampshire Feb, 20th, 2003 Examining Committee: ____________________________ Sean Smith (chair) ____________________________ Edward Feustel ____________________________ Christopher Hawblitzel !!!!!!!!!____________________________ !!!!!!!!!Carol Folt !!!!!!!!! Dean of Graduate Studies 1 2 Abstract SSH, the Secure Shell, is a popular software-based approach to network security. It is a protocol that allows user to log into another computer over a network, to execute commands in a remote machine, and to move files from one machine to another. It provides authentication and encrypted communications over unsecured channels. However, SSH protocol has an inherent security flaw. It is vulnerable to the “man-in-the- middle Attack”, when a user establishes his first SSH connection from a particular client to a remote machine. My thesis entails designing, evaluating and prototyping a public key infrastructure which can be used with the SSH2 protocol, in an academic setting, thus eliminating this vulnerability due to the man in the middle attack. The approach presented is different from the one that is based on the deployment of a Certificate Authority. My scheme does not necessarily require third party verification using a Certificate Authority; it is decentralized in nature and is relatively easy to set up. Keywords used: SSH, PKI, digital certificates, Certificate Authority, certification path, LDAP servers, Certificate Revocation List, X509v3 Certificate, OpenSSL, mutual authentication, and tunneled authentication. 3 Acknowledgments I want to thank Professor Sean Smith for his guidance, assistance and unremitting support over the last two years. He has always been always there to guide me and provide his insight on all the technical and non-technical aspects of this thesis work. His intellectual creativity, and insightful suggestions have been invaluable for my thesis research. I could never thank him enough for being an excellent mentor and a wonderful person. Sometimes words are just not enough to express one’s heartfelt gratitude. My thanks also go to the other members of my thesis committee – Professor Chris Hawblitzel, who assisted me in setting up and using Java Cryptographic API for my project; Professor Edward Feustel for the discussions during the course of the project, which helped me clarify my concepts. I would like to thank Professor Fillia Makedon for guiding and assisting me in every possible way throughout my course of study. I would also like to thank Carl Ellison for his discerning suggestions that were incorporated in developing a solution that is proposed in this paper. This work was supported in part by the Mellon Foundation, Internet2/AT&T, and by the U.S. Department of Justice, contract 2000-DT-CX-K001. Therefore, special thanks to them for making this project possible. 4 TABLE OF CONTENTS 1. INTRODUCTION............................................................................................................. 7 1A. This Paper: ...................................................................................................................... 8 2. BACKGROUND............................................................................................................... 8 2A. The Man in the middle Attack ....................................................................................... 14 2B. Spoofing Attack ............................................................................................................. 15 2C. A lesser significant Knownhosts2 file Attack................................................................ 16 2D. SSH Security Vulnerability............................................................................................ 16 3. POSSIBLE SOLUTIONS .............................................................................................. 17 3A. Solution 1: TLS (Transport Layer Security using User Certificates)........................... 17 3B. Solution 2: Smart Cards/Tokens (using X509 Certificates)......................................... 21 3C. Solution 3: TTLS (Tunneled Transport Layer Security using Passwords) ................. 22 3D. Solution 4: LEAP (Lightweight Extensible Authentication Protocol) ........................... 23 3E. Our Solution:.................................................................................................................. 26 i. HMAC...................................................................................................................... 28 ii. Centralized Minimal PKI Approach:........................................................................ 29 iii. How it works:............................................................................................................ 31 iv. A Decentralized Non-CA Approach:....................................................................... 32 v. Is it really secure? ................................................................................................... 34 4. DESIGN CHOICES FOR DARTMOUTH’S ACADEMIC SETTING............................ 40 4A. Suitable and scalable Certificate deployment methodology ....................................... 40 5. IMPLEMENTATION ...................................................................................................... 45 5A. Application Interface...................................................................................................... 46 5 6. SCOPE OF FUTURE WORK ....................................................................................... 50 7. RELATED WORK AND CONCLUSION....................................................................... 52 7A. F-SSH: ........................................................................................................................... 52 7B. SSH Client 3.2.0, SSH Communications Security Inc:................................................ 52 8. REFERENCES.............................................................................................................. 55 9. Appendix - Code Snippets Used to Extend TTSSH. .................................................. 58 6 LIST OF FIGURES Figure 1 Transport Layer Key Exchange ........................................................... 11 Figure 2 User Authentication Layer................................................................... 12 Figure 3 Semi-centralized minimal PKI Scheme................................................ 39 Figure 4 Decentralized Scheme ......................................................................... 39 Figure 5 Extended SSH2 ................................................................................... 46 Figure 6 Authentication Setup ........................................................................... 48 Figure 7 Retrieve Hash Store............................................................................. 49 Figure 8 Verify server public key received ........................................................ 49 Figure 9 Enter Pass Phrase ................................................................................ 49 Figure 10 Certificate Selection Window ............................................................ 50 LIST OF TABLES Table 1 Authentication Schemes........................................................................................... 25 7 1. INTRODUCTION This research thesis develops a minimal and yet scalable solution for the “Man-in-the- Middle-Attack” problem in the SSH protocol. This paper offers guidelines to develop a practicable and a viable Infrastructure for certificate usage for SSH so that the user can be certain that he has mutually1 authenticated with the server, in an academic setting such as the one at Dartmouth. Design constraints on the certificate authority, such as certificate generation, distribution, verification, revocation, and management have also been evaluated so that the user has to make a minimal effort to set up an ssh client and use it securely. The deployment constraints, design considerations and scalability issues with in a Dartmouth scenario are also evaluated in this paper. Dartmouth College has a decentralized network with several LANS each independent of the other and connected to a central backbone. Each building has its own LAN that is independently managed. The network at Dartmouth has roughly 8,000 ports. The design and code that was written to develop the prototype is open source and modifies the SSH protocol. It is based on the original open source code