The Secure Socket API: TLS as an Operating System Service Mark O’Neill, Scott Heidbrink, Jordan Whitehead, Tanner Perdue, Luke Dickinson, Torstein Collett, Nick Bonner, Kent Seamons, and Daniel Zappala, Brigham Young University https://www.usenix.org/conference/usenixsecurity18/presentation/oneill This paper is included in the Proceedings of the 27th USENIX Security Symposium. August 15–17, 2018 • Baltimore, MD, USA ISBN 978-1-939133-04-5 Open access to the Proceedings of the 27th USENIX Security Symposium is sponsored by USENIX. The Secure Socket API: TLS as an Operating System Service Mark O’Neill Scott Heidbrink Jordan Whitehead Tanner Perdue Luke Dickinson Torstein Collett Nick Bonner Kent Seamons Daniel Zappala Brigham Young University [email protected], [email protected], [email protected], tanner [email protected] [email protected], [email protected], [email protected] [email protected], [email protected] Abstract Significant efforts to catalog developer mistakes and the complexities of modern security APIs have been pub- SSL/TLS libraries are notoriously hard for developers to lished in recent years [8, 12, 23, 4, 19]. As a result, use, leaving system administrators at the mercy of buggy projects have emerged that reduce the size of security and vulnerable applications. We explore the use of the APIs [20], enhance library security [1], and perform cer- standard POSIX socket API as a vehicle for a simpli- tificate validation checks on behalf of vulnerable applica- fied TLS API, while also giving administrators the abil- tions [3, 18, 9, 5]. A common conclusion of these works ity to control applications and tailor TLS configuration is that TLS libraries need to be redesigned to be simpler to their needs. We first assess OpenSSL and its uses in for developers to use securely. open source software, recommending how this function- In this work we present the Secure Socket API (SSA), ality should be accommodated within the POSIX API. a TLS API for applications designed to work within the We then propose the Secure Socket API (SSA), a min- confines of the existing standard POSIX socket API al- imalist TLS API built using existing network functions ready familiar to network programmers. We extend the and find that it can be employed by existing network POSIX socket API in a natural way, providing backwards applications by modifications requiring as little as one compatibility with the existing POSIX socket interface. line of code. We next describe a prototype SSA imple- This effort required an analysis of current security library mentation that leverages network system calls to provide use to guide our efforts, and careful interaction with ker- privilege separation and support for other programming nel network code to not introduce undue performance languages. We end with a discussion of the benefits and overhead in our implementation. The SSA enables devel- limitations of the SSA and our accompanying implemen- opers to quickly build TLS support into their applications tation, noting avenues for future work. and administrators to easily control how applications use TLS on their machines. We demonstrate our prototype SSA implementation across a variety of use cases and 1 Introduction also show how it can be trivially integrated into existing programming languages. Transport Layer Security (TLS1) is the most popular Our contributions are as follows: security protocol used on the Internet. Proper use of TLS allows two network applications to establish a se- • An analysis of contemporary use of TLS by 410 cure communication channel between them. However, Linux packages and a qualitative breakdown of improper use can result in vulnerabilities to various at- OpenSSL’s 504 API endpoints for TLS functional- tacks. Unfortunately, popular security libraries, such as ity. These analyses are accompanied by design rec- OpenSSL and GnuTLS, while feature-rich and widely- ommendations for the Secure Socket API, and may used, have long been plagued by programmer misuse. also serve as a guide for developers of security li- The complexity and design of these libraries can make braries to improve their own APIs. them hard to use correctly for application developers and even security experts. For example, Georgiev et al. find • A description of the Secure Socket API and how it that the “terrible design of [security library] APIs” is the fits within the existing POSIX socket API, with de- root cause of authentication vulnerabilities [11]. scriptions of the relevant functions, constants, and administrator controls. We also provide example 1Unless otherwise specified, we use TLS to indicate TLS and SSL usages and experiences creating new TLS applica- USENIX Association 27th USENIX Security Symposium 799 tions using the SSA that require less than ten lines from developers, which may not ever be provided due to of code and as little as one. We modify existing ap- project shutdown, financial incentive, or other reasons. plications to use the SSA, resulting in the removal Thus TLS connection security is at the mercy of appli- of thousands of lines of existing code. cation developers, despite their inability to properly use security APIs and unfamiliarity with the specific secu- • A description of and source code for a prototype rity needs of system administrators. One illustration of implementation of the Secure Socket API. We also the demand for administrator control is the Redhat-led provide a discussion of benefits and features of this effort to create a system-wide “CryptoPolicy” configu- implementation, and demonstrate the ease of adding ration file [15]. Through custom changes in OpenSSL SSA support to other languages. and GNUTLS, this configuration file allows developers • A description of and source code for a tool that to defer some security settings to administrators. dynamically ports existing OpenSSL-using applica- The synthesis of these two problem spaces is that de- tions to use the SSA without requiring modification. velopers lack a common, usable security API and admin- istrators lack control over secure connections. In this pa- Previous findings have motivated the work for simpler per we explore a solution space to this problem through TLS APIs and better administrator controls. This work the POSIX socket API and operating system control. We explores utilization of the POSIX socket API as a possi- seek to improve on prior endeavors by reducing the TLS ble avenue to address these needs. API to a handful of functions that are already offered to We also discuss some finer points regarding the im- and used by network programmers, effectively making plementation and use of the SSA. We outline the ben- the TLS API itself nearly transparent. This drastically efits and drawbacks of our chosen implementation, and reduces the code required to use TLS. We also explore do the same for some suggested alternative implemen- supporting programming languages beyond C/C++ with tations. For users of the SSA, we discuss the avenues a singular API implementation. Developers merely se- for SSA configuration and its deployment with respect to lect TLS as if it were a built-in protocol such as TCP or different platforms and skill levels of users. UDP. Moreover, this enables administrators to configure TLS policies system-wide, while allowing developers to 2 Motivation use options to add configuration and request stricter se- curity policies. TLS use by applications is mired by complicated APIs Shifting control of TLS to the operating system and and developer mistakes, a problem that has been well administrators may be seen as controversial. However, documented. The libssl component of the OpenSSL most operating systems already offer critical services 1.0 library alone exports 504 functions and macros for to applications to reduce code redundancy and to en- use by TLS-implementing applications. This problem is sure that the services are run in a manner that does not likely to persist, as the unreleased OpenSSL 1.1.1 has in- threaten system stability or security. For example, ap- creased this number substantially. This and other TLS plication developers on Linux and Windows are not ex- APIs have been criticized for their complexity [11, 12] pected to write their own TCP implementation for net- and, anecdotally, our own explorations find many func- working applications or to implement their own file sys- tions within libssl that have non-intuitive semantics, tem functionality when writing to a file. Moreover, oper- confusing names, or little-to-no use in applications. A ating systems and system administrators have been found body of work has cataloged developer mistakes when to focus more attention on security matters [17]. Thus using these libraries to validate certificates, resulting in we believe establishing operating system and adminis- man-in-the-middle vulnerabilities [4, 11, 8]. trator control of TLS and related security policies is in A related problem is that the reliance on application line with precedent and best practice. developers to implement security inhibits the control ad- ministrators have over their own machines. For exam- 3 SSA Design Goals ple, an administrator cannot currently dictate what ver- sion of TLS is used by applications she installs, what Our primary goal in developing the SSA is to find a so- cipher suites and key sizes are used, or even whether ap- lution that is both easy to use for developers and grants plications use TLS at all. This coupling of application a high degree of control to system administrators. Since functionality with security policy can make otherwise de- C/C++ developers on Linux and other Unix-like systems sirable applications unadoptable by administrators with already use the POSIX socket API to create applications incompatible security requirements. This problem is ex- that access the network, this API represents a compelling acerbated when security flaws are discovered in applica- path for simplification of TLS APIs. Other languages tions and administrators must wait for security patches use this API directly or indirectly, either through imple- 800 27th USENIX Security Symposium USENIX Association mentation of socket system calls or by wrapping another the component of OpenSSL responsible for TLS func- implementation.