Experiences Deploying Multi-Vantage-Point Domain Validation at Let’s Encrypt Henry Birge-Lee Liang Wang Daniel McCarney Princeton University Princeton University Square Inc., Prev. Let’s Encrypt Roland Shoemaker Jennifer Rexford Prateek Mittal Unaffiliated, Prev. Let’s Encrypt Princeton University Princeton Unversity Abstract CA (vulnerable to BGP hijacking) An attacker can obtain a valid TLS certificate for a domain 4. HTTP GET by hijacking communication between a certificate authority 5. HTTP response (CA) and a victim domain. Performing domain validation from multiple vantage points can defend against these 1. Certificate request attacks. We explore the design space of multi-vantage-point 2. Challenge 6. Certificate domain validation to achieve (1) security via sufficiently 3. Server modifications diverse vantage points, (2) performance by ensuring low latency and overhead in certificate issuance, (3) manageability Domain owner Domain’s web server by complying with CA/Browser forum requirements, and requiring minimal changes to CA operations, and (4) a Figure 1: Domain control validation by Certificate Authority. low benign failure rate for legitimate requests. Our open- source implementation was deployed by the Let’s Encrypt CA in February 2020, and has since secured the issuance of more than half a billion certificates during the first year domain owner to demonstrate control of a core resource of its deployment. Using real-world operational data from associated with the domain (e.g., a web server, email address, Let’s Encrypt, we show that our approach has negligible or DNS record). latency and communication overhead, and a benign failure rate comparable to conventional designs with one vantage point. Finally, we evaluate the security improvements using a 1.1 Domain Validation Attacks and Defenses combination of ethically conducted real-world BGP hijacks, Internet-scale traceroute experiments, and a novel BGP Domain control validation is vulnerable to localized and simulation framework. We show that multi-vantage-point targeted Border Gateway Protocol (BGP) attacks that allow domain validation can thwart the vast majority of BGP attacks. adversaries to obtain bogus certificates [21,29]. These attacks Our work motivates the deployment of multi-vantage-point are possible because validation is often performed over domain validation across the CA ecosystem to strengthen insecure HTTP connections (since domain validation is a TLS certificate issuance and user privacy. necessary step that must occur before establishing an HTTPS connection). The BGP attack puts the adversary on the path between the CA and the victim domain or the victim domain 1 Introduction and the CA. Then, the adversary asks the CA for a certificate for the victim domain. The validation request (e.g., an HTTP Certificate Authorities (CAs) establish trust on the Internet GET to the victim domain) is directed to the adversary where by associating domains with the correct public keys through it is maliciously answered (Figure1). With its validation digitally signed certificates. A certificate contains domain request successful, the adversary can obtain a certificate for name(s) and the associated public key. The CAs must the domain. These attacks are particularly effective because ensure that a certificate is only given to the legitimate the BGP attack can be localized (affecting the target CA) owner of a domain. Otherwise, an adversary with a bogus and short-lived (during domain validation) but lead to the certificate could circumvent the assurances of confidentiality adversary obtaining a universally-valid multi-year certificate and integrity offered by HTTPS, and then decrypt or for the victim’s domain. These attacks are well within the modify sensitive user data (e.g., logins, financial information, capability of repressive regimes which have been accused cryptocurrency credentials [22]). To prevent these attacks, of launching BGP attacks [26] and have motive to intercept CAs perform domain control validation that requires the encrypted communications [1]. Internet NOT affected by MultiVA design. Validating from multiple vantage points the adversary’s BGP attacks CA Internet affected by seems like a simple idea. Yet, creating a production-grade the adversary’s BGP attacks Decision : 3 Validation OK system is challenging, due to competing trade-offs: Issue the cert • Security. The multiple vantage points must offer VP1 sufficiently diverse perspectives on routing to ensure that Validation OK 1 some vantage points can reach the legitimate domain. (A) 2 Cert request: Validation I own domain A Also, the quorum policy (i.e., the “vote” among the Response vantage points before signing a certificate) must be Challenge strong enough to thwart attacks, without sacrificing performance and robustness. Victim domain A Adversary • Manageability. Validating from multiple vantage points CA requires more server and network resources, spread 3 Decision : across more locations. This may require billing Two validation failures Can’t issue the cert arrangements with multiple cloud providers. In addition, Validation fail VP3 the CA/Browser forum, which decides the rules for the VP1 Validation fail operation of publicly trusted CAs, places security and VP2 Validation OK 1 Cert request: auditing requirements on the data centers CAs use [25]. (B) I own domain A Thus, a system with multiple vantage points may require ? 2 Response Challenge Validation the maintenance and auditing of multiple data centers. Challenge ? Challenge • Performance. The latency introduced by additional vantage points should not significantly slow down the Victim domain A Adversary overall domain control validation process. Similarly, the approach should have low communication overhead. Figure 2: A localized BGP attack affects a portion of the The implementation of multi-vantage-point domain Internet. If the CA has only one vantage point (A), the validation should also be incrementally deployable. adversary successfully gets the certificate. With multiple Performance constraints are particularly sensitive when vantage points (B), the CA detects the attack as two vantage deploying on a live production system, requiring careful points reach the legitimate server and fail the validations. system monitoring and a phased deployment. • Benign failure. A benign (validation) failure is a non- To mitigate these attacks, CAs need to defend themselves malicious validation request that should have been from routing attacks on domain control validation. A successful but was blocked because of validation failures promising approach is to perform validation from multiple caused by external factors in some vantage points. The diverse vantage points, to make it hard for the adversary’s failures are mostly caused by DNS propagation delay attack to “fool” all (or many) of the vantage points [21]. and configuration errors; see §4.2. A multi-vantage-point Vantage points unaffected by the BGP attack reach the validation system should not throw a significant number legitimate victim domain and observe that domain control of benign failures. validation has not been completed (see Figure2). This would We explore the complex design space of multi-vantage- stop the CA from issuing a certificate to the adversary. With point domain validation to balance the trade-offs among effective multi-vantage-point validation in place, an adversary these challenges. We propose to use a deployment of only capable of launching localized BGP attacks will have multiple vantage points within a single cloud provider to significant difficulty obtaining a bogus certificate as the achieve good performance and manageability, as the site adversary cannot have topological proximity to all of the reliability engineering (SRE) and billing departments only CA’s diverse vantage points. Thus, successful attacks would need to interface with a single cloud provider. We satisfy require announcing BGP routes with broad scope (readily the compliance requirements imposed by the CA/Browser visible in public BGP monitoring platforms [11, 13]), such as forum by carefully tracking validation results from the advertising smaller sub-prefixes (which is infeasible for /24 original CA and the cloud vantage points, respectively. IP prefixes). We carefully select cloud vantage points across diverse geographic locations to ensure sufficient diversity and system 1.2 MultiVA Design, Deployment, & Analysis security, and connect them to existing CA components using mutually-authenticated TLS. Our design balances the number This paper presents the design and evaluation of multiVA, and location of vantage points to control the trade-off between the first real-world deployment of the multi-vantage-point security, manageability, performance, and benign failures. countermeasure to secure domain control validation. More vantage points would improve security, but may increase validation overhead and manageability difficulty. Finally, we majority of BGP attacks (additionally see Table3 for a incorporate a configurable domain validation quorum policy summary of lessons learned through design, deployment, to strike a balance between security and benign failures. and evaluation aspects of this project). Finally, we also make recommendations for further enhancing the multiVA MultiVA deployment. We develop a fully open-source deployment via additional diverse vantage points. implementation of our multiVA design. Notably, our imple- mentation does not require any changes to the Automated We hope our work motivates industry-wide adoption