The Broken Shield: Measuring Revocation Effectiveness in the Windows Code-Signing PKI Doowon Kim and Bum Jun Kwon, University of Maryland, College Park; Kristián Kozák, Masaryk University, Czech Republic; Christopher Gates, Symantec; Tudor Dumitras, University of Maryland, College Park https://www.usenix.org/conference/usenixsecurity18/presentation/kim This paper is included in the Proceedings of the 27th USENIX Security Symposium. August 15–17, 2018 • Baltimore, MD, USA ISBN 978-1-939133-04-5 Open access to the Proceedings of the 27th USENIX Security Symposium is sponsored by USENIX. The Broken Shield: Measuring Revocation Effectiveness in the Windows Code-Signing PKI Doowon Kim Bum Jun Kwon Kristian´ Kozak´ University of Maryland University of Maryland Masaryk University Christopher Gates Tudor Dumitras, Symantec Research Labs University of Maryland Abstract code. A common security policy is to trust executables that carry valid signatures from unsuspicious publishers. Recent measurement studies have highlighted security The premise for trusting these executables is that the threats against the code-signing public key infrastructure signing keys are not controlled by malicious actors. (PKI), such as certificates that had been compromised Unfortunately, anecdotal evidence and recent measure- or issued directly to the malware authors. The primary ments of the Windows code-signing ecosystem have doc- mechanism for mitigating these threats is to revoke the umented cases of signed malware [8, 9, 12, 23, 26] and abusive certificates. However, the distributed yet closed potentially unwanted programs (PUPs) [1, 13, 17, 28], nature of the code signing PKI makes it difficult to evalu- where the trusted certificates were either compromised ate the effectiveness of revocations in this ecosystem. In or issued directly to the malware authors. The pri- consequence, the magnitude of signed malware threat is mary defense against these threats is to revoke the cer- not fully understood. tificates involved in the abuse. For the better studied In this paper, we collect seven datasets, including the Web’s PKI, prior measurements have uncovered impor- largest corpus of code-signing certificates, and we com- tant problems with this approach, including long revo- bine them to analyze the revocation process from end to cation delays [6, 29, 30], large bandwidth costs for dis- end. Effective revocations rely on three roles: (1) discov- seminating the revocation information [19], and clients ering the abusive certificates, (2) revoking the certificates that do not check whether certificates are revoked [19]. effectively, and (3) disseminating the revocation infor- In contrast, little is currently known about the effec- mation for clients. We assess the challenge for discover- tiveness of revocations in the code signing PKI. With- ing compromised certificates and the subsequent revoca- out this understanding, platform security protections risk tion delays. We show that erroneously setting revocation making incorrect assumptions about how critical revo- dates causes signed malware to remain valid even after cations are for end-host security and about the practical the certificate has been revoked. We also report failures challenges for implementing effective revocations in the in disseminating the revocations, leading clients to con- code-signing ecosystem. tinue trusting the revoked certificates. Code signing uses a default-valid trust model, where certificate chains remain trusted until proven compro- 1 Introduction mised. Due to this fact, missing or delayed revocations for a certificate involved in abuse allow bad actors to gen- The code-signing Public Key Infrastructure (PKI) is a erate trusted executables until the certificate expires or is fundamental building block for establishing trust in com- successfully added to a revocation list. puter software [22]. This PKI allows software publish- Abusive code-signing certificates may also present a ers to sign their executables and to embed certificates security threat beyond their expiration dates, which is an that bind the signing keys to the publishers’ real-world important distinction from the Web’s PKI where the ex- identities. In turn, client platforms can verify the signa- piration date limits the use of a compromised certificate tures and check the publishers, to confirm the integrity and also puts a limit on how long a revocation for that of third-party programs and to avoid executing malicious certificate must be maintained. To avoid re-signing and USENIX Association 27th USENIX Security Symposium 851 Role Finding Implication The mark-recapture estimation for the number of compro- Discovery There might be malware with compromised certificates that mised certificates suggests that even a large AV vendor can of remain a threat for a long time without being detected. Potentially only see about 36.5% of the population. CAs took on average 171.4 days to revoke the compro- Compromised Compromised certificates are not discovered and revoked mised certificates after the malware signed with the cer- Certificates for a long time. tificates appeared in the wild. Setting CAs erroneously set effective revocation dates for 62 cer- Wrong effective revocation date setting results in the sur- Revocation Date tificates, causing 402 signed malware to remain valid. vival of signed malware although its certificates is revoked. Clients have no way to check the revocation status of the 788 certificates contain neither CRLs nor OCSP points. certificates. 13 CRLs and 15 OCSP servers had reachability issues. Dissemination OCSP servers responded with unknown or unauthorized of messages. CAs improperly maintain their CRLs and OCSP servers. Revocation 19 certificates have inconsistent responses from CRLs and Information OCSP; they are valid from OCSP but are revoked in CRLs. Errors in the revocation process are made, and later re- 278 revoked certificates were added and then later removed tracted. CAs misunderstood the code signing PKI and re- from 18 CRLs. moved expired certificates from CRLs. Table 1: Summary of findings. distributing binaries when a signing certificate expires, files, while soft revocations that set a revocation date af- Windows developers may extend the validity of binaries ter the issuance date may not cover undiscovered signed they release by including a trusted timestamp, provided malware. Moreover, the CAs also must properly main- by a Time-Stamping Authority (TSA), that certifies the tain their revocation infrastructure so that the informa- signing time of a binary. If a malicious binary is cor- tion of compromise can be disseminated to the clients. If rectly signed and timestamped before the expiration date the dissemination is not handled as it should be, it may of the certificate, it will remain trusted even after its cer- reduce the incentives for revoking code signing certifi- tificate expires—unless the certificate is revoked. This cates. These challenges render the code signing ecosys- means that prompt and effective revocations, even of ex- tem opaque and difficult to audit, which contributes to an pired certificates, are critical in the code signing PKI. under-appreciation of the security threats that result from An effective revocation process faces additional chal- ineffective revocations. lenges in the code signing ecosystem. This process in- In this paper, we present an end-to-end measurement volves three roles: (1) discovering certificates that are of certificate revocations in the code signing PKI; in compromised or controlled by malicious actors; (2) re- particular, how effective is the current revocation pro- voking these certificates effectively; and (3) disseminat- cess from discovery to dissemination, and what threats ing the revocation information so that it is broadly avail- are introduced if the process is not properly done. Our able. work extends prior works in the code signing PKI; previ- Unlike in the Web’s PKI, where potentially com- ous studies have focused on signed PUPs [1, 13, 28] and promised certificates can be discovered systematically signed malware [12], but there is no study of code sign- through network scanning [6, 29, 30], in the code sign- ing certificate revocation process yet. Unlike the prior ing PKI this requires discovering signed malware or PUP studies in the Web’s PKI [2, 6, 7, 10, 19] where TLS cer- samples on end-hosts around the world. Security compa- tificate can be collected by scanning the Internet, we are nies involved in this discovery process cannot observe all unable to utilize a comprehensive corpus of code sign- the hosts where a maliciously signed binary may appear. ing certificates since there is no official repository for This also makes it a challenge to detect the total number code signing certificates. To overcome the challenge, of certificates that are actively being used to sign mal- we utilize data sets that are publicly released from prior ware, which leads to an incorrect perception about the research [1, 13] and increase our coverage with Syman- need and urgency of revocations. Even though a signed tec’s internal repository of binary samples. We extract malicious binary is discovered, it is difficult to determine 145,582 unique leaf code signing certificates from the the date when a certificate revocation should become ef- data sets. From the code signing certificates, we also ex- fective. Hard revocations that invalidate the entire life tract 215 Certificate Revocation Lists (CRLs) used only of the certificate may invalidate too many benign signed for code signing certificates,