practice

DOI:10.1145/3347446 is additional code a programmer wants  Article development led by queue.acm.org to call. Adding a dependency avoids repeating work: designing, testing, de- bugging, and maintaining a specific Software reuse is finally here unit of code. In this article, that unit of but comes with risks. code is referred to as a package; some systems use the terms library and mod- BY RUSS COX ule instead. Taking on externally written depen- dencies is not new. Most programmers have at one point in their careers had to go through the steps of manually Surviving installing a required library, such as C’s PCRE or zlib; C++’s Boost or Qt; or Java’s JodaTime or JUnit. These pack- ages contain high-quality, debugged Software code that required significant exper- tise to develop. For a program that needs the functionality provided by one of these packages, the tedious work of manually downloading, in- Dependencies stalling, and updating the package is easier than the work of redeveloping that functionality from scratch. The high fixed costs of reuse, however, mean manually reused packages tend to be big; a tiny package would be easier to reimplement. FOR DECADES, DISCUSSION of software reuse was more A dependency manager (a.k.a. pack- common than actual software reuse. Today, the situation age manager) automates the download- ing and installation of dependency is reversed: developers reuse software written by others packages. As dependency managers every day, in the form of software dependencies, and the make individual packages easier to download and install, the lower fixed situation goes mostly unexamined. costs make smaller packages economi- My background includes a decade of working with cal to publish and reuse. For example, ’s internal source code system, which treats the Node.js dependency manager NPM 17 provides access to more than 750,000 software dependencies as a first-class concept, as packages. One of them, escape-string- well as developing support for dependencies in the Go regexp, consists of a single function programming language.2 that escapes regular expression opera- tors in its input. The entire implemen- Software dependencies carry with them serious tation is: risks that are too often overlooked. The shift to easy, var matchOperatorsRe = fine-grained software reuse has happened so quickly /[|\\{}( )[ \ ]^ $ + * ?.]/ g ; that we do not yet understand the best practices for module.exports = function (str) { choosing and using dependencies effectively, or even if (typeof str !== ’string’) { throw new TypeError( for deciding when they are appropriate and when not. ’Expected a string’); The purpose of this article is to raise awareness of the } return str.replace( risks and encourage more investigation of solutions. matchOperatorsRe, ’\\$&’); In software development today, a dependency };

36 COMMUNICATIONS OF THE ACM | SEPTEMBER 2019 | VOL. 62 | NO. 9 Before dependency managers, and RubyGems (Ruby) each host more practice. But there are important dif- publishing an eight-line code library than 100,000 packages. The arrival of ferences that are being ignored. would have been unthinkable: too this kind of fine-grained, widespread Decades ago, most developers trust- much overhead for too little benefit. software reuse is one of the most con- ed others to write the software they NPM, however, has driven the over- sequential shifts in software develop- depended on, such as operating sys- head approximately to zero, with the ment over the past two decades. And if tems and compilers. That software was result that nearly trivial functionality we are not more careful, it will lead to purchased from known sources, often can be packaged and reused. In late serious problems. with some kind of support agreement. April 2019, the escape-string-regexp There was still a potential for bugs or package was explicitly depended What Could Go Wrong? outright mischief,20 but at least the de- upon by almost a thousand other A package, for this discussion, is code velopers knew who they were dealing NPM packages, not to mention all the downloaded from the Internet. Adding with and usually had commercial or packages developers write for their a package as a dependency outsources legal recourses available. own use and don’t share. the work of developing that code—de- The phenomenon of open source Dependency managers now exist signing, writing, testing, debugging, software, distributed at no cost over the for essentially every programming lan- and maintaining—to someone else on Internet, has displaced many of those guage: Maven Central (Java), NuGet the Internet, often unknown to the pro- earlier software purchases. When reuse (.NET), Packagist (PHP), PyPI (Python), grammer. Using that code exposes the was difficult, there were fewer projects program to all the failures and flaws publishing reusable code packages. in the dependency. The program’s ex- Even though their licenses typically dis- ecution now literally depends on code claimed, among other things, any “im- downloaded from this stranger on the plied warranties of merchantability and Internet. Presented this way, it sounds fitness for a particular purpose,” the incredibly unsafe. Why would anyone projects built up well-known reputa- do this? tions that often factored heavily into Because it’s easy, it seems to work, people’s decisions about which to everyone else is doing it, and, most use. The commercial and legal sup- importantly, it seems like a natural port for trusting software sources

IMAGE BY T. EMIL EMIL T. BY IMAGE continuation of age-old established was replaced by reputational support.

SEPTEMBER 2019 | VOL. 62 | NO. 9 | COMMUNICATIONS OF THE ACM 37 practice

Many common early packages still en- No matter what the expected cost, joy good reputations: consider BLAS experiences with larger dependencies (published in 1979), Netlib (1987), suggest some approaches for estimat- libjpeg (1991), LAPACK (1992), HP STL ing and reducing the risks of adding a (1994), and zlib (1995). software dependency. Better tooling is Dependency managers have scaled Developers trust likely needed to help reduce the costs down this open source code reuse mod- more code with of these approaches, much as depen- el. Now, developers can share code at dency managers have focused to date the granularity of individual functions less justification on reducing the costs of downloading consisting of tens of lines of code. This for doing so. and installation. is a major technical accomplishment. Myriad packages are available, and Inspect the Dependency writing code can involve a large num- You would not hire a software devel- ber of them, but the commercial, legal, oper you have never heard of and know and reputational support mechanisms nothing about. You would learn more for trusting the code have not carried about the person first: check referenc- over. Developers trust more code with es, conduct a job interview, run back- less justification for doing so. ground checks, and so on. Before you The cost of adopting a bad depen- depend on a package found on the In- dency can be viewed as the sum, over ternet, it is similarly prudent to learn a all possible bad outcomes, of the cost bit about it first. of each bad outcome multiplied by A basic inspection can provide a its probability of happening (risk), as sense of how likely you are to run into shown in the equation. problems trying to use this code. If the inspection reveals likely minor prob- expected cost = cost(b) × probability(b) lems, you can take steps to prepare ∑ for or perhaps avoid them. If the in- b ∈ bad outcomes spection reveals major problems, it The context in which a dependency may be best not to use the package; will be used determines the cost of a maybe you will find a more suitable bad outcome. At one end of the spec- one, or maybe you need to develop trum is a personal hobby project, one yourself. Remember that open where the cost of most bad outcomes source packages are published by is near zero: you are just having fun, their authors in the hope they will bugs have no real impact other than be useful but with no guarantee of wasting time, and even debugging can usability or support. In the middle be fun. So, the risk probability almost of a production outage, you will be doesn’t matter—it’s being multiplied the one debugging the package. As by a failure cost of almost zero. At the the original GNU General Public Li- other end of the spectrum is produc- cense warned, “The entire risk as to tion software that must be maintained the quality and performance of the for years. Here, the cost of a bug in a program is with you. Should the pro- dependency can be very high: servers gram prove defective, you assume the may go down, sensitive data may be di- cost of all necessary servicing, repair vulged, customers may be harmed, or or correction.”7 companies may fail. High failure costs The following are some consider- make it much more important to esti- ations when inspecting a package and mate and then reduce any risk of a seri- deciding whether to depend on it: ous failure. Design. Is the documentation clear? Does the API have a clear design? If the authors can explain the package’s API and its design well in the documen- tation, that increases the likelihood they have explained the implementa- tion well to the computer in the source code. Writing code using a clear, well- designed API is also easier, faster, and hopefully less error-prone. Have the authors documented what they expect from client code in order to make fu-

38 COMMUNICATIONS OF THE ACM | SEPTEMBER 2019 | VOL. 62 | NO. 9 practice ture upgrades compatible? (Examples caught. If you insist on tests in code at least mean more people for whom include the C++23 and Go8 compatibil- you write (you do, right?), you should the code works well enough, along with ity documents.) insist on tests in code you outsource to faster detection of new bugs. Wide- Code quality. Is the code well writ- others. spread usage is also a hedge against the ten? Read some of it. Does it look like Assuming the tests exist, run, and question of continued maintenance; if the authors have been careful, consci- pass, you can gather more information a widely used package loses its main- entious, and consistent? Does it look by running them with runtime instru- tainer, an interested user is likely to like code you would want to debug? mentation such as code coverage analy- step forward. You may need to. sis, race detection,16 memory-allocation For example, libraries such as PCRE Develop your own systematic ways checking, and memory-leak detection. or Boost or JUnit are incredibly widely to check code quality. For example, used. That makes it more likely—al- something as simple as compiling a C though certainly not guaranteed—that or C++ program with important com- bugs you might otherwise run into piler warnings enabled (for example, have already been fixed, because others –Wall) can give you a sense of how se- ran into them first. riously the developers work to avoid Security. Will you be processing un- various undefined behaviors. Recent trusted inputs with the package? If so, languages such as Go, Rust, and Swift does it seem to be robust against mali- use an unsafe keyword to mark code cious inputs? Does it have a history of that violates the type system; look to security problems listed in the NVD see how much unsafe code there is. (National Vulnerability Database)?13 More advanced semantic tools such as Debugging. Find the package’s is- For example, in 2006 when Jeff Infer6 or SpotBugs19 are helpful, too. sue tracker. Are there many open bug Dean and I started work on Google Linters are less helpful: you should reports? How long have they been Code Search5—grep over public source ignore rote suggestions about topics open? Are there many fixed bugs? code—the popular PCRE regular ex- such as brace style and focus instead Have any bugs been fixed recently? If pression library seemed like an ob- on semantic problems. you see lots of open issues about what vious choice. In an early discussion Keep an open mind about unfamil- look like real bugs, especially if they with Google’s security team, however, iar development practices. For exam- have been open for a long time, that’s we learned that PCRE had a history ple, the SQLite library ships as a single not a good sign. On the other hand, of problems such as buffer overflows, 200,000-line C source file and a single if the closed issues show that bugs especially in its parser. We could have 11,000-line header called the amal- are rarely found and promptly fixed, learned the same by searching for gamation. The sheer size of these files that’s great. PCRE in the NVD. That discovery did should raise an initial red flag, but closer Maintenance. Look at the pack- not immediately cause us to abandon investigation would turn up the actual age’s commit history. How long has PCRE, but it did make us think more development source code, a traditional the code been actively maintained? Is carefully about testing and isolation. file tree with more than 100 C source it actively maintained now? Packages files, tests, and support scripts. It turns that have been actively maintained out the single-file distribution is built for an extended amount of time are automatically from the original sources more likely to continue to be main- and is easier for end users, especially tained. How many people work on those without dependency managers. the package? Many packages are per- (The compiled code also runs faster, be- sonal projects that developers create cause the compiler can see more optimi- and share for fun in their spare time. zation opportunities.) Others are the result of thousands of Testing. Does the code have tests? hours of work by a group of paid de- Can you run them? Do they pass? Tests velopers. In general, the latter kind of establish the code’s basic functionality package is more likely to have prompt Licensing. Is the code properly li- is correct, and they signal the developer bug fixes, steady improvements, and censed? Does it have a license at all? is serious about keeping it correct. For general upkeep. Is the license acceptable for your proj- example, the SQLite development tree On the other hand, some code really ect or company? A surprising fraction has an incredibly thorough test suite is “done.” For example, NPM’s escape- of projects on GitHub have no clear with more than 30,000 individual test string-regexp, shown earlier, may nev- license. Your project or company may cases, as well as developer documenta- er need to be modified again. impose further restrictions on the al- tion explaining the testing strategy.10 Usage. Do many other packages de- lowed licenses of dependencies. For On the other hand, if there are few tests pend on this code? Dependency man- example, Google disallows the use of or no tests, or if the tests fail, that’s a agers can often provide statistics about code licensed under AGPL-like licens- serious red flag. Future changes to usage, or you can use a Web search to es (too onerous), as well as WTFPL-like the package are likely to introduce re- estimate how often others write about licenses (too vague).9 gressions that could easily have been using the package. More users should Dependencies. Does the code have

SEPTEMBER 2019 | VOL. 62 | NO. 9 | COMMUNICATIONS OF THE ACM 39 practice

dependencies of its own? Flaws in in- package’s API and that it does what tute a different, equally appropriate direct dependencies are just as bad you think it does. (If you can’t or it dependency later, by changing only for your program as flaws in direct doesn’t, turn back now!) It is worth the wrapper. Migrating your per-proj- dependencies. Dependency managers making the extra effort to turn those ect tests to use the new interface will can list all the transitive dependen- programs into automated tests that test the interface and wrapper imple- cies of a given package, and each of can be run against newer versions mentation, as well as making it easy them should ideally be inspected as of the package. If you find a bug and to test any potential replacements for described here. A package with many have a potential fix, you will want to the dependency. dependencies incurs additional in- be able to rerun these project-specific For Code Search, we developed an spection work, because those same tests easily, to ensure the fix did not abstract Regexp class that defined dependencies incur additional risk break anything else. the interface Code Search needed that needs to be evaluated. It is especially worth exercising the from any regular expression engine. likely problem areas identified by the Then we wrote a thin wrapper around basic inspection. For Code Search, PCRE implementing that interface. we knew from past experience that The indirection made it easy to test PCRE sometimes took a long time to alternate libraries, and it prevented execute certain regular expression accidentally introducing knowledge searches. The initial plan was to have of PCRE internals into the rest of the separate thread pools for “simple” source tree. That in turn ensured it and “complicated” regular expres- would be easy to switch to a different sion searches. One of the first tests dependency if needed. was a benchmark comparing pcre- grep with a few other grep implemen- Isolate the Dependency Many developers have never looked tations. For one basic test case, pcre- Isolating a dependency at runtime may at the full list of transitive dependen- grep was 70 times slower than the also be appropriate in order to limit cies of their code and do not know fastest grep available, so we started the possible damage caused by bugs. what they depend on. For example, to rethink the plan to use PCRE. Even For example, Google Chrome allows the NPM user community discovered though PCRE was eventually dropped users to add dependencies—extension in March 2016 that many popular entirely, that benchmark remains in code—to the browser. When Chrome projects—including Babel, Ember, the code base today. launched in 2008, it introduced the and React—all depended indirectly critical feature (now standard in all on a tiny package called left-pad, con- Abstract the Dependency browsers) of isolating each extension sisting of a single eight-line function Depending on a package is a decision in a sandbox running in a separate op- body. They discovered this when the likely to be revisited later. Perhaps erating-system process.18 author of left-pad deleted that pack- updates will take the package in a An exploitable bug in a badly writ- age from NPM, inadvertently break- new direction. Perhaps serious secu- ten extension therefore did not au- ing most Node.js users’ builds.22 rity problems will be found. Perhaps a tomatically have access to the entire And left-pad is hardly exceptional in better option will come along. For all memory of the browser itself and this regard. For example, 30% of the these reasons, it is worth the effort to could be stopped from making inap- 750,000 packages published on NPM make it easy to migrate your project to propriate system calls.12 For Code depend—at least indirectly—on es- a new dependency. Search, until we dropped PCRE en- cape-string-regexp. Adapting Leslie If the package will be used from tirely, the plan was to isolate at least Lamport’s observation about distrib- many places in your project’s source the PCRE parser in a similar sandbox. uted systems, a dependency manager code, migrating to a new dependen- Today, another option would be a can easily create a situation in which cy would require making changes to lightweight hypervisor-based sand- the failure of a package you did not all those different source locations. box such as gVisor.11 Isolating depen- even know existed can render your Worse, if the package will be exposed dencies reduces the associated risks own code unusable. in your own project’s API, migrat- of running that code. ing to a new dependency would re- Even with these examples and oth- Test the Dependency quire making changes in all the code The inspection process should in- calling your API, which you might clude running a package’s own tests. not control. To avoid these costs, it If the package passes the inspection makes sense to define an interface of and you decide to make your project your own, along with a thin wrapper depend on it, the next step should implementing that interface using be to write new tests focused on the the dependency. Note that the wrap- functionality needed by your appli- per should include only what your cation. These tests often start out as project needs from the dependency, short stand-alone programs written not everything the dependency of- to ensure you can understand the fers. Ideally, that allows you to substi-

40 COMMUNICATIONS OF THE ACM | SEPTEMBER 2019 | VOL. 62 | NO. 9 practice er off-the-shelf options, runtime iso- developer community has a proverb lation of suspect code is still too dif- about this: “A little copying is better ficult and rarely done. True isolation than a little dependency.”14 would require a completely memory- safe language, with no escape hatch Upgrade the Dependency into untyped code. That’s challeng- Even after For a long time, the conventional wis- ing not just in entirely unsafe lan- all that work, dom about software was, “If it ain’t guages such as C and C++, but also in broke, don’t fix it.” Upgrading carries a languages that provide restricted un- you are not done chance of introducing new bugs; with- safe operations, such as Java when tending your out a corresponding reward—such as including JNI (Java Native Interface), a new feature you need—why take the or Go, Rust, and Swift when includ- dependencies. risk? This analysis ignores two costs. ing their “unsafe” features. Even in The first is the cost of the eventual a memory-safe language such as Ja- It’s important upgrade. In software, the difficulty of vaScript, code often has access to to continue to making code changes does not scale far more than it needs. In November linearly: making 10 small changes is 2018, the latest version of the NPM monitor them less work and easier to get right than package event-stream, which pro- and perhaps making one equivalent large change. vided a functional streaming API for The second is the cost of discovering JavaScript events, was discovered to even re-evaluate already-fixed bugs the hard way. Es- contain obfuscated malicious code your decision pecially in a security context, where that had been added 2.5 months ear- known bugs are actively exploited, ev- lier. The code, which harvested large to use them. ery day you wait is another day that at- Bitcoin wallets from users of the Co- tackers can break in. pay mobile app, was accessing system resources entirely unrelated to pro- cessing event streams.1 One of many possible defenses to this kind of prob- lem would be to better restrict what dependencies can access.

Avoid the Dependency If a dependency seems too risky and you can’t find a way to isolate it, the best answer may be to avoid it entirely, or at least to avoid the parts you have For example, consider what hap- identified as most problematic. pened at Equifax in 2017, as recounted For example, as we better under- by executives in detailed Congressional stood the risks and costs associated testimony.21 On March 7, a new vulner- with PCRE, our plan for Google Code ability in Apache Struts was disclosed, Search evolved from “use PCRE di- and a patched version was released. rectly,” to “use PCRE but sandbox the On March 8, Equifax received a notice parser,” to “write a new regular ex- from US-CERT (United States Comput- pression parser but keep the PCRE ex- er Emergency Readiness Team) about ecution engine,” to “write a new pars- the need to update any uses of Apache er and connect it to a different, more Struts. Equifax ran source code and efficient open source execution en- network scans on March 9 and March gine.” Later we rewrote the execution 15, respectively; neither scan turned engine as well, so that no dependen- up a particular group of public-facing cies were left, and we open sourced Web servers. On May 13, attackers the result: RE2.4 found the servers that Equifax’s se- If you need only a tiny fraction curity teams could not. They used the of a dependency, the simplest solu- Apache Struts vulnerability to breach tion may be to make a copy of what Equifax’s network and then steal de- you need (preserving appropriate tailed personal and financial informa- copyright and other legal notices, of tion about 148 million people over the course). You are taking on responsi- next two months. Equifax finally no- bility for fixing bugs, maintenance, ticed the breach on July 29 and publicly and so on, but you are also completely disclosed it on September 4. By the end isolated from the larger risks. The Go of September, Equifax’s CEO, CIO, and

SEPTEMBER 2019 | VOL. 62 | NO. 9 | COMMUNICATIONS OF THE ACM 41 practice

CSO had all resigned, and a Congres- reach production, then in most cases sional investigation was underway. delaying an upgrade is riskier than up- Equifax’s experience drives home grading quickly. the point that although dependency The window for security-critical managers know the versions they are upgrades is especially short. In the using at build time, other arrange- If a dependency aftermath of the Equifax breach, fo- ments must be made to track that seems too risky and rensic security teams found evidence information through the produc- that attackers (perhaps different tion deployment process. For the you can’t find a ones) had successfully exploited the Go language, we are experimenting way to isolate it, Apache Struts vulnerability on the with automatically including a ver- affected servers on March 10, only sion manifest in every binary, so that the best answer three days after it was publicly dis- deployment processes can scan bi- closed, but they had run only a single naries for dependencies that need may be to avoid whoami command. upgrading. Go also makes that in- it entirely, or at formation available at runtime, so Watch Your Dependencies that servers can consult databases of least to avoid the Even after all that work, you are not known bugs and self-report to moni- parts you have done tending your dependencies. It’s toring software when they are in need important to continue to monitor of upgrades. identified as most them and perhaps even re-evaluate Upgrading promptly is important, problematic. your decision to use them. but it means adding new code to your First, ensure you keep using the project, which should mean updating specific package versions you think your evaluation of the risks of using you are. Most dependency managers the dependency based on the new ver- now make it easy or even automatic to sion. At minimum, you would want to record the cryptographic hash of the skim the diffs showing the changes expected source code for a given pack- being made from the current version age version and then to check that to the upgraded versions, or at least hash when redownloading the pack- read the release notes, to identify the age on another computer or in a test most likely areas of concern in the up- environment. This ensures your build graded code. If a lot of code is chang- uses the same dependency source ing, so that the diffs are difficult to di- code you inspected and tested. These gest, you can incorporate that fact into kinds of checks prevented the event- your risk-assessment update. stream attacker, described earlier, You will also want to rerun the tests from silently inserting malicious code you have written that are specific to in the already-released version 3.3.5. your project, to ensure the upgraded Instead, the attacker had to create a package is at least as suitable for the new version, 3.3.6, and wait for people project as the earlier version. Rerun- to upgrade (without looking closely at ning the package’s own tests also the changes). makes sense. If the package has its It is also important to watch for own dependencies, it is entirely pos- new indirect dependencies creeping sible that your project’s configuration in. Upgrades can easily introduce new uses versions of those dependencies packages upon which the success of (either older or newer ones) different your project now depends. They de- from those used by the package’s au- serve your attention as well. In the thors. Running the package’s own tests case of event-stream, the malicious can quickly identify problems specific code was hidden in a different pack- to your configuration. age, flatmap-stream, which the new Again, upgrades should not be event-stream release added as a new completely automatic. You must ver- dependency. ify the upgraded versions are appro- Creeping dependencies can also priate for your environment before affect the size of your project. Dur- deploying them.3 ing the development of Google’s If your upgrade process includes re- Sawzall15—a JIT’ed logs processing running the integration and qualifica- language—the authors discovered tion tests you have already written for at various times that the main inter- the dependency, so that you are likely preter binary contained not just Saw- to identify new problems before they zall’s JIT but also (unused) PostScript,

42 COMMUNICATIONS OF THE ACM | SEPTEMBER 2019 | VOL. 62 | NO. 9 practice

Python, and JavaScript interpreters. have done only a subset of them for a Tracking and Controlling Each time, the culprit turned out to subset of my own dependencies. Most Microservice Dependencies be unused dependencies declared of the time the entirety of the decision Silvia Esparrachiari, Tanya Reilly, and Ashleigh Rentz by some library Sawzall did depend is, “Let’s see what happens.” Too often, https://queue.acm.org/detail.cfm?id=3277541 on, combined with the fact that anything more than that seems like too Thou Shalt Not Depend on Me Google’s build system eliminated much effort. Tobias Lauinger, Abdelberi Chaabane, any manual effort needed to start us- The Copay and Equifax attacks are and Christo B. Wilson ing a new dependency. This kind of clear warnings of real problems in the https://queue.acm.org/detail.cfm?id=3205288 error is the reason the Go language way software dependencies are con- makes importing an unused package sumed today. We should not ignore References a compile-time error. the warnings. Here are three broad 1. Baldwin, A. Details about the event-stream incident. The npm Blog (Nov. 2018); https://bit.ly/2DRjySJ Upgrading is a natural time to re- recommendations: 2. Cox, R. Go & Versioning, 2018; https://research.swtch. visit the decision to use a dependency 1. Recognize the problem. If noth- com/vgo. 3. Cox, R. The principles of versioning in Go. GopherCon that’s changing. It’s also important ing else, this article hopefully con- Singapore (May 2018); https://www.youtube.com/ to periodically revisit any dependen- vinced you that there is a problem watch?v=F8nrpe0XWRg. 4. Cox, R. RE2: A principled approach to regular cy that isn’t changing. Does it seem here worth addressing. We need expression matching. Google Open Source Blog (Mar. plausible that there are no security many people to focus significant ef- 2010); https://bit.ly/2XoLFzC. 5. Cox, R. Regular expression matching with a trigram problems or other bugs to fix? Has the fort on solving it. index or how Google Code Search worked. Swtch.com project been abandoned? Maybe it’s 2. Establish best practices for today. (Jan. 2012); https://swtch.com/~rsc/regexp/regexp4. html. time to start planning to replace that Best practices are needed for manag- 6. Facebook. Infer: A tool to detect bugs in Java and C/ dependency. ing dependencies using what is avail- C++/Objective-C code before it ships; https://fbinfer. com/. It’s also important to recheck the able today. This means working out 7. GNU Project. GNU General Public License, version security history of each dependency. processes that evaluate, reduce, and 1, 1989; https://www.gnu.org/licenses/old-licenses/ gpl-1.0.html. For example, Apache Struts disclosed track risk, from the original adoption 8. Go Project. Go 1 and the future of Go programs, 2013; https://golang.org/doc/go1compat. different major remote code execution decision through production use. In 9. Google Open Source. Using third-party licenses; vulnerabilities in 2016, 2017, and 2018. fact, just as some engineers specialize https://opensource.google.com/docs/thirdparty/ licenses/#banned. Even if you have a list of all the servers in testing, others may need to special- 10. Hipp, D. R. How SQLite is tested; https://www.sqlite. that run it and update them promptly, ize in managing dependencies. org/testing.html. 11. Lacasse, N., Open-sourcing gVisor, a sandboxed that track record might make you re- 3. Develop better dependency tech- container runtime. Google Cloud (May 2018); http:// think using it at all. nology for tomorrow. Dependency bit.ly/2wzA84D. 12. Langley, A. ’s seccomp sandbox. managers have essentially eliminated ImperialViolet (Aug. 2009); https://www. the cost of downloading and install- imperialviolet.org/2009/08/26/seccomp.html. 13. National Institute of Standards and Technology. ing a dependency. Future develop- National Vulnerability Database—Search and ment efforts should focus on reducing Statistics; https://nvd.nist.gov/vuln/search. 14. Pike, R. Go Proverbs, 2015; https://go-proverbs. the cost of the kind of evaluation and github.io/. maintenance necessary to use a de- 15. Pike, R., Dorward, S., Griesemer, R. and Quinlan, S. Interpreting the data: Parallel analysis with Sawzall. pendency. For example, package-dis- Scientific Programming J. 13, 4 (2005), 277–298; covery sites might work to find more https://doi.org/10.1155/2005/962135. 16. Potapenko, A. Testing Chromium: ThreadSanitizer v2, ways to allow developers to share a next-gen data race detector. Chromium Blog (Apr. 2014); http://bit.ly/2WN29o0. their findings. Build tools should, at 17. Potvin, R., Levenberg, J. Why Google stores billions of the least, make it easy to run a pack- lines of code in a single repository. Commun. ACM 59, 7 (July 2016), 78–87; https://doi.org/10.1145/2854146. Conclusion age’s own tests. More aggressively, 18. Reis, C. Multi-process architecture. Chromium Blog Software reuse is finally here, and its build tools and package-management (Sept. 2008); https://blog.chromium.org/2008/09/ multi-process-architecture.html. benefits should not be understated. It systems could also work together to 19. SpotBugs: Find bugs in Java programs; https:// has brought an enormously positive allow package authors to test new spotbugs.github.io/. 20. Thompson, K. Reflections on trusting trust.Commun. transformation for software develop- changes against all public clients of ACM 27, 8 (Aug. 1984), 761–763; https://doi. ers. Even so, we have accepted this their APIs. Languages should also org/10.1145/358198.358210. 21. U.S. House of Representatives Committee on transformation without completely provide easy ways to isolate a suspect Oversight and Government Reform. The Equifax Data thinking through the potential conse- package. Breach, Majority Staff Report, 115th Congress (Dec. 2018); http://bit.ly/2Gf53IJ. quences. The old reasons for trusting There is a lot of good software out 22. Willis, N. A single Node of failure. LWN.net (Mar. 2016); dependencies are becoming less valid there. Let’s work together to find out https://lwn.net/Articles/681410/. 23. Winters, T. SD-8: Standard library compatibility, C++ at exactly the same time there are more how to reuse it safely. standing document, 2018; http://bit.ly/2QNhT5k. dependencies than ever. The kind of critical examination of Russ Cox leads the development of the Go programming specific dependencies outlined in this language at Google, with a current focus on improving the Related articles security and reliability of using software dependencies. article is a significant amount of work on queue.acm.org With Jeff Dean, he created Google Code Search. He and remains the exception rather than worked for many years on the Plan 9 from Bell Labs The Calculus of Service Availability . the rule. It’s unlikely that any develop- Ben Treynor, Mike Dahlin, ers actually make the effort to do this Vivek Rau, and Betsy Beyer for every possible new dependency. I https://queue.acm.org/detail.cfm?id=3096459 Copyright held by author/owner.

SEPTEMBER 2019 | VOL. 62 | NO. 9 | COMMUNICATIONS OF THE ACM 43