viewpoints VDOI:10.1145/2935880 Thomas Haigh Historical Reflections How Charles Bachman Invented the DBMS, a Foundation of Our Digital World His 1963 Integrated Data Store set the template for all subsequent database management systems. IFTY-THREE YEARS AGO a small team working to automate the business processes of the General Electric Company built the first database man- Fagement system. The Integrated Data Store—IDS—was designed by Charles W. Bachman, who won the ACM’s 1973 A.M. Turing Award for the accomplish- ment. Before General Electric, he had spent 10 years working in engineering, finance, production, and data process- ing for the Dow Chemical Company. He was the first ACM A.M. Turing Award winner without a Ph.D., the first with a background in engineer- ing rather than science, and the first to spend his entire career in industry rather than academia. Some stories, such as the work of Babbage and Lovelace, the creation of the first electronic computers, and the emergence of the personal computer industry have been told to the public again and again. They appear in popu- lar books, such as Walter Isaacson’s recent The Innovators: How a Group of Hackers, Geniuses and Geeks Created the Digital Revolution, and in museum exhibits on computing and innova- tion. In contrast, perhaps because da- tabase management systems are rarely Figure 1. This image, from a 1962 internal General Electric document, conveyed the idea COURTESY OF CHARLES W. BACHMAN AND THE CHARLES BABBAGE INSTITUTE. BABBAGE AND THE CHARLES BACHMAN W. OF CHARLES COURTESY experienced directly by the public, of random access storage using a set of “pigeon holes” in which data could be placed. JULY 2016 | VOL. 59 | NO. 7 | COMMUNICATIONS OF THE ACM 25 viewpoints database history has been largely ne- establish a “totally integrated man- glected. For example, the index of Isaa- agement information system.”8 This cson’s book does not include entries If any technology would integrate and automate all the for “database” or for any of the four was essential to the core operations of a business, ideally people to have won Turing Awards in with advanced management report- this area: Charles W. Bachman and Ed- rebuilding of our daily ing and simulation capabilities built gar F. Codd (1981), James Gray (1988), lives around digital right in. The latest and most expensive or Michael Stonebraker (2014). computers of the era had new capabili- That’s a shame, because if any tech- infrastructures, ties that seemed to open the door to a nology was essential to the rebuild- it was the database more aggressive approach. Compared ing of our daily lives around digital to the machines of the 1950s they infrastructures, which I assume is management system. had relatively large memories. They what Isaacson means by “the Digital featured disk storage as well as tape Revolution,” then it was the database drives, could process data more rap- management system. Databases un- idly, and some were even used to drive dergird the modern world of online interactive terminals. information systems and corporate The reality of data processing intranet applications. Few skills are of subjectivity in judgments about changed much more slowly than the more essential for application develop- “firsts,” particularly as IDS predated hype, and remained focused on simple ers than a basic familiarity with SQL, the concept of a database management administrative applications that batch the standard database query language, system. As a fusty historian I value nu- processed large files to accomplish and a database course is required for ance and am skeptical of the idea that tasks such as weekly payroll process- most computer science and informa- any important innovation can be fully ing, customer statement generation, tion systems degree programs. Within understood by focusing on a single or accounts payable reporting. ACM, SIGMOD—the Special Interest breakthrough moment. I have docu- Many companies announced their Group for Management of Data—has mented many ways in which IDS built intention to build totally integrated a long and active history fostering da- on earlier file management and report management information systems, tabase research. Many IT professionals generation systems.7 However, if any but few ever claimed significant suc- center their entire careers on database system deserves the title of “first data- cess. A modern reader would not be technology: the census bureau esti- base management system” then it is shocked to learn that firms were un- mates the U.S. alone employed 120,000 clearly IDS. It became a model for the able to create systems of comparable database administrators in 2014 and earliest definitions of “data base man- scope to today’s Enterprise Resources predicts faster than average growth for agement system” and included most of Planning and data warehouse proj- this role. the core capabilities later associated ects using computers with perhaps the Bachman’s IDS was years ahead of with the concept. equivalent of 64KB of memory, no real its time, implementing capabilities operating system, and a few megabytes that had until then been talked about What Was IDS For? of disk storage. Still, even partially in- but never accomplished. Detailed func- Bachman created IDS as a practical tegrated systems covering significant tional specifications for the system tool, not an academic research project. portions of a business would have real were complete by January 1962, and In 1963 there was no database research value. The biggest roadblocks to even Bachman was presenting details of the community. Computer science was just modest progress toward this goal were planned system to his team’s in-house beginning to emerge as an academic the sharing of data between applica- customers by May of that year. It is less field, but its early stars focused on pro- tions and the difficulties application clear from archival materials when the gramming language design, theory of programmers faced in exploiting ran- system first ran, but Bachman tells me computation, numerical analysis, and dom access disk storage. that a prototype installation of IDS was operating system design. In contrast Getting a complex job done might tested with real data in the summer of to this academic neglect, the efficient involve dozens of small programs and 1963, running twice as fast as a custom- and flexible handling of large collec- the generation of many working tapes built manufacturing control system tions of structured data was the central full of intermediate data. These banks performing the same tasks. challenge for what we would now call of whirring tape drives provided com- The details of IDS, Bachman’s life corporate information systems depart- puter centers with their main source story, and the context in which it arose ments, and was then called business of visual interest in the movies of the have been explored elsewhere.2,6 In this data processing. era. Tape-based processing techniques column, I focus on two specific ques- During the early 1960s the hype and evolved directly from those used with tions: reality of business computing diverged pre-computer mechanical punched ˲ Why do we view IDS as the first da- dramatically. Consultants, visionaries, card machines: files, records, fields, tabase management system, and business school professors, and com- keys, grouping, merging data from ˲ What were its similarities and dif- puter salespeople had all agreed that two files, and the hierarchical combi- ferences versus later systems? the best way to achieve real economic nation of master and detail records There will always be an element payback from computerization was to within a single file. These applied to 26 COMMUNICATIONS OF THE ACM | JULY 2016 | VOL. 59 | NO. 7 viewpoints magnetic tape much as they had done support for structuring data in tape what we still consider the core task of to punched cards, except that tape files but lacked comparable support a database management system. Pro- storage made sorting much harder. for random access storage. Harnessing grams could not manipulate data files The formats of tape files were usually the power of disks meant finding ways directly, instead making calls to IDS so fixed by the code of the application to sequence, insert, delete, or search that it would perform the data opera- programs working with the data. Ev- for records that did not simply repli- tions on their behalf. ery time a field was added or changed cate the sequential techniques used Like modern database manage- all the programs working with the file with tape. Solutions such as hashing, ment systems, IDS explicitly stored would need to be rewritten. If applica- linked lists, chains, indexing, inverted and manipulated metadata about the tions were integrated, for example, by files, and so on were quickly devised records and their relationships, rather treating order records from the sales but these were relatively complex to than expecting each application pro- accounting system as input for the pro- implement and demanded expert gram to understand and respect the duction scheduling application, the judgment to select the best method for format of every data file it worked with. resulting web of dependencies made a particular task (see Figure 1). It enforced relationships between dif- it increasingly difficult to make even IDS was intended to substantially ferent record types, and would protect minor changes when business needs solve these two problems, so that ap- database integrity. Database design- shifted. plications could be integrated to share ers specified record clusters, linked The other key challenge was mak- data files and ordinary programmers list sequencing, indexes, and other ing effective use of random access stor- could effectively develop random ac- details of record organization to boost age in business application programs. cess applications using high-level lan- performance based on expected usage Sequential tape storage was conceptu- guages.