electing Storage Media for Long-Term Access to Digital Records

CHARLES M. DOLLAR, PH.D., CA

The periodic replacement of digital media is a his study explores the selection of storage media for long-term access to digital records from four different but threshold solution that helps mitigate the consequences interrelated perspectives. The first viewpoint is an overview of the current of media fragility and technological obsolescence. digitalT storage environment with an emphasis on magnetic and media. It is followed by a discussion of the so-called problem of digital The replacement of digital storage media involves the storage media. The third perspective is a review of evaluation criteria for media selection. An use of selection criteria by information professionals to assessment of current storage media available today comprises the fourth perspective. (Author’s note: much of this discussion is indebted to the assess current and future magnetic and optical storage author’s soon-to-be released work Authentic Electronic Records: Long-Term Access Strategies, media. The criteria include (1) high storage capacity, which is being published by Cohasset Associates in 1999.) Although media selection is a threshold issue (2) high data-transfer rate, (3) life expectancy of at for long-term access to digital records, it is only one component of a long-term strategy for access to least 20 years, (4) established and stable market- digital records. Other components include mainte- nance of a stable storage environment, periodic media renewal, conversion to new technology place presence, (5) affordability, and (6) suitability. platforms, and quality control procedures.

Application of these criteria leads to the conclusion The Digital Storage Environment The hierarchical storage model (HMS) defines digital storage media as (1) on-line, (2) near-line, that digital linear tape is the preferred storage medium and (3) off-line, based upon frequency of access. In this model, on-line storage creates higher costs, to support the long-term storage of digital records. offers faster speed, but provides lower capacity

36 THE INFORMATION MANAGEMENT JOURNAL / July 1999 than near-line storage. Near-line times (ROM), recordable (write once metal mold into which molten storage consists of optical media read many or WORM), and polycarbonate is injected. After the jukeboxes and automated tape- rewritable (RW). These three types of molded plastic disk cools, its retrieval libraries, while off-line optical storage media share several underside is coated with a very thin, storage consists of shelved optical features. Their shared physical media highly reflective aluminum coating. and magnetic media. Off-line storage characteristics include Reading occurs when a low-power has lower costs, slower speed, 1. a rigid translucent polycarbonate laser beam is focused on a track and and higher capacity than near- substrate containing thousands the presence or absence of pits and line storage of tracks lands is measured by the amount of Electronic records that have been reflected light. Hundreds of discs can 2. a reflective coating over the set aside for long-term storage are be fabricated with the same mold substrate no longer required for operational without any error. The pits and lands purposes so they fall into the off-line 3. spots in the tracks, usually called in CD-ROM are irreversible, leading storage environment. Two current pits, that represent binary 1s and to the term “read only.” In addition, recording technologies – magnetic “lands” (spaces without pits) no new digital information can be and optical – are used for off-line that represent 0s later added to the disc. storage. 4. a protective clear lacquer or acrylic overcoat Read-Only Systems Off-line Magnetic Media There are two read-only technolo- There are three different kinds of These three types of optical stor- gies in use today. One is the off-line magnetic media in use age media also use a low-power laser CD-ROM disc that enjoys a wide today: longitudinal, longitudinal to read the representations of binary market penetration and is an serpentine, and helical. Longitudinal 1s and 0s. Reading occurs when established technology. The second recording media have a fixed number a low-power laser beam is focused on is the DVD disc that is an emerging of parallel tracks (e.g., 18 tracks for a track and reflectivity is measured. storage technology with considerable 3480 tape cartridges) laid across the Alterations (e.g., a bubble or a pit) future potential for the long-term length of the tape that are written to in the smooth recording surface storage of electronic records because or read from the beginning of the causes the laser beam’s reflected light of its high storage capacity and high tape to its end. to disperse. A timing mechanism data-transfer rate. DVD has even Longitudinal serpentine record- tells the optical head when to expect begun to appear in the consumer ing media also have a fixed number reflectance, and the optical head market in both computer and enter- of tracks (128 to 204) for digital linear interprets the absence of reflected tainment functions. As DVD technol- tape (DLT) laid down the length of light as a transition from one or more ogy matures over the next several the tape. Multiple write/read heads binary 1s to one or more binary 0s. years, it will likely achieve the same record or read simultaneously the The presence of reflected light level of market penetration that first two or four tracks sequentially denotes a transition from one or CD-ROM technology has today. from the beginning to the end of the more binary 0s to one or more binary (For more information on DVD tape, and the next two or four tracks 1s. ROM, WORM, and RW optical technology, see Taylor 1997 and are read from the end of the tape, storage media can be differentiated Silver 1999.) back to beginning. This stepping by the way a binary stream of 1s down technique continues until all and 0s is recorded and whether Recordable Optical Media the tracks are read or are written on. the recorded binary bit stream can Recordable optical media, known Helical recording media involves be changed. generally as “write once read many” short tracks that are recorded diago- (WORM), have the same features of nally (at an angle of about 11 degrees) Read-Only Technology read-only optical media with one across the width of a 4mm or The manufacture of read-only key difference: the method of 8mm tape. Both helical and longitu- optical media involves a process recording 1s and 0s. In WORM dinal serpentine recording media whereby the binary bit stream of an recording technology, the on and have a storage capacity and data information object (e.g., a book, soft- off states of a laser replicates the transfer rate that is five to six times ware, or music) is replicated in the on 1s and 0s of a binary bit stream. greater than that of longitudinal and off modulations of a high-power In the on state, the heat from a recording media. laser beam that etches pits and lands focused laser causes microscopic, on a glass master disk coated with a irreversible physical alterations on Off-Line Optical Media special photoresistant material. The the surface of the recording material, There are three different types of pits and lands on the tracks of the which represent 1s. No alteration optical storage media: read many master disk are replicated in a occurs in the off state.

38 THE INFORMATION MANAGEMENT JOURNAL / July 1999 Rewritable Optical Media readability and longevity are at risk. with prolonged exposure to high The essential characteristic of the A readable digital record is one temperature and high humidity rewritable optical media is that its whose underlying bit stream can be (NIST 1991). In addition, the binder data-bearing surface deformations processed on that holds the recording material to are reversible. Rewritable optical 1. the computer system or device the polycarbonate substrate is subject disks are available as magneto- that initially created it to hydrolysis when exposed to high optical (MO) and phase-change. humidity. This condition can lead to 2. the computer system or device In MO technology, recording small particles of the recording ma- that currently stores it occurs when a laser beam heats terial separating from the substrate. a microscopic area of thin-film 3. a computer system or device that Research conducted by the magnetic material in a track and its will be used to store the digital National Media Laboratory (NML) coercivity is reduced so that as the 1s information in the future on predicted life expectancy of and 0s of a bit stream pass through a magnetic and optical storage media write-head magnet, the polarity of Digital records can become confirms the negative impact of tem- the area heated by the laser beam is unreadable in two different ways. peratures in excess of 26º C (74° F) changed to match that of the bit One is the degradation that results and a relative humidity of 70 percent stream. (Coercivity is the capacity of from exposure to a hostile storage or more (Bogart 1995). According to a magnetic field to resist erasure or environment. So far as magnetic this research, lower temperatures alteration. The level of coercivity media are concerned, their composi- and lower relative humidity levels depends upon the chemical proper- tion – the magnetic particles on can contribute significantly to the ties of the recording material.) which signals are recorded, the predicted life expectancy of all media Reading occurs when a low- substrate, and the binder that but particularly to the life expectancy power laser beam is focused on the holds the magnetic particles to the of magnetic and optical media. John thin-film magnetic material in a substrate – becomes unstable in a Van Bogart, director of media track. A phenomenon known as the hostile storage environment. The research at NML, who conducted “Kerr effect” causes a different angle strength of signals recorded on the these studies, believes that 10º C or incidence of reflectance for 1s and magnetic particles naturally degrades (50° F) and 20 percent relative 0s respectively that is detected by a or fades over time, but the rate humidity provide the ideal storage sensor and then translated back into of degradation generally increases environment for those electronic a bit stream. with elevated temperatures. Also, the records retained for long periods In phase-change recording tech- substrate tends to expand and of time. (This is also the recommen- nology, a laser beam heats a thin-film contract as temperatures cycle dation in ANSI/PIMA IT9.23-1998: material that can be in one of two from high to low and back to high. Imaging Materials – Polyester Base states: (1) amorphous, in which there This event can lead to “cinches” – Storage Standard.) is very low reflectance, or (2) crys- (wrinkles) or scratches in the Attaining the predicted life talline, in which there is very high magnetic particles. The binder that expectancy of digital storage media reflectance. The power of a focused holds the magnetic particles to the through storage in a controlled envi- laser beam is modulated (high substrate is susceptible to binder ronment is important but can not power/low power) so as to duplicate the 1s and 0s of a bit stream. When the focused laser beam is at the high- Magnetic and optical storage media power level, it causes the film to form an amorphous structure about 1 are inherently fragile, so their micron in diameter. When the focused laser beam is at the lower readability and longevity are at risk. power, it causes the amorphous structure to relax and return to a crystalline state. (If the structure of hydrolysis when exposed to high resolve a more fundamental aspect the spot is already crystalline, the relative humidity. The effect causes of maintaining the readability of elec- low-power laser beam does not the magnetic particles and the bit tronic records: media obsolescence. affect it.) Phase change recording stream recorded on them to separate Media obsolescence occurs when technology is used in DVD discs. from the substrate. the storage medium used (e.g., a tape Optical media, of course, are or disk) is physically incompatible The Storage Media Problem relatively less vulnerable than mag- with the available computer hard- Magnetic and optical storage netic media. However, the recording ware and therefore cannot be read. media are inherently fragile, so their material of optical media can degrade Media obsolescence seems inevitable

THE INFORMATION MANAGEMENT JOURNAL / July 1999 39 because advances in digital storage High Storage Capacity High Data-Transfer Rate technology have introduced changes Over the last three decades, an The data-transfer rate of the drive in the way the underlying bit stream exponential growth in the storage for a particular digital storage that constitutes the records is physi- capacity of digital storage media has medium is defined as the period of cally represented. Consequently, been accompanied by a decrease in time required to transfer one older storage media are incompatible physical dimensions. In other words, megabyte of data. The higher the with those used in the present, and more and more bytes can be stored in data-transfer rate, the less time those in use today are likely to be smaller and smaller surface areas of required to read or transfer data from incompatible with those developed storage media. A decade or so ago one storage medium to another, an in the future. this was measured in megabytes activity required for media renewal. The significance of a high data- transfer rate can be illustrated In some respects, the notion of suitability with DLT and CD-R discs. It would take about 55 hours of continuous is in opposition to the general practice of operation (assuming no read errors) to transfer the content of 50 DLT discovering new uses of existing technologies. tapes containing 500 gigabytes of electronic records. To transfer the same amount of information from A case in point is the IBM 3590 (one million bytes). In the early 1,600 CD-Rs would require approxi- MagStar tape. Although the 3590 1990s it was measured in gigabytes mately 800 hours. (This estimate MagStar tape cartridge has the same (1,000 megabytes); in the late 1990s it assumes that a 2X drive is being physical dimensions as the IBM 3480 is measured in terabytes (1,000 giga- used. For more information on this and 3490/E tape cartridges, it is not bytes). In the early years of the 21st topic see Pahwa 1994.) Typically, the backward compatible. A primary century it will be measured in higher the relative cost of the drive, reason for this is that the 3590 has 128 petabytes (1,000 terabytes). the higher the data-transfer rate. A recording tracks, the 3490 36 tracks, The storage capacity of magnetic relatively low data-transfer rate and the 3480 18 tracks. storage media has tended to match, entails a lower front-end cost but a There is no ultimate or permanent if not exceed, that of optical media, higher back-end cost. In this context, solution to media obsolescence although DVD technology offers a it is indeed a question of paying now (Ross 1995). The most effective way to significant increase of storage or paying later. mitigate its impact is to periodically capacity in a small physical form reformat or copy electronic records (i.e., the CD-ROM form factor of Life Expectancy from old media to newer media. If 4.72 inches in diameter). of at Least 20 Years executed correctly, this solution to Intuitively, a high-storage capacity A projected life expectancy of media obsolescence is likely to entail seems to be an absolute benefit; yet 20 years seems rather modest given considerable cost each time such a it does involve certain trade-offs and vendor claims of 50 to 100 years life risks in implementation. Take, for large-scale update takes place. expectancy of certain CD-ROM and example, what is involved in trans- WORM optical media. Bear in mind, ferring the content of 3480 tapes however, that the longevity of digital Selection Criteria to DLT with a storage capacity storage media exceeds the life The selection of a storage of 10 gigabytes. It would take expectancy of drives to read them, medium for long-term storage of approximately 50 3480 tapes to fully and the life expectancy of the drives digital records is critical. Media use the 10 gigabytes of storage on selection criteria for long-term a single DLT. The metadata and to read the media exceeds the life access to digital records include associated documentation for each of expectancy of the software applica- tion used to process and render the • high storage capacity the 50 3480 tapes, including where each begins and ends, must be trans- digital records. Given the rate of • high data-transfer rate ferred to or otherwise associated technology changes during the last three decades, a combined life • minimum projected life expect- with the single DLT. This is possible but must be done carefully to avoid expectancy of 20 years for specific ancy of 20 years inadvertent misidentification of storage media, along with the neces- • established and stable market- digital records that could have the sary drive and software, is a prudent place presence practical effect of their being lost. requirement. This means that the This problem can become acute only way to extend the usability of • affordability when the number of 3480 tapes is digital records is to transfer them • suitability quite large (Olsen 1999). periodically to new storage media

40 THE INFORMATION MANAGEMENT JOURNAL / July 1999 Helical recording technology For example, the data transfer rate CD-Recordable optical media. UDF involves exposure of large parts of for DLT drives is 5MB per second conforms to ISO 13346, which is the tape surface to the read/write while that of the IBM 3590 Magstar is the logical file-format standard for head(s), which subjects the tape to 9MB per second, a data-transfer rate CD-Rs. In sharp contrast, DVD stor- substantial mechanical stress and that no optical media and drives age technology vendors have formed necessitates periodic replacement of and no low-cost magnetic storage a DVD Forum that is promoting the read/write head(s). In addition, media and drives can match. Third, industrywide standards for interop- most of the error correction codes although the predicted life expect- erability and backward compatibility. (ECC) for helical recording are not ancy of magnetic media is less than However, until DVD becomes widely as robust as those for longitudinal that of optical media, it still is within available, the use of optical storage digital recording technology (e.g., a 20-year time period within which media for long-term access to digital 3480 tape cartridge). Based upon his digital storage technologies are likely records remains problematic. research experience with 4mm and to remain relatively stable. Two other considerations militate 8mm tape used for backup purposes, The fourth factor is marketplace against the use of optical storage NML’s Van Bogart is doubtful that penetration and stability. Except for media for long-term access to digital it would be possible to read the infor- the 3490, 3590 Magstar, 4mm and records. Even though optical storage mation back from 4mm and 8mm 8mm tapes and drives, magnetic media may satisfy the criteria for tapes 20 years later (Bogart 1998). media have an established track affordability and suitability, the record of reliability when compared storage capacity and data-transfer Digital Storage Media with optical media. The magnetic rate of optical media are substantially Assessment marketplace is much more stable less than those of most magnetic What digital storage medium than the optical media marketplace storage media. In the long run, there- should be used? WORM optical as suggested by established product fore, the use of optical storage media media, it is sometimes argued, lines that tend to have a relatively for long-term storage could entail should be the preferred long-term long life. For example, DLT and substantial costs when it becomes storage medium because with drives, which were introduced in necessary to transfer them to new WORM the underlying bit stream 1985, had a substantial market share storage media. that comprises digital records cannot in 1997, largely because of backward Today, the mainstream of the be changed or otherwise altered. This compatibility within the product line. off-line digital storage marketplace is magnetic recording technology; this argument, however, does not take Some analysts suggest that the DLT situation appears likely to remain into account the fact that storage cartridge’s strong market share is the case until DVD recording tech- media are vulnerable to alteration likely to persist for another decade nology has multiple vendors and a or corruption when electronic records or so (Bucholtz 1999). The same substantial customer base. However, are reformatted, copied, converted, observations apply as well to 3480 not all magnetic storage media are or migrated. In each of these activi- tape, which was introduced in 1984. acceptable for long-term storage of ties, the so-called “permanent” The Nordic Council Report to digital records. recording of WORM optical media Preserve and Provide Access to A prudent guideline to follow provides no protection because the Electronic Records notes that 3480 when selecting a specific magnetic physical representations of 1s and 0s tapes “probably will be around for storage medium is to avoid both must be translated into electrical sig- many years” (Nordic Council of established media on the verge of nals that computers can understand Ministers 1996). becoming obsolete and recently and process. In this regard, therefore, On the other hand, with the introduced media that are not yet WORM optical media are no differ- exception of CD-ROM media and firmly established in the market- ent than magnetic media; on balance, drives, the market life of WORM and place. Implementation of this guide- the latter provides greater support rewritable optical media and drives line would eliminate 6250 bpi tape for greater long-term access. has been relatively brief and vendors and the 3590 Magstar tape from There are several reasons for this have introduced new products with consideration; 4mm and 8mm tapes conclusion. First, the storage capacity little concern for interoperability and are excluded from consideration of magnetic storage media, such as backward compatibility. largely because they employ helical DLT, continues to exceed that of Optical media vendors have recording technology. In addition, optical media. Second, the data- created an organization called Optical the benefit of 4mm and 8mm media transfer rate for DLT, 3490 tape, Storage Technology Association (OSTA), and drives being affordable is more and 3590 Magstar tape is quite high which is promoting backward com- than outweighed by their relatively when compared with most remov- patibility. OSTA recently announced low storage capacity and slow data- able magnetic media such as 4mm its adoption of the Universal transfer rate vis-a-vis other magnetic and 8mm tape and optical media. Disk Format (UDF), primarily for storage media.

42 THE INFORMATION MANAGEMENT JOURNAL / July 1999 The magnetic storage technology that best satisfies the selection criteria REFERENCES reviewed earlier is DLT, followed Bucholtz, Chris. “DLT: Tape’s Future.” 1999. Available at closely by 3480 tape. Both enjoy a http://www.dynoteck.co.uk/pages/dhari.html. substantial marketplace presence Dollar, Charles M. Authentic electronic Records: Long-Term Access Strategies. 1999. that is likely to persist for some time. The commitment of vendors to back- National Institute of Standards and Technology (NIST). Development of a Testing ward compatibility, as enhancements Methodology to Predict Optical Disk Life Expectancy. NIST Special Publication 500-200. 1991. are made in their product lines, along with high suitability and Nordic Council of Ministers. To Preserve and Provide Access to Electronic Records. 1996 affordability factors, make them the Olsen, Florence. “USCS Juggles Data Requests.” Government Computer News. 8 February preferred magnetic storage media. 1999. Of these two, DLT should be the Pahwa, Ash. The CD-Recordable Bible, An Essential Guide for Any Business. 1994. storage medium of choice because of its high storage capacity and fast Ross, Seamus. “Preserving and Maintaining Electronic Resources in the Visual Arts for the Next Century?” Information Services & Use. 1995. data-transfer rate. Silver, Bruce. DVD, CRD-R, CD-RW, and MO Media in Document Management Conclusion Applications. 1999. This article has attempted to Taylor, Jim. DVD Demystified: The Guidebook for DVD-Video and DVD-ROM. 1997. provide a framework within which Van Bogart, John. Magnetic Tape Storage and Handling. 1995. archivists, records managers, and other information professionals can –––.Electronic message to author Charles Dollar. 28 January 1998. make informed decisions about the digital storage technology that best ABOUT THE AUTHOR: Charles M. Dollar, Ph.D., CA, heads Dollar Consulting. He is a supports long-term access to digital consultant in archives and records management with 25 years’ experience, and specializes in records. Although the author recom- electronic records and archival education. He was previously with the U.S. National Archives mends the use of DLT for the and Records Administration and the University of British Columbia. He received a Ph.D. from long-term storage of digital records, the University of Kentucky. The author may be reached at [email protected]. its use does involve a trade-off between increased storage capacity and the requirement to ensure full intellectual control of the content on one or more DLT. This problem can only be exacerbated as new storage media with greater storage capacities and data transfer rates enter the marketplace. This circumstance is illustrative of a factor many information managers fail to take into account when think- ing about long-term access to digital records: there is no free lunch. Whatever information managers do to extend the usability of digital records over time involves trade-offs between costs and benefits and vary- ing degrees of risk. It is unlikely that a final or permanent solution to this problem will be achieved in the foreseeable future. Consequently, information managers need to remind themselves continually and also caution other decision makers to think long and hard about the consequences associated with the selection of storage media for long- term access to digital records. J

THE INFORMATION MANAGEMENT JOURNAL / July 1999 43