Archiving the Digital Image: Today's Best Practices of File Preparation
ARCHIVING THE DIGITAL IMAGE: TODAY'S BEST PRACTICES OF FILE PREPARATION
Frank Wiewandt
A Thesis
Submitted to the Graduate College of Bowling Green State University in partial fulfillment of the requirements for the degree of
MASTER OF EDUCATION
December 2005
Committee:
Dr. Gene Poor, Advisor
Dr. Kathyrn Hoff
Dr. Paul Cessarini
© 2005
Frank Wiewandt
All Rights Reserved iii
ABSTRACT
Dr. Gene Poor, Advisor
Unlike images created with traditional film-based technology, those created in today’s digital environment run the risk of being lost to future generations because there are no universally practiced standards for archiving them. Photographers must make decisions concerning this critical step with few sources of information to guide them.
This study is intended to present the current best practices of four professional photographer/educators in an effort to present proven, if only temporary, solutions to this problem.
Qualitative research methods were used for this descriptive study. Participant interviews were used to collect the data. Subject matter experts were found by searching moderated Internet forums for candidates demonstrating a high level of experience in the field of digital imaging, the ability to articulate their views, and a willingness to share their knowledge. Additional screening of potential participants was accomplished by visiting individual’s websites to gain further evidence of their professionalism. Four candidates were approached for the study, and all agreed to participate.
The participants reported using digital technology almost exclusively to capture their images. Also, every job was delivered digitally to clients. This data demonstrates the pervasiveness of digital technology in today’s photography marketplace.
Each participant had his own method of archiving digital image files based upon his business model and workflow. They disagreed about the viability of RAW file formats for archival use. RAW file formats leave the imaging sensor data intact rather iv
than converting it in-camera to one of several commonly used file formats. Unfortunately each manufacturer uses proprietary data in the RAW files from its cameras making it unreadable to all but its own imaging software.
The majority of images being created today by professional photographers are digital. This ever-expanding body of work needs to be protected from becoming digital technology “orphans.” Until industry standards evolve, it is increasingly important for photographers to take precautions when archiving digital image files. Even though there is no singular method of file preparation for digital image archiving, the practices outlined by the participants of this study are a good place to start.
v
To Molly,
Not a letter, just a start. vi
ACKNOWLEDGMENTS
The success of this thesis in no small part was due to the visionary notions of Dr.
Gene Poor, Dr. Kathryn Hoff's caring dedication to to the details, and the "bleeding-edge" technology expertise of Dr. Paul Cesarini. Thank you for being honorable examples, dedicated instructors, and great friends. vi T h TABLE OF CONTENTS e f Page o ll INTRODUCTION ...... 1 o w CHAPTER I ...... 1 i n Context of the Problem...... 2 g p Problem of the Study ...... 4 a g Significance of the Study...... 4 e s Objectives of the Study...... 5 a r Assumptions...... 6 e f Limitations...... 6 o r Definition of Terms...... 7 m a CHAPTER II ...... 9 tt e Review of Literature ...... 9 d p Historical Context...... 10 r o Relevant Theory...... 12 p e Current Literature...... 13 rl y Summary...... 14 . P CHAPTER III...... 16 l e Methodology...... 16 a s Restatement of the Problem...... 16 e u Research Design...... 16 s e Data Collection Instrument...... 16 t h Participants...... 17 i s a s a g u i d vii
Procedure of Data Analysis ...... 17
Protection of Human Subjects ...... 18
Budget...... 19
Timeline...... 20
CHAPTER IV...... 21
Findings...... 21
Biographies ...... 21
Digital Usage ...... 22
Capture ...... 23
Delivery...... 24
Archiving Practices...... 24
The RAW Format ...... 27
Considerations...... 28
CHAPTER V ...... 29
Summary...... 29
Conclusions...... 30
Discussion...... 31
Recommendations for Future Studies...... 33
REFERENCES ...... 34
APPENDIX A. Consent E-mail ...... 36
APPENDIX B. Interview Guidelines ...... 37
APPENDIX C. Verification E-mails ...... 38
APPENDIX D. Peer Review E-mail ...... 42 viii
LIST OF TABLES
Table Page
1 Estimated expenses for study...... 19
2 Deadlines for the completion of the study ...... 20
1
Chapter I
Introduction
Photographs have provided generations of grateful viewers with lasting memories by recording important images on film. Photographers have established profitable careers by creating images that can be sold not once but again and again. Neither of these scenarios would be possible without carefully archiving, or preserving, the materials that are needed to successfully reproduce an image (Ostroff, 1976).
Over time, archival methods have been established to guarantee the long-term survival of traditional film and printed materials. Even though the various groups directly involved in the production and preservation of traditional imaging materials differ in their concerns, national and international organizations, for example, like the International
Standards Organization (ISO) and the American National Standards Institute (ANSI) have taken the lead and developed standards (Adelstein, 1999). In comparison, today’s digital cameras create digital files instead of latent images on film. No such universally accepted practices exist as standards for either capturing or archiving these digital image files.
Digital imaging technologies are in their relative infancy. As the demands and capabilities of digital cameras continue to escalate and evolve, photographers are faced with the problem of saving their images in a manner assuring viability in a digital environment that has yet to be established. “While the majority of photographers recognize the vulnerability of the digital environment, they cannot identify a singular source for information regarding proper methods and procedures to assist in the protection of their digital assets” (Bushey, 2005, p. 24). 2
Context of the Problem
As the shift in photography continues from tradition film technologies to digital imaging technologies, the search for universally accepted archiving techniques is becoming very important. In reference to the Walt Disney Photo Library, Lerner (2001) wrote, “It is the long-term issue of archiving the digital assets that looms ominously overhead, waiting to strike the collection down with one small change in format or hardware” (p. 171). Thus, although the methods used in archiving film have evolved into a set of universally accepted standards, the ways to archive digital image files, on the other hand, have no such industry-accepted and industry approved set of methods and practices.
The archiving techniques currently accepted by the film industry evolved along with the rest of its unique technology, and so it will be with digital imaging. Film technology advanced along the parallel, but separate, paths of camera and film development. As a result, two separate industries evolved. Similarly, digital imaging is split between hardware and software development, but there is a paradigm shift away from the shared responsibility of the film industry for the creation and long-term preservation of images, to a proprietary approach favored by the current professional digital camera manufacturers. Never before has the current and future income that photographers must generate from their images been so captive to the specific technology that facilitates the creation of those image files. The same could be said about the long- term survival of important images that are digitally archived. Consider the following definition of the word archive from the American Heritage Dictionary (2004):
3
Archive - Computer Science
a. A long-term storage area, often on magnetic tape, for backup
copies of files or for files that are no longer in active use.
b. A file containing one or more files in compressed format for
more efficient storage and transfer. [CD]
Thus, there are two technical issues involved in the digital archiving process. The first issue is physical storage. As computer technology continues to rapidly advance, physical storage media has become a moving target. The American Heritage Dictionary’s definition is a good example. In the not-too-distant past magnetic tape was, indeed, the storage media of choice for digital image files. While it continues to be used in many industries, other storage technologies have evolved that offer the archivist better alternatives (Fraser, 2005, p. 144). Digital photography pioneers moved image files from floppy disks to Syquest and Zip disks because the new technology provided larger capacities and improved physical properties. As individual image files grew larger, photographers needed to store them on CD-R/RW and DVD-R/RW media, again because these technologies offered larger capacities and a better way to keep the media viable over the long term. A current trend is to store image files on separate large capacity hard drives and servers. On this topic Frasier (2005) wrote:
Strictly speaking, there’s no such thing as an archival medium for digital storage –
any of the even slightly convenient solutions available for recording ones and
zeros will degrade over time. . . . Any archiving strategy must include periodic
refreshing of the data onto new media, preferably taking advantage of
improvements in technology. (p. 144) 4
One issue that is currently providing a roadblock to a universally accepted archival solution is the file format itself. Even though high-end digital cameras have had the ability to output RAW files since the early 1990s (DigiCamHistory.Com, n.d.), most image files were converted to large, uncompressed or minimally (lossless) compressed files (TIFF and PSD are most common) used for the highest quality reproduction, or more greatly compressed files (JPEG) that are of lower image quality but take up less storage space and are easier to transmit over the Internet. The larger files were saved and acted as archival files that could be reused as necessary, and the smaller files were either saved for convenience or deleted. As the demand for higher quality image files increased, photographers increasingly relied on RAW format files that allowed them to access and manipulate the unprocessed digital image data separately from capture and processing data. The RAW format is widely viewed as a significant breakthrough in image formatting. It has led to problems, though. Each camera manufacturer has taken the RAW format concept and developed proprietary versions for its own specific camera models.
As a result, the digital imaging industry has no universally accepted RAW format
(OpenRAW, 2005).
Problem of the Study
The problem of this study was to discover the current best practices of digital image file preparation and to examine the limitations and implications of these practices.
Significance of the Study
Photography pioneers built their own wooden cameras. They formulated and mixed sensitized materials and coated various substrates like metal or glass plates. When exposed to light reflected off subjects, these plates held the latent images of those 5 subjects (London, 1989). The plates that remain are irreplaceable artifacts of an era that has long ago passed from distant memory only to be found, if at all, in history books. Had more of these plates been preserved, or archived, valuable visual data would be available for researchers of all persuasions to use in their academic endeavors.
In a similar vein, archiving digital images is an immediate problem that directly affects a photographer’s current and future income as well as the long-term survival of important images. Finding a universally acceptable archiving solution is key to the continued advancement of digital imaging technologies, and over the long term the ways in which these digital images will be accessed and utilized. When surveyed, 95.9% of photographers indicated they would follow archival standards if they are implemented
(Bushey, 2005). In a conclusion of this survey the author wrote the following:
Photographers have begun to understand the challenges to continuing access and
long-term preservation presented by the use of proprietary digital systems and
technical obsolescence, as revealed by their habits of saving digital images in
more than one format, refreshing digital media, and upgrading older file formats
to operate on newer versions of image applications; by their willingness to adopt a
standard for image creation and preservation; and by their eagerness to participate
in the survey. (Bushey, 2005, p. 24)
Objectives of the Study
The primary objective of the study was to examine how some leading photographers archive their digital image files. A secondary objective was to gather educated opinions about the future of digital image archiving. 6
Assumptions
The photographers selected to participate were engaged in professional-level digital imaging and were willing to talk candidly for the record. Their opinions were expressly their own and not those of past or present employers, clients, or affiliates.
Considering the fluid nature of change in the imaging industry, the situation is very likely to have changed in significant and unforeseen ways after the interviews have been done and the data compiled.
Limitations
This paper does not cover the issue of Digital Asset Management, specifically the cataloguing and retrieving of archived image files. It also does not include information concerning file security or protection technologies.
The physical storage of image files is not examined beyond a brief historical treatment of the subject. Advances in digital storage technology are assumed to continue well into the future making current media obsolete.
Only born digital images (Bushey, 2005) are discussed, not those created by scanning traditionally exposed film or prints. “The term ‘born digital image’ refers to a digital image that never physically existed before becoming a digital file” (Bushey, 2005, p. 3). 7
Definition of Terms
A number of terms and acronyms are used repeatedly throughout this document.
In order to facilitate the understanding and use of these terms, the author’s operational definitions are listed below:
– Archival refers to photographic materials designed for long-term storage and future use.
– Realistic model research paradigm is simply the idea that even though individual
subjects may be involved in shared situations each one will generate unique
perceptions, and it is the researcher’s task is to distinguish among these
perceptions (Birley & Moreland, 1998).
– Open source is software that is created by a development community rather than a
single vendor (TechEncyclopedia, n.d.).
– Open standards are specifications for hardware and/or software that are publicly
available (TechEncyclopedia, n.d.).
– Proprietary standards are specifications for hardware and/or software that are
developed and controlled by one company (TechEncyclopedia, n.d.).
– TIFF (Tagged Image File Format) is a widely used bitmapped file format that uses
lossless compression methods. Lossless compression decompresses data back to
its original form without any loss. The decompressed file and the original are
identical (TechEncyclopedia, n.d.).
– JPEG (Joint Photographic Experts Group) is an ISO/ITU standard still image format
that is popular due to its excellent but lossy compression capabilities. Lossy
methods provide high degrees of compression and result in very small 8
compressed files, but there is a certain amount of loss when they are restored
(TechEncyclopedia, n.d.).
– RAW file refers to several uncompressed proprietary formats that contain data that
have not been processed (TechEncyclopedia, n.d.).
– PSD refers to Adobe Photoshop’s native, layered file format (TechEncyclopedia, n.d.).
– DNG (Digital NeGative) is a digital camera file format from Adobe that was designed
to provide a standard for RAW files (TechEncyclopedia, n.d.). 9
Chapter II
Review of Literature
There has been much relevant writing about the permanence of film and print but little of relevance concerning the longevity of digital image files. This is partly because traditional imaging technology has been used since the mid-1800s (London, 1989). As sophisticated and mature as digital imaging appears to be at this moment, it is still a new technology. Literature that was once timely and relevant has quickly become outdated.
Today such literature’s significance is mainly historical. As Bushey (2005) stated, “In general, the literature reveals a lack of sources that address the practical aspects of how to create and manage digital images as reliable records and preserve their authenticity over the long-term” (p. 2).
The first section of this literature review deals with the historical context of the problem. The brief inspection of archival issues in film-based technology will be helpful in understanding the significance of archiving; and it will reveal parallels to digital imaging technology. Some historical review of digital imaging technologies will also be needed in order to shed some light on the difficulties currently being encountered in the field.
Any treatment of digital technology can get very technical, very fast. It is not the purpose of this paper to explore the technology itself; rather, it has been written to examine the challenges today’s technology will present to current and future users. It will be necessary, though, to explain some of the fundamental theory behind current image file formats in order to consider their place in the evolution of archival digital file formats. That information will be found in the Relevant Theory section. 10
It should come as no surprise that much of the current literature about such a
“bleeding-edge” topic as digital image formats will be found on the Internet and not in scholarly journals where the peer-review process alone is enough to delay publishing long past immediate relevancy. To an extent this is also true of much of what is found on the Web. Because the reality of the problem is changing even as this paper is being written, significant information concerning these changes is often a moving target. The sources of this information, though, are not. We live in a world of digital communication with dialogue via the Internet, which includes newsgroups, forums, and lists as the conduit of this instantaneous flow of information. The Current Literature section of this paper explores these sources.
Historical Context
It does not matter how recent or removed the film technology being considered is; as soon as it was created, it began to deteriorate. About this subject Young (2002) wrote the following, “Since the birth of photography, many processes and materials have been used, all subject to deterioration through time” (p. 281). Preservation and restoration of historical photographs was generally considered the province of museums and serious collectors. The first step was to find out what caused photographic materials to degrade
(Ostroff, 1976). Next, measures could be taken to slow or stop the deterioration.
Researchers have revealed the causes of the deterioration and have developed methods proven effective in the conservation of photographic materials, but there are other issues to consider. Nothing has significantly changed since Ostroff (1976) wrote:
Ideally, thorough conservation of photographs requires complete removal of all
residual chemicals and gold toning, and storage in an inert atmosphere maintained 11
at 50% relative humidity, 50° F, within a sterilized, stainless steel container that is
sealed by welding. Obviously this is an impracticable approach, one that requires
compromises; these are based primarily on practicability and economic
considerations. The question is, how to achieve the high quality needed while
economically insuring the longest life for any photographic collection? (p. 2)
The answers to Ostroff’s (1976) question are found in balancing the value of the work with the efforts to preserve it.
In order to balance the interests of those in charge of managing traditional photography archives, ISO and ANSI standards were established (Adelstein, 1999). This author explained that, “a key purpose of standards is to resolve such differences and to arrive at a consensus which is technically sound, practical, and which will provide maximum benefit” (Adelstein, 1999, p. 41). No such universal standards currently exist for digital imaging technology (Minister of Public Works and Government Services
Canada, 2000). As a result, interested organizations must generate their own standards but even the U.S. Government admits the limitations of this approach. Consider what
Purglia, Reed, and Rhodes (2004) wrote:
The Technical Guidelines are intended to be informative, and not intended to be
prescriptive. We hope to provide a technical foundation for digitization activities,
but further research will be necessary to make informed decisions regarding all
aspects of digitizing projects. These guidelines provide a range of options for
various technical aspects of digitization, primarily relating to image capture, but
do not recommend a single approach. (p. 1)
It is interesting to take a brief look at how traditional and digital imaging 12 technologies coexist at the present time. There seems to be a fear of digital imaging that may be well founded. Young (2002) explained, “Where it might take 100 years or more for a photograph to decay, digital storage is dependent on specific technology, and this could be inaccessible in a matter of five years or less” (p. 282). One author listed first among her rants that “digital is not archival” (Lerner, 2001, p. 172). Lerner (2001) continued her evaluation of digital imaging technology when she wrote:
The greatest strength of, and the biggest threat to, any collection is this ongoing
and relentless move towards digital photography. The strength is obviously found
in the ability to process data quickly and at a reasonable cost, and the danger is
not being able to keep up with the evolving technology. (p. 172)
Unless these issues are overcome, digital archiving may retain its reputation as a short- term solution at best.
Relevant Theory
Until recently organizations like the U.S. Government recommended archiving digital images by saving them as TIFF formatted files (Purglia, Reed, & Rhodes 2004).
Such a format allowed the files to be saved with the greatest amount of data and the least amount of compression, assuring the highest image quality. RAW is not a singular format, rather it designates a group of formats. RAW is a reference to the raw sensor data combined with metadata. Metadata is the information needed to process the raw data
(Fraser, 2005). Once the camera manufacturer develops in-camera software that combines raw sensor data with their own proprietary metadata it becomes a RAW format and is one of the many RAW formats specific to camera brand (Fraser, 2005). Nikon uses the NEF designation, Canon uses CRW, Leaf calls its MOS, Minolta uses MRW, and so 13 forth. The list continues on for every camera manufacturer making cameras that support
RAW file formatting. The separation of raw sensor data and metadata could make RAW files ideal for use as an archival format because all the data is pristine (unaltered or converted). The ability of future generations of software to create even better image files from the untouched raw data is the advantage that Steinmueller (2003) considered when he wrote that, “the raw files also act more like the digital version of an undeveloped film negative. Over time we will get improved RAW file converters to get better and better results from the same data” (p. 150). Unfortunately the manufacturer’s use of proprietary and encrypted data in their RAW formats is making it unlikely that any of these current
RAW formats will ever be acceptable as an archival solution (OpenRAW, 2005). A number of advocacy groups have petitioned digital camera manufacturers requesting that they “open” their formats by releasing all the data third-party software developers need in order to create format conversion applications. If the manufacturers agree, this would be the first step toward creating a RAW format that could be considered archival.
Current Literature
Current literature with such a rapidly changing subject as digital imaging is mainly found on manufacturers’ and organizations’ websites. Even though these websites were created to promote a specific product or point of view, they can be used to obtain meaningful information. Adobe’s http://www.adobe.com/ and OpenRAW’s http://www.openraw.org/ include technical information about file formats, specifically the RAW file formats that are central to the issues raised in digital image archiving. Other websites of interest are photographer Michael Reichmann’s The Luminous Landscape 14 http://luminous-landscape.com/, Avondale Media’s rawformat http://www.rawformat.com/, and PhotoshopNews.com http://photoshopnews.com/.
The photography industry is in the midst of rapid evolution. It seems like new cameras and software are announced daily and the Internet has provided the opportunity for instant feedback through forums, lists, and newsgroups. With the number of moderated forums and private lists available, newsgroups are rarely considered sources of serious dialogue. However, moderated forums like those found on Photo News Network http://www.photonews.net/forums/forums.html require permission to join and attract top names in the industry; these have been around long enough to be considered credible sources of information. All the websites mentioned previously have affiliated forums or lists that often beat the mainstream press on important stories. Finally, there are private lists like Juergen Specht’s http://www.juergenspecht.com/lists/ D1scussion and
OpenRAW that are include more than 1500 professional photographers from around the world and have the well-deserved reputation of being extremely valuable resources.
Summary
It is easy to research archival practices for traditional film technologies. The problems associated with deteriorating photographic materials have been isolated and solutions have been developed. Standards are in place to mitigate the differing views involved in the conservancy of film and photographs. In contrast there are no such industry standards for digital images.
The archival format of choice, the TIFF, falls short of the potential promised by
RAW formats. Unfortunately, the camera manufacturers are promoting RAW format solutions that include proprietary and encrypted data that limits their archival usefulness. 15
Several interested groups (e.g., OpenRAW and rawformat) have campaigned to persuade the manufacturers to open their files and share all the data in order to give all software developers the opportunity to make converters for their RAW files.
We are living in a time of instantaneous communication. The problem of this study demonstrates the unsettled reality that photographers must face moment by moment. The only way for them to keep up with the flow of information is to tap into the resources of the Internet. As long as a skeptical approach is maintained, moderated forums and private lists offer the best sources of current, available information. Some manufacturers’ and concerned groups’ websites also can provide valuable information that is not readily available elsewhere.
Currently, one trend is to save digital image files in their native, proprietary RAW format as well as one or more of the highest quality format conversions like TIFF or
PSD, Adobe Photoshop’s native file format. Because there is no one universally recognized standard file format, the only way to assure even short-term access to these digital image files is by saving them in multiple formats. 16
Chapter III
Methodology
For this study, qualitative research methods were used to investigate what digital imaging experts are doing to prepare their image files for archival storage. The data was collected through in-depth interviews. Participants selected for these interviews were determined based on the emic perspective (Pike, n.d.) where each interviewee was expected to have an insider’s perspective about the subject.
Restatement of the Problem
The problem of this study was to discover the current best practices of digital image file preparation and to examine the limitations and implications of these practices.
Research Design
This is a descriptive study. It utilized a realist model research paradigm that explores the current thinking and experience of digital imaging subject matter experts who are directly involved in acquiring/archiving digital image files. All the participants had extensive professional experience in the field of digital imaging, but each brought a unique perspective to the dialogue.
Data Collection Instrument
The research methodology employed for this paper was based on interviews with subject matter experts. The first step in carrying out the research was to select the participants. Professional forums, newsgroups, and lists relating to digital imaging in general and RAW file formats specifically were searched. Authors whose posts demonstrated a high level of experience, a willingness to share their knowledge, and the ability to articulate their views were put on a list. Further research was done on each 17 photographer; visits to their websites screened out non-professionals and those not engaged primarily in the work of digital imaging. A broad potential participant list was divided into two groups. The first group included the four photographers who best suited the purpose of this thesis. The second group formed a pool of alternates.
Participants
Bushey (2005) found photographers eager to participate in her survey, and this was true with the photographers asked to participate in this study. The four photographers selected as the first group immediately agreed to be interviewed. Some background information on each participant is presented to establish credibility concerning the opinions that were expressed during the interviews. All four considered the topic of archival file preparation a critical issue for today’s digital photographer.
The data was collected through telephone interviews with the participants. The interviews lasted between 45 and 90 minutes. They were digitally audio recorded on a minidisk recorder attached to a telephone handset. The participants had agreed to be recorded for this study. The minidisk data was later converted to AIFF audio format files for archiving purposes and possible digital distribution.
The project was reviewed by Bowling Green State University’s (BGSU) Human
Subjects Review Board (HSRB) and assigned Project Number H06T035GE7 upon approval. The participants indicated their consent by replying to an HSRB-approved e- mail request (Appendix A).
Procedures of Data Analysis
The first step in the analysis of the data collected in the interviews was to recognize the bias of the interviewer and to minimize its impact. As a professional 18 photographer, the interviewer is very close to the situation being studied, and a personal stakeholder in any outcome of this study. He is a loyal user of a specific brand of camera equipment and has taught workshops using the industry’s largest-selling digital imaging software.
In order to limit the effect of the interviewer’s bias, an initial set of questions was drawn up and reviewed for relevance to the problem of the study (Appendix B). Upon completion of the interviews, summaries were e-mailed to the participants for review and comment (Appendix C). This checking of the interviews by participants eliminated any distortion of the subjects’ responses caused by the interviewer’s bias. After the interview data had been analyzed, it was shared with a peer debriefer, Brent Cavanaugh, of
Bowling Green State University’s Center for Applied Technology, for review and comment in an effort to neutralize any remaining personal bias (Appendix D).
Protection of Human Subjects
All the rules and regulations of BGSU’s HSRB were followed. 19
Budget
All recording, and distributing expenses required to complete the research were presented in Table 1.
Table 1
Expenses for Study
Supplies
CD-R Media 50 X $ .50 $25.00
Minidisk Media 10 X $2.00 $20.00
CD Sleeves 20 X $1.50 $30.00
Subtotal $75.00
Telephone charges
Long Distance 400 minutes X $0.10 $40.00
Subtotal $40.00
Grand Total $115.00
20
Timeline
A calendar of events listing the major deadlines for completion of this thesis is presented in Table 2.
Table 2
Deadlines for the Completion of the Study
Target Date Task
May 26, 2005 Proposal Defended
July15, 2005 Proposal Submitted
September 12, 2005 HSRB Approval
September 29, 2005 Interviews Completed
October 17, 2005 Thesis Completed
October 25, 2005 Thesis Defended
October 31, 2005 Submit Error-Free Copy
November 7, 2005 Electronic Submission
21
Chapter IV
Findings
It is important to examine the environment in which professional photographers work to help understand the problem of the study. The pervasiveness of the technology in the field of photography is revealed by the extensive use of digitally captured, distributed, and archived files.
Participants provided valuable information concerning their experience in archiving digital image files. Specific data concerning their best practices in preparing image files for archiving is shared. An examination of the controversy surrounding proprietary RAW file formats rounds out their conversations.
Biographies
The interview group included Will Crockett, David Riecks, Ellis Vener, and
David Dentry. The first three are full-time professional photographers and educators.
Although Dentry is a part-time photographer, his contributions were important to this paper because he is a digital imaging industry insider. He is a General Manager of
Technical Support for one of the major manufacturers of professional digital cameras and was selected for that background and insight as well as his work as a photographer.
Will Crockett is a successful commercial, industrial, and advertising photographer working in the Chicago area. He is also a photography educator with a history of instructing corporate-sponsored national seminars and most recently as the founder of
ShootSmarter University, a workshop and training facility located in a Chicago suburb.
David Riecks began his career as a photojournalist before working at The Maine
Photographic Workshops. He left there to begin his long-running career as a professional 22 photographer. Riecks holds national seminars on analog/digital workflow, website building, and most recently, building stock photography databases. He is a frequent contributor to many high-level forums and lists and is an active member of several photography trade organizations. Riecks chairs the Stock Artists Alliance (SAA) Imaging
Technology Standards board and the American Society of Media Photographers (ASMP)
Digital Photography Standards and Practices committee (Riecks, 2004).
Ellis Vener began his career nearly 25 years ago assisting other photographers and soon after started his own commercial photography business. He, like the other three participants, contributes regularly to photography forums and lists on the Internet. He also reviews imaging technology for several print and on-line publications (Greenspun,
2002).
Digital Usage
In an effort to gauge the participant’s current use of digital imaging in their businesses, a series of questions were asked about image file usage. All four participants are working primarily with digital cameras, capturing 90 to 99% of their work digitally.
Film was used either for personal work or when no practical digital solution was available. Crockett (personal communication, September 24, 2005) still shoots advertising for a film manufacturer, which accounted for most of the film (10%) he reported. Riecks (personal communication, September 22, 2005), until recently, photographed much of his table-top food assignments using 4” X 5” large-format sheets of film, but has been forced to increase his digital usage because his preferred film lab continues to reduce its services as a direct result of the increased popularity of digital cameras. 23
All photographers questioned stated they delivered digital files in some form to
100% of their clients. They also regularly delivered some form of printed reference sheet(s) and/or proof print(s).
Capture
The choice of file format for image capture varied among the participants. Vener
(personal communication, September 29, 2005) shoots everything in the RAW file format and converts to other formats depending on the intended use. “It just doesn’t make any sense to handicap yourself from the start by shooting JPEG, and shooting TIFFs just takes up too much storage in the camera at the original capture, so it make sense to shoot
RAW” said Vener (personal communication, September 29, 2005). Crockett (personal communication, September 24, 2005) only shoots RAW when he is using his Leaf digital medium-format camera back. The Leaf camera back Crockett uses outputs a proprietary
RAW file that can only be opened in a special application. Everything else is shot as a large, fine-quality JPEG. Riecks is leery of annotating RAW files so shoots a RAW formatted file plus JPEG formatted file for each image (D. Riecks, personal communication, September 22, 2005). “I wouldn’t consider shooting straight JPEG because I just don’t have that confidence,” said Riecks (personal communication,
September 22, 2005). Dentry (personal communication, September 20, 2005) shoots mostly JPEGs for wedding, portrait, and event photography. He reserves shooting RAW for situations requiring the very highest quality and the ability to convert the file to multiple formats for delivery (D. Dentry, personal communication, September 20, 2005). 24
Delivery
When it comes time to deliver the image files to the client all the participants acted in a similar way. They delivered low-resolution JPEG files for previewing images on a computer screen. These thumbnails, as they are often called, are not good enough to produce a quality print and are only meant to be viewed on-screen to aid in final selection or to be used as composites in advertising layouts. When larger files are submitted for end use, the client (or printer) will receive processed, high-resolution, 8-bit or 16-bit, color managed, RGB TIFFs.
Archiving Practices
Each participant was asked how he approached the issue of archiving digital image files, and each had his own perspective on the subject and approach to file preparation. They are all aware that digital imaging technology is still evolving, though, and understand the probable need to both migrate their files to future formats and upgrade their storage media. “The matter of backward compatibility is something I think we all are concerned with,” said Vener (personal communication, September 29, 2005). “I know
I’m concerned with it.” He continued, “It’s a forced migration. Either you have to accept a certain level and say ‘I’m not going any further’ and will get left behind or you have to keep learning it” (E. Vener, personal communication, September 29, 2005). Dentry
(personal communication, September 20, 2005) said, “I personally feel confident enough that the NEF format (Nikon’s proprietary RAW format) will be around long enough where I’ll be able to process or convert or whatever with it. I’m not worried about it.” 25
Crockett quipped,
Photographers who panic because, you know, CDs aren’t going to be around
forever, and PSD files aren’t going to be around forever, that’s just ‘blond talk’
because its not just going to disappear one night and there’s not going to be a
bridge to allow you to get to whatever the new format is. There’s always going to
be converters, so even if we do have an image that’s an MOS file (Leaf’s
proprietary RAW format) and ten years from now we need to resurrect it, there
will be a way to resurrect that MOS file. There’s no problem there (personal
communication, September 24, 2005).
Riecks (personal communication, September 22, 2005) referred to his archive masters as pre-purposed, “a file that has no definite destination but is at that crux where you’ve got to take one of six different forks in the road to get to your end use.” For him this takes form as a high-resolution, 8-bit or 16-bit, wide color-spaced RGB TIFF with little sharpening or manipulation. “The thing is I can always go back and I can fix it, the same thing with the RAW files. I mean as long as you’ve got the RAW file you can go back and fix it” (D. Riecks, personal communication, September 22, 2005). The RAW file format is discussed in greater detail later in this study.
Dentry (personal communication, September 20, 2005) converts his files to the
PSD format and processes them in Adobe Photoshop. He archives both the original RAW or JPEG file and the processed PSD file.
The method described by Vener was the closest to the common practices revealed in Bushey’s (2005) survey: 26
I save the RAW and record that into three places, and then if I do conversions to
JPEG or to TIFF. I will save those as well, although I’m more likely to save a
TIFF for a client on a CD-R and just file it away and not let it take up hard drive
space. I can always come back to the RAW and reprocess it later. As I mentioned
earlier, my skill level is changing, so as I learn more about how to do things a
better way it pays off sometimes to revisit the work” (E. Vener, personal
communication, September 29, 2005).
Earlier in his career Crockett (personal communication, September 24, 2005) photographed many famous personalities and celebrities. He has a concrete vault in his studio to store those transparencies he considers to have long-term value. He said he shoots little these days that needs saving beyond a three-year period. With a bit of a twist, though, he lets his clients take the responsibility of archiving important images. Crockett said:
“Usually our bigger jobs, that’s going to be more product oriented or
environmental oriented types of images, we do something kind of crazy. We
actually give them the RAW file, because it helps us with our archiving and our
storage. Heaven forbid we were to lose images for whatever reason like a flood, a
fire, even a hurricane, heaven forbid. We want to be able to retrieve the full size
RAW file for a period of three years, that’s as long as we figure that these images
will be really useful. So we will give a DVD filled with RAW files back to the
client. Now the client can’t open those RAW files because they are an unusual
image format. So they’ve got them, they can’t use them, so there’s no reason that 27
we can loose any money with them going ahead and using them,” (personal
communication, September 24, 2005).
The RAW Format
The biggest hot-button question is whether or not to use RAW files as archival masters. As reported previously, Vener (personal communication, September 29, 2005) is a RAW format evangelist. He shoots and archives every image in at least the RAW file format. Dentry (personal communication, September 20, 2005) archives both RAW and converted files but appreciated that others do not share his enthusiasm when he stated,
“The RAW format is a tough subject because some people swear by it and say it can give you much better results all around, some people never use it because they don’t know how to use it.” Crockett (personal communication, September 24, 2005) sees RAW as a tool, but his views on archiving in general leave the question of format open. He is as comfortable archiving a large, fine-quality JPEG as a RAW formatted file.
Not everyone sees the RAW format as the archival format of choice. The core problem in Riecks’ (personal communication, September 22, 2005) mind is the proprietary nature of the data. “If the manufacturers and developers would embrace an open standard I think you would have a lot more going, but because everybody is proprietary, everyone wants to protect their trade secrets, everybody wants you to turn to them. They don’t get it. They don’t get it that people don’t want to have something locked up where they can’t get it.” He continued, “Basically, RAW is, unless it’s converted to something else that’s a little bit more open, it’s just a ticking time-bomb. . . .
Anything I’m really concerned about I tend to convert as soon as possible & keep it as my archive color master” (personal communication, September 22, 2005). 28
Considerations
Digital is here to stay. All of the participants shoot, deliver, and archive the vast majority of their images digitally. They capture files formatted as RAW, JPEG, or both simultaneously depending on the subject matter and the image’s intended use. They deliver low-resolution JPEGs for viewing, and high-resolution, 8-bit or 16-bit, color managed RGB TIFFs for reproduction.
Each photographer considered the archiving process in a slightly differently way compared to each another. All were at least mildly concerned with the rapidly changing technology and have plans to migrate their image file’s format as well as physical storage at some point in the future.
The RAW file format is prominent in any discussion of digital image archiving.
The ability to revisit a file at a later date and work on pristine, unprocessed, image data is attractive to all participants. Some of them find it dangerous, though, to rely on a proprietary format and are calling for manufacturers to open their files so they can continue to be supported by software developers in the future. The others have faith that there will always be a conversion solution available to them when they eventually need it. 29
Chapter V
Summary
A brief summary of the important issues raised in this paper precedes a summary of the general findings revealed in the study. Conclusions based on the testimonies of the professional photographers interviewed are followed by a discussion of implications drawn from their comments. The areas of concern raised in this study and deserving of further study are addressed at the end of this chapter.
Today’s world of photography is divided into two groups identified by their use of technology. The use of traditional, film and chemistry-based technology identifies the first group. For this group, methods and practices for conservation have been developed and industry standards established (London, 1989) assuring the survival of even the most delicate of materials required to reproduce photographs. The second group embraces new, still-evolving digital imaging technology; the use of this media is growing steadily, rapidly replacing film as the choice of professional photographers. The long-term viability and ultimate survival of important images created using this transitioning technology is in question due to the lack of industry standards and practices regarding the archiving of these digital assets.
In large part, this study is relevant because of the lack of current literature addressing this issue. There is much discussion on Internet forums and lists about specific problems as they are discovered, but the big picture is only now coming into view. The most relevant piece of literature related to this research study was published less than two months before this paper was completed. Bushey’s (2005) report documented photographer’s real-world practices using digital imaging technology. She, too, noted the 30 lack of sources relating to the management of digital images over the long term (Bushey,
2005).
The purpose of this study was to document the practices of a select group of working photographers in an effort to share their well-considered solutions to the problem of digital image preservation in an environment void of industry standards.
There is no singular best practice, though. The case studies presented offer some examples of best practices that can be used as starting points for others seeking advice about where to begin the process of archiving digital image files. They also serve to further the discussion about this increasingly important issue.
Conclusions
Bushey (2005) indicated that 100% of the professional photographers the researcher surveyed had integrated some form of digital imaging into their business models, and nearly 70% used only digital imaging. All the participants interviewed for this paper reinforced this earlier finding. Considering the growing number of images being created in this digital environment the issue of how to keep digital images viable for the long term may be reaching a critical point.
Photographers understand the dilemma and are becoming increasingly concerned
(Bushey, 2005), although every photographer interviewed for this study believed that future technological advances would provide answers. Crockett thought that when the current technology begins to show signs of failure, a revenue-hungry industry would come up with a solution. “Will there be a problem? Yes. Will the problem be solved?
Absolutely!” he said (personal communication, September 24, 2005). 31
No one file format that can be considered an industry standard for archiving digital image files. Even the ubiquitous, re-purposed TIFF favored by Riecks is not universally used for archiving as demonstrated by Dentry’s use of PSD, Crockett’s use of
JPEG, and Vener’s use of RAW.
The appeal of the RAW-formatted file as an archival solution remains unfulfilled primarily because of its proprietary nature. Without an open RAW standard, depending on the viability of individual companies to support their proprietary secrets indefinitely is unsound. At least one attempt has been made to develop and promote an open source
RAW format. Adobe, the software developer responsible for Photoshop, named it DNG for digital negative, but so far none of the participants in this study have embraced it.
Discussion
Because of the pervasive use of digital imaging technology in professional photography it is imperative to find a solution to the long-term archiving dilemma. We cannot gamble the future of these cultural artifacts on the confidence of future technology. That being said, it is market forces that move companies to action, so the demand for a solution will have to come from current users of the technology and not future beneficiaries of it. Although professionals make up a small portion of all users of digital imaging technology, a large population of consumers considers them as leaders.
These professionals also can call on their powerful trade organizations to lobby companies to reconsider their positions concerning proprietary RAW formats.
The cry for more open source RAW formats from manufacturers is falling on mostly unresponsive ears. The desire to protect trade secrets is strong in spite of the eventuality of unsupported formats. Riecks commented, “As manufacturers discard 32 things left and right as they often do, or just abandon projects, you get these orphans”
(personal communication, September 22, 2005). He argued for a compromise solution involving time-limited protection of proprietary secrets. For example, after a RAW file format is superseded or discarded, all the proprietary information that hobbled competing software developers should be revealed allowing them to fully support the format. This would be a winning situation for all interested parties. The original manufacturer keeps its secrets while they are still marketable, secondary software developers will find a revenue stream previously unavailable to them, and photographers no longer have to worry about their RAW files becoming “orphans.”
Saving multiple copies of image files converted to multiple formats on multiple storage media and stored in multiple locations is as close to a universally recommended practice as any. “It’s the salmon approach to photography,” Riecks said, “lay a lot of eggs and stick them all over the place. When you need one, one of them you’ll be able to open” (personal communication, September 22, 2005).
The participants also agreed that eventually their archives would need to
“migrate” to new storage media and most probably new file formats as well. This is not a new requirement. Each photographer had already dealt with the issue of keeping up with a technology still in the early stages of development. The probability of migrating file formats could be eliminated or at least delayed indefinitely, though, if an open specification RAW format were developed and accepted as a standard by both the manufacturers and end users.
Finally, there is a question of what and how much really must be saved indefinitely. Photographers are not psychic – they may or may not be able to tell what 33 images are important to archive for the future. As a result, most seem determined to save everything! Crockett’s (personal communication, September 24, 2005) take on the short- and long-term value of his work is a refreshing departure from that attitude. “We do have times when there are certain things that do need to have long-term archive, and there, that’s where we rely on the client,” said Crockett. This is an approach that puts the responsibility of protecting the digital assets into the hands of those who value it the most, an idea that merits serious consideration.
Recommendations for Future Studies
The major concern shared by the photographers who participated in this study was the unreliability of the hardware used to physically store the image files. It does not matter what format the image is saved as, if the file becomes corrupted or destroyed by malfunctioning hardware, it is lost. The need for redundant, reliable storage was evident from the examples of hardware failure relayed to the researcher during the interviews. As individual image files get progressively larger so does the storage space needed to archive burgeoning collections. A study of storage habits and concerns as well as up-to- date survey data would be welcome to those researching photography-related technology issues.
The issues raised in this thesis are a long way from being resolved. More studies need to be undertaken in a continuing effort to guide the manufacturers and digital imagers toward a mutually acceptable archiving solution. In the meantime, more survey data needs to be collected in an effort to shed more light on the activities and concerns of those on the front lines of digital image preservation, the image archivists and the professional photographers. 34
References
American Heritage® Dictionary, 4th edition with thesaurus. [CD]. (2004). Boston:
Houghton Mifflin Company.
Adelstein, P. (1999). The why, how, what and where of image permanence standards.
Journal of the Society of Archivists, 20, 1, 41-47.
Birley, G., & Moreland, N. (1998). A practical guide to academic research. London:
Korgan Page.
Bushey, J. (2005, August). International Research on Permanent Authentic Records in
Electronic Systems (InterPARES) 2 project: Experimental, interactive, dynamic
records: Survey of record-keeping practices of photographers using digital
technology (Final report for participants). Retrieved September 22, 2005 from
http://www.interpares.org/display_file.cfm
DigiCamHistory.Com homepage. (n.d.). Retrieved October 8, 2005 from
http://www.digicamhistory.com/
Fraser, B. (2005). Realworld camera raw with Adobe Photoshop CS. Berkeley, CA:
Peachpit Press.
Greenspun, P. (2002). photo.net Interview: Ellis Vener. Retrieved September 15, 2005
from http://www.photo.net/interviews/vener-ellis/
Lerner, A. (2001). Business archives and digital images: Preservation issues versus
getting the job out. The Imaging Science Journal, 49, 171-175.
London, B. (1989). History of photography. In B. L. Upton, & J. Upton Photography:
Adapted from the Life library of photography (4th ed., pp. 351-389). Glenville, IL:
Scott, Foresman and Company. 35
Minister of Public Works and Government Services Canada. (2000). Capture your
collections: A guide for managers planning and implementing digitization
projects. Canada: Author.
OpenRAW.s (2005). The raw problem. Retrieved April 23, 2005, from
http://www.openraw.org/info/
Ostroff, E. (1976). Conserving and restoring photographic collections. Washington, DC:
American Association of Museums.
Pike, K. (n.d.). A stereoscopic window on the world. Retrieved April 25, 2005 from
http://personalwebs.oakland.edu/~kitchens/150d/pike.html
Purglia, S., Reed, J., & Rhodes, E. (June 2004). U.S. national archives and records
administration (NARA) technical guidelines for digitizing archival materials for
electronic access: Creation of production master files – raster images. College
Park, MD: Author.
Riecks, D. (2004). David Riecks Photography. Retrieved September 13, 2005 from
http://www.riecks.com/about.html
Steinmueller, U. (2003, August). The raw deal: Why you should use the raw file format
with digital cameras [Electronic version]. Shutterbug, 94 – 95, 150. Retrieved
April 20, 2005, from http://www.jirvana.com/shutterbug/Aug_002_raw_deal.pdf
TechEncyclopedia (n.d.). Retrieved October 17, 2005 from http://www.techweb.com/encyclopedia/
Young, E. (2002). What ate my image? Royal Photographic Society Journal, 142, 6, 281-
282. 36
Appendix A. Consent E-mail
Office of Graduate Studies College of Technology Bowling Green , OH 43403-0303 B ow l i ng G r e e n S t a t e (419) 372-7613 Fax: (419) 372-7570 University
PARTICIPANT E-MAIL CONSENT FORM
A Research Study of Best Practices for Archiving Digital Image Files
You are invited to participate in a study to determine the current best practices of saving digital image files. I am conducting this study in order to find out how imaging professionals are saving their image files for future use. I, Frank Wiewandt, am a Graduate Student at Bowling Green State University’s College of Technology. The information obtained will be presented in my Master’s Thesis and possible future writings on this subject matter. You were selected as a possible participant because of your current involvement in the field of digital imaging. If you decide to participate you will be asked to submit to an interview lasting approximately 45 minutes. All effort will be made to conduct a face-to- face interview but a telephone interview will be conducted if necessary. The interview will be audio recorded and I will take notes. I will send you a summary of our interview for review and comment before publishing the results. I will be using direct quotes from our interview in my writings. A complete transcription of our interview will be included in an Appendix to my thesis. No attempt will be made to conceal your identity. Your participation in this study is voluntary. If you decide to participate and later change your mind, you may withdraw your consent and discontinue your participation. To withdraw from the study you should notify me in writing. If you have any questions regarding this research study, please contact:
Frank Wiewandt Dr. Gene Poor, Advisor Graduate Student Professor, VC&TE Bowling Green State University College of Technology [email protected] [email protected] (419) 588-1008 (419) 372-7577
You may also contact the Chair, Human Subjects Review Board, Bowling Green State University, (419) 372-7716 ([email protected]), if you have any questions about the conduct of this study or your rights as a participant. Returning this e-mail signifies your agreement to participate in this study. Remember that email is NOT 100% secure. HSRB Project # H06T035GE7 expires 8/25/2006 37
Appendix B. Interview Guidelines
Start off with date & IDs – It’s approximately __:00AM/PM on __/__/2005. My name is FW & I’m conducting an interview with ______on the subject of digital image file preparation for archival storage.
____, tell me a little about your background in photography in general, & digital imaging specifically.
- What % of your photography is digitally captured (v. film)?
- What % of your photography is delivered digitally (v. film or print)?
- What camera hardware do you use?
- What imaging software do you use?
- Capture? Manipulation? File conversion? Other (Sharpening, WWW, etc.)?
- What file format do you use to capture images to the camera?
- Size & Quality.
- What file format(s) do you deliver to clients on a regular basis?
- What file format(s) (& other considerations) do you save your image files as?
- Short term? Long term? Archival?
- Describe your digital workflow?
- Do you have concerns about proprietary RAW file formats? DNG file formats?
- Are you familiar with the JPEG 2000 format?
- If so, what are your impressions for digital image files?
- Are you concerned about the long-term viability of your digital images? 38
Appendix C. Verification E-mails
39
40
41
42
Appendix D. Peer Review E-mail