Framing Digital Image Credibility: Image Manipulation Problems, Perceptions and Solutions
Total Page:16
File Type:pdf, Size:1020Kb
Sabrina Bleecker Caldwell The Australian National University Research School of Computer Science November, 2016 Framing digital image credibility: image manipulation problems, perceptions and solutions A thesis submitted for the degree of Doctor of Philosophy of The Australian National University © Copyright by Sabrina Bleecker Caldwell 2016 Framing digital image credibility: image manipulation problems, perceptions and solutions I certify that this thesis is my own original work, resulting from research undertaken by me as a PhD candidate at the Australian National University Words: 47250 _______________________________________ Sabrina Bleecker Caldwell Caldwell Page | 2 Framing digital image credibility: image manipulation problems, perceptions and solutions Acknowledgements To create an active tempo in my writing and to reflect that it is my own research, I have written this thesis largely in first person, but I wish to acknowledge that just as it takes a village to raise a child, it takes a community to provide the intellectual contributions, financial and logistical support, and pastoral care required for the epic journey that is a PhD. I am grateful to many people for their thoughtful and inspirational assistance to me in undertaking this research. First and foremost I wish to acknowledge my principal supervisor and advisor Professor Tamás Gedeon for his sage advice and practical support. Secondly, I want to express my appreciation to Dr Richard Jones, whose support was also invaluable and treasured. Together, Tom and Richard provided me with years of encouragement, consultation, understanding, and even friendship, and provided me with many exciting opportunities to advance and promote my research. I also thank Dr Martyn Jolly of the School of Art for his expertise in photography and cross-disciplinary perspectives in respect of my investigations. As my research progressed, I found myself working with a fellow PhD student, Leana Copeland, who also used eye gaze as a tool of investigation (in her case for reading comprehension), and together we were able to undertake joint experimentation in which she gained outcomes for her research and I gained outcomes for mine. It was wonderful working with her and I hope that an opportunity arises to do so again in future. On a personal note, I wish to thank my husband Brian Caldwell for his endless and steadfast support. It is seldom that a man is called upon to endure the trials of supporting a wife undertaking PhD studies not just once, but twice. Lastly, I want to thank the Research School of Computer Science, the College of Engineering and Computer Science and ANU for their faith in me, and for allowing me this extraordinary opportunity to follow my passion to find a practical solution to the problem of credibility in photography, and the visual information it captures, for posterity. Caldwell Page | 3 Framing digital image credibility: image manipulation problems, perceptions and solutions Abstract Image manipulation is subverting the credibility of photographs as a whole. Currently there is no practical solution for asserting the authenticity of a photograph. People express their concern about this when asked but continue to operate in a ‘business as usual’ fashion. While a range of digital forensic technologies has been developed to address falsification of digital photographs, such technologies begin with ‘sourceless’ images and conclude with results in equivocal terms of probability, while not addressing the meaning and content contained within the image. It is interesting that there is extensive research into computer-based image forgery detection, but very little research into how we as humans perceive, or fail to perceive, these forgeries when we view them. The survey, eye-gaze tracking experiments and neural network analysis undertaken in this research contribute to this limited pool of knowledge. The research described in this thesis investigates human perceptions of images that are manipulated and, by comparison, images that are not manipulated. The data collected, and their analyses, demonstrate that humans are poor at identifying that an image has been manipulated. I consider some of the implications of digital image manipulation, explore current approaches to image credibility, and present a potential digital image authentication framework that uses technology and tools that exploit social factors such as reputation and trust to create a framework for technologically packaging/wrapping images with social assertions of authenticity, and surfaced metadata information. The thesis is organised into 6 chapters. Chapter 1: Introduction I briefly introduce the history of photography, highlighting its importance as reportage, and discuss how it has changed from its introduction in the early 19th century to today. I discuss photo manipulation and consider how it has changed along with photography. I Caldwell Page | 4 Framing digital image credibility: image manipulation problems, perceptions and solutions describe the relevant literature on the subject of image authentication and the use of eye gaze tracking and neural nets in identifying the role of human vision in image manipulation detection, and I describe my area of research within this context. Chapter 2: Literature review I describe the various types of image manipulation, giving examples, and then canvas the literature to describe the landscape of image manipulation problems and extant solutions, namely: the nature of image manipulation, investigations of human perceptions of image manipulation, eye gaze tracking and manipulated images, known efforts to create solutions to the problem of preserving unadulterated photographic representations and the meanings they hold. Finally, I position my research activities within the context of the literature. Chapter 3: The research I describe the survey and experiments I undertook to investigate attitudes toward image manipulation, research human perceptions of manipulated and unmanipulated images, and to trial elements of a new wrapper-style file format that I call .msci (mobile self- contained image), designed to address image authenticity issues. Methods, results and discussion for each element are presented in both explanatory text and by presentation of papers resulting from the experiments. Chapter 4: Analysis of eye gaze data using classification neural networks I describe pattern classifying neural network analysis applied to selected data obtained from the experiments and the insights this analysis provided into the opaque realm of cognitive perception as seen through the lens of eye gaze. Chapter 5: Discussion I synthesise and discuss the outcomes of the survey and experiments. Caldwell Page | 5 Framing digital image credibility: image manipulation problems, perceptions and solutions I discuss the outcomes of this research, and consider the need for a distinction between photographs and photo art. I offer a theoretical formula within which the overall authenticity of an image can be assessed. In addition I present a potential image authentication framework built around the .msci file format, designed in consideration of my investigation of the requirements of the image manipulation problem space and the experimental work undertaken in this research. Chapter 6: Conclusions and future work This thesis concludes with a summary of the outcomes of my research, and I consider the need for future experimentation to expand on the insights gained to date. I also note some ways forward to develop an image authentication framework to address the ongoing problem of image authenticity. Caldwell Page | 6 Framing digital image credibility: image manipulation problems, perceptions and solutions Table of Contents 1 Introduction .............................................................................................................. 19 1.1 Why photographs are important ...................................................................... 21 1.1.1 History, memories and the human experience ............................................ 21 1.1.2 Evidence, security and justice ..................................................................... 22 1.1.3 Science and medicine .................................................................................. 23 1.1.4 Visual culture .............................................................................................. 23 1.2 Photo manipulation: a two-edged sword ......................................................... 24 1.3 Digital image forensics vs. authentication ...................................................... 28 1.4 Summary and Scope ........................................................................................ 29 2 Literature Survey ..................................................................................................... 33 2.1 Nature of image manipulation ......................................................................... 33 2.1.1 The rise and rise of photography and concomitant photo manipulation ..... 33 2.1.2 Types of photograph manipulation ............................................................. 43 2.1.3 Factors outside of the image affecting credibility ....................................... 47 2.2 Human perceptions and perspectives of photo manipulation ......................... 51 2.3 Peak photographic organisations’ responses ................................................... 53 2.4 Eye gaze tracking and manipulated images ...................................................