UCAM-CL-TR-261 Technical Report ISSN 1476-2986 Number 261 Computer Laboratory Image resampling Neil Anthony Dodgson August 1992 15 JJ Thomson Avenue Cambridge CB3 0FD United Kingdom phone +44 1223 763500 http://www.cl.cam.ac.uk/ c 1992 Neil Anthony Dodgson This technical report is based on a dissertation submitted by the author for the degree of Doctor of Philosophy to the University of Cambridge, Wolfson College. Technical reports published by the University of Cambridge Computer Laboratory are freely available via the Internet: http://www.cl.cam.ac.uk/TechReports/ ISSN 1476-2986 3 Summary Image resampling is the process of geometrically transforming digital images. This dissertation considers several aspects of the process. We begin by decomposing the resampling process into three simpler sub-processes: reconstruction of a continuous intensity surface from a discrete image, transformation of that continuous surface, and sampling of the transformed surface to produce a new discrete image. We then consider the sampling process, and the subsidiary problem of intensity quantisation. Both these are well understood, and we present a summary of existing work, laying a foundation for the central body of the dissertation where the sub-process of reconstruction is studied. The work on reconstruction divides into four parts, two general and two specific: 1. Piecewise local polynomials: the most studied group of reconstructors. We examine these, and the criteria used in their design. One new derivation is of two piecewise local quadratic reconstructors. 2. Infinite extent reconstructors: we consider these and their local approximations, the problem of finite image size, the resulting edge effects, and the solutions to these problems. Amongst the reconstructors discussed are the interpolating cubic B-spline and the interpolating Bezier cubic. We derive the filter kernels for both of these, and prove that they are the same. Given this kernel we demonstrate how the interpolating cubic B-spline can be extended from a one- dimensional to a two-dimensional reconstructor, providing a considerable speed improvement over the existing method of extension. 3. Fast Fourier transform reconstruction: it has long been known that the fast Fourier transform (FFT) can be used to generate an approximation to perfect scaling of a sample set. Donald Fraser (in 1987) took this result and generated a hybrid FFT reconstructor which can be used for general transformations, not just scaling. We modify Fraser’s method to tackle two major problems: its large time and storage requirements, and the edge effects it causes in the reconstructed intensity surface. 4. A priori knowledge reconstruction: first considering what can be done if we know how the original image was sampled, and then considering what can be done with one particular class of image coupled with one particular type of sampling. In this latter case we find that exact reconstruction of the image is possible. This is a surprising result as this class of images cannot be exactly reconstructed using classical sampling theory. The final section of the dissertation draws all of the strands together to discuss transformations and the resampling process as a whole. Of particular note here is work on how the quality of different reconstruction and resampling methods can be assessed. 4 About this document This report is, apart from some cosmetic changes, the dissertation which I submitted for my PhD degree. I performed the research for this degree over the period January 1989 through May 1992. The page numbering is different in this version to versions prior to 2005 because I have been able to incorporate many of the photographs which were previously unavailable in the online version. Figures 1.4, 2.1, 2.2, 4.34, 7.9, and 7.10 are either incomplete or missing, owing to the way in which the original manuscript was prepared. Appendix C is also missing for the same reason. Acknowledgements I would like to thank my supervisor, Neil Wiseman, for his help, advice, and listening ear over the course of this research. I am especially grateful for his confidence in me, even when I had little of my own, and for reading this dissertation and making many helpful comments. I would also like to thank David Wheeler and Paul Heckbert for useful discussions about image resampling and related topics, which clarified and stimulated my research. Over the last three and a third years several bodies have given me financial assistance. The Cambridge Commonwealth Trust paid my fees and maintenance for the first three years, for which I am grateful. Wolfson College, through the kind offices of Dr Rudolph Hanka, and the Computer Laboratory, have both contributed towards travel, conference, and typing expenses; and I am grateful for these too. Finally, I am extremely grateful to my wife, Catherine Gibbs, for her financial support, without which that last third of a year would have been especially difficult. This dissertation was typed in by four people, not a pleasant task. Many thanks go to Lynda Murray, Eileen Murray, Annemarie Gibbs, and Catherine Gibbs for taking this on, and winning! The Rainbow graphics research group deserve many thanks for their support and friendship over my time in Cambridge; they have made the research much more enjoyable. Thanks go to Olivia, Stuart, Jeremy, Heng, Alex, Adrian, Dan, Tim, Rynson, Nicko, Oliver, Martin, Jane, and Neil. Equal, if not more, thanks go to Margaret Levitt and Eileen Murray, who have helped to keep me sane; and also to the other inmates of Austin 4 and 5 for the interesting working environment they provide. Final heartfelt thanks go to my parents and to Catherine, to whom this dissertation is dedicated. Their support, and encouragement, has been invaluable in getting me to this point. Contents Notation 8 Preface 9 1 Resampling 11 1.1 Introduction ........................................ 11 1.2 Images ........................................... 11 1.3 Sources of discrete images ................................ 13 1.4 Resampling: the geometric transformation of discrete images . ......... 16 1.5Thepracticalusesofresampling............................ 16 1.6 Optical image warping . ................................ 18 1.7 Decomposition ...................................... 18 1.8 How it really works .................................... 27 1.9 The literature ....................................... 29 1.10 Summary ......................................... 31 2 Sampling 33 2.1Convertingthecontinuousintothediscrete...................... 33 2.2 The problem of discrete intensities ........................... 33 2.3 Producing a discrete image ............................... 37 2.4Thesamplingprocess.................................. 39 2.5 Area vs pointsampling.................................. 43 2.6 Practical considerations . ................................ 46 2.7 Anti-aliasing ....................................... 48 2.8 Sampling which considers the reconstruction method ................. 49 2.9 Summary ......................................... 50 3 Reconstruction 52 3.1 Classification ....................................... 52 3.2 Skeleton reconstruction . ................................ 55 5 6 CONTENTS 3.3 Perfect interpolation ................................... 58 3.4 Piecewise local polynomials ............................... 59 3.5WhatshouldwelookforinaNAPKreconstructor?................. 72 3.6 Cubic functions ...................................... 87 3.7Higherorders....................................... 95 3.8 Summary ......................................... 96 4 Infinite Extent Reconstruction 98 4.1 Introduction to infinite reconstruction . ....................... 98 4.2Edgeeffects........................................ 98 4.3 Sinc and other perfect interpolants ........................... 117 4.4Gaussianreconstructors................................. 123 4.5 Interpolating cubic B-spline ............................... 125 4.6 Local approximations to infinite reconstructors .................... 142 4.7 Summary ......................................... 144 5 Fast Fourier Transform Reconstruction 145 5.1Thereconstructedintensitysurface........................... 145 5.2DFTsampleratechanging................................ 148 5.3 FFT intensity surface reconstruction . ....................... 154 5.4 Problems with the FFT reconstructor . ....................... 156 5.5 Summary ......................................... 169 6 A priori knowledge reconstruction 171 6.1Knowledgeaboutthesampler..............................171 6.2 Knowledge about the image content . ....................... 174 6.3 One-dimensional sharp edge reconstruction ...................... 175 6.4 Two-dimensional sharp edge reconstruction ...................... 191 6.5 Summary ......................................... 210 7 Transforms & Resamplers 211 7.1Practicalresamplers................................... 211 7.2 Transform dimensionality ................................ 213 7.3 Evaluation of reconstructors and resamplers ...................... 215 7.4 Summary ......................................... 227 8 Summary & Conclusions 228 8.1Majorresults....................................... 228 8.2Otherresults....................................... 229 CONTENTS 7 8.3Futurework........................................ 229 A Literature Analysis 230 B Proofs 238 B.1 The inverse Fourier transform of a box function .................... 238 B.2 The Fourier transform of the sinc function ....................... 238 √ B.3 Derivation of a =