A Fully Automated Geometric Lens Distortion Correction Method

A FULLY AUTOMATED GEOMETRIC LENS

DISTORTION CORRECTION METHOD

Thesis

Submitted to

The School of Engineering of the

UNIVERSITY OF DAYTON

In Partial Fulﬁllment of the Requirements for

The Degree of

Master of Science in Electrical Engineering

Sravanthi Mannuru, M.S.

UNIVERSITY OF DAYTON

Dayton, Ohio

December, 2011 A FULLY AUTOMATED GEOMETRIC LENS DISTORTION

CORRECTION METHOD

Name: Mannuru, Sravanthi

APPROVED BY:

Eric Balster, Ph.D. Frank A. Scarpino, Ph.D. Advisor Committee Chairman Committee Member Electrical & Computer Engineering Electrical & Computer Engineering Assistant Professor Professor

Vijayan K. Asari, Ph.D. Committee Member Electrical & Computer Engineering Professor & Ohio Research Scholars Chair Wide-Area Surveillance

John Weber, Ph.D. Tony Saliba, Ph.D. Associate Dean Dean, School of Engineering School of Engineering & Wilke Distinguished Professor

ii ABSTRACT

A FULLY AUTOMATED GEOMETRIC LENS DISTORTION CORRECTION METHOD

Name: Mannuru, Sravanthi University of Dayton

Advisor: Dr. Eric Balster

In applications such as computer vision and robotics, camera calibration is required to correct geometric lens distortion of images. The problem with most techniques is that they require human involvement in the calibration process. This thesis proposes a new algorithm for camera calibration with no human involvement.

Typically in camera calibration process, an image of a calibration target (usually a checkerboard) is acquired for distortion correction. The checkerboard is used because it has known features and is easily segmented. If the image of checkerboard pattern undergoes distortion when the image is captured, and the distortion may be determined by analyzing the image of the checkerboard.

The proposed process for coefficient estimation is accomplished by segmenting out the checkerboard of a acquired image. The segmentation is done by finding the connected pixels (components), labeling the connected components and filtering out the unnecessary components from the acquired image. Then the algorithm uses sobel edge detection to detect the vertical and horizontal edges of the checkerboard, because

iii the lines can be used to measure the displacement of image coordinates from their ideal location.

Next, the proposed distortion-correction model is applied to the edges of the image with a set of correction coeﬃcients, resulting a set of corrected images. Next the best

ﬁt line (synthesized line) is found for each observed line in the each corrected image, and the squared distance between each synthesized and observed line is calculated in each corrected image. The average squared distance is then calculated for each corrected image. Finally, the minimum average distance is found for a set of corrected images in order to obtain the respective image correction coeﬃcients.

Both synthetically generated images and natural images have been used to measure the performance of the proposed algorithm. The amount of distortion present in images before and after correction are represented graphically, and results show that the proposed, fully automated algorithm provides equivalent results when compared to other methods which require human involvement.

iv This thesis is dedicated to my professor Dr. Eric Balster who has been my friend,

guide and philosopher.

v ACKNOWLEDGMENTS

First and foremost I would like to thank God for sustaining me as I completed this work, and for giving me strength to overcome seemingly insurmountable obstacles. I owe my deepest gratitude to my thesis advisor, Dr. Eric Blaster and I couldn’t have asked for better. He is incredibly helpful and willing to go way above and beyond. I am eternally grateful for all his help. Thanks for expressing his conﬁdence in abilities to complete the task at hand.

I would like to thank my committee members, Dr. Frank A Scarpino and Dr.

Vijayan K Asari, for generously oﬀering their time, support, guidance and review of this document.

I would like to thank my parents and my brother for their love and support throughout the years.

Last, but not the least, I thank my colleagues and friends for their support, help and valuable hits.

vi TABLE OF CONTENTS

Page

ABSTRACT ...... iii

DEDICATION ...... v

ACKNOWLEDGMENTS ...... vi

LIST OF FIGURES ...... ix

LIST OF TABLES ...... xi

CHAPTER:

1. INTRODUCTION ...... 1

1.1 Problems Caused by Radial Distortion ...... 3 1.2 Previous Work ...... 4 1.3 Outline ...... 7

2. RADIAL DISTORTION-CORRECTION MODEL ...... 8

2.1 Distortion Model ...... 8 2.2 Distortion-Correction Model ...... 9

3. CORRECTING RADIAL DISTORTION ...... 12

3.1 Coeﬃcients Estimator ...... 13 3.1.1 Extract Checkerboard ...... 14 3.1.2 Edge Detection ...... 17 3.1.3 Synthesized Lines Generator and Line Distance Calculator . 19 3.1.4 Minimum Average Squared Distance ...... 20 3.2 Correction ...... 21

vii 4. RESULTS ...... 22

4.1 Experiments Conducted using Synthetically Generated Imagery . . 22 4.1.1 Experiment 1 ...... 22 4.1.2 Experiment 2 ...... 30 4.2 Experiments Conducted using Natural Imagery ...... 34 4.2.1 Experiment 1 ...... 34 4.2.2 Experiment 2 ...... 36

5. CONCLUSION AND FUTURE WORK ...... 39

5.1 Conclusions ...... 39 5.2 Future Work ...... 39

BIBLIOGRAPHY ...... 41

viii LIST OF FIGURES

Figure Page

1.1 Two types of radial distortion ...... 1

1.2 Barrel distortion ...... 2

1.3 Example images of barrel distortion and pincushion distortion . . . . 3

1.4 Image mosaicing without lens distortion correction ...... 4

1.5 Checkerboard ...... 5

1.6 Clicking on the four extreme corners of the checkerboard ...... 6

3.1 Real-world image ...... 12

3.2 General algorithm ...... 13

3.3 Coeﬃcients estimator block diagram ...... 15

3.4 Extracting checkerboard algorithm ...... 15

3.5 Threshold image M(xd, yd)...... 16

3.6 Connected components ...... 16

3.7 Connected components labeling ...... 17

3.8 Extracted checkerboard ...... 17

3.9 Edge detection ...... 18

3.10 Generating synthesized for observed lines ...... 19

3.11 The squared distance is calculated between synthesized and observed

lines in each corrected image ...... 20

ix 3.12 Proposed distortion-correction algorithm ...... 21

4.1 Synthetic checkerboard image of size 328x328 ...... 23

4.2 Distortion process ...... 23

4.3 A synthetic radially distorted image ...... 23

4.4 Correction process ...... 24

4.5 Correcting radially distorted image ...... 24

4.6 Comparing original and corrected image using MSE ...... 25

4.7 Diﬀerence between image coordinates in x-direction with respect to

y-coordinates before distortion-correction ...... 26

4.8 Diﬀerence between image coordinates along x-axis with respect to y-

coordinates after distortion-correction ...... 26

4.9 Synthetic checkerboard image of size 720x720 ...... 30

4.10 Distorted image ...... 31

4.11 Corrected image ...... 31

4.12 Mean square error between original image and distorted image . . . . 32

4.13 Correcting radial distorted image using proposed distortion-correction

algorithm ...... 35

4.14 Diﬀerence between the distorted and corrected curves ...... 35

4.15 Correcting radial distorted image using CCTM algorithm ...... 36

4.16 Diﬀerence between the proposed algorithm and CCTM algorithm . . 37

4.17 Correcting radial distorted image using proposed correction algorithm 37

4.18 Diﬀerence between the distorted and corrected curves ...... 38

x LIST OF TABLES

Table Page

4.1 Execution time required to distort and correct an image using proposed

distortion-correction algo