DOCSLIB.ORG

Visual Object Recognition

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Visual Object Recognition Kristen Grauman University of Texas at Austin Bastian Leibe RWTH Aachen University SYNTHESIS LECTURES ON COMPUTER VISION # 1 Abstract The visual recognition problem is central to computer vision research. From robotics to information retrieval, many desired applications demand the ability to iden- tify and localize categories, places, and objects. This tutorial overviews computer vision algorithms for visual object recognition and image classification. We introduce primary representations and learning approaches, with an emphasis on recent advances in the field. The target audience consists of researchers or students working in AI, robotics, or vision who would like to understand what methods and representations are available for these problems. This lecture summarizes what is and isn’t possible to do reliably today, and overviews key concepts that could be employed in systems requiring visual categorization. Keywords: More specifically, the topics we cover in detail within the above are: global representations versus local descriptors; detection and description of local invari- ant features; efficient algorithms for matching local features, including tree-based and hashing-based search algorithms; visual vocabularies and bags-of-words; methods to ver- ify geometric consistency according to parameterized geometric transformations; deal- ing with outliers in correspondences, including RANSAC and the Generalized Hough transform; window-based descriptors, including histograms of oriented gradients and rectangular features; part-based models, including star graph models and fully con- nected constellations; pyramid match kernels; detection via sliding windows; Hough voting; Generalized distance transform; the Implicit Shape Model; and the Deformable Part-based Model. v Contents Contents............................................ .........................v 1 Introduction........................................ .........................1 1.1 Overview......................................... ......................1 1.2 Challenges.............................................................3 1.3 The State of the Art...................................................5 2 Overview: Recognition of Specific Objects . ....................7 2.1 Global Image Representations . ...................7 2.2 Local Feature Representations..........................................9 3 Local Features: Detection and Description. ....................11 3.1 Introduction..................................... .....................11 3.2 Detection of Interest Points and Regions . 13 3.2.1 Keypoint Localization 13 3.2.2 Scale Invariant Region Detection 15 3.2.3 Affine Covariant Region Detection 21 3.2.4 Orientation Normalization 23 3.2.5 Summary of Local Detectors 23 3.3 Local Descriptors.....................................................24 3.3.1 The SIFT Descriptor 24 3.3.2 The SURF Detector/Descriptor 25 3.4 Concluding Remarks..................................................27 4 Matching Local Features............................... .....................29 4.1 Efficient Similarity Search.......................... ...................30 4.1.1 Tree-based Algorithms 30 4.1.2 Hashing-based Algorithms and Binary Codes 33 vi 4.1.3 A Rule of Thumb for Reducing Ambiguous Matches 36 4.2 Indexing Features with Visual Vocabularies ........................... 37 4.2.1 Creating a Visual Vocabulary 37 4.2.2 Vocabulary Trees 39 4.2.3 Choices in Vocabulary Formation 40 4.2.4 Inverted File Indexing 42 4.3 Concluding Remarks..................................................42 5 Geometric Verification of Matched Features . ..................44 5.1 Estimating Geometric Models........................ .................44 5.1.1 Estimating Similarity Transformations 45 5.1.2 Estimating Affine Transformations 45 5.1.3 Homography Estimation 46 5.1.4 More General Transformations 48 5.2 Dealing with Outliers.................................................49 5.2.1 RANSAC 50 5.2.2 Generalized Hough Transform 52 5.2.3 Discussion 53 6 Example Systems: Specific-Object Recognition . ..................55 6.1 Image Matching.................................... ...................55 6.2 Object Recognition ...................................................55 6.3 Large-Scale Image Retrieval...........................................56 6.4 Mobile Visual Search..................................................56 6.5 Image Auto-Annotation...............................................58 6.5.1 Conclusions 59 7 Overview: Recognition of Generic Object Categories. ..................61 8 Representations for Object Categories . .....................62 8.1 Window-based Object Representations . ................62 8.1.1 Pixel Intensities and Colors 62 vii 8.1.2 Window Descriptors: Global Gradients and Texture 63 8.1.3 Patch Descriptors: Local Gradients and Texture 64 8.1.4 A Hybrid Representation: Bags of Visual Words 67 8.1.5 Contour and Shape Features 68 8.1.6 Feature Selection 68 8.2 Part-based Object Representations....................................69 8.2.1 Overview of Part-Based Models 70 8.2.2 Fully-Connected Models: The Constellation Model 73 8.2.3 Star Graph Models 75 8.3 Mixed Representations................................................76 8.4 Concluding Remarks..................................................76 9 Generic Object Detection: Finding and Scoring Candidates . ................78 9.1 Detection via Classification . ...................78 9.1.1 Speeding up Window-based Detection 79 9.1.2 Limitations of Window-based Detection 80 9.2 Detection with Part-based Models.....................................82 9.2.1 Combination Classifiers 82 9.2.2 Voting and the Generalized Hough Transform 83 9.2.3 RANSAC 84 9.2.4 Generalized Distance Transform 85 10 Learning Generic Object Category Models . .................87 10.1 Data Annotation.................................. ....................87 10.2 Learning Window-based Models.......................................89 10.2.1Specialized Similarity Measures and Kernels 89 10.3 Learning Part-based Models ......................................... 100 10.3.1Learning in the Constellation Model 100 10.3.2Learning in the Implicit Shape Model 100 10.3.3Learning in the Pictorial Structure Model 101 11 Example Systems: Generic Object Recognition . ................104 viii CONTENTS 11.1 The Viola-Jones Face Detector.......................................104 11.1.1Training Process 104 11.1.2Recognition Process 106 11.1.3Discussion 107 11.2 The HOG Person Detector...........................................108 11.3 Bag-of-Words Image Classification ................................... 109 11.3.1Training Process 109 11.3.2Recognition Process 110 11.3.3Discussion 110 11.4 The Implicit Shape Model ........................................... 111 11.4.1Training Process 111 11.4.2Recognition Process 112 11.4.3Vote Backprojection and Top-Down Segmentation 112 11.4.4Hypothesis Verification 114 11.4.5Discussion 115 11.5 Deformable Part-based Models.......................................116 11.5.1Training Process 116 11.5.2Recognition Process 116 11.5.3Discussion 118 12 Other Considerations and Current Challenges . .................120 12.1 Benchmarks and Datasets........................... .................120 12.2 Context-based Recognition...........................................123 12.3 Multi-Viewpoint and Multi-Aspect Recognition . 124 12.4 Role of Video........................................................126 12.5 Integrated Segmentation and Recognition . 126 12.6 Supervision Considerations in Object Category Learning . 127 12.6.1Using Weakly Labeled Image Data 128 12.6.2Maximizing the Use of Manual Annotations 129 12.6.3Unsupervised Object Discovery 129 12.7 Language, Text, and Images ......................................... 130 CONTENTS ix 13 Conclusions......................................... .......................132 Preface This lecture summarizes the material in a tutorial we gave at AAAI 2008 (Grau- man & Leibe 2008)1. Our goal is to overview the types of methods that figure most prominently in object recognition research today, in order to give a survey of the concepts, algorithms, and representations that one might use to build a visual recognition system. Recognition is a rather broad and quickly moving field, and so we limit our scope to methods that are already used fairly frequently in the literature. As needed, we point the reader to outside references for more details on certain topics. We assume that the reader has basic familiarity with machine learning algo- rithms for supervised classification, and some background in low-level image processing. Outline: The text is divided primarily according to the two forms of recogni- tion: specific and generic. In the first chapter, we motivate those two problems and point out some of their main challenges. Chapter 2 then overviews the problem of specific object recognition, and the global and local representations used by current approaches. Then, in Chapters 3 through 5, we explain in detail how today’s state-of- the-art methods utilize local features to identify an object in a new image. This entails efficient algorithms for local feature extraction, for retrieving candidate matches, and for performing geometric verification. We wrap up our coverage of specific objects by outlining example end-to-end systems from recent work in Chapter 6, pulling together the key steps from the prior chapters on local features and matching. Chapter 7 introduces the generic (category-level) recognition problem, and
Recommended publications
  • Hough Transform, Descriptors Tammy Riklin Raviv Electrical and Computer Engineering Ben-Gurion University of the Negev Hough Transform

    Hough Transform, Descriptors Tammy Riklin Raviv Electrical and Computer Engineering Ben-Gurion University of the Negev Hough Transform

    DIGITAL IMAGE PROCESSING Lecture 7 Hough transform, descriptors Tammy Riklin Raviv Electrical and Computer Engineering Ben-Gurion University of the Negev Hough transform y m x b y m 3 5 3 3 2 2 3 7 11 10 4 3 2 3 1 4 5 2 2 1 0 1 3 3 x b Slide from S. Savarese Hough transform Issues: • Parameter space [m,b] is unbounded. • Vertical lines have infinite gradient. Use a polar representation for the parameter space Hough space r y r q x q x cosq + ysinq = r Slide from S. Savarese Hough Transform Each point votes for a complete family of potential lines: Each pencil of lines sweeps out a sinusoid in Their intersection provides the desired line equation. Hough transform - experiments r q Image features ρ,ϴ model parameter histogram Slide from S. Savarese Hough transform - experiments Noisy data Image features ρ,ϴ model parameter histogram Need to adjust grid size or smooth Slide from S. Savarese Hough transform - experiments Image features ρ,ϴ model parameter histogram Issue: spurious peaks due to uniform noise Slide from S. Savarese Hough Transform Algorithm 1. Image à Canny 2. Canny à Hough votes 3. Hough votes à Edges Find peaks and post-process Hough transform example http://ostatic.com/files/images/ss_hough.jpg Incorporating image gradients • Recall: when we detect an edge point, we also know its gradient direction • But this means that the line is uniquely determined! • Modified Hough transform: for each edge point (x,y) θ = gradient orientation at (x,y) ρ = x cos θ + y sin θ H(θ, ρ) = H(θ, ρ) + 1 end Finding lines using Hough transform
  • Scale Invariant Feature Transform (SIFT) Why Do We Care About Matching Features?

    Scale Invariant Feature Transform (SIFT) Why Do We Care About Matching Features?

    Scale Invariant Feature Transform (SIFT) Why do we care about matching features? • Camera calibration • Stereo • Tracking/SFM • Image moiaicing • Object/activity Recognition • … Objection representation and recognition • Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging parameters • Automatic Mosaicing • http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html We want invariance!!! • To illumination • To scale • To rotation • To affine • To perspective projection Types of invariance • Illumination Types of invariance • Illumination • Scale Types of invariance • Illumination • Scale • Rotation Types of invariance • Illumination • Scale • Rotation • Affine (view point change) Types of invariance • Illumination • Scale • Rotation • Affine • Full Perspective How to achieve illumination invariance • The easy way (normalized) • Difference based metrics (random tree, Haar, and sift, gradient) How to achieve scale invariance • Pyramids • Scale Space (DOG method) Pyramids – Divide width and height by 2 – Take average of 4 pixels for each pixel (or Gaussian blur with different ) – Repeat until image is tiny – Run filter over each size image and hope its robust How to achieve scale invariance • Scale Space: Difference of Gaussian (DOG) – Take DOG features from differences of these images‐producing the gradient image at different scales. – If the feature is repeatedly present in between Difference of Gaussians, it is Scale Invariant and should be kept. Differences Of Gaussians
  • Arxiv:2108.09823V1 [Cs.AI] 22 Aug 2021 1 Introduction

    Arxiv:2108.09823V1 [Cs.AI] 22 Aug 2021 1 Introduction

    Embodied AI-Driven Operation of Smart Cities: A Concise Review Farzan Shenavarmasouleh1, Farid Ghareh Mohammadi1, M. Hadi Amini2, and Hamid R. Arabnia1 1:Department of Computer Science, Franklin College of arts and sciences, University of Georgia, Athens, GA, USA 2: School of Computing & Information Sciences, College of Engineering & Computing, Florida International University, Miami, FL, USA Emails: [email protected], [email protected], amini@cs.fiu.edu, [email protected] August 24, 2021 Abstract A smart city can be seen as a framework, comprised of Information and Communication Technologies (ICT). An intelligent network of connected devices that collect data with their sensors and transmit them using wireless and cloud technologies in order to communicate with other assets in the ecosystem plays a pivotal role in this framework. Maximizing the quality of life of citizens, making better use of available resources, cutting costs, and improving sustainability are the ultimate goals that a smart city is after. Hence, data collected from these connected devices will continuously get thoroughly analyzed to gain better insights into the services that are being offered across the city; with this goal in mind that they can be used to make the whole system more efficient. Robots and physical machines are inseparable parts of a smart city. Embodied AI is the field of study that takes a deeper look into these and explores how they can fit into real-world environments. It focuses on learning through interaction with the surrounding environment, as opposed to Internet AI which tries to learn from static datasets. Embodied AI aims to train an agent that can See (Computer Vision), Talk (NLP), Navigate and Interact with its environment (Reinforcement Learning), and Reason (General Intelligence), all at the same time.
  • Zero-Shot Compositional Concept Learning

    Zero-Shot Compositional Concept Learning

    Zero-Shot Compositional Concept Learning Guangyue Xu Parisa Kordjamshidi Joyce Y. Chai Michigan State University Michigan State University University of Michigan [email protected] [email protected] [email protected] Abstract Train Phase: Concept of Sliced Concept of Apple In this paper, we study the problem of rec- Sliced Tomato Sliced Bread Sliced Cake Diced Apple Ripe Apple Peeled Apple Localize, Learn and Compose Regional ognizing compositional attribute-object con- Visual Features cepts within the zero-shot learning (ZSL) framework. We propose an episode-based cross-attention (EpiCA) network which com- Test Phase: bines merits of cross-attention mechanism and Sliced Apple Compose the Learnt Regional Visual Diced Pizza Features episode-based training strategy to recognize … novel compositional concepts. Firstly, EpiCA bases on cross-attention to correlate concept- Figure 1: Given the concepts of sliced and apple in the visual information and utilizes the gated pool- training phase, our target is to recognize the novel com- ing layer to build contextualized representa- positional concept slice apple which doesn’t appear in tions for both images and concepts. The up- the training set by decomposing, grounding and com- dated representations are used for a more in- posing concept-related visual features. depth multi-modal relevance calculation for concept recognition. Secondly, a two-phase episode training strategy, especially the trans- with other objects or attributes, the combination ductive phase, is adopted to utilize unlabeled of this attribute-object pair is not observed in the test examples to alleviate the low-resource training set. learning problem. Experiments on two widely- used zero-shot compositional learning (ZSCL) This is a challenging problem, because objects benchmarks have demonstrated the effective- with different attributes often have a significant di- ness of the model compared with recent ap- versity in their visual features.
  • I Hope This Is Helpful": Understanding Crowdworkers’ Challenges and Motivations for an Image Description Task

    I Hope This Is Helpful": Understanding Crowdworkers’ Challenges and Motivations for an Image Description Task

    105 "I Hope This Is Helpful": Understanding Crowdworkers’ Challenges and Motivations for an Image Description Task RACHEL N. SIMONS, Texas Woman’s University DANNA GURARI, The University of Texas at Austin KENNETH R. FLEISCHMANN, The University of Texas at Austin AI image captioning challenges encourage broad participation in designing algorithms that automatically create captions for a variety of images and users. To create large datasets necessary for these challenges, researchers typically employ a shared crowdsourcing task design for image captioning. This paper discusses findings from our thematic analysis of 1,064 comments left by Amazon Mechanical Turk workers usingthis task design to create captions for images taken by people who are blind. Workers discussed difficulties in understanding how to complete this task, provided suggestions of how to improve the task, gave explanations or clarifications about their work, and described why they found this particular task rewarding or interesting. Our analysis provides insights both into this particular genre of task as well as broader considerations for how to employ crowdsourcing to generate large datasets for developing AI algorithms. CCS Concepts: • Information systems → Crowdsourcing; • Computing methodologies → Computer vision; Computer vision tasks; Image representations; Machine learning; • Human- centered computing → Accessibility. Additional Key Words and Phrases: Crowdsourcing; Computer Vision; Artificial Intelligence; Image Captioning; Accessibility; Amazon Mechanical Turk
  • Exploiting Information Theory for Filtering the Kadir Scale-Saliency Detector

    Exploiting Information Theory for Filtering the Kadir Scale-Saliency Detector

    Introduction Method Experiments Conclusions Exploiting Information Theory for Filtering the Kadir Scale-Saliency Detector P. Suau and F. Escolano {pablo,sco}@dccia.ua.es Robot Vision Group University of Alicante, Spain June 7th, 2007 P. Suau and F. Escolano Bayesian filter for the Kadir scale-saliency detector 1 / 21 IBPRIA 2007 Introduction Method Experiments Conclusions Outline 1 Introduction 2 Method Entropy analysis through scale space Bayesian filtering Chernoff Information and threshold estimation Bayesian scale-saliency filtering algorithm Bayesian scale-saliency filtering algorithm 3 Experiments Visual Geometry Group database 4 Conclusions P. Suau and F. Escolano Bayesian filter for the Kadir scale-saliency detector 2 / 21 IBPRIA 2007 Introduction Method Experiments Conclusions Outline 1 Introduction 2 Method Entropy analysis through scale space Bayesian filtering Chernoff Information and threshold estimation Bayesian scale-saliency filtering algorithm Bayesian scale-saliency filtering algorithm 3 Experiments Visual Geometry Group database 4 Conclusions P. Suau and F. Escolano Bayesian filter for the Kadir scale-saliency detector 3 / 21 IBPRIA 2007 Introduction Method Experiments Conclusions Local feature detectors Feature extraction is a basic step in many computer vision tasks Kadir and Brady scale-saliency Salient features over a narrow range of scales Computational bottleneck (all pixels, all scales) Applied to robot global localization → we need real time feature extraction P. Suau and F. Escolano Bayesian filter for the Kadir scale-saliency detector 4 / 21 IBPRIA 2007 Introduction Method Experiments Conclusions Salient features X HD(s, x) = − Pd,s,x log2Pd,s,x d∈D Kadir and Brady algorithm (2001): most salient features between scales smin and smax P.
  • Teaching Visual Recognition Systems

    Teaching Visual Recognition Systems

    Teaching visual recognition systems Kristen Grauman Department of Computer Science University of Texas at Austin Work with Sudheendra Vijayanarasimhan, Prateek Jain, Devi Parikh, Adriana Kovashka, and Jeff Donahue Visual categories Beyond instances, need to recognize and detect classes of visually and semantically related… Objects Scenes Activities Kristen Grauman, UT-Austin Learning-based methods Last ~10 years: impressive strides by learning appearance models (usually discriminative). Novel image Annotator Training images Car Non-car Kristen Grauman, UT-Austin Exuberance for image data (and their category labels) 14M images 1K+ labeled object categories [Deng et al. 2009-2012] ImageNet 80M images 53K noisily labeled object categories [Torralba et al. 2008] 80M Tiny Images 131K images 902 labeled scene categories 4K labeled object categories SUN Database [Xiao et al. 2010] Kristen Grauman, UT-Austin And yet… • More data ↔ more accurate visual models? • Which images should be labeled? X. Zhu, C. Vondrick, D. Ramanan and C. Fowlkes. Do We Need More Training Data or Better Models for Object Detection? BMVC 2012. Kristen Grauman, UT-Austin And yet… • More data ↔ more accurate visual models? X. Zhu, C. Vondrick, D. Ramanan and C. Fowlkes. Do We Need More Training Data or Better Models for Object Detection? BMVC 2012. Kristen Grauman, UT-Austin And yet… • More data ↔ more accurate visual models? • Which images should be labeled? • Are labels enough to teach visual concepts? “This image has a cow in it.” Human annotator [tiny image montage
  • Hough Transform 1 Hough Transform

    Hough Transform 1 Hough Transform

    Hough transform 1 Hough transform The Hough transform ( /ˈhʌf/) is a feature extraction technique used in image analysis, computer vision, and digital image processing.[1] The purpose of the technique is to find imperfect instances of objects within a certain class of shapes by a voting procedure. This voting procedure is carried out in a parameter space, from which object candidates are obtained as local maxima in a so-called accumulator space that is explicitly constructed by the algorithm for computing the Hough transform. The classical Hough transform was concerned with the identification of lines in the image, but later the Hough transform has been extended to identifying positions of arbitrary shapes, most commonly circles or ellipses. The Hough transform as it is universally used today was invented by Richard Duda and Peter Hart in 1972, who called it a "generalized Hough transform"[2] after the related 1962 patent of Paul Hough.[3] The transform was popularized in the computer vision community by Dana H. Ballard through a 1981 journal article titled "Generalizing the Hough transform to detect arbitrary shapes". Theory In automated analysis of digital images, a subproblem often arises of detecting simple shapes, such as straight lines, circles or ellipses. In many cases an edge detector can be used as a pre-processing stage to obtain image points or image pixels that are on the desired curve in the image space. Due to imperfections in either the image data or the edge detector, however, there may be missing points or pixels on the desired curves as well as spatial deviations between the ideal line/circle/ellipse and the noisy edge points as they are obtained from the edge detector.
  • M2P3: Multimodal Multi-Pedestrian Path Prediction by Self-Driving Cars with Egocentric Vision

    M2P3: Multimodal Multi-Pedestrian Path Prediction by Self-Driving Cars with Egocentric Vision

    M2P3: Multimodal Multi-Pedestrian Path Prediction by Self-Driving Cars With Egocentric Vision Atanas Poibrenski Matthias Klusch iMotion Germany GmbH German Research Center for Artificial Intelligence (DFKI) German Research Center for Artificial Intelligence (DFKI) Saarbrücken, Germany Saarbrücken, Germany [email protected] [email protected] Igor Vozniak Christian Müller German Research Center for Artificial Intelligence (DFKI) German Research Center for Artificial Intelligence (DFKI) Saarbrücken, Germany Saarbrücken, Germany [email protected] [email protected] ABSTRACT effective and efficient multi-pedestrian path prediction intraffic Accurate prediction of the future position of pedestrians in traffic scenes by AVs. In fact, there is a plethora of solution approaches scenarios is required for safe navigation of an autonomous vehicle for this problem [65] to be employed in advanced driver assistance but remains a challenge. This concerns, in particular, the effective systems of AVs. Currently, these systems enable an AV to detect if and efficient multimodal prediction of most likely trajectories of a pedestrian is actually in the direction of travel, warn the control tracked pedestrians from egocentric view of self-driving car. In this driver and even stop automatically. Other approaches would allow paper, we present a novel solution, named M2P3, which combines a ADAS to predict whether the pedestrian is going to step on the conditional variational autoencoder with recurrent neural network street, or not [46]. encoder-decoder architecture in order to predict a set of possible The multimodality of multi-pedestrian path prediction in ego-view future locations of each pedestrian in a traffic scene. The M2P3 is a challenge and hard to handle by many deep learning (DL) mod- system uses a sequence of RGB images delivered through an internal els for many-to-one mappings.
  • Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift

    Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift

    Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift Remi Tachet des Combes∗ Han Zhao∗ Microsoft Research Montreal D. E. Shaw & Co. Montreal, QC, Canada New York, NY, USA [email protected] [email protected] Yu-Xiang Wang Geoff Gordon UC Santa Barbara Microsoft Research Montreal Santa Barbara, CA, USA Montreal, QC, Canada [email protected] [email protected] Abstract Adversarial learning has demonstrated good performance in the unsupervised domain adaptation setting, by learning domain-invariant representations. However, recent work has shown limitations of this approach when label distributions differ between the source and target domains. In this paper, we propose a new assumption, generalized label shift (GLS), to improve robustness against mismatched label distributions. GLS states that, conditioned on the label, there exists a representation of the input that is invariant between the source and target domains. Under GLS, we provide theoretical guarantees on the transfer performance of any classifier. We also devise necessary and sufficient conditions for GLS to hold, by using an estimation of the relative class weights between domains and an appropriate reweighting of samples. Our weight estimation method could be straightforwardly and generically applied in existing domain adaptation (DA) algorithms that learn domain-invariant representations, with small computational overhead. In particular, we modify three DA algorithms, JAN, DANN and CDAN, and evaluate their performance on standard and artificial DA tasks. Our algorithms outperform the base versions, with vast improvements for large label distribution mismatches. Our code is available at https://tinyurl.com/y585xt6j. 1 Introduction arXiv:2003.04475v3 [cs.LG] 11 Dec 2020 In spite of impressive successes, most deep learning models [24] rely on huge amounts of labelled data and their features have proven brittle to distribution shifts [43, 59].
  • Histogram of Directions by the Structure Tensor

    Histogram of Directions by the Structure Tensor

    Histogram of Directions by the Structure Tensor Josef Bigun Stefan M. Karlsson Halmstad University Halmstad University IDE SE-30118 IDE SE-30118 Halmstad, Sweden Halmstad, Sweden [email protected] [email protected] ABSTRACT entity). Also, by using the approach of trying to reduce di- Many low-level features, as well as varying methods of ex- rectionality measures to the structure tensor, insights are to traction and interpretation rely on directionality analysis be gained. This is especially true for the study of the his- (for example the Hough transform, Gabor filters, SIFT de- togram of oriented gradient (HOGs) features (the descriptor scriptors and the structure tensor). The theory of the gra- of the SIFT algorithm[12]). We will present both how these dient based structure tensor (a.k.a. the second moment ma- are very similar to the structure tensor, but also detail how trix) is a very well suited theoretical platform in which to they differ, and in the process present a different algorithm analyze and explain the similarities and connections (indeed for computing them without binning. In this paper, we will often equivalence) of supposedly different methods and fea- limit ourselves to the study of 3 kinds of definitions of di- tures that deal with image directionality. Of special inter- rectionality, and their associated features: 1) the structure est to this study is the SIFT descriptors (histogram of ori- tensor, 2) HOGs , and 3) Gabor filters. The results of relat- ented gradients, HOGs). Our analysis of interrelationships ing the Gabor filters to the tensor have been studied earlier of prominent directionality analysis tools offers the possibil- [3], [9], and so for brevity, more attention will be given to ity of computation of HOGs without binning, in an algo- the HOGs.
  • Kernel Methods for Unsupervised Domain Adaptation

    Kernel Methods for Unsupervised Domain Adaptation

    Kernel Methods for Unsupervised Domain Adaptation by Boqing Gong A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (COMPUTER SCIENCE) August 2015 Copyright 2015 Boqing Gong Acknowledgements This thesis concludes a wonderful four-year journey at USC. I would like to take the chance to express my sincere gratitude to my amazing mentors and friends during my Ph.D. training. First and foremost I would like to thank my adviser, Prof. Fei Sha, without whom there would be no single page of this thesis. Fei is smart, knowledgeable, and inspiring. Being truly fortunate, I got an enormous amount of guidance and support from him, financially, academically, and emotionally. He consistently and persuasively conveyed the spirit of adventure in research and academia of which I appreciate very much and from which my interests in trying out the faculty life start. On one hand, Fei is tough and sets a high standard on my research at “home”— the TEDS lab he leads. On the other hand, Fei is enthusiastically supportive when I reach out to conferences and the job market. These combined make a wonderful mix. I cherish every mind-blowing discussion with him, which sometimes lasted for hours. I would like to thank our long-term collaborator, Prof. Kristen Grauman, whom I see as my other academic adviser. Like Fei, she has set such a great model for me to follow on the road of becoming a good researcher. She is a deep thinker, a fantastic writer, and a hardworking professor.