Color and Geometrical Structure in Images Applications in microscopy Jan-Mark Geusebroek This book was typeset by the author using LATEX 2". Cover: Victory Boogie Woogie, by Piet Mondriaan, 1942{1944, oil-painting with pieces of plastic and paper. Reproduction and permission for printing kindly pro- vided by Gemeentemuseum Den Haag. Copyright c 2000 by Jan-Mark Geusebroek. ° All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission from the author ([email protected]). ISBN 90-5776-057-6 Color and Geometrical Structure in Images Applications in microscopy ACADEMISCH PROEFSCHRIFT ter verkrijging van de graad van doctor aan de Universiteit van Amsterdam, op gezag van de Rector Magni¯cus prof. dr J. J. M. Franse ten overstaan van een door het College voor Promoties ingestelde commissie, in het openbaar te verdedigen in de Aula der Universiteit op donderdag 23 november 2000 te 12.00 uur door Jan-Mark Geusebroek geboren te Amsterdam Promotiecommissie: Prof. dr ir A. W. M. Smeulders Dr H. Geerts Prof. dr J. J. Koenderink Prof. dr G. D. Finlayson Prof. dr ir L. van Vliet Prof. dr ir C. A. Grimbergen Prof. dr ir F. C. A. Groen Prof. dr P. van Emde Boas Faculteit: Natuurwetenschappen, Wiskunde & Informatica Kruislaan 403 1098 SJ Amsterdam Nederland The investigations described in this thesis were carried out at the Janssen Research Foundation, Beerse, Belgium. The study was supported by the Janssen Research Foundation. Advanced School for Computing and Imaging The work described in this thesis has been carried out at the Intelligent Sensory Information Systems group. This work was carried out in graduate school ASCI. ASCI dissertation series number 54. Contents 1 Introduction 1 1.1 Part I: Color . 2 1.2 Part II: Geometrical Structure . 4 2 Color and Scale 13 2.1 Color and Observation Scale . 14 2.1.1 The Spectral Structure of Color . 14 2.1.2 The Spatial Structure of Color . 16 2.2 Colorimetric Analysis of the Gaussian Color Model . 17 2.3 Conclusion . 19 3 A Physical Basis for Color Constancy 23 3.1 Color Image Formation Model . 25 3.1.1 Color Formation for Reflection of Light . 25 3.1.2 Color Formation for Transmission of Light . 27 3.1.3 Special Cases . 29 3.2 Illumination Invariant Properties of Object Reflectance or Transmittance 30 3.3 Experiments . 32 3.3.1 Overview . 32 3.3.2 Small-Band Experiment . 35 3.3.3 Broad-Band Experiment . 36 3.3.4 Colorimetric Experiment . 36 3.4 Discussion . 38 4 Measurement of Color Invariants 43 4.1 Color Image Formation Model . 45 4.2 Determination of Color Invariants . 46 4.2.1 Invariants for White but Uneven Illumination . 46 4.2.2 Invariants for White but Uneven Illumination and Matte, Dull Surfaces . 48 i ii CONTENTS 4.2.3 Invariants for White, Uniform Illumination and Matte, Dull Surfaces . 49 4.2.4 Invariants for Colored but Uneven Illumination . 51 4.2.5 Invariants for a Uniform Object . 52 4.2.6 Summary of Color Invariants . 53 4.2.7 Geometrical Color Invariants in Two Dimensions . 54 4.3 Measurement of Color Invariants . 55 4.3.1 Measurement of Geometrical Color Invariants . 56 4.3.2 Discriminative Power for RGB Recording . 61 4.3.3 Evaluation of Scene Geometry Invariance . 63 4.3.4 Localization Accuracy for the Geometrical Color Invariants . 64 4.4 Conclusion . 66 5 Robust Autofocusing in Microscopy 73 5.1 Material and Methods . 74 5.1.1 The Focus Score . 74 5.1.2 Measurement of the Focus Curve . 75 5.1.3 Sampling the Focus Curve . 77 5.1.4 Large, Flat Preparations . 77 5.1.5 Preparation and Image Acquisition . 78 5.1.6 Evaluation of Performance for High NA . 81 5.2 Results . 82 5.2.1 Autofocus Performance Evaluation . 82 5.2.2 Evaluation of Performance for High NA . 83 5.2.3 Comparison of Performance with Small Derivative Filters . 85 5.2.4 General Observations . 85 5.3 Discussion . 86 6 Segmentation of Tissue Architecture by Distance Graph Matching 91 6.1 Materials and Methods . 93 6.1.1 Hippocampal Tissue Preparation . 93 6.1.2 Image Acquisition and Software . 93 6.1.3 K-Nearest Neighbor Graph . 94 6.1.4 Distance Graph Matching . 94 6.1.5 Distance Graph Comparison . 96 6.1.6 Cost Functions . 97 6.1.7 Evaluation of Robustness on Simulated Point Patterns . 98 6.1.8 Algorithm Robustness Evaluation . 99 6.1.9 Robustness for Scale Measure . 100 6.1.10 Cell Detection . 100 6.1.11 Hippocampal CA Region Segmentation . 100 CONTENTS iii 6.2 Results . 101 6.2.1 Algorithm robustness evaluation . 101 6.2.2 Robustness for Scale Measure . 105 6.2.3 Hippocampal CA Region Segmentation . 105 6.3 Discussion . 107 6.4 Appendix: Dynamic Programming Solution for String Matching . 109 7 A Minimum Cost Approach for Segmenting Networks of Lines 115 7.1 Network Extraction Algorithm . 116 7.1.1 Vertex Detection . 116 7.1.2 Line Point Detection . 116 7.1.3 Line Tracing . 118 7.1.4 Graph Extraction . 119 7.1.5 Edge Saliency and Basin Coverage . 120 7.1.6 Thresholding the Saliency Hierarchy . 121 7.1.7 Overview . 122 7.1.8 Error Analysis . 122 7.2 Illustrations . 125 7.2.1 Heart Tissue Segmentation . 125 7.2.2 Neurite Tracing . 125 7.2.3 Crack Detection . 125 7.2.4 Directional Line Detection . 126 7.3 Conclusion . 127 8 Discussion 137 8.1 Color . 137 8.2 Geometrical Structure . 139 8.3 General Conclusion . 140 Samenvatting 143 Chapter 1 Introduction When looking at Victory Boogie Woogie, by the Dutch painter Piet Mondrian, the yellow blocks appear jumpy and unstable, as if they move [33]. As the painting hangs ¯rmly ¯xed to the wall, the visual e®ect results from within the brain as it processes the incoming visual information. In fact, a visual scene which enters the brain fed into three subsystems [24, 34]. One subsystem segments the scene in parts by the apparent color contrast. The subsystem gives the ability to see the various colored patches as di®erent entities. A second subsystem provides us with the color of the parts. The subsystem is used for identifying the patches based on their color. The third subsystem localizes objects in the world. It tells us where the patches are in the scene. In contrast, the latter system is color blind, judging the scene on intensity variations only. Cooperation between the ¯rst subsystem, segmenting the di®erent colored parts, and the latter subsystem, localizing the di®erent patches, results in ambiguity when the intensity of neighboring color patches is similar. The phenomenon is in e®ect in Victory Boogie Woogie by the yellow stripes on a white background, as described by Livingstone [33]. Apart from the color appearance of the blocks, Mondrian arranged blocks to form a pattern of perpendicular lines. The visual arrangement is sifted out by the third, monochromatic subsystem which extracts the spatial organization of the scene. The lines are e®ectuated by an intensity contrast with the background. The yellow stripes have no such contrast, but lines appear as the gaps are supplemented by the brain. In Victory Boogie Woogie, Mondrian combined local color contrast and the geometrical arrangement of details to stimulate a visual sensation in the brain. Like Victory Boogie Woogie, this thesis deals with both color and spatial structure. Part I describes the spatial interaction between colors. Color is discussed in its phys- ical environment of light. Consequently, the physics of light reflection are included in the human subsystem dealing with shape extraction. Part II describes the quan- ti¯cation of geometrical structure speci¯cally applied to microscopy, although some 1 2 Introduction of the concepts may have a broader application span. Tissue at the microscopical level often exhibits a regular pattern. Automatic extraction of such arrangements is considered, aiming at drug screening for pharmaceutical research. The two parts are mostly separated from one another, as is the case for perception. Using the parts in future research in conjunction may have synergy on color image processing. 1.1 Part I: Color Color seems to be an unalienable property of objects. It is the orange that has that color. However, the heart of the matter is quite di®erent. Human perception actively assigns colors to an observed scene. There is a discrepancy between the physics of light, and color as signi¯ed by the brain. One undeniable fact is that color perception is bootstrapped by a physical cause: it results from light falling onto the eye. Objects in the world respond to daylight by reflecting di®erently part of the incoming light spectrum. The speci¯c component of reflection mainly instantiates the color appearance of the object. Another fact is that color perception results from experience. We assign the color of an orange that label as we have learned by experience, as we are capable to do so by the biological mechanism. Experience has led to the denominations of signs to colors. It would have given language no advantage to label colors when we could not compare them with memory. A last contribution to color as we know it is evolution that has shaped the actual mechanism of color vision. Evolution, such that a species adapts to its environment, has driven the use of color by perception. Color is one of the main cues for segmenting objects in a scene.