Image Processing Processing Organizational Issues

Image Image Processing Processing Organizational issues

• What type of lecture? – 2 hours classroom lecture, 2 hours tutorials – Tutorial consisting of mostly programming assignments and some theoretical assignments – Introductory course in the fields of image processing, computer vision, and Image Processing pattern recognition – (Soft) requirement for starting a diploma thesis in this field

• Prerequisites Class 1 – Undergraduate mathematics Introduction – Some programming experience in C/C++ (for the programming assignments)

Image Image Processing Organizational issues Processing Other courses

• Grading policy • This winter term: – Oral or written exam (depends on the number of participants) – Pattern Matching in Computer Vision (2+2, called Mustererkennung) after the lecturing period – Regular attendance in the tutorials required to qualify for the exam • Next summer term (planned): – Computer Vision (2+2) • Tutorials start next week – Image Segmentation (2+2) or Pattern Recognition and Machine Learning (2+2) • These slides will be made available at http://www.inf.tu-dresden.de/content/institutes/ki/is/bv_ws0708.de.html

• They are password protected. Access via… – username: open – password: sesame

• Imaging (e.g. photography, ultrasound, x-ray, magnetic resonance)

• Image enhancement (e.g. Adobe Photoshop)

• Image and video compression

• Computer graphics (model Æ image)

• Computer vision (image Æ model) Author: Daniel Cremers • Digital image: regular grid of intensity values • Focus of this and the following courses: image enhancement and • Continuous function computer vision

Image Image Processing Imaging – ultrasound, MRI Processing Image enhancement - denoising

Ultrasound image of a human embryo Magnetic resonance image of a human head

Welk et al. 2005: variational deblurring

Image Image Processing Computer graphics – water simulation Processing Computer vision – 3D reconstruction

Kolev et al. 2007: multiview reconstruction

Guendelman et al. 2005: Simulating the flow of water with level sets

Brox et al. 2007: 3D human pose tracking

Wedel et al. 2007: obstacle detection and segmentation

Image Image Processing Computer vision – human tracking Processing Difference between image processing and computer vision

• Usually, the term “image processing” is used for the general subject of processing images with a computer

• Computer vision signifies the specific task to make the computer interpret the content of images

• This is a difficult and so far unsolved task

• Computer vision has similar motivations as artificial intelligence and cognitive neuroscience

Brox et al. 2007: 3D human pose tracking

© 2007 Thomas Brox 15 © 2007 Thomas Brox 16 Image Image Processing Why is computer vision difficult? Processing Images are only a structured grid of numbers • Vision is a natural and easy task for humans (and many animals) • What’s that?

• This is not for free: ~50% of the primate’s cortex deals with the processing of visual information (Felleman-van Essen 1991)

• Making a computer see like humans see means to solve a large part of the AI problem (this cannot be easy)

• What do you see in this image? Author: Daniel Cremers

• A lot of high level knowledge • This is how the computer sees Einstein and content information necessary • Demonstrates what our brain achieves at the unconscious level

Image Image Processing Importance of the spatial arrangement Processing Importance of image processing and computer vision

INT J COMPUT VISION 6.085 • Computer vision is a very young research field ACM T INFORM SYST 5.059 BIOINFORMATICS 4.894 MIS QUART 4.731 • Image content is defined by the spatial arrangement of intensities – Main computer vision conference (ICCV) founded in 1987 IEEE T PATTERN ANAL 4.306 ACM COMPUT SURV 4.13 ACM T GRAPHIC 4.081 – Main computer vision journal (IJCV) founded in 1988 J AM MED INFORM ASSN 3.979 IEEE T EVOLUT COMPUT 3.77 IEEE T MED IMAGING 3.757 NEUROINFORMATICS 3.541 • It is not sufficient to treat images as vectors and to analyze these vectors J CHEM INF MODEL 3.423 VLDB J 3.289 • Nonetheless, it belongs already to the most influential MED IMAGE ANAL 3.256 J ACM 2.917 J CRYPTOL 2.833 subfields of computer science; tendency: growing rapidly IEEE T IMAGE PROCESS 2.715 MACH LEARN 2.654 IEEE T NEURAL NETWOR 2.62 IEEE WIREL COMMUN 2.577 COGNITIVE BRAIN RES 2.568 IEEE T MOBILE COMPUT 2.55 CHEMOMETR INTELL LAB 2.45 • By 2006, the International Journal of Computer Vision IEEE INTELL SYST 2.413 HUM-COMPUT INTERACT 2.391 J MOL GRAPH MODEL 2.371 had the highest impact factor of all computer science J BIOMED INFORM 2.346 J COMPUT PHYS 2.328 DATA MIN KNOWL DISC 2.295 journals: 6.085 IEEE ACM T COMPUT BI 2.283 ARTIF INTELL 2.271 J MACH LEARN RES 2.255 NEURAL COMPUT 2.229 IEEE NETWORK 2.211 ACM T DATABASE SYST 2.143 COMPUT BIOL CHEM 2.135 • Impact factor in 2002: 2.03 IEEE T SOFTWARE ENG 2.132 INFORM MANAGE-AMSTER 2.119 QUANTUM INF COMPUT 2.105 J COMPUT AID MOL DES 2.089 IEEE T KNOWL DATA EN 2.063 IEEE PERVAS COMPUT 2.062 J COMPUT BIOL 2 • Computer vision applications not restricted by a small MATCH-COMMUN MATH CO 2 market but by the limited quality provided so far NEURAL NETWORKS 2 Zebra image Same image with a different Impact factors of computer science row length • More research Æ more products journals 2006

© 2007 Thomas Brox 19 © 2007 Thomas Brox 20 Image Image Processing Some image processing applications Processing Related sciences • Quality control, visual inspection • Information systems • Computer science • Physics – Document analysis – Audio processing –Optics • Security systems – Image Google – Pattern recognition – Computational physics – Fingerprint recognition – Optimization – Iris recognition • Earth surveillance – Machine learning • Electrical engineering – Face recognition – Weather forecasts – Data retrieval – Signal processing –Tracking – Google Earth – Computer graphics –Robotics • Neuroscience – Control theory • Medical systems • Driver assistance systems – Neurophysiology – Software engineering – Image enhancement – Lane control – Computational neuroscience – Data analysis – Collision avoidance – Psychophysics • Mathematics – Routine diagnostics – Autonomous driving – Numerics • Medicine – Statistics • Entertainment industry • Robotics – Linear algebra – 3D reconstruction – Autonomous robots (e.g. pathfinder) • Philosophy – Functional analysis – Motion capturing – Domestic robots – Graph theory – Augmented reality – Geometric algebra – Human-machine interaction • Artificial intelligence

Image Image Processing Conferences and Journals Processing Overview (planned) • Conferences – ICCV: International Conference on Computer Vision • Class 1: Introduction – ECCV: European Conference on Computer Vision – CVPR: Int. Conference on Computer Vision and Pattern Recognition • Class 2: Image acquisition and representation – NIPS: Neural Information Processing Systems • Class 3: Noise, point operations – DAGM: Tagung der Deutschen Arbeitsgemeinschaft für Mustererkennung • Class 4: Fourier transform – ICIP: International Conference on Image Processing – ICPR: International Conference on Pattern Recognition • Class 5: Linear filters – SSVM: Int. Conf. on Scale Space and Variational Methods • Class 6: Wavelets • Class 7: Morphological filters • Journals – International Journal of Computer Vision • Class 8: Nonlinear diffusion filters – IEEE Transactions on Pattern Analysis and Machine Intelligence • Class 9: Variational methods I – IEEE Transactions on Image Processing • Class 10: Variational methods II – Computer Vision and Image Understanding – Image and Vision Computing • Class 11: Deconvolution – Journal of Mathematical Imaging and Vision • Class 12: Texture analysis and filtering – Journal of Visual Communication and Image Representation – Realtime Imaging • Class 13: Color spaces – Pattern Recognition • Continued next term with the course Computer Vision

• Image processing, and particularly computer vision, are important • R. C. Gonzalez, R. E. Woods: Digital Image Processing, Addison-Wesley, Reading, 2nd Edition, 2002. research fields • J. Bigün: Vision with Direction, Springer, Berlin, 2006. • Their importance grows rapidly with the number of successful applications • K. D. Tönnies: Grundlagen der Bildverarbeitung (in German), Pearson Studium, Munich, 2005. • Image processing makes use of techniques from various other sciences • CV Online: Online compendium on numerous image processing and computer vision topics, • Especially mathematics provides many helpful tools http://homepages.inf.ed.ac.uk/rbf/CVonline/