Distributed Computing Automated Resistor Classification Group Thesis Pascal Niklaus, Gian Ulli [email protected], [email protected] Distributed Computing Group Computer Engineering and Networks Laboratory ETH Zürich Supervisors: Tobias Langner, Jochen Seidel Prof. Dr. Roger Wattenhofer January 16, 2015 Abstract In this group thesis, we develop an algorithm that allows the automated classifi- cation of resistors using image recognition. The algorithm is implemented in an Android app and therefore usable on every Android smartphone. The problem is split into parts and the result is obtained step by step be- ginning with the localization of the resistor in the captured image and cropping the image such that only the resistor’s body remains. In the following analysis, the color rings are separated from the background and finally assigned a specific color. In the end, it is possible to determine the resistance of the resistor. Our evaluation of the algorithm shows that the resistor localization works with nearly every image and also under different conditions. The detection of the color rings on the body of the resistor also performs well on most of the images. However, our approach to the color assignment turned out to be unreliable. The task is quite difficult because every camera has different saturation and bright- ness settings. Also, external conditions like shadows and the background have a significant impact on the colors in an image. i Contents Abstracti 1 Introduction1 1.1 Motivation...............................1 1.2 Related Work.............................1 2 Preliminaries3 2.1 Resistor Decoding..........................3 2.2 Lab Color Space...........................4 2.3 Software and Hardware for the Implementation..........5 3 Problems and Methods6 3.1 Resistor Localization.........................6 3.1.1 Noise Filtering........................6 3.1.2 Hough Line Detection....................7 3.1.3 Statistical Analysis......................9 3.2 Color Ring Detection......................... 11 3.2.1 Edge Detection........................ 11 3.2.2 Penalty System........................ 13 3.3 Color Classification.......................... 14 4 Implementation 16 5 Evaluation and Discussion 19 5.1 Evaluation Results.......................... 19 5.1.1 Color Classification...................... 20 6 Future Work 23 Acknowledgements 24 ii Contents iii Bibliography 25 Chapter 1 Introduction 1.1 Motivation While working in the lab, one often faces the problem of having a bunch of different resistors which might be used in a circuit. Unfortunately, these parts are all too often neither sorted nor labeled according to their values. However, the resistance of a resistor is encoded by a color combination. The combination either consists of four or five color rings, which allows the determination of the resistor’s resistance. The number of possible colors is restricted and every resistor has in theory a unique color ring assignment. Therefore, we thought that this problem could be solved automatically using image recognition, which is more comfortable than using an ohmmeter or looking up the value in a table. In particular, it would be useful to have an app which does all that work for you. Since we are interested in image recognition and smartphone app development, this project is a good combination of both topics. This group project focuses on the development of an algorithm that can be used for the automated classification of resistors. The algorithm systematically analyzes an image, searches the resistors and then calculates the resistances ac- cording to their color code. Additionally, this algorithm is implemented in an Android app. In addition to the improvement of comfort, this app can also be useful for people who have difficulties to identify the small color rings or who suffer from color blindness. Another advantage is that no additional hardware is needed, since one can just measure resistors on the go with a smartphone. 1.2 Related Work There exist several previous approaches to this topic. Most of those implemen- tations are experimental. A notable project is described in [1] and was primarily thought to be an Android app. Later on, it was optimized for a specific webcam with manual focus and an additional light source with code running on a regular 1 1. Introduction 2 computer. The main reason for the platform change was the poor focus of the smartphone camera at low distances, which made it nearly impossible to detect any color rings. The algorithm searches the resistor as a line between its leads. From there, it locates the rings perpendicular to the alignment of the resistor’s body and finally determines their colors. Due to the fact that the camera is fixed and an additional light source is provided, the algorithm could be calibrated accurately. Therefore, it was possible to obtain reliable values. Another realization can be found in [2]. This app was developed for iOS, which has the advantage that there exist fewer devices and cameras, which simplifies the development. Further restrictions are made about the number of color rings and the orientation of the resistor. Namely, the algorithm requires the exact positioning of the resistor in the captured image, realized by a tiny box seen on the camera preview window. The user has to assure that the resistor is inside this box. Moreover, only resistors with four rings are allowed. As the hardware is basically restricted to one device, the colors can be determined pretty accurately by having calibrated values for each possible color. Due to incompatibilities with a new iOS version, further development was abandoned. Chapter 2 Preliminaries 2.1 Resistor Decoding As mentioned before, the resistance of a resistor is encoded by a color combina- tion. As there exists no standard that defines the direction the color combination has to be read, most combinations have two corresponding resistances. Our al- gorithm solves this inconsistency by returning two possible values if neither of those can be rejected due to further restrictions explained below. Color Ring 1 Ring 2 Ring 3 Ring 4 Ring 5 Ring 1 Ring 2 Ring 3 Ring 4 black 0 0 100 brown 1 1 1 101 1% red 2 2 2 102 2% orange 3 3 3 103 yellow 4 4 4 104 green 5 5 5 105 0:5% blue 6 6 6 106 0:25% violet 7 7 7 107 0:1% gray 8 8 8 0:05% white 9 9 9 gold 10−1 5% silver 10−2 10% Table 2.1: The resistance of a five ring resistor or a four ring resistor is based on the color rings. Within this thesis, we define a resistor as a set Rn of color rings where n denotes the total amount of rings (either four or five) and ci is the i-th color ring 3 2. Preliminaries 4 of the resistor. Rn = fc1; c2; : : : ; cng ; n = 4; 5 (2.1) Furthermore, val(ci) denotes the value and tol(ci) the tolerance of the color be- longing to color ring i according to Table 2.1. The corresponding resistance res(Rn) can then be calculated as follows: val(c3) res(R4) = 10 · val(c1) + val(c2) · 10 ± tol(c4) (2.2) val(c4) res(R5) = 100 · val(c1) + 10 · val(c2) + val(c3) · 10 ± tol(c5) (2.3) The full decoding scheme can be seen in Table 2.1. A blank cell means that this specific color is not possible for that ring. In consequence, there are some color combinations that yield an invalid resistance. If detected, the algorithm can reject such a combination. 2.2 Lab Color Space A normal image captured with a camera is stored in the RGB color space. There exist several alternative color spaces, like the Lab color space that has some advantages especially useful for our purposes. The Lab space consists of three channels. L measures the lightness, A the amount of green or red and B the amount of blue or yellow. Since green and red as well as blue and yellow are color-opponents, they cannot occur at the same time and each pair can therefore be described by one coordinate. In [3], one can find a comprehensive overview of the Lab space. One advantage of Lab is that all information about color is stored in only two channels. For us, this is helpful because the color is an important criteria for the way we localize the resistor in the image. The biggest benefit, however, is that the measured color differences in the Lab color space correspond to the ones perceived by our eyes. This helps us to find the color changes that the human eye would also notice. For example, we don’t weight aberrations in the background, like variations of blue, that much but rather color changes from ∗ blue to a different color like brown. To calculate color distances, the CIE ∆E94 color difference formula is used (see [3] for further information). This function compares a reference color (L1; a1; b1) to a color (L2; a2; b2). The metric is calculated as follows: s 2 2 2 ∗ ∆L ∆Cab ∆Hab ∆E94 = + + (2.4) kL · SL kC · SC kH · SH where ∆L = L1 − L2 2. Preliminaries 5 q q 2 2 2 2 ∆Cab = a1 + b1 − a2 + b2 q 2 2 2 ∆Hab = (a1 − a2) + (b1 − b2) − ∆Cab q q 2 2 2 2 SL = 1;SC = 1 + 0:045 · a1 + b1;SH = 1 + 0:015 · a1 + b1 kL = 1; kC = 1; kH = 1 It is important to note that this metric is not commutative, since some weight- ing factors only depend on the reference color. This means that the reference color is taken into account stronger than the color it is compared to. 2.3 Software and Hardware for the Implementation The algorithm is implemented in Java for the Android platform (API version ≥ 14).