Author(S) Repeat Author and Affiliation Boxes As Needed

Send your completed paper to Sandy Rutter at [email protected] by 13 April 2007 to be included in the ASABE Online Technical Library. If you can't use this Word document and you'd like a PDF cover sheet please contact Sandy. Please have Word's AutoFormat features turned OFF and do not include live hyperlinks. Your paper should be no longer than 12 pages. For general information on writing style, please see http://www.asabe.org/pubs/authguide.html. This page is for online indexing purposes and should not be included in your printed version.

Author(s)

First Name Middle Name Surname Role Email Junxiong Zhang no zhang_junxio [email protected]

Affiliation

Organization Address Country China Agricultural University P.O. Box 339#, College of China Engineering, China Agricultural University, Beijing 100083, China

Author(s) – repeat Author and Affiliation boxes as needed--

First Name Middle Name Surname Role Email Yi Xun no xun_yi@126. com

Affiliation

Organization Address Country China Agricultural University P.O. Box 339#, College of China Engineering, China Agricultural University, Beijing 100083, China

First Name Middle Name Surname Role Email Wei Li Yes [email protected] du.cn

Affiliation

Organization Address Country China Agricultural University P.O. Box 49#, College of China Engineering, China Agricultural University, Beijing 100083, China Phone: +86 10 62736527

Publication Information

Pub ID Pub Date 073090 2007 ASABE Annual Meeting Paper

An ASABE Meeting Presentation

Paper Number: 073090

Identification and Detection for Surface Cracks of Corn Kernel Based on Computer Vision

Junxiong Zhang College of Engineering, China Agricultural University, Beijing 100083, China. [email protected] Yi Xun College of Engineering, China Agricultural University, Beijing 100083, China. [email protected] Wei Li

Written for presentation at the 2007 ASABE Annual International Meeting Sponsored by ASABE Minneapolis Convention Center Minneapolis, Minnesota 17 - 20 June 2007 Abstract. Surface cracks detection of corn kernel has been studied based on BP neural network segmentation of surface color characteristics and morphology algorithm. The seeds of NongDa-4967 and NongDa-3138 (two novel varieties of corn developed by China Agricultural University) were taken as research objectives. Firstly, binary image including the information of cracks, boundary and non-cracks were obtained by horizontal and vertical Sobel operators. Subsequently, by analyzing the color characteristics, a BP neural network model with three layers was built, R, G, B color components were the inputs of the network, and the outputs were background, corn kernel tip cap and other parts. The tip point of the kernel could be identified from the kernel tip cap. And a majority of non-cracks information was eliminated by subtracting a circular area with the tip point as the center. Finally, according to the crack lengths and positions, the crack was extracted, and the lengths were calculated. An experiment has been carried out with 80 kernels with cracks and 80 kernels without cracks selected from NongDa-4967 and NongDa-3138 respectively. The identification results showed that the surface cracks detection of corn kernels could be realized and the detecting accuracy was 92.5% and 88.8%. Keywords. corn kernel, surface cracks, computer vision, BP neural network.

The authors are solely responsible for the content of this technical presentation. The technical presentation does not necessarily reflect the official position of the American Society of Agricultural and Biological Engineers (ASABE), and its printing and distribution does not constitute an endorsement of views which may be expressed. Technical presentations are not subject to the formal peer review process by ASABE editorial committees; therefore, they are not to be presented as refereed publications. Citation of this work should state that it is from an ASABE meeting paper. EXAMPLE: Author's Last Name, Initials. 2007. Title of Presentation. ASABE Paper No. 07xxxx. St. Joseph, Mich.: ASABE. For information about securing permission to reprint or reproduce a technical presentation, please contact ASABE at [email protected] or 269-429-0300 (2950 Niles Road, St. Joseph, MI 49085-9659 USA). Introduction Cracks of corn kernel can lead to the decline of corn starch content, and the corn with cracks can not be made into samp and other foods. Furthermore, during the process of storage, the kernel with cracks has strong ability of moisture absorbing, which tends to make the temperature of corn higher and occurrence of fungal diseases. For corn seeds, the cracks will influence the rate of germination, and even as the forage material, the cracking rate should be restricted strictly. The traditional methods of corn crack detection include illumination, radiation and others. Most of these methods have many disadvantages, such as the complicated detecting process, time- consuming, destructive, which are unfit for all samples detecting. In recent years, the digital image processing technology has been used widely for varied products quality detecting and controlling. And it also has been an effective method for object surface crack detection in many fields, such as crack detection of concrete and pipeline surface (Iyer, 2005; Sinha, 2006; Ammouche, 2001).Computer vision technology can make products non-destructive detection realized, and it has the virtues of non-touch, fast and flexibleness, which provides another new detecting approach for cracks detection of rice and corn kernels. Zhu (1998) observed corn kernel stress crack by using scanning electron microscope, analyzed crack expanding and the affect on the endosperm structure. Gunasekaran (1987) investigated the identification method of corn seed interior crack through corn kernel image collected with commercial vision system. However, these detection methods demand skilled techniques and costly equipments, which are difficult to be applied in practice. Corn kernel surface crack detection using image identification was addressed in this paper. Cracks could be extracted and measured automatically based on images edge detection, segmentation by BP neural network model of color characteristics and morphologic operation. This method has been proved to be practical in corn seeds processing.

Materials and Methods

Materials NongDa-4967 and NongDa-3138 are two novel varieties of corn seed developed by China Agricultural University. 80 kernels with different cracks and 80 kernels without cracks selected from each variety are taken as research objectives. Figure 1 is the profile of corn kernel. O is the centroid, T is the tip point, TP is the long axis through the centroid O. Define the front side and back side of the kernel as shown in Figure 1. P Endosperm

Pericarp

M O N Germ

Tip Cap T Front Back

Figure 1. Profile of corn kernel

2 Image Acquisition System Figure 2 is the sketch of corn kernel image acquisition system, which is composed of camera, image grabbing card, computer, illumination room, light source, and carrier. A BASLER A602fc color CCD camera and a Matrox Meteor-II/1394 image grabbing card are used to capture images. The illumination room with circular profile has sub-white interior wall, which has perfect diffuse reflection effect. The black rubber is used as the background of the carrier which corn kernel is put on. After the acquisition system has been calibrated, set spatial resolution as 0.075mm/pixel to collect image.

1 2 3 4 5 6 7

Figure 2. Images acquisition system of corn kernel. 1. CCD camera, 2. grabbing card, 3. computer, 4. acquisition box, 5. illumination room, 6. lighting, 7. carrier.

Identification Algorithm of Corn Kernel Surface Crack

Image Edge Detection According to statistic analysis, surface cracks of corn kernel mostly appear on the back of the kernel, and rarely appear in the tip cap. As a result, when collecting images, put the germ of corn kernel on the carrier downward and keep them non-contact each other. Figure 3(a) is the image of single corn kernel. From figure 3(a), the gray value difference between corn kernel and background color is obvious in R channel. The binarization target of the whole corn kernel is obtained by thresholding in R channel (figure 3(b)). The centroid coordinates (xo , y o ) of target can be calculated by the formulas: 1 1 xo=邋 xk, y o = y k (1) K(xk , y k )挝 A K ( x k , y k ) A where A is the pixels set belonging to the kernel object, K is the total number of pixels, which denotes the area of the object. The gray value of crack of corn kernel in R channel changes greatly. The cracks can be enhanced by edge detection, and numerous operators are introduced to detect edges with strong direction randomicity (Bhand, 2005; Lin, 2006, giakoumis, 2006). Horizontal and vertical Sobel operators are used to edge detection based on R channel in this paper. Figure 3(c) is the result of the binary edges.

(a) (b) (c) Figure 3. Image of corn kernel. (a) primary image, (b) binary image of corn kernel, (c) edges of corn kernel.

Non-crack Elimination of Kernel Tip Cap In figure 3(c), large quantities of non-crack targets exist in the corn kernel tip cap, which will influence crack identification and extraction and need to be eliminated. The method is: make the kernel tip point as the center, subtract the circular area with the radius r , namely, the pixel within the circular area are regarded as background.

Determination of the Tip Point Ning (2004) found the position of tip point through curvature change on the basis of the specialty that corn kernel tip is sharper than other parts. However, this algorithm can not identify the tip point effectively for some kernels with sharp edge or unobvious tip characteristics. Quan (2006) used wavelet transform to seek the tip point according to the distribution of the distance from kernel boundary points to the centroid, but it is difficult to calculate. Basing on corn kernel color characteristics, the neural network is used to segment the images and extract the region of kernel tip cap. And a method determining the corn kernel tip position is brought forward through centroid coordinates. In figure 3(a), the image can be divided into three parts according to color: background, kernel tip cap and other parts. BP neural network with only one hidden layer is utilized to segment images. The input layer has three nodes, which are R、G、B color components, three regions are the output nodes. Choosing flexible BP algorithm to train the network and using Nguyen- Widrow to generate the initial thresholds and biases for each layer. The transform function of hidden layer and output layer are sigmoid function. The hidden layer nodes number can be determined by the experiential formula (2) and it can be adjusted with segmentation effect of image and the network convergence speed.

1 h= ( n� m )2 a , a [0,10] (2) where h , n and m are the hidden, input and output layer nodes respectively. Set the network train goal 0.01, the learn rate 0.01and the epochs 10000. For assuring the network’s reliability and generalization, make N≥ W /e , N is the sample number for training, W is the number of thresholds need to be adjusted and e is the train goal. Selecting 20 corn kernel images from each kind of corn seed randomly and choosing 100 pixel points as train samples by hand from three regions of each image above-mentioned. Making use of the neural network has been trained to segment images and figure 4(a) is the three different regions.

4 The largest connected component of the tip cap region is picked up through region labeling, and assume its pixels set is At . According to figure 1, making the kernel tip point T as the point which has the longest distance from centroid O in At , the calculating formula is:

TO= max{dk }, ( x k , y k ) A t (3) where TO is the distance from the kernel tip point T to the centroid O， dk is the distance from the point (xk , y k ) in At to the centroid O.

(a) (b) (c) Figure 4. Removing the tip cap of the corn kernel. (a) result of segmentation, (b) region of the tip cap, (c) edges after removing the tip cap

Determination of Radius r

Position the centroid Ot of the region in figure 4(b) using the formula similar to (1), and extract the contour of the region. Calculating the distance rk from the points of contour to the tip points T, the radius r is determined by the formula as follows: 1 r= rk, ( r k > TO t ) (4) K where TOt is the distance from the tip point T to the centroid Ot , K is the number of pixels which rk is larger than TOB . Make the kernel edges image in figure 3(c) subtract the circular area with the tip cap point T as the center, r as the radius. Figure 4 (c) is the image acquired which has been removed a great deal of non-crack information of the tip cap.

Removing the Kernel Contour The contour of the kernel should be removed for extracting the crack. Firstly, image in figure 4(c) is thinned to get the single pixel thick edge (the result is shown in figure 5(a)). Then use a 3×3 template to dilate the contour of the kernel object in figure 3(b) (the result is shown in figure 5(b)). After substraction of the results obtained by the two steps above, the result is displayed in figure 5(c). Most contour pixels have been removed.

(a) (b) (c) (d)

5 Figure 5. Removing contour of corn kernel. (a) thinned edge of figure 4(c), (b) dilated contour of corn kernel, (c) result of (a)-(b), (d) extracted crack

Extraction of Cracks According to the Chinese national standard GB/T16714-1996, only the length of the crack is longer than the half length of kernel, it would be regarded as crack. Combining with image resolution, define the connected component which satisfy with the two rules below as crack:(1) the pixel number of connected component is larger than 20; (2) the average distance L from all 3 points of the connected component to the tip point T is larger than TO . 2

Label every connect component of figure 5(c), and record the number of pixels Ni . If Ni < 20 , then regard the component is not crack, and make the pixel as background. If Ni≥20 , calculate 3 the average distance L from all points of connected component to the tip point. If L< TO , it 2 shows the component is near the tip of kernel and is not regarded as crack. Then it should be removed. Figure 5(d) is the crack extracted. Calculating the whole length L of crack target obtained, and the relative length is defined as:

L c = (5) l where l is the length of the kernel long axis. Larger is the value of c , more seriously is the seed surface crack.

Results and Discussion According to the method mentioned above, a detecting experiment has been made with 80 kernels with cracks and 80 kernels without cracks selected from NongDa-4967 and NongDa- 3138 respectively. After identification, the number of those kernels which regarded as with cracks was recorded. And the identification accuracy and error was shown in table 1. Table 1. Surface cracks detecting results. Variety Kernels with cracks Kernels without cracks Number of Identification Accuracy Number of Identification Error (%) sample result (%) sample result NongDa-4967 80 74 92.5 80 5 6.3 NongDa-3138 80 71 88.8 80 3 3.4 In table 1, the identification accuracy of NongDa-4967 reaches 92.5% and the error was 6.3%; the identification accuracy of NongDa-3138 reaches 88.8% and the error was 3.4%. The main factor of influencing identification accuracy was that a few kernel cracks appear on the other surfaces. And owing to the existence of misshapen kernels, feculence of the surface, etc., the identification error came into being.

Conclusion A BP neural network model was constructed to segment the image into background, kernel tip cap and other parts based on color characteristics of kernel. The method of removing the tip area was used to eliminate the influence of non-crack in the tip cap. In other areas of kernel,

6 edge detection and morphological analysis were brought to extract the crack object effectively, and crack absolute and relative lengths were calculated. The identification experiment of corn kernel with crack and without crack chosen from NongDa-4967 and NongDa-3138 has been carried out, and the accuracy was 92.5% and 88.8% respectively. The results showed the method is significant in theory and practice for corn kernel crack detection.

Acknowledgements This study was supported by National Natural Science Foundation of China, project No. 30471011, and by the Research Fund for the Doctoral Program of Higher Education, project No. 20050019005.

References Ammouche, A., J. Riss, D. Breysse, et al.. 2001. Image analysis for the automated study of microcracks in concret. Cement and Concrete Composites, 23:267-278. Bhandarkar, S. M., X. Luo, R. Daniels, et al.. 2005. Detection of Cracks in Computer Tomography Images of Logs. Pattern Recognition Letters, 26(14): 2282-2294. Giakoumis, I., N. Nikolaidis, I. Pitas. 2006. Digital image processing techniques for the detection and removal of cracks in digitized paintings. Transactions on Image Processing, 15(1): 178-188. Gunasekaran, S., M. Coopert, A. G. Berlage, et al.. 1987 Image Processing for Stress Cracks in Corn Kernels. Trans. ASAE, 28(1): 266-271. Iyer, S., S. K. Sinha. A robust approach for automatic detection and segmentation of cracks in underground pipeline images. 2005. Image and Vision Computing, 23(10): 921-933. Lin, Y. CH., Y. P. Cui, Y. G. Huang. 2006. Study on edge detection and target recognition in complex background. Optics and Precision Engineering, 14(3): 509-514 (in Chinese). Ning, J. F., D. J. He, SH. Q. Yang. 2004. Identification of tip cap and germ surface of corn kernel using computer vision. Transactions of the Chinese society of Agricultural Engineering, 20(3): 117-119 (in Chinese). Quan, L. ZH., X. Y. Ma. 2006. Adjusting the Shape of Corn Kernels Based on Wavelet Analysis. Journal of Agricultural Mechanization Research, 2: 154-156 (in Chinese) Sinha, S. K., P. W. Fieguth. 2006. Automated detection of cracks in buried concrete pipe images. Automation in Construction, 15(1): 58-72. Wan, Y. N., C. M. Lin, J. F. Chou. 2002. Rice quality classification using an automatic grain quality inspect ion system. Trans. ASAE, 45(2): 379-387， Zheng,H. D., M. H. Liu, Y. H. Wu, et al.. 2006. Rice fissure detection using computer vision. Transactions of the Chinese Society of Agricultural Engineering, 22(7):129-133 Zhu, W. X., CH. W. Cao. 1998. Microscopic Structure Analysis on Stress Cracks in Corn [J]. Transactions of the Chinese of Agricultural Engineering, 14(2):198-202 (in Chinese)