Reducing Workload by Defective Product Classification

Author Date Published Dr. Nguyen Ngoc Tam February 7th, 2020 Content Abstract 3 Introduction 4 Background 5 Solution 7 Results 10 Conclusion 11 How can FPT help? 13 About the author 13

Reducing Workload by Defective Product Classification 2 AbstractPlaying the roles of eyes and ears in the manufacturing process, /Machine Learning (AI/ML) is contributing to addressing the growing challenges of product quality control. According to the corporate insurance carrier, AGCS, “defective products not only pose a serious safety risk to the public but can also cause significant financial and reputational damage to the companies concerned. Defective product incidents have caused insured losses in excess of $2B over the past five years, making them the largest generator of liability losses.” With over 50% of manufacturers planning to increase Artificial Intelligence/ Machine Learning spending in the coming years, according to Forbes Insights research, the industry is leaving behind its stagnant reputation to dive into automation.

Reducing Workload by Defective Product Classification 3 IntroductionA very common challenge of ML application in manufacturing is data acquisition and data processing. While the accuracy of data plays a strong role in the performance of ML algorithms, the challenge is to go through massive data sets and label each unit with a high level of accuracy. This paper sought to solve the problem of using AI/ML in the process of defective product classification by using Autoencoder model and One-class classification algorithm to classify normal and abnormal ones. The solution does not only help to extract important features, thereby precisely detecting abnormalities, but also reduces the amount of time spent on these tasks. Experiments on client’s data set showed a reduction of approximately 35% on user workload, allowing manufacturers to achieve higher productivity and lower burden on quality control workers.

Reducing Workload by Defective Product Classification 4 BackgroundProduct quality control plays an important role in maintaining and strengthening the link between manufacturers and final customers. Statistics show that 45% of companies using smart manufacturing technologies have experienced increased customer satisfaction. The visual quality impression of a product has a strong influence on the decision of whether a product is purchased or not. Traditionally, the verification process, when manually handled by human eyes, is time-consuming and takes Usually, a lot of effort. Today, the automated verification system using artificial intelligence is becoming more and more defects occur popular among smart technology manufacturers. very rarely, A common approach using artificial intelligence for the verification process is to use supervised learning so experts model. However, the challenge of this approach is need to spend in the training process of faulty sample collection. Usually, defects occur very rarely, so experts need extreme labor to spend extreme labor effort to label sample into a unit level. To overcome this challenge, this paper effort to label proposes an unsupervised learning approach for sample into a defect detection, which can save much more human labor effort during the training process. unit level. This unsupervised learning model uses Autoencoder network, which will reconstruct the normal input image based on a sample of normal data, and if it meets the abnormal data, it will not reconstruct defect areas. This process will leave images after the reconstruction process with strange bright spots on the CT image. By subtracting the corresponding query region, a pixel-wise anomaly score is obtained, which is then used to detect defects, therefore experts can save time going through all the normal sample. This approach however, still has some limitations:

In some cases, anomalies are small bright dots which hardly recognizable even by human eyes. There is also no apparent difference between the abnormal image and the normal image. Patterns of data set vary, which causes difficulty for the Autoencoder network to learn the main patterns. To overcome the above limitations, this paper proposes a combination model in which the outcome of Autoencoder network will be the input of One-class model to identify objects in a specific concept. This combination approach will be able to reduce the number of images that need to be inspected hence save more human workload and make it easier for experts to detect defect sample.

Reducing Workload by Defective Product Classification 5 This combination approach will be able to reduce the number of images that need to be inspected, and hence save more human workload as well as make it easier for experts to detect defect sample.

Reducing Workload by Defective Product Classification 6 Solution The solution is the combination of an Autoencoder network and a One-class model in training models. The idea behind this approach is like how we break an object into multiple pieces and then reconstruct the object in a predefined way, if the object is not the same then it is defected. Fig 1 shows the overall architecture of this model in the training and testing phases, which includes:

Training Phase This phase will help AI/ML to learn how to classify normal and abnormal data. Normal process in supervised model would require manual effort to label the data to help improve the model. With the unsupervised approach, U-net will improve the model overtime.

Testing Phase In this phase, the learned classifiers① model from training phase, which are the output of the One-class classification with Local outlier factor algorithm, will be used to discriminate between normal images and abnormal images. Overtime when the model is improved through the 2 phases, the accuracy output of image processing will also improve. In this paper, we run the testing phase on the real data set of 25,088 images from our client.

②Reducing Workload by Defective Product Classification 7

Loss SSIM

U-net Training AutoEncoder Encode Decode

Same batch Shared data weights

Training One Class Encode Encoder Model

1×256 Compactness Loss Hyper Sphere (boundary)

Hyper Sphere (boundary)

Normal Images

Unet - Encoder Normal One Class Encode Model Abnormal

Abnormal Images

Figure 1: Overall architecture of the proposed model

The training is conducted separately by using U-net and One-class model. We illustrate the procedures as follows:

Step 1 Step 2 Step 3

For each batch sample, Encode block was trained Use the Encode block training U-net with SSIM with compactness loss. that was trained in Step loss to update weight Training Autoencoder 1 as a pre-train model for for the Autoencoder. with SSIM loss function training the One-class SSIM loss function plays and encoder with model to identify the the role to improve the Compactness loss is normal objects out of prediction accuracy of carried out in parallel. the data set. the model. The lower the This step aims to drag weight loss, the better the characteristics of the accuracy the model will image in the same class achieve. as close as possible.

Reducing Workload by Defective Product Classification 8 An autoencoder is a neural network that consists of Autoencoders 3 main parts: The encoder, the latent representation with U-net and the decoder. It learns to reconstruct an input image x⋹Rk×h×w to its output. Therefore, Autoencoders are unsupervised learning models (do not require labelled inputs to enable learning). An Autoencoder consists of an encoder function E: Rk×h×w → Rd and a decoder function D: Rd → Rk×h×w, where d denotes the dimensionality of the latent space and k,h,w denote the number of channels, height, and width of the input image, respectively. In our experiments, the functions E and D are parameterized by U-net and SSIM loss function.

The other evaluation metrics are all based on per- pixel that assume unrealistic independence between SSIM loss neighbouring pixels. Therefore, they fail to detect function structural differences between the inputs and their reconstructions. By adapting the loss and evaluation functions to capture local inter-dependencies between image, we specifically motivate the use of the structural similarity metric SSIM (x,y) as both the loss and evaluation metric for Autoencoder. To evaluate local dependence between image regions, SSIM (x,y) is used as a loss and evaluation metric for Autoencoder.

All data used in training are the vector outputs of Compactness loss the encoder network which belong to the same class. Therefore, they all share the same class label. Compactness loss calculates the average similarity between the constituent samples of a given batch. We use this loss function to drag the characteristics of the image in the same class as close as possible.

Reducing Workload by Defective Product Classification 9 Results We define the most important class which must predict exactly is Positive (P-anomaly), and Negative (N-normal). Values that are correctly classified by the model are labelled true (T), while incorrect predictions are false (F). Our focus is to make sure that an abnormal image was not predicted as normal. Therefore, we define the ratio betweenTN (True Negative) and the total normal images as the user workload reduction. Instead of checking all the images, our method proposes people to only check images which are considered abnormal by model.

TN workload reduction = total of normal images testing

During the testing phase, a data set includes 25088 normal images and 92 abnormal images have been used. The model predicts exactly 8724 normal images out of total 25088 normal images, leaving 16,364 normal images that require manual verification. The manual workload reduction in the above test is approximated 35%:

8724 workload reduction = = 3 25088

In a larger scale of application, this workload reduction may be further enhanced and will have a considerable impact regarding cost saving comparing to the traditional approach.

Reducing Workload by Defective Product Classification 10 Conclusion

The findings of this study can be understood as a combination of the Autoencoder and One-class model We achieved the to resolve problems in which the complex dataset is a basic success in huge obstacle. Our method ignored the decoder block and only used encoder output from Autoencoder notifying normal with a U-net network for One-class with Local outlier images and factor algorithm. We achieved the basic success in notifying normal images and abnormal images. These abnormal images. findings provide a potential mechanism for reducing These findings an approximate 35% user workload. provide a potential This is the key component in future attempts to mechanism for overcome these complexities in the dataset. Future work could continue to explore and improve to increase reducing an the number of normal images prediction. Research approximate does not just stop there; the problem is also open the giant potential room for automation in product testing. It is not only classified normal and abnormal % images but also highlights the region of the defect in abnormal images result. In the future, deploying this 35user workload. automatic system will bring tremendous benefits to manufacturing enterprises.

Reducing Workload by Defective Product Classification 11 On a business perspective, using automated quality control can replace humans who manually spot defects - a process which is often error-prone. Manufacturing companies can benefit from this approach to improve quality control through automated visual inspection tools. This application improves production time and reduces the burden for manual workers to discover defective products.

Reducing Workload by Defective Product Classification 12 About FPT In the past 30 years, FPT has particularly pioneered in building and developing Vietnamese-branded software; applying technology to life, creating benefits to people and businesses; modernizing national backbone industries; providing young people with education and practical training. In Industry 4.0, FPT is the pioneer in researching and developing new technologies including Artificial Intelligence, Big Data, , etc. FPT is also the pioneer company in Vietnam when being in partnership with world-leading technology corporations such as GE (Predix), (MinSphere), (Skywise), Amazon AWS, etc., to create the most advanced digital platforms. To contribute to bringing Vietnamese wisdom to the world, FPT goal to invests in artificial intelligence (AI) center in Quy Nhon which is known as a center of mathematics in Viet Nam. Establish by the end of 2018, FPT Quy Nhon reached 200 employees. This branch aims to increase the number of people to 500 people at the end of this year. Besides, FPT has started to build the campus for the FPT university branch of artificial intelligence in December 2019. Quy Nhon is expected to become a leading AI destination over the world.

About the Author Nguyen Ngoc Tam Nguyen Ngoc Tam graduated Bachelor’s Degree at the University of Transport and Communications in 2010. He achieved the third prize in the Vietnamese Student Technology Information Olympic in 2009. He worked for Gameloft as a developer until 04/2018. In 2019, He received a Computer Science Master Degree at Ho Chi Minh University of Science. Currently, he is a scientist in computer vision as well as the leader of Analytics And AI (FWI.AAA.AA.QNH) unit in FPT software Quy Nhon.

Early Fire Detection in Outdoor Environments Using Real-time Video Footage 13 References:

K. Simonyan and A. Zisserman. (2015). Very deep convolutional networks for large-scale image recognition. In.

Jonathan Long, Evan Shelhamer, Trevor Darrell. (2015). The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (pp. 3431-3440).

Larochelle H, Lajoie I, Bengio Y, Manzagol PA. (2010). Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research (pp. 3371−3408).

M. Haselmann, D. P. Gruber and P. Tabatabai. (2018). Anomaly Detection Using Deep Learning Based Image Completion. 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA) (pp. 1237-1242).

Schlegl T., Seeböck P., Waldstein S.M., Schmidt-Erfurth U., Langs G. (2017). Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery. In: Niethammer M. et al. (eds) Information Processing in Medical Imaging. IPMI 2017. Lecture Notes in Computer Science, vol 10265. Springer, Cham.

Houssam Zenati, Chuan Sheng Foo, Bruno Lecouat, Gaurav Manek, Vijay Ramaseshan Chandrasekhar. (2018). Efficient GAN-Based Anomaly Detection. ICLR Workshop 2018.

Kingma, D.P.; Welling, M. (2013). Auto-Encoding Variational Bayes. arXiv:1312.6114.

Paul Bergmann, Sindy Lowe, Michael Fauser, David Sattlegger, and Carsten Steger. (2019). Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders. Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications.

Mei, Shuang; Wang, Yudan; Wen, Guojun. (2018). Automatic Fabric Defect Detection with a Multi-Scale Convolutional Denoising Autoencoder Network Model. Sensors 18, no. 4: 1064.

P. Perera and V. M. Patel. (2019). Learning Deep Features for One-Class Classification. IEEE Transactions on Image Processing, vol. 28, no. 11 (pp. 5450-5463).

Baur C., Wiestler B., Albarqouni S., Navab N. (2019). Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images. In: Crimi A., Bakas S., Kuijf H., Keyvan F., Reyes M., van Walsum T. (eds) Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2018. Lecture Notes in Computer Science, vol 11383. Springer, Cham.

Markus M. Breunig, Hans-Peter Kriegel, Raymond T. Ng, and Jörg Sander. (2000). LOF: identifying density- based local outliers. In Proceedings of the 2000 ACM SIGMOD international conference on Management of data (SIGMOD ‘00) (pp. 93-104).

Reducing Workload by Defective Product Classification 14 References:

Allianz Global Corporate & Specialty SE. (2017). Product Recall: Managing the Impact of the New Risk Landscape Report

Jinjiang Wanga, Yulin Maa, Laibin Zhanga, Robert X. Gao b, Dazhong Wuc. (2018). Deep Learning for Smart Manufacturing: Methods and Applications

Forbes Insight Team. (2018). How AI Builds A Better Manufacturing Process

Reducing Workload by Defective Product Classification 15 FPT Software is part of FPT Corporation, a technology and IT services provider headquartered in Vietnam with nearly US$1.2 billion in revenue and 28,000 employees. FPT is a pioneer in and delivers world-class services in Smart factory, Digital platforms, RPA, AI, IoT, Mobility, Cloud, , Testing, more. The company has served over 700 customers worldwide, 83 of which are Fortune 500 companies in the industries of Aerospace & Aviation, Automotive, Banking and Finance, Communications, Media and Services, Logistics & Transportation, Utilities, Consumer Packaged Goods, Healthcare, Manufacturing, Public sector, Technology and more.

For further information, please contact us at: [email protected]

© 2020 FPT Software. All rights reserved.