DEGREE PROJECT IN INFORMATION AND COMMUNICATION TECHNOLOGY, FIRST CYCLE, 15 CREDITS STOCKHOLM, SWEDEN 2020 Interpretation of Swedish Sign Language using Convolutional Neural Networks and Transfer Learning Gustaf Halvardsson & Johanna Peterson KTH ROYAL INSTITUTE OF TECHNOLOGY ELECTRICAL ENGINEERING AND COMPUTER SCIENCE Authors Gustaf Halvardsson & Johanna Peterson
[email protected] &
[email protected] The School of Electrical Engineering and Computer Science KTH Royal Institute of Technology Place for Project KTH Royal Institute of Technology & Prevas AB Stockholm, Sweden Examiner Benoit Baudry - KTH Royal Institute of Technology Supervisors César Soto Valero - KTH Royal Institute of Technology Maria Månsson - Prevas AB ii Abstract The automatic interpretation of signs of a sign language involves image recognition. An appropriate approach for this task is to use Deep Learning, and in particular, Convolutional Neural Networks. This method typically needs large amounts of data to be able to perform well. Transfer learning could be a feasible approach to achieve high accuracy despite using a small data set. The hypothesis of this thesis is to test if transfer learning works well to interpret the hand alphabet of the Swedish Sign Language. The goal of the project is to implement a model that can interpret signs, as well as to build a user-friendly web application for this purpose. The final testing accuracy of the model is 85%. Since this accuracy is comparable to those received in other studies, the project’s hypothesis is shown to be supported. The final network is based on the pre-trained model InceptionV3 with five frozen layers, and the optimization algorithm mini-batch gradient descent with a batch size of 32, and a step-size factor of 1.2.