ML for EV3 with Scratch3 Camp II - Check-In
Total Page:16
File Type:pdf, Size:1020Kb
ML for EV3 with Scratch3 Camp II - Check-In • Pre-survey for students: https://forms.gle/YuYgVMpmWKpRMgsp7 Check the Zoom Chat for the link (Only for new students. Not for returning students) • Test your ML account login. Ask your coach if you do not have credentials • Scratch to EV3 communication is NOT required today • We need a mask for a ML project today Copyright © Chung 2020 1 Pre-survey for students: https://forms.gle/YuYgVMpmWKpRMgsp7 Check the Zoom Chat for the link. Not for returning students. Copyright © Chung 2020 2 Start Recording!! Introduction to ML for EV3 with Scratch3 II Focusing on real-time video image applications using ML Aug 14, 2020 Copyright © Chung 2020 3 This file can be accessed at: http://www.robofest.net/2020/ML_EV3_Scratch3.pdf This link can be found in the Chat. Recorded lecture video will be available after the camp. Copyright © Chung 2020 4 Introduction to ML for EV3 with Scratch3 II Focusing on real-time Aug 14, 2020 vision applications CJ Chung, Ph.D. Professor of CS Founder & Director of Robofest Lawrence Technological University Copyright © Chung 2020 5 Robofest Camp = workshop + mini project competition Winners from the 1st Camp on July 10th . 1st place: https://youtu.be/BZzX7x26NII (Smart Mask Detector) . 2nd place: https://youtu.be/CmnwfPzYXIE (Smart Shopper) Copyright © Chung 2020 6 Robofest Camp = workshop + mini project competition Winners from the 2nd Camp on July 24 . Co-1st place: https://youtu.be/UEgeDiKJ6rc (Recognizing handwriting to turn on/off a EV3 robotic fan) . Co-1st place: https://youtu.be/LH5pInLapMo (Mood Recognition) . 3rd place: https://youtu.be/N14C1yYEP2U (Recognizing text as Sanitize or Sanitized) Copyright © Chung 2020 7 Robofest Camp = workshop + mini project competition . Details about the mini project competition will be announced at the end of today’s meeting . Top teams will win prizes sponsored by Copyright © Chung 2020 8 This camp is sponsored by Copyright © Chung 2020 9 Zoom Class Rules .Mute your microphone .If you have questions or to answer instructor’s questions •Use Zoom chat. •It is also OK to unmute and ask questions anytime Copyright © Chung 2020 10 A Machine Learning (ML) Example In ML, Machines are taught to learn from example data Copyright © Chung 2020 11 2 3 4 4 (2+3+4) mod 5 = 4 3 5 9 2 6 3 4 3 Examples with correct answers 4 3 8 0 (labels) 3 4 1 3 2 8 9 4 7 3 1 1 2 12 Copyright © Chung 2020 3 5 4 ? Concept of ML Math Model on computer After training Trained Math Model unseen data on computer (d0+d1+d2) % 5 Copyright © Chung 2020 13 (6 3 4) (2 3 4) (4 3 8) (731) (3 5 9) (3 4 1) (2 8 9) Group label: 0 Group label: 1 Group label: 2 Group label: 3 Group label: 4 ? (3 5 4) unseen data Copyright © Chung 2020 14 unseen image Copyright © Chung 2020 15 What is Machine Learning (ML) - A subfield of AI AI: mimics the intelligence of behavioral pattern of humans or other living entity Copyright © Chung 2020 16 What is Machine Learning (ML) . ML is a technique where computers can Math be trained by using sample data to Model output answers without being explicitly on computer programmed. ML algorithms build a mathematical model based on "training data", in order to make predictions or decisions without After training being explicitly programmed to do so. Trained Math Model Unseen data on computer If n < 4.5, then dark Copyright © Chung 2020 17 3 new Scratch Extensions to use today Copyright © Chung 2020 18 Today’s Projects .Project 1: Face detection - Red Nose (Using pretrained ML model) .Project 2: Mask NoMask •2a: Images from the web, your ML model, no coding •2b: Images from your webcam, your own ML model, and Scratch coding .Project 3: Pose detection – Wrist Up or Down (Using pretrained ML model & Your ML model) Copyright © Chung 2020 19 Using machinelearningforkids.co.uk . Click on first . No need to sign up. Your coach has a username and password for each team . It is a paid account by usage. If you do not use it for 30 days, it may be closed . Your account can have up to 4 projects . Multiple devices can use the account. But projects in the account must be shared . If you do not have an account, please ask by Zoom-chat NOW! Copyright © Chung 2020 20 Please watch an ML4Kids Overview video (after this camp) . https://youtu.be/EjbHXMzeX4c (22min) Copyright © Chung 2020 21 ML for Kids Video explains: .Text .Number .Image “Turn on the fan” or .Sound “It is hot here” Pac-Man: to avoid Ghost. See next slide “&…..@$^” sound in alien language to go left Rock, Paper, Scissors Copyright © Chung 2020 22 Scratch Review & Simple Face Detection Demo Do not need to follow. Please Just watch Copyright © Chung 2020 23 Review: Scratch Block Categories . Motion blocks deal with the movement of sprites. They relate mainly to the x and y position and direction of the sprite. Looks blocks are related to the appearance of sprites and the stage. Sound blocks control sound. Events blocks are related to various triggers in a project, or when one part signals another to run. Control blocks run the basic flow of a project in the desired fashion. Sensing blocks associate with sprites and the stage detecting conditions. Operators blocks deal with many mathematical functions. Variables blocks are for storing and accessing data. My blocks are blocks that hold custom procedures for a selected sprite. Images blocks contain costume & backdrop image reporters Copyright © Chung 2020 24 Review: Six Scratch Block Shapes Hat blocks are the C blocks loop the blocks blocks that start every or check if a condition is script. true. Reporter blocks can hold Stack blocks perform numbers and strings. commands. Boolean blocks are the Cap blocks are the blocks conditions — they are that end scripts. either true or false Copyright © Chung 2020 25 x and y coordinates in Scratch Copyright © Chung 2020 26 x and y positions of my nose in Scratch using a pre-trained model Code on backdrop It will take a few seconds to start the code. If no x & y updates, restart the script Copyright © Chung 2020 27 Review: what does this reporter block do? 1. Send current image to pre-trained model 2. The model sends back the x coordinate of the nose on the image Copyright © Chung 2020 28 Project #1: Red Nose – move a red strawberry to your nose . Use a pre-trained machine learning model to do face detection on a live webcam video, and code animated effects on your nose . No need to use ML account. Go to: https://machinelearningforkids.co.uk/pretrained Copyright © Chung 2020 29 Project #1 Steps (1/5) . Click on “Get Started” button. Click on button at the bottom left to open the Extensions window. Click on the Video Sensing extension. Open the Extensions window again. Click on the Face detection extension to use the pre-trained model. Delete the cat sprite by clicking the trash can icon. Click on the green “cat+” button to choose a Sprite. Copyright © Chung 2020 30 Project #1 Steps (2/5) . Click on “Food” button and click on Strawberry . Click on “Costumes” tab . Select the leaves of strawberry and . press Delete key to remove the leaves . Click on Flip Vertical, then you will get . Reduce the size from 100 to 50 . Click on “Code” tab Copyright © Chung 2020 31 Project #1 Steps (3/5) . Check the New blocks from the 2 Extensions Copyright © Chung 2020 32 Project #1 Steps (4/5) Set x coordinate of the current Sprite Copyright © Chung 2020 33 Project #1 Steps (5/5) for dynamic red nose size The size of the red- nose is proportional to the size of my nose Copyright © Chung 2020 34 Group Photo Time Turn on your camera and say “ML” Copyright © Chung 2020 35 Project #2a - Mask NoMask (Images from the web, your ML model, no coding) Model Trained Mask Model Unseen image Copyright © Chung 2020 Project 2a: Mask NoMask (1/3) . Go to: https://machinelearningforkids.co.uk/ . Click on the blue “Get Started” button . Log in your ML account. Please Zoom-chat if your team does not have an account . Click on Go to your Projects . Enter Project Name and Recognising: . Click on CREATE button . Click on the Project: . Click on “Train” button Copyright © Chung 2020 37 Project 2a: Mask NoMask (2/3) . Click on ; Create “Mask” and “NoMask” buckets . Open another browser tab and Google search “Mask face”. Click on “Images” . Drag 10 jpg or png pictures from the other browser windows and drop them to “Mask” bucket . Now Google search “Face” and click on Images . Drag 10 pictures from the other browser windows and drop them to “NoMask” bucket . Back to project . Click on “Learn & Test” button . Click on . Quiz time! (Around 10 minutes) Copyright © Chung 2020 38 Project 2a: Mask NoMask (3/3) . When training is done, enter a web image address and press “Test with www” button Copyright © Chung 2020 39 Time for a 3 minute Stop Recording Copyright © Chung 2020 40 Restart Recording Project #2b Mask NoMask (Collect data using camera; Your ML model & your code) Model Scratch Trained webcam Mask code Model Copyright © Chung 2020 41 Project 2b: Mask NoMask (1/4) . Go to: https://machinelearningforkids.co.uk/ . Click on the blue “Get Started” button . Log in your ML account. Please chat if your team does not have an account . Click on Go to your Projects . Enter Project Name and Recognising . Click on button .