Machine Learning and Communication § § § § Fraunhofer HeinrichHertzInstitute
10 technology inventedormadebyFraunhoferHHI Technologies Every secondbitontheinternettouchesVideo orPhotonic Research &DevelopmentinPhotonics,Video &Wireless Annual budgetof50M€/450Researchers Globally activeplayerindigitalinfrastructureresearch 0 –10 2 –10 4 Gbps H.264 –H.265 H.266 Machine Learning andCommunication Thomas Wiegand : 3G –4G5G
slide 2 n n n Machine Learningand Outline Decision MakingExplained Data Communication Video CodingStandards Machine Learning andCommunication Thomas Wiegand :
slide 3 Machine Learning and Video Coding Standards Visual CommunicationSystems Capture Display
Encoder Decoder Machine Learning andCommunication Video Video Video Video Thomas Wiegand : Encoder Channel Decoder Channel Demodulator Modulator Channel
slide 5 Visual CommunicationSystems Capture Display
Encoder Decoder Machine Learning andCommunication Video Video Video Video Thomas Wiegand : Encoder Channel Decoder Channel Demodulator Modulator Channel
slide 6 n n standards: Implementations ofvideo coding Video CodingStandards developed by manufacturers Real-time video encodingis Only decoderisspecified Machine Learning andCommunication n coding: International standardizationofvideo n n (12/2016: about1Billiondevices) Every 2 H.266 isinfutureplanningstage H.265 isstartingtobecomerelevant Thomas Wiegand : nd bitontheInternetisH.264
slide 7 PSNR [dB] Performance ofVideo Standards 50% 50% Machine Learning andCommunication Thomas Wiegand : Bit Rate[kbit/s]
slide 8 Time <= T, ... Rate<=R, n n n Machine Learning Boundary conditions H.265/MPEG-HEVC Natural video Data Machine Learning andCommunication Thomas Wiegand : Algorithm Learning Algorithm Encoder
slide 9 Fraunhofer HHIH.265Encoder First Results: benchmark starting pointforlearning Machine Learning andCommunication Thomas Wiegand : learned algorithm speed up 40 %
slide 10 Machine Learning and Data Communication Visual CommunicationSystems Capture Display
Encoder Decoder Machine Learning andCommunication Video Video Video Video Thomas Wiegand : Encoder Channel Decoder Channel Demodulator Modulator Channel
slide 12 The NextGeneration:5GNetwork Machine Learning andCommunication Communications Car2Car &Car2X Industrial Wireless Mobile HighSpeedInternet Thomas Wiegand : • • • • Requirements • • Technology 1 mslatency 10 xbatterylife 100 xdevices 1000 xthroughput Optical fiber lights become senders DSL boxes andstreet
slide 13 Wireless FiberandLocationSensing • • sensors MIMO Antennas Location ofusersvia 3D beamformingwith Machine Learning andCommunication Thomas Wiegand :
slide 14 Control &UserPlane Split: Example: Networked Future MobileDigitalInfrastructure Prediction using Maschine Learning Machine Learning andCommunication Safety Thomas Wiegand : Autonomous Driving Localization , Securityand Data Transfer & Routing Trust Antennas Wireless Fiber-
slide 15 The Tactile Internet n n n Can be realisedaspart ofWiFi,5Gor fixednetworks Ultra highreliability Very lowend-to-end latencies(1ms) Human reactionstimes Machine Learning andCommunication Thomas Wiegand : source: ITU TechWatch Report: The Tactile Internet source: https://netzoekonom.de
slide 16 Collaborative Driving Driver assistance with AR ofpotentially dangerous objects and situations Machine Learning andCommunication Thomas Wiegand : Source: ITU TechWatch Report: The Tactile Internet
slide 17 Interpretable Machine Learning 14.2 Millionimages,22.000classes Do wetrustthemachine ??? Classification usingMachine Learning Big Data + Machine Learning Machine Learning andCommunication Thomas Wiegand : = Automatic Annotation “no cancer” “cancer”
slide 19 Revert theDeepNeuralNetwork Machine Learning andCommunication Thomas Wiegand :
slide 20 Interpretability ofMachineLearning that ML algorithmsdotheright Interpretability isfirststeptowardsmaking sure(i.e.verifying) Machine Learning andCommunication thing! Thomas Wiegand :
slide 21 classification decisions. general W. Samek,K.-RMülleretal.: Main idea: Idea forInterpretableMachineLearning methodtoexplain individual Machine Learning andCommunication Thomas Wiegand :
…. Samek etal.,TNNLS Lapuschkin etal., Bach etal.,PLOSONE,2015 “ladybug” CVPR, 2016 , 2016
slide 22 Classification Machine Learning andCommunication Thomas Wiegand : ladybug cat dog
slide 23 Explanation Machine Learning andCommunication Thomas Wiegand : ladybug dog cat Initialization =
slide 24 (Deep) Taylor Decomposition (Montavon etal.,arXiv2015) Theoretical interpretation Relevance Propagation Machine Learning andCommunication layers proportionally Relevance ofupper layersisredistributedtolower ? Thomas Wiegand : (depending onactivations &weights). ladybug dog cat
slide 25 Relevance Conservation Property Relevance ConservationProperty Machine Learning andCommunication Thomas Wiegand : ladybug dog cat
slide 26 blue color:evidenceagainst prediction red color:evidenceforprediction [number ML DecompositionExamples classification as“3” what speaksfor/against ]: explanationtargetclass ML Decomposition distinguishes between positive and negativeevidence Machine Learning andCommunication Thomas Wiegand : classification as“9” what speaksfor/against (Bach etal.,PLOSONE2015)
slide 27 n n n Communication areconverging Summary: MachineLearningand Interpretable Machine Learning: Data CommunicationandMachineLearning: Video CodingStandards and MachineLearning: • • • • • • (Tactile Internet), Sensors Explanation requiredforDecision Making! Decomposition explains classificationresults Machine Learningnecessary forefficient Next Generation5G:Highbitrates,lowlatencies Improve Video Encodingusing ML H.264 è H.265 Machine Learning andCommunication è H.266 Thomas Wiegand :
slide 28 § HHI/TUB membersandresearchassociates ITU-T VCEG &ISO/IECMPEGcolleagues § § § §
H. Schwarz,D.Marpe, T. Hinz,P. Helle T. Schierl,C.Hellge,R.Skupin, Y. Sanchez S. Bosse,B.Blankertz, A. Norcia,G.Curio K.-R. Müller, W. Samek … Machine Learning andCommunication Thomas Wiegand :
slide 29