DOCSLIB.ORG

Cooperative Edge Deepfake Detection

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Cooperative Edge Deepfake Detection Cooperative edge deepfake detection Main Subject area: Computer Engineering Authors: Enis Hasanaj, William Söder, Albert Aveler Supervisor: Garrit Schaap JÖNKÖPING June 2021 This final thesis has been carried out at the School of Engineering at Jönköping University within Computer Engineering. The authors are responsible for the presented opinions, conclusions, and results. Examiner: Rachid Oucheikh Supervisor: Garrit Schaap Scope: 15 hp (first-cycle education) Date: 2021-06-12 Abstract ........................................................................................................... 1 1 Introduction ........................................................................................... 2 1.1 PROBLEM STATEMENT ...................................................................................... 2 1.1.1 Deepfakes ................................................................................................. 2 1.1.2 Computational power in machine learning .............................................. 2 1.2 LITERATURE REVIEW ........................................................................................ 4 1.2.1 Edge training and federated learning ....................................................... 4 1.2.2 Deepfake detection ................................................................................... 4 1.3 PURPOSE AND RESEARCH QUESTIONS ............................................................... 5 1.4 SCOPE AND LIMITATIONS .................................................................................. 6 1.5 DISPOSITION ..................................................................................................... 7 2 Method and implementation ............................................................ 8 2.1 DATA COLLECTION ........................................................................................... 8 2.2 DATA ANALYSIS ............................................................................................... 9 2.3 VALIDITY AND RELIABILITY ........................................................................... 10 2.4 CONSIDERATIONS ........................................................................................... 10 2.5 PROPOSED SOLUTION OF COOPERATIVE EDGE DEEPFAKE DETECTION ............. 11 3 Theoretical framework .................................................................... 12 3.1 DARKNET YOLOV2 ....................................................................................... 12 3.1.1 Basics ..................................................................................................... 12 3.1.2 Details .................................................................................................... 13 3.2 COMBINING THE MODELS ............................................................................... 15 3.3 ENSEMBLE METHODS ...................................................................................... 16 3.3.1 Bagging .................................................................................................. 16 3.3.2 Boosting ................................................................................................. 17 3.3.3 Stacking .................................................................................................. 18 3.4 PROGRAMMING LANGUAGES AND PLATFORMS ............................................... 18 3.5 DEEPFAKES EXPLAINED .................................................................................. 19 3.6 DATASET ........................................................................................................ 21 3.7 TERMINOLOGY ............................................................................................... 22 3.7.1 Bounding box ......................................................................................... 22 3.7.2 True/False Positive/Negative ................................................................. 23 3.7.3 Recall ..................................................................................................... 23 3.7.4 Precision ................................................................................................. 23 3.7.5 Accuracy ................................................................................................ 23 3.7.6 Confidence ............................................................................................. 23 3.7.7 Aggregated vs. Non-Aggregated models ............................................... 24 3.7.8 Supervised vs. Unsupervised learning ................................................... 24 3.7.9 Overfitting .............................................................................................. 24 3.8 CONVOLUTIONAL NEURAL NETWORK (CNN) ................................................ 24 3.8.1 Convolutional layers .............................................................................. 24 3.8.2 Pooling layers ......................................................................................... 25 3.8.3 Fully connected layers ........................................................................... 25 4 Results .................................................................................................... 27 4.1 TRAINING WITH DIFFERENT SUBSETS .............................................................. 27 4.2 TRAINING WITH DIFFERENT NUMBER OF ITERATIONS ...................................... 29 4.3 EDGE TRAINING RESULTS ............................................................................... 32 4.4 ENSEMBLE RESULTS ....................................................................................... 34 5 Discussion ............................................................................................. 36 5.1 LIMITATIONS .................................................................................................. 36 5.2 RESULT DISCUSSION ....................................................................................... 36 5.3 METHOD DISCUSSION ..................................................................................... 38 6 Conclusions and further research ................................................. 39 6.1 CONCLUSIONS ................................................................................................ 39 6.1.1 Practical implications ............................................................................. 40 6.1.2 Scientific implications............................................................................ 40 6.2 FURTHER RESEARCH ....................................................................................... 40 6.2.1 Android .................................................................................................. 40 6.2.2 Apple support for object detection on the edge ..................................... 41 6.2.3 Federated learning .................................................................................. 41 6.2.4 Another ensemble method ..................................................................... 42 7 References............................................................................................. 44 Abstract Deepfakes are an emerging problem in social media and for celebrities and political profiles, it can be devastating to their reputation if the technology ends up in the wrong hands. Creating deepfakes is becoming increasingly easy. Attempts have been made at detecting whether a face in an image is real or not but training these machine learning models can be a very time-consuming process. This research proposes a solution to training deepfake detection models cooperatively on the edge. This is done in order to evaluate if the training process, among other things, can be made more efficient with this approach. The feasibility of edge training is evaluated by training machine learning models on several different types of iPhone devices. The models are trained using the YOLOv2 object detection system. To test if the YOLOv2 object detection system is able to distinguish between real and fake human faces in images, several models are trained on a computer. Each model is trained with either different number of iterations or different subsets of data, since these metrics have been identified as important to the performance of the models. The performance of the models is evaluated by measuring the accuracy in detecting deepfakes. Additionally, the deepfake detection models trained on a computer are ensembled using the bagging ensemble method. This is done in order to evaluate the feasibility of cooperatively training a deepfake detection model by combining several models. Results show that the proposed solution is not feasible due to the time the training process takes on each mobile device. Additionally, each trained model is about 200 MB, and the size of the ensemble model grows linearly by each model added to the ensemble. This can cause the ensemble model to grow to several hundred gigabytes in size. Keywords: Machine learning, deepfake, artificial intelligence, ensemble, convolutional neural networks, edge, YOLOv2 1 1 Introduction In this study, deepfake detection models are trained locally on multiple iPhone devices using the YOLOv2 system (Redmon, 2016), which is part of the Darknet open-source machine learning framework. YOLOv2 is used to train models using parts of an image dataset and the models are combined into an ensemble once the models have been trained. A method is proposed for how this can be done, and the feasibility of the
Load more

Recommended publications
  • RISC-V Core Out-Clocks Apple, Sifive; Available As IP
    RISC-V core out-clocks Apple, SiFive; available as IP Movember 05, 2020 //By Peter Clarke Micro Magic Inc. (Sunnyvale, Calif.) has functioning silicon of its 5GHz-capable 64bit RISC-V processor and is offering the design as intellectual property. The Micro Magic processor out-clocks the Apple A14 bionic, the processor at the heart of the iPhone 12 and one of the first processors on 5nm silicon. It also goes faster than a quad-core U84 CPU that SiFive states can operate at up to 2.6GHz clock frequency when implemented in a 7nm process. Micro Magic has a history that goes back to Sun Microsystems and beyond (see EDA company claims world’s fastest 64bit RISC-V core). It is reportedly one of Silicon Valley’s well-kept secrets and a go-to resource for design teams trying to remove bottlenecks in their datapath designs. Andy Huang, an independent contractor who supports Micro Magic for marketing and business development functions, contacted eeNews Europe and demonstrated the processor running EEMBC CoreMark benchmarks over a Facetime connection. Huang was founder and CEO of ACAD Corp., the developer of the Finesim simulator, one of the first and fastest of parallel SPICE simulators. ACAD was acquired by Magma Design Automation in 2006 before Magma itself was acquired by Synopsys in 2012. Huang declined to say which foundry had manufactured silicon for Micro Magic or in what manufacturing process it had been implemented. Huang said the that processor is made in a FinFET process and was manufactured using a multiproject wafer (MPW) run.
    [Show full text]
  • An Optimized H.266/VVC Software Decoder on Mobile Platform
    An Optimized H.266/VVC Software Decoder On Mobile Platform Yiming Li, Shan Liu, Yu Chen, Yushan Zheng, Sijia Chen, Bin Zhu, Jian Lou Tencent Media Lab, Shenzhen, China and Palo Alto, CA, USA, fmarcli, [email protected] Abstract—As the successor of H.265/HEVC, the new versatile standard. Therefore, it is essential to have an efficient and video coding standard (H.266/VVC) can provide up to 50% optimized software decoder implementation to support the bitrate saving with the same subjective quality, at the cost of emerging applications. In [3] [4], an independent VVC soft- increased decoding complexity. To accelerate the application of the new coding standard, a real-time H.266/VVC software ware decoder implemented by Tencent demonstrated real-time decoder that can support various platforms is implemented, HD/UHD decoding capability on x86 platform. Considering where SIMD technologies, parallelism optimization, and the that mobile devices have become an essential carrier and acceleration strategies based on the characteristics of each coding display tool for video services, extensive optimization efforts tool are applied. As the mobile devices have become an essential were made on top of the framework of [3] to achieve real- carrier for video services nowadays, the mentioned optimization efforts are not only implemented for the x86 platform, but more time HD/UHD decoding on the mobile platform. As a result, importantly utilized to highly optimize the decoding performance a uniform-designed software H.266/VVC decoder that can on the ARM platform in this work. The experimental results show run real-time on different platforms and supports versatile that when running on the Apple A14 SoC (iPhone 12pro), the av- functionalities such as screen content coding (SCC) is ac- erage single-thread decoding speed of the present implementation complished.
    [Show full text]
  • Downloaded from Adobe Stock [51]
    big data and cognitive computing Article Fast and Effective Retrieval for Large Multimedia Collections Stefan Wagenpfeil 1,* , Binh Vu 1 , Paul Mc Kevitt 2 and Matthias Hemmje 1 1 Faculty of Mathematics and Computer Science, University of Hagen, Universitätsstrasse 1, D-58097 Hagen, Germany; [email protected] (B.V.); [email protected] (M.H.) 2 Academy for International Science & Research (AISR), Derry BT48 7JL, UK; [email protected] * Correspondence: [email protected] Abstract: The indexing and retrieval of multimedia content is generally implemented by employing feature graphs. These graphs typically contain a significant number of nodes and edges to reflect the level of detail in feature detection. A higher level of detail increases the effectiveness of the results, but also leads to more complex graph structures. However, graph traversal-based algorithms for similarity are quite inefficient and computationally expensive, especially for large data structures. To deliver fast and effective retrieval especially for large multimedia collections and multimedia big data, an efficient similarity algorithm for large graphs in particular is desirable. Hence, in this paper, we define a graph projection into a 2D space (Graph Code) and the corresponding algorithms for indexing and retrieval. We show that calculations in this space can be performed more efficiently than graph traversals due to the simpler processing model and the high level of parallelization. As a consequence, we demonstrate experimentally that the effectiveness of retrieval also increases substantially, as the Graph Code facilitates more levels of detail in feature fusion. These levels of detail also support an increased trust prediction, particularly for fused social media content.
    [Show full text]
  • Download Magazine
    Magazine Issue Porsche Engineering 1/2021 www.porsche-engineering.com AUTOMOTIVE DEVELOPMENT ON THE MOVE Next level As always, our strongest motivation: our own standards. The new Panamera S E-Hybrid. Drive defines us. How can you measure progress? The new Panamera and its numbers make for a good start. Its .-litre twin-turbo V engine combines with the electric motor to produce a system output of kW (PS). And this new power is backed with an increased electric range. Only the best to drive your next project forward. Discover more at www.porsche.com/Panamera Fuel consumption (in l/km) combined . – .; CO₂ emissions (in g/km) combined –; electricity consumption (in kWh/km) combined .–. Porsche Engineering Magazine 1/2021 EDITORIAL 3 Dear Reader, “Next level”—scarcely any other expression sums the situation in which we currently find ourselves more succinctly: In many fields, we simply have to reach the next level. We are familiar with the term from the worlds of computer games and sports. Those who have reached a certain level there often move on to the next level right away. This also applies to our work in automotive development: We cannot rest on our laurels. We see this in trends such as artificial intelligence, the use of game engines in vehicle development and future E/E architectures, which we examine in the title series. They exemplify the increasing complexity of our work. We also have to reach the next level in our methods. In his article, Marius Mihailovici, Managing Director of our digital branch in Cluj, explains what this looks like in the software development sector.
    [Show full text]
  • Thread-Level Parallelism I
    Great Ideas in UC Berkeley UC Berkeley Teaching Professor Computer Architecture Professor Dan Garcia (a.k.a. Machine Structures) Bora Nikolić Thread-Level Parallelism I Garcia, Nikolić cs61c.org Improving Performance 1. Increase clock rate fs ú Reached practical maximum for today’s technology ú < 5GHz for general purpose computers 2. Lower CPI (cycles per instruction) ú SIMD, “instruction level parallelism” Today’s lecture 3. Perform multiple tasks simultaneously ú Multiple CPUs, each executing different program ú Tasks may be related E.g. each CPU performs part of a big matrix multiplication ú or unrelated E.g. distribute different web http requests over different computers E.g. run pptx (view lecture slides) and browser (youtube) simultaneously 4. Do all of the above: ú High fs , SIMD, multiple parallel tasks Garcia, Nikolić 3 Thread-Level Parallelism I (3) New-School Machine Structures Software Harness Hardware Parallelism & Parallel Requests Achieve High Assigned to computer Performance e.g., Search “Cats” Smart Phone Warehouse Scale Parallel Threads Computer Assigned to core e.g., Lookup, Ads Computer Core Core Parallel Instructions Memory (Cache) >1 instruction @ one time … e.g., 5 pipelined instructions Input/Output Parallel Data Exec. Unit(s) Functional Block(s) >1 data item @ one time A +B A +B e.g., Add of 4 pairs of words 0 0 1 1 Main Memory Hardware descriptions Logic Gates A B All gates work in parallel at same time Out = AB+CD C D Garcia, Nikolić Thread-Level Parallelism I (4) Parallel Computer Architectures Massive array
    [Show full text]
  • Warrior Times One School, Two Campuses
    I​ ssue 1 . November . 2020 Warrior Times One School, Two Campuses On the Road to Reopening - Finally! Plans for Nov. 30th By: Eliana Montas and Ivy Pennyfeather Back in the summer, we never expected this school year to start remotely. Although we are currently not in the most ideal situation for learning, more decisions have been made to inform us as to how Franklin School District will be continuing our 2020-2021 school year. Hybrid -- Blue and Gold Teams If you choose hybrid school, this means that you will be spending part of your time in school, and part of your time online. Not sure what to decide? Here are some facts about hybrid school at FMS. Hybrid school will begin on Nov. 30. Parents and students will have the choice to either start hybrid or continue to be fully remote. Students who choose hybrid will be separated into two teams; Blue and Gold. Blue would be in school for one week and then attend online school for the following week, and the same with the Gold team on alternating weeks. Everyone will still be asynchronous on Fridays, meaning that there will be no formal teaching that day. At the start of each hybrid student’s day, they’ll need to complete a health screening form that’ll confirm whether or not they have a COVID-19 symptom or not. Both campuses will continue bussing which means that while you are on the bus you’ll need to follow the bus protocol in order to make sure that you and others around you stay safe.
    [Show full text]
  • Warrior Times One School, Two Campuses
    I​ ssue 1 . November . 2020 Warrior Times One School, Two Campuses On the Road to Reopening - Finally! Plans for Nov. 30th By Eliana Montas and Ivy Pennyfeather Back in the summer, we never expected this school year to start remotely. Although we are currently not in the most ideal situation for learning, more decisions have been made to inform us as to how Franklin School District will be continuing our 2020-2021 school year. Hybrid -- Blue and Gold Teams If you choose hybrid school, this means that you will be spending part of your time in school, and part of your time online. Not sure what to decide? Here are some facts about hybrid school at FMS. Hybrid school will begin on Nov. 30. Parents and students will have the choice to either start hybrid or continue to be fully remote. Students who choose hybrid will be separated into two teams; Blue and Gold. Blue would be in school for one week and then attend online school for the following week, and the same with the Gold team on alternating weeks. Everyone will still be asynchronous on Fridays, meaning that there will be no formal teaching that day. At the start of each hybrid student’s day, they’ll need to complete a health screening form that will confirm whether or not they have a COVID-19 symptom. Both campuses will continue bussing which means that while you are on the bus you’ll need to follow the bus protocol in order to make sure that you and others around you stay safe.
    [Show full text]
  • Multicore Semantics: Making Sense of Relaxed Memory
    Multicore Semantics: Making Sense of Relaxed Memory Peter Sewell1, Christopher Pulte1, Shaked Flur1,2 with contributions from Mark Batty3, Luc Maranget4, Alasdair Armstrong1 1 University of Cambridge, 2 Google, 3 University of Kent, 4 INRIA Paris October { November, 2020 Slides for Part 1 of the Multicore Semantics and Programming course, version of 2021-06-30 Part 2 is by Tim Harris, with separate slides Contents 1 These Slides These are the slides for the first part of the University of Cambridge Multicore Semantics and Programming course (MPhil ACS, Part III, Part II), 2020{2021. They cover multicore semantics: the concurrency of multiprocessors and programming languages, focussing on the concurrency behaviour one can rely on from mainstream machines and languages, how this can be investigated, and how it can be specified precisely, all linked to usage, microarchitecture, experiment, and proof. We focus largely on x86; on Armv8-A, IBM POWER, and RISC-V; and on C/C++. We use the x86 part also to introduce some of the basic phenomena and the approaches to modelling and testing, and give operational and axiomatic models in detail. For Armv8-A, POWER, and RISC-V we introduce many but not all of the phenomena and again give operational and axiomatic models, but omitting some aspects. For C/C++11 we introduce the programming-language concurrency design space, including the thin-air problem, the C/C++11 constructs, and the basics of its axiomatic model, but omit full explanation of the model. These lectures are by Peter Sewell, with Christopher Pulte for the Armv8/RISC-V model section.
    [Show full text]
  • Zacks Equity Research Zacks Equity Research 10 S
    February 3, 2021 Zacks Equity Research Zacks Equity Research www.zacks.com 10 S. Riverside Plaza, Suite 1600 - Chicago, Il 60606 Industry Outlook Coronavirus outbreak has been beneficial for the Zacks Computer – Mini Computers industry as it raised demand for PCs and tablets significantly. Despite massive supply-chain disruption the ongoing work-from-home and online learning wave have been beneficial for industry participants like Apple (AAPL), HP (HPQ) and Lenovo Group ( LNVGY). Although extensive job losses are expected to hurt demand of high-end laptops and smartphones in the near term, the availability of 5G-based iPhones has been a key catalyst. Further, launch of foldable as well as AI and ML-infused smartphones, tablets, wearables and hearables are major growth drivers for the industry participants. Industry Description The Zacks Computer – Mini Computers industry comprises prominent companies like Apple and HP that offer devices including smartphones, desktops, laptops, printers, wearables and 3-D printers. Such devices are based either on Apple’s iOS, MacOS, iPadOS, WatchOS or on Microsoft Windows or on Google Chrome and Android operating systems. They predominantly use processors from Apple, Intel (INTC), AMD, Qualcomm (QCOM), NVIDIA (NVDA), Samsung, Broadcom and MediaTek, among others. 3 Mini Computer Industry Trends to Watch Out For Bring Your Own Device (BYOD) Aids Momentum: The industry is benefiting from the rapid adoption of BYOD in workplaces. Enterprises practicing BYOD allow employees to use their personal devices, including mobiles, laptops and tablets, for work purposes. BYOD helps in bridging communication gaps between remote workers and desk-bound employees, thereby improving process management and workflow.
    [Show full text]
  • National Information Assurance Partnership Common Criteria
    National Information Assurance Partnership Common Criteria Evaluation and Validation Scheme ® TM Validation Report for the Apple iOS 14 and iPadOS 14: Contacts Report Number: CCEVS-VR-VID11191-2021 Dated: August 20, 2021 Version: 1.0 National Institute of Standards and Technology Department of Defense Information Technology Laboratory ATTN: NIAP, SUITE: 6982 100 Bureau Drive 9800 Savage Road Gaithersburg, MD 20899 Fort Meade, MD 20755-6982 ACKNOWLEDGEMENTS Validation Team Patrick Mallett, Ph.D. Jerome Myers, Ph.D. DeRon Graves Seada Mohammed J David D Thompson The Aerospace Corporation Common Criteria Testing Laboratory Kenji Yoshino Thibaut Marconnet Acumen Security, LLC 2 Table of Contents 1 Executive Summary ............................................................................................................... 4 2 Identification .......................................................................................................................... 5 3 Architectural Information .................................................................................................... 6 4 Security Policy........................................................................................................................ 7 4.1 Cryptographic Support .................................................................................................................................. 7 4.2 User Data Protection .....................................................................................................................................
    [Show full text]
  • Apple Platform Security
    Apple Platform Security May 2021 Contents Introduction to Apple platform security 5 A commitment to security 6 Hardware security and biometrics 7 Hardware security overview 7 Apple SoC security 8 Secure Enclave 9 Touch ID and Face ID 17 Hardware microphone disconnect 24 Express Cards with power reserve 25 System security 26 System security overview 26 Secure boot 26 Secure software updates 48 Operating system integrity 50 Additional macOS system security capabilities 52 System security for watchOS 63 Random number generation 67 Apple Security Research Device 68 Encryption and Data Protection 70 Encryption and Data Protection overview 70 Passcodes and passwords 70 Data Protection 73 FileVault 85 How Apple protects users’ personal data 89 Digital signing and encryption 91 App security 93 App security overview 93 Apple Platform Security 2 App security in iOS and iPadOS 94 App security in macOS 99 Secure features in the Notes app 103 Secure features in the Shortcuts app 104 Services security 105 Services security overview 105 Apple ID and Managed Apple ID 105 iCloud 107 Passcode and password management 110 Apple Pay 119 iMessage 132 Secure Business Chat using the Messages app 135 FaceTime security 136 Find My 137 Continuity 140 Car keys security in iOS 143 Network security 146 Network security overview 146 TLS security 146 IPv6 security 147 Virtual private network (VPN) security 148 Wi-Fi security 149 Bluetooth security 152 Ultra Wideband security in iOS 154 Single sign-on security 154 AirDrop security 155 Wi-Fi password sharing security on
    [Show full text]
  • Iphone 12 Pro
    iPhone 12 Pro Versions: A2407 (International); A2341 (USA); A2406 (Canada, Japan); A2408 (China, Hong Kong) NETWORK Technology GSM / CDMA / HSPA / EVDO / LTE / 5G LAUNCH Announced 2020, October 13 Status Released 2020, October 23 BODY Dimensions 146.7 x 71.5 x 7.4 mm (5.78 x 2.81 x 0.29 in) Weight 189 g (6.67 oz) Build Glass front (Gorilla Glass), glass back (Gorilla Glass), stainless steel frame SIM Single SIM (Nano-SIM and/or eSIM) or Dual SIM (Nano-SIM, dual stand-by) - for China IP68 dust/water resistant (up to 6m for 30 mins) Apple Pay (Visa, MasterCard, AMEX certified) DISPLAY Type Super Retina XDR OLED, HDR10, 800 nits (typ), 1200 nits (peak) 2 Size 6.1 inches, 90.2 cm (~86.0% screen-to-body ratio) Resolution 1170 x 2532 pixels, 19.5:9 ratio (~460 ppi density) Protection Scratch-resistant glass, oleophobic coating Dolby Vision Wide color gamut True-tone PLATFORM OS iOS 14 Chipset Apple A14 Bionic (5 nm) CPU Hexa-core GPU Apple GPU (4-core graphics) MEMORY Card slot No Internal 128GB 6GB RAM, 256GB 6GB RAM, 512GB 6GB RAM NVMe MAIN Quad 12 MP, f/1.6, 26mm (wide), 1.4µm, dual pixel PDAF, OIS CAMERA 12 MP, f/2.0, 52mm (telephoto), 1/3.4", 1.0µm, PDAF, OIS, 2x optical zoom 12 MP, f/2.4, 120˚, 13mm (ultrawide), 1/3.6" TOF 3D LiDAR scanner (depth) Features Quad-LED dual-tone flash, HDR (photo/panorama) Video 4K@24/30/60fps, 1080p@30/60/120/240fps, 10-bit HDR, Dolby Vision HDR (up to 60fps), stereo sound rec.
    [Show full text]