0
Applied Imagery Pattern Recognition
Workshop49th Annual IEEE AIPR 2020 Trusted Computing, Privacy, and Securing Multimedia
Washington DC. (On-line) 13-15 October, 2020 1 2020 Applied Imagery Pattern Recognition Workshop
AIPR is sponsored by:
2 2020 Applied Imagery Pattern Recognition Workshop
Table of Contents
Schedule At-a-Glance ...... 3 Welcome ...... 4 AIPR Executive Committee ...... 5 Guest Speakers ...... 5 Schedule ...... 15 Tuesday, October 13 ...... 16 Session I AI and Cyber Physical ...... 16 Session II Security with IoT/Cloud/BlockChains ...... 16 Wednesday, October 14 ...... 18 Session III Deep Learning for Multimedia Applications ...... 18 Session IV Geospatial Analytics and Computer Vision ...... 18 Thursday, October 15 ...... 20 Session V BioMedical / Emerging Areas ...... 20 Abstracts ...... 21
3 2020 Applied Imagery Pattern Recognition Workshop
4 2020 Applied Imagery Pattern Recognition Workshop
Welcome
Welcome and thank you for attending the AIPR 2020 Workshop. “Trusted Computing, Privacy, and Securing Multimedia” are key concerns in the unpredictable year that 2020 has been. The on-going COVID-19 pandemic has not only changed the way we work and interact but has also shown the importance of securing our cyber infrastructure. New technologies such as 5G, Internet of Things (IoT), and cloud have become everyday terms. Deep fakes, security of cyber infrastructure, and social media manipulation are key concerns for reliance on the public networks. The shift to work-at-home and school-at-home have tested the resiliency and security of the multimedia systems and networks. Rather than fail, we have seen success in terms of innovations and the society’s ability to adapt to new situations and technologies. We believe that the topics of security, image detection and manipulation are very relevant to the challenges we are facing today across the world. We hope that you and your families are safe in this time of uncertainty.
The Trusted Computing, Privacy, and Securing Multimedia domains represented at the IEEE AIPR 2020 include artificial intelligence, cyber physical, security, deep learning, geo-spatial analytics, and applications including national security and biomedical. We highly encourage attendees to interact with speakers across different domains to potentially learn about and explore new ideas and techniques that they may not have encountered or fully appreciated in their own domains. Additionally, please provide us feedback on the topics, organization, and future interest in Applied Imagery Pattern Recognition.
We hope you enjoy your experience at virtual AIPR 2020!
Prasad Calyam (University of Missouri) and Jon Rolf (NSA) | Program Co-Chairs
Kannappan Palaniappan (University of Missouri) and Travis Axtell (Ball Aerospace)| Conference Chairs
5 2020 Applied Imagery Pattern Recognition Workshop
AIPR Executive Committee
Chair: K. Palaniappan, University of Missouri-Columbia Program Chairs for 2020: Prasad Calyam, University of Missouri-Columbia, Jon Rolf, National Security Agency Secretary: Carlos Maraviglia, NRL Treasurer: James Aanstoos, Mississippi State University Emeritus Local Arrangements: Donald J. Gerson, Gerson Photography Publicity: Peter Costianes, AFRL, Emeritus Web Master: Charles J. Cohen, Cybernet External Support: John Irvine, MITRE Registration Chair: Jeff Kretsch, Raytheon BBN Technologies Proceedings Chair: Franklin Tanner, Raytheon Student Paper Award Chair: Paul Cerkez, Coastal Carolina University
Members:
James Aanstoos, Mississippi State University Emeritus Murray H. Loew, GWU Travis Axtell, Ball Aerospace Carlos Maraviglia, NRL Bob Bonneau, AFOSR Paul McCarley, AFRL - Eglin AFB Christoph Borel-Donohue, Army Research Laboratory Daniela Moody, Ursa Prasad Calyam, University of Missouri Abhishek Murthy, Signify John Caulfield, Cyan Systems Carsten Oertel, Mitre Paul Cerkez, Coastal Carolina K. Palaniappan, University of Missouri Charles Cohen, Cybernet Robert Pless, GWU Peter Costianes, AFRL, Emeritus Mark Pritt, Lockheed Martin Peter Doucette, US Geological Survey Mike Pusateri, LSI Corporation Donald J. Gerson, Gerson Photography Katie Rainey, Naval Information Warfare Center Pacific Neelam Gupta, Army Research Lab Raghuveer Rao, Army Research Lab Mark Happel, Johns Hopkins University Jon Rolf, NSA John Irvine, MITRE Guna Seetharaman, NRL Steve Israel, Draper Alan Schaum, NRL Andrew Kalukin, NGA Franklin Tanner, Raytheon Kevin T. Kornegay, Morgan State University Karl Walli, USAF (Retired) Michael D. Kelly, IKCS Jeff Kretsch, Raytheon BBN Technologies (retired)
Emeritus: In Memoriam: Larry Davis Larry Clarke Robert Haralick Michael Hord Joan Lurie Heidi Jacobus Robert Mericsko David Schaefer William Oliver Elmer "Al" Williams J. Michael Selander
5 2020 Applied Imagery Pattern Recognition Workshop
Guest Speakers
6 2019 Applied Imagery Pattern Recognition Workshop
Keynote Speaker Securing Cyber-Physical and IoT Systems in Smart Living Environments
Dr. Sajal K. Das, whose academic genealogy includes Thomas Alva Edison, is a professor of Computer Science and the Daniel St. Clair Endowed Chair at Missouri University of Science and Technology, where he was the Chair of Computer Science during 2013-2017. Prior to 2013, he was a University Distinguished Scholar Professor of Computer Science and Engineering, and founding director of the Center for Research in Wireless Mobility and Networking at the University of Texas at Arlington. During 2008-2011, Dr. Das served the National Science Foundation as a Program Director in the Computer and Network Systems Division. His research interests include wireless sensor networks, mobile and pervasive computing, smart environments (smart city, smart grid, smart healthcare), cyber-physical systems; IoT, crowdsensing, cloud computing, security and trustworthiness, social and biological networks, and applied graph theory and game theory. He has contributed significantly to these areas, having published 300+ research articles in high quality journals and 400+ papers in peer-reviewed conferences, and 52 book chapters. A holder of 5 US patents, Dr. Das has directed numerous funded projects totaling over $16 million and coauthored four books – Smart Environments: Technology, Protocols, and Applications (John Wiley, 2005); Handbook on Securing Cyber-Physical Critical Infrastructure: Foundations and Challenges (Morgan Kaufman, 2012); Mobile Agents in Distributed Computing and Networking (Wiley, 2012); and Principles of Cyber-Physical Systems: An Interdisciplinary Approach (Cambridge University Press, 2020). According to DBLP, Dr. Das is one of the most prolific authors in computer science. His h- index is 86 with 33,000+ citations according to Google Scholar. He is the founding Editor-in-Chief of Elsevier’s Pervasive and Mobile Computing journal, and serves as an Associate Editor of several journals including the IEEE Transactions on Mobile Computing, IEEE Transactions on Dependable and Secure Computing, and ACM Transactions on Sensor Networks. A founder of IEEE PerCom, WoWMoM, SMARTCOMP and ICDCN conferences, Dr. Das served as General and Program Chair of numerous conferences. He is a recipient of 10 Best Paper Awards in prestigious conferences, and numerous awards for teaching, mentoring and research including IEEE Computer Society’s Technical Achievement award for pioneering contributions to sensor networks and mobile computing, and University of Missouri System President’s Award for Sustained Career Excellence. He graduated 43 PhD, 32 MS thesis students, and 9 postdoctoral fellows. Dr. Das is an IEEE Fellow.
Abstract: Our daily lives are becoming increasingly dependent on a variety of smart cyber-physical infrastructures, such as smart cities and buildings, smart energy grid, smart transportation, smart healthcare, etc. Alongside, smartphones and sensor-based IoTs are empowering humans with fine- grained information and opinion collection through crowdsensing about events of interest, resulting in actionable inferences and decisions. This synergy has led to the cyber-physical-social (CPS) convergence with human in the loop, the goal of which is to improve the “quality” of life. However, CPS and IoT systems are extremely vulnerable to failures, attacks and security threats. This talk will highlight unique research challenges in securing such systems, followed by novel defense mechanisms. Our proposed frameworks and solutions are based on a rich set of theoretical and practical design principles, such as secure data fusion, uncertainty reasoning, information theory, prospect theory, reputation scoring, and belief and trust models. Two case studies will be considered: (1) Security forensics and lightweight statistical anomaly detection in the smart grid CPS to defend against organized and persistent adversaries that can launch data falsification attacks on the smart meters using stealthy 7 2020 Applied Imagery Pattern Recognition Workshop
strategies. The novelty of our approach lies in a newly defined information-theoretic metric that helps quantify robustness and security, thus minimizing the attacker’s impact on the customers and utilities with low false alarm rates; (2) Secure and trustworthy decision making in mobile crowd sensing to detect false (or spam) contributions due to selfish and malicious behavior of users. Based on the cumulative prospect theory and reputation/trust model, our approach prevents revenue loss owing to undue incentives and improves the operational reliability and decision accuracy. The talk will be concluded with directions for future research.
Keynote Speaker
Evasion Attacks against Machine Learning in Practical Settings
Lujo Bauer is a Professor of Electrical and Computer Engineering, and of Computer Science, at Carnegie Mellon University. He received his B.S. in Computer Science from Yale University in 1997 and his Ph.D., also in Computer Science, from Princeton University in 2003. Dr. Bauer is a member of CyLab, Carnegie Mellon's computer security and privacy institute, and serves as the director of CyLab's Cyber Autonomy Research Center. Dr. Bauer's research examines many aspects of computer security and privacy, including developing high-assurance access-control systems, building systems in which usability and security co-exist, and designing practical tools for identifying software vulnerabilities. His recent work focuses on developing tools and guidance to help users stay safer online and on examining how advances in machine learning can (or might not) lead to a more secure future. Dr. Bauer served as the program chair for the flagship computer security conferences of the IEEE (S&P 2015) and the Internet Society (NDSS 2014) and is an associate editor of ACM Transactions on Privacy and Security.
Abstract: Much research over the past decade has shown that machine learning algorithms are susceptible to adversarial examples---carefully crafted, minimal perturbations to training-time inputs that lead to misclassification at test time. The majority of such research, however, has been carried out with toy datasets such as MNIST and without consideration for practical constraints that need to be overcome when attacking a real-world system. In this talk I'll examine two real-world uses of machine learning algorithms---for face recognition and for malware classification. I'll describe the constraints that attacks on these systems would have to overcome in practice, and I'll show that overcoming these constraints is, unfortunately, well within attackers' capabilities.
Keynote Speaker
AERPAW: A Unique Research Platform for Advanced Wireless Communications and Networking
Rudra Dutta received a B.E. in Electrical Engineering from Jadavpur University, Kolkata, India, in 1991, a M.E. in Systems Science and Automation from Indian Institute of Science, Bangalore, India in 1993, and a Ph.D. in Computer Science from North Carolina State University, Raleigh, USA, in 2001. From 1993 to 1997 he worked for IBM as a software developer and programmer in various networking related projects. He has been employed from 2001 - 2007 as Assistant Professor, from 2007 - 2013 as Associate Professor, and since 2013 as Professor, in the 8 2019 Applied Imagery Pattern Recognition Workshop department of Computer Science at the North Carolina State University, Raleigh. As of Fall, 2018, he is serving as Associate Department Head on an interim basis. His current research interests focus on design and performance optimization of large networking systems, Internet architecture, wireless networks, and network analytics. He is a senior member of IEEE and a distinguished member (distinguished engineer) of ACM. His work has been supported by grants from the National Science Foundation, the Army Research Office, the National Security Agency, and industry, most recently including a PAWR grant from NSF (the AERPAW project). He has served as a reviewer for many premium journals, on NSF, DoE, ARO, and NSERC (Canada) review panels, as part of the organizing committee of many premium conferences, most recently as General Co-Chair for the IEEE Sarnoff Symposium in 2019. He previously served on the editorial boards of the Elsevier Journal of Optical Switching and Networking, and the Springer Photonic Communication Networks journal, for several years, and is currently serving as Program Co-Chair of the Optical Networking Symposium at IEEE Globecom 2021.
Abstract: The Aerial Experimentation and Research Platform for Advanced Wireless (AERPAW) facility, in early design and construction at the Centennial Campus of North Carolina State University and adjoining areas in the City of Raleigh and the Town of Cary, is planned to be a wireless testbed embedded in the real world, initially spanning an area of a few square miles that contains urban, suburban, and agricultural landforms. The experimental facilities will consist of a variety of radio resources (including programmable software-defined radios), installed semi-permanently at over 30 locations, and over 20 portable nodes, with the portable nodes being mountable on a variety of vehicles, including ground and aerial robots whose trajectory can be programmatically controlled. We expect that it will enable a rich variety of wireless research experiments with a high degree of realism, many of which would be impossible without such a facility. This talk will briefly describe the architecture and design of AERPAW, and the experiment scenarios that we envision AERPAW will be able to house.
Keynote Speaker
FABRIC: More than Just a Pretty Testbed
Anita Nikolich is a Research Scientist at UIUC, Cyber Policy Fellow at the University of Chicago, Co-Director of the DEFCON AI Village and a security consultant at a cryptocurrency exchange.
Abstract: FABRIC creates a unique research infrastructure to enable cutting-edge, at-scale research in networking, cybersecurity, distributed computing and storage, machine learning and AI. We are crafting a rich tapestry of ‘everywhere-programmable’ global nodes equipped with large amounts of network, compute and storage, interconnected by high speed, dedicated optical links connecting FABRIC to specialized testbeds (5G/IoT PAWR, NSF Clouds), HPC, science facilities and the Internet. I’ll give a high-level introduction to FABRIC and some thoughts about the types of ‘out of the box’ security and machine learning experiments we’d like the community to consider.
9 2020 Applied Imagery Pattern Recognition Workshop
Keynote Speaker Security and Privacy for Distributed Optimization and Learning
Nitin Vaidya is the McDevitt Chair of Computer Science at Georgetown University. He received his Ph.D. from the University of Massachusetts at Amherst. He previously served as a Professor and Associate Head in Electrical and Computer Engineering at the University of Illinois at Urbana-Champaign. He has co-authored papers that received awards at several conferences, including 2015 SSS, 2007 ACM MobiHoc and 1998 ACM MobiCom. He is a fellow of the IEEE. He has served as the Chair of the Steering Committee for the ACM PODC conference, as the Editor-in-Chief for the IEEE Transactions on Mobile Computing, and as the Editor-in-Chief for ACM SIGMOBILE publication MC2R.
Abstract: Consider a network of agents wherein each agent has a private cost function. In the context of distributed machine learning, the private cost function of an agent may represent the “loss function” corresponding to the agent’s local data. The objective here is to identify parameters that minimize the total cost over all the agents. In machine learning for classification, the cost function is designed such that minimizing the cost function should result in model parameters that achieve higher accuracy of classification. Similar optimization problems arise in the context of other applications as well. Our work addresses privacy and security of distributed optimization with applications to machine learning. In privacy-preserving machine learning, the goal is to optimize the model parameters correctly while preserving the privacy of each agent’s local data. In security, the goal is to identify the model parameters correctly while tolerating adversarial agents that may be supplying incorrect information. When a large number of agents participate in distributed optimization, security compromise or failure of some of the agents becomes increasingly likely. The talk will provide intuition behind the design and correctness of the algorithms.
Keynote Speaker Trusted Computing, Privacy, and Securing Multimedia: A Perspective from the National Science Foundation (NSF)
M Prabha (Balakrishnan) Prabhakaran is currently a Program Director in the Human Centered Computing (HCC) program of Information and Intelligent Systems (IIS) Division of the CISE (Computer and Information Science and Engineering) Directorate of the National Science Foundation (NSF). He is also involved with Secure and Trustworthy Computing as well as other programs such as Fairness in Artificial Intelligence, Future of Work. He is also a Professor in the faculty of Computer Science Department, University of Texas at Dallas. Dr. Prabhakaran received the prestigious NSF CAREER Award FY 2003 for his proposal on Animation Databases. He was selected as an ACM Distinguished Scientist in 2011 and is currently an IEEE Senior Member. He is an Associate Editor of IEEE Transactions on Multimedia. He is Member of the Editorial board of Multimedia Systems Journal (Springer), Multimedia Tools and Applications journal (Springer), and other multimedia systems journals. He received the Best Associate Editor for 2015, from Springer’s Multimedia Systems Journal. Dr Prabhakaran is a Member of the Executive Council of the ACM Special Interest Group on Multimedia (SIGMM) and is the Co-Chair of IEEE Technical Committee on Multimedia Computing (TCMC) Special 10 2019 Applied Imagery Pattern Recognition Workshop
Interest Group on Video Analytics (SIGVA). Dr. Prabhakaran served the General Co-Chair of the IEEE International Conference on Health Informatics (ICHI) 2015. He was also a General Co-Chair of ACM International Conference on Multimedia Retrieval 2013 (ICMR 2013), IEEE Haptic, Audio, and Visual Environments (HAVE) 2014, a General Co-chair of ACM Multimedia 2011, and ACM Multimedia and Security (MM&Sec) 2007. Prof Prabhakaran's research has been funded by Federal Agencies such as the National Science Foundation (NSF), USA Army Research Office (ARO), and the US-IGNITE Program, apart from industries and consortiums.
Abstract: This talk starts off with a brief background on Prabhakaran’s personal research in multimedia media forensics, using watermarking and other related approaches using machine learning. He primarily dealt with tamper proofing and authentication of 3D data: 3D meshes, 3D point clouds, and 3D human motion capture data. Based on these research results, Prabhakaran and his research team designed ALERT (Authentication, Localization, and Estimation of Risks and Threats), as a secure layer in the decision support system used in the navigation control of vehicles and robots. His current research includes exploration of deep learning techniques for tamper detection in 3D data. With this research background, Prabhakaran has been involved National Science Foundation (NSF) programs such as Human Centered Computing (HCC) and Secure and Trustworthy Computing (SaTC). He is also involved with Fairness in Artificial Intelligence (AI), Future of Work, Smart and Connected Communities. The talk will provide an overview of NSF’s leadership in AI and NSF CISE (Computer and Information Science and Engineering) Directorate’s activities on Secure and Trustworthy Computing.
Special Sessions and Tutorials
Tutorial Title: Approaches for Multi-modal Synthetic Media Detection Tutorial Abstract: The proposed IEEE AIPR tutorial will include a 90 minute professional development course on approaches for multi-modal synthetic media detection. The course will provide the opportunity for participants to build a deep learning pipeline to include data pre-processing, neural network design, training methodology and algorithm benchmarking. Participants will be working in the Python programming language, and will be using open source modules including OpenCV, SciKit-learn, Pandas, and the PyTorch deep learning framework. This tutorial will cover some basic concepts of data engineering, data science, machine learning and deep learning. Participants will gain insights into issues specific to these technologies via interactive exercises as they relate to synthetic media detection with a specific case study applied to the Facebook Deepfake Detection Challenge. Tutorial Presenter Bios:
Zigfried Hampel-Arias is a Data Scientist at IQT Labs. Zig’s research interests include generative modeling, quantum computing and contributing to the open source community as an author and reviewer. Prior to joining IQT Labs, Zig spent a decade as an astrophysicist, traveling around the Americas to work on remote, high-altitude experiments. In his spare time, Zig enjoys exploring the desert, kayaking on northern lakes, and learning new languages on Duolingo. 11 2020 Applied Imagery Pattern Recognition Workshop
Michael Lomnitz is a Senior Data Scientist at IQT Labs, focusing on applied research to help the Intelligence Community better understand technology challenges. He has worked on a series of projects applying deep learning for tasks in computer vision, automatic speech recognition, speaker verification, image compression and encryption. Prior to his career as a researcher in A.I., Michael studied the big bang as an experimental nuclear physicist at the Relativistic Heavy Ion Collider in Long Island NY.
Special Session
Title: Future of Healthcare Policy and Regulation in the Digital Age Session Abstract: The proposed IEEE AIPR special session will include a 90-minute agenda that provides for expert speakers and a panel discussion. The special session will allow participants to focus on healthcare issues, privacy, current regulation and policy, and the system's future to protect and utilize health data. The topics covered in the session include blockchain for data security in healthcare and resolving the disputes. Participants will gain insights into issues related to the impact of health data privacy in the world around healthcare, including rare diseases, databases, risks/consequences of health data used in social media, policy, data ownership, and HIPAA compliance. Moderator Bio:
Monica Weldon President/CEO/Founder Consultant/NWU Pritzker MSL Law Student Bridge the Gap – SYNGAP Education and Research Foundation
BTG Email: [email protected]
NWUEmail: [email protected]
MW Consulting: [email protected]
LinkedIn: https://www.linkedin.com/in/monica-weldon/ 12 2019 Applied Imagery Pattern Recognition Workshop
In November of 2012, Monica's twin son, Beckett, was the first to be diagnosed at Texas Children's Genetics Clinic with the gene mutation SYNGAP1 (6p21.3). When Beckett was 4 months old, she noticed he was not meeting the same milestones as his twin sister. Monica then began a journey to find answers to help her son. She started to blog about his progress and this led to building a community of parents and caregivers that are now a strong support group. She is the Founder and President/CEO of Bridge the Gap – SYNGAP Education and Research Foundation. It is her passion to help support these families by raising awareness and creating a strong foundation that will accelerate a path to better therapies. She retired in 2016 after 23 years in education teaching secondary science. Her new focus is on building the programs and mission of Bridge the Gap – SYNGAP Education and Research Foundation. She is the Primary Investigator on the SYNGAP1 (MRD5) Registry and Natural History Study. She is a life member of the Worldwide Association of Female Professionals and a member of the first class of 2017 Illumina Ambassadors established in the United States. In addition to leading the foundation, she is an author, public speaker, consultant on rare disease business strategies and advocates for rare disease legislation at both the federal and state levels. Several of her authored scientific publications include Nature Neuroscience, The Journal of Neurodevelopmental Disorders, and The Journal of Pediatrics. She has authored a book about her son Beckett’s diagnostic journey called “Slow Moving Stream - My Special Boy”. She is a graduate from East Texas Baptist University with a Bachelor's of Science in Biology/Psychology (1991) and Secondary Certification in Education (1995). She will be attending Northwestern University Pritzker School of Law in the Fall of 2020 to earn her Masters in Science Law. She has five beautiful children, Haleigh (28), Taylor, USMC (25), Sawyer (23), and the twins Beckett & Pyper (12). She was a Global Genes 2015 & 2016 RARE Champion in Advocacy Award nominee for her work in the rare disease community and Wego Health Awards Nominee for years 2015, 2017, 2018, & 2019. 2020 Global Shakers Rare Disease Champion.
Panelists Bio:
Candace Lerman, Esq. Lerman Law Firm – Lawyer of Counsel Consultant/Patient Advocate
Rare Candace Email: [email protected]
Law Firm Email (FL Bar): [email protected]
LinkedIn: https://www.linkedin.com/in/candace-lerman-esq/ Candace Lerman, Esq., is an attorney, consultant, and author of the rare disease blog, RareCandace.com. She earned her Juris Doctorate with a concentration in Health Law at the Shepard 13 2020 Applied Imagery Pattern Recognition Workshop
Broad School of Law at Nova Southeastern University and a certificate in Healthcare Compliance from Seton Hall University School of Law. In addition, she holds a Bachelor’s and Master’s degree in International Affairs from Florida State University. Candace researched and repurposed a drug to put her rare blood disorder into remission twice! Inspired by her struggle of living with Immune Thrombocytopenia, she launched the ITP Patient Driven Research Initiative, which works with multiple stakeholders to improve treatments for the ITP community. She works closely with members of Congress on federal healthcare policy and can often be found on Capitol Hill. Candace is Of Counsel to the Lerman Law Firm, assisting clients in healthcare, compliance, and corporate law. Besides practicing law, Candace consults on projects for pharmaceutical companies and non-profit organizations on topics ranging from patient registries, natural history studies, clinical trial design, patient engagement, Real World Evidence, and patient assistance programs. Coming from a diverse background as both an attorney and patient, she provides unique insights while breaking down the silos that plague drug development.
Andrea Downing Title: Co-Founder, The Light Collective
Email: [email protected]
LinkedIn: https://www.linkedin.com/in/andreadowning/ Andrea Downing is a BRCA Community Data Organizer and an ePatient security researcher. In 2018, Andrea discovered a security vulnerability which affected the privacy and safety of all closed groups on Facebook and launched a congressional inquiry. Her work has been featured on CNN, Fortune, and The Verge, and has catalyzed an urgent dialogue on national health privacy policy and the need for protections outside of HIPAA. Andrea has co-founded a nonprofit called The Light Collective to work with vulnerable patient groups seeking digital rights and safe spaces for patient support communities on social media.
14 2019 Applied Imagery Pattern Recognition Workshop
Michael Mittelman, MBA Healthcare Consultant Cybersecurity, Digital Health Executive
Email: [email protected]
LinkedIn: https://www.linkedin.com/in/michael-mittelman/ Michael Mittelman works in healthcare access, and patient and family inclusion with a special focus on cybersecurity, data governance, patient data sharing, and patient privacy. He leverages his own 30 plus years of lived experience as a rare disease patient and 3 time kidney transplant recipient. He has formal training in human behavior, healthcare management, human-centered design, and cybersecurity. He has not only been the CEO of a digital health company, but he has served as the Patient Editor at The BMJ, a DoD Consumer Reviewer, a Stanford MedX e-Patient Scholar, and Chairman of The American Living Organ Donor Fund. He is also a PCORI Ambassador and a special advisor to the UNOS Board of Directors.
15 2020 Applied Imagery Pattern Recognition Workshop
Schedule NOTE: All times for sessions noted in the Program are in EDT Time Zone.
16 2019 Applied Imagery Pattern Recognition Workshop
Tuesday, October 13 8:45a Welcome and Workshop Overview
9:00a Keynote: Securing Cyber-Physical and IoT Systems in Smart Living Environments Dr. Sajal K. Das, Professor and Daniel St. Clair Endowed Chair, Department of Computer Science Missouri University of Science and Technology, USA
Session I AI and Cyber Physical – Chair: K. Palaniappan 9:50a T50: Using models of cortical development based on sparse coding to discriminate between real and synthetically generated faces Nga T Nguyen (Los Alamos National Laboratory)*; Juston Moore (Los Alamos National Laboratory); Garrett Kenyon (Los Alamos National Laboratory)
10:10a T56: Transductive Learning with Graphical Models: Characterizing Adversarial Data in the Wild Diane Oyen (LANL)*
10:30a T35: IC-ChipNet: Deep Embedding Learning for Fine-grained Retrieval, Recognition, and Verification of Microelectronic Images Md Alimoor Reza (Indiana University)*; David Crandall (Indiana University)
10:50a T59: Efficient Training of Object Detection Models for Automated Target Recognition Charlene Cuellar Vite (Lockheed Martin)*; Gary Chern (Lockheed Martin); Austen Groener (Lockheed Martin); Mark Pritt (Lockheed Martin)
11:10a T13: Occupancy Detection for Smart HVAC Efficiency in Building Energy: A Deep Learning Neural Network Framework using Thermal Imagery Yaa T. Acquaah (North Carolina Agricultural and Technical State University)*; Raymond Tesiero (North Carolina Agricultural and Technical State University); Balakrishna Gokaraju (North Carolina Agricultural and Technical State University); Jonathan Steele (North Carolina Agricultural and Technical State University); Gregory Monty (North Carolina A&T state University)
11:30a T15: Understanding the ability of deep neural networks to detect connected components in images SHUYUE GUAN (George Washington University); Murray H Loew (George Washington University)*
11:50a T61: Improving Robustness to Audio Perturbations Using Biological Receptive Fields Ben Migliori (Los Alamos National Laboratory); Derek Zhang (University of Maryland)*
12:10p Lunch Break
Session II Security with IoT/Cloud/BlockChain – Chair: Kevin Kornegay 1:00p Keynote: Evasion Attacks against Machine Learning in Practical Settings Lujo Bauer, Professor of Electrical and Computer Engineering, and of Computer Science, at Carnegie Mellon University
1:50p T12: Enhancing Network-edge Connectivity Security in Drone Video Analytics Alicia Esquivel (University of Missouri); Prasad Calyam (University of Missouri-Columbia)*; Deniz Kavzak Ufuktepe (University of Missouri-Columbia); Chengyi Qu (University of Missouri-Columbia)
2:10p T19: A Framework for Public Blockchain-based Trustworthy Federated Learning Muhammad Habib ur Rehman (Khalifa University of Science and Technology)*; Khaled Salah (Khalifa University of Science and Technology ); Ernesto Damiani (Khalifa University); Davor Svetinović (Khalifa University of 17 2020 Applied Imagery Pattern Recognition Workshop
Science and Technology)
2:30p T25: Mitigating Security Risk of IoT with A Blockchain-Based Solution Soundararajan Ezekiel (IUP)*
2:50p T39: Autonomous Navigation Assurance with Explainable AI and Security Monitoring Denzel Hamilton (Morgan State University)*; Lanier Watkins (Johns Hopkins University); Kevin Kornegay (Morgan State University); Denzel Hamilton (Morgan State University)
3:10p T60: Optimizing Deep Learning Based Intrusion Detection Systems Defense Against White-Box and Backdoor Adversarial Attacks Through a Genetic Algorithm Khaled Alrawashdeh (University of Cincinnati)*
3:30p T66: Black-box Adversarial Attacks in Autonomous Vehicles K. Naveen Kumar (Indian Institute of Technology Hyderabad); Reshmi Mitra (Southeast Missouri State University)*; C. Krishna Mohan (Indian Institute of Technology Hyderabad)
4:00p Poster Session I - P4: Detection of unknown maneuverability hazards in low-altitude UAS color imagery using linear features - Jeffrey J Dale (University of Missouri)*; Grant Scott (University of Missouri); James Keller (University of Missouri, Columbia, USA) - P5: Deep Convolutional Neural Network Ensembles for Improved Malaria Parasite Detection - Hussin K Ragb (Christian Brothers University)*; Ian Dover (Christian Brothers University); Redha Ali (University of Dayton
- P7: A Review on Image Dehazing Algorithms for Vision based Applications in Outdoor Environment - Teena Sharma (Indian Institute of Technology Kanpur, India)*; Tejashwani Shah (Indian Institute of Technology Kanpur, India); Dr. Nishchal K Verma (Indian Institute of Technology Kanpur, India); Shantaram Vasikarla (California State University, Northridge, CA)
- P11: Shape and Texture Aware Graph Processing - Michael J Reale (SUNY Polytechnic Institute)*; MARIA CORNACCHIA (AFRL); Micah Church (SUNY Polytechnic Institute); Nicholas Goutermout (SUNY Polytechnic Institute); John Rubado (SUNY Polytechnic Institute); Matthew Ochrym (SUNY Polytechnic Institute) - P16: DeepFakes Detection in Videos using Feature Engineering Techniques in Deep Learning Convolution Neural Network Frameworks - Balakrishna Gokaraju (University of West Alabama)*; Balakrishna Gokaraju (North Carolina Agricultural and Technical State University); Kaushik Roy (North Carolina A&T State University); Sonya Burroughs (North Carolina A&T State Univ.) - P17: Improving Anomaly Based Intrusion Detection System for IoT by Exploring Multi-Class Machine Learning Techniques - Balakrishna Gokaraju (University of West Alabama)*; Balakrishna Gokaraju (North Carolina Agricultural and Technical State University); Kaushik Roy (North Carolina A&T State University); Xiaohong Yuan (North Carolina A&T State University); addison shaver ( North Carolina A&T State University); Zhipeng Liu ( North Carolina A&T State University) - P20: Interactive Graphical Visualization of Deep Learning Models - Akhil Vyas (University of Illinois Urbana-Champaign); Prasad Calyam (University of Missouri-Columbia)* - P30: Analyzing Information Flow in the IoT Network for Threat Detection Using DCNN and Clustering Techniques - SOUNDARARAJAN EZEKIEL (IUP)*; Laurent Njilla (Air Force Research Lab)
- P31: A Framework for Hierarchical Detection in IoT Networks - SOUNDARARAJAN EZEKIEL (IUP)*; Laurent Njilla (Air Force Research Lab) 18 2019 Applied Imagery Pattern Recognition Workshop
Wednesday, October 16 9:00a Keynote: AERPAW: A Unique Research Platform for Advanced Wireless Communications and Networking Rudra Dutta, Professor, North Carolina State University Session III Deep Learning for Multimedia Applications – Chair: John Irvine 9:50a T2: Tamper-Proofing Imagery from Distributed Sensors using Learned Blockchain Consensus Alexander D Wissner-Gross (Harvard University)*; Jared Willard (U.S. Army Research Laboratory); Noah Weston (US ARMY CERDEC)
10:10a T6: Spatiotemporal Maneuverability Hazard Analytics from Low-Altitude UAS Sensors Trevor M Bajkowski (University of Missouri - Columbia); David Huangal (University of Missouri - Columbia); J. Alex Hurt (University of Missouri - Columbia); Jeffrey J Dale (University of Missouri); James Keller (University of Missouri, Columbia, USA); Grant Scott (University of Missouri)*; Stanton Price (U.S. Army Engineer Research and Development Center)
10:30a T18: Multimodal Approach for the DeepFake Detection Challenge Michael Lomnitz (IQT Labs)*; Zigfried Hampel-Arias (In-Q-Tel); Vishal Sandesara (IQT Labs)
10:50a T23: SMVNet: Deep Learning Architectures for Accurate and Robust Multi-View Stereopsis Shizeng Yao (University of Missouri - Columbia)*
11:10a T55: Evaluation of Different Data and Decision Fusion Mechanisms for Robust Moving Object Detection Gani Rahmon (University of Missouri-Columbia)*; Filiz Bunyak (University of Missouri-Columbia); Kannappan Palaniappan (University of Missouri); Guna Seetharaman (US Naval Research Laboratory)
11:30a T41: MACHINE LEARNING ON SMALL UAVS Steven Israel (Draper Laboratory)*; Alison Cleary (Draper Laboratory)
11:50a T71: Multi-modal Data Analysis and Fusion for Robust Object Detection in 2D/3D Sensing Jonathan Schierl (University of Dayton)*; Theus H Aspiras (University of Dayton); Vijayan K Asari (University of Dayton); Quinn Graehling (University of Dayton); Andre Van Rynbach (Air Force Research Laboratory); David Rabb (Air Force Research Laboratory)
12:10p Lunch Break
1:00p Keynote: FABRIC: More than Just a Pretty Testbed Anita Nikolich – Research Scientist at UIUC, Cyber Policy Fellow at the University of Chicago
Session IV Geospatial Analytics and Computer Vision – Chair: Prasad Calyam 1:50p T36: Video streaming in ad hoc networks Andrei Sukhov (National Research University Higher School of Economics)*; Evgeny Sagatov (Samara National Research University); Aleksei Gladkii (Samara National Research University)
2:10p T38: Aiding Material Design Through Machine Learning Stanton Price (U.S. Army Engineer Research and Development Center)*; Christina Young ( U.S. Army Engineer Research and Development Center); Matthew Maschmann (University of Missouri-Columbia); Steven Price (Mississippi College)
2:30p T58: Methods of Exploiting Multispectral Imagery for the Monitoring of Illicit Coca Fields Emily E Berkson (Lockheed Martin)*; Charlene Cuellar Vite (Lockheed Martin); Austen Groener (Lockheed Martin); Michael Harner (Lockheed Martin); Stephen O'Neill (Lockheed Martin); Tyler Bartelmo (Lockheed Martin) 19 2020 Applied Imagery Pattern Recognition Workshop
2:50p T62: Person-Deidentification Versus Reidentification: A Novel Approach Towards Performance Evaluation of Privacy Enhancements in Images Ankur Chattopadhyay (Northern Kentucky University)*; Levi Pfantz (University of Wisconsin -White Water); Robert Ruska (Northern Kentucky University)
3:10p T68: Machine Learning Architecture and Image Quality Steven Israel (Draper Laboratory)*; John Irvine (Mitre); Steven Tate (Draper)
3:30p T69: Adversarial Examples in Deep Learning Regression Models for Multivariate Time Series Forecasting Gautam Raj mode (University of Missouri); Khaza Anuarul Hoque (University of Missouri)*
4:00p Poster Session II - P34: Real-Time Traffic Management System using an Object-detection based Signal Logic - Murthy V Jonnalagedda (Jawaharlal Nehru Technological University Kakinada)*; Sashank Taduri (Kakinada Smart City Corporation Ltd.,); Rachana Reddy Dereddy (VIT Vellore )
- P44: Determining the Impact of Cybersecurity Failures During and Attributable to Pandemics and other Emergency Situations - Maria Clare Lusardi (University of Missouri); Isaac Dubovoy (North Dakota State University); Jeremy Straub (Department of Computer Science North Dakota State University)*
- P47: A Semi-automatic System for Moving Objects Annotation using Deep Detections and Multi-object Tracking Techniques- NOOR M AL-SHAKARJI (University of Missouri Columbia)*; Filiz Bunyak (University of Missouri-Columbia); Hadi Aliakbarpour (University of Missouri-Columbia); Guna Seetharaman (Naval Research Laboratory); Kannappan Palaniappan (University of Missouri)
- P49: Monitoring and Detection Time Optimization of Man in the Middle Attacks using Machine Learning - Otily Toutsop (Morgan State University)*; Paige Harvey (Morgan State University); Kevin Kornegay (Morgan State University)
- P53: Autonomous Navigation Under Adversarial Attack - Md Tanvir Arafin (Morgan State University)*; Kevin Kornegay (Morgan State University)
- P64: A Hybrid Approach for Human Activity Recognition with Support Vector Machine and 1D Convolutional Neural Network - Md Maruf Hossain Shuvo (University of Missouri-Columbia)*; Nafis Ahmed (University of Missouri-Columbia); Rohit Nouduri (University of Missouri-Columbia); Kannappan Palaniappan (University of Missouri) - P65: Benchmark and Evaluation of Low Resource Object Detection in Biomedical Images - Sanjoy Kundu (Oklahoma State University); Nikhil Gunti (Oklahoma State University); Bailey Hendrickson (Oklahoma State University); Sunil More (Oklahoma State University); Sathya Narasimhan (OK State)*
- P70: RefinedFed: A Refining Algorithm for Federated Learning - Mohamed Gharibi (University of Missouri-Kansas City)*; Praveen Rao (University of Missouri-Columbia)
- P54: Investigating Global Mosaic Generation from Mesentery Video Sequences using Deep Learning - Rumana Aktar (University of Missouri-Columbia)*; Jaired Collins (University of Missouri- Columbia); Giovanna Guidoboni (University of Missouri); Virginia Huxley (University of Missouri- Columbia); Kannappan Palaniappan (University of Missouri)
20 2019 Applied Imagery Pattern Recognition Workshop
Thursday, October 17 Session V BioMedical / Emerging Areas – Chair: Murray H. Loew
9:00a Keynote: Security and Privacy for Distributed Optimization and Learning Nitin Vaidya, Professor and McDevitt Chair of Computer Science, Georgetown University
9:50a T1: Adaptive Data Generation and Bidirectional Mapping for Polyp Images HAOQI GAO (Wakayama University)
10:10a T9: Ensemble Lung Segmentation System Using Deep Neural Networks Redha Ali (University of Dayton)*; Russell Hardie (University of Dayton); Hussin K Ragb (Christian Brothers University)
10:30a T40: Deep-Learning Semantic Segmentation for High-Resolution Medical Volumes Imad Eddine Toubal (University of Missouri)*; Ye Duan (University of Missouri - Columbia); Deshan Yang (Washington School of Medicine St. Louis)
10:50a T57: A Cell Augmentation Tool for Blood Smear Analysis Yasmin M. Kassim (National Institutes of Health )*; Stefan Jaeger (National Institutes of Health)
11:10a T37: Deep Fakes for Histopathology Images: Myth or Reality? Nouf Alrasheed (University of Missouri-Kansas City); Arun Zachariah (University of Missouri-Columbia); Shivika Prasanna (UNIVERSITY OF MISSOURI-COLUMBIA); Deepthi Rao (UNIVERSITY OF MISSOURI-COLUMBIA); Praveen Rao (University of Missouri-Columbia)*
11:30a T21: Deepfake Video Detection Based on Spatial, Spectral, and Temporal Inconsistencies Using Multimodal Deep Learning John K Lewis (University of Missouri-Columbia)*; Helen Chen (University of Missouri); Imad Eddine Toubal (University of Missouri); Vishal Sandesara (IQT Labs); Michael Lomnitz (IQT Labs); Zigfried Hampel-Arias (In-Q- Tel); Prasad Calyam (University of Missouri-Columbia); Kannappan Palaniappan (University of Missouri)
11.50a T52: Broad Area Damage Assessment Zachary J DeSantis (Lockheed Martin Space)*; Matthew Reisman (Lockheed Martin Space); Latisha Konz (Lockheed Martin Space); Shabab Siddiq (Lockheed Martin Space)
12.10a Lunch Break
1:00p Keynote: Trusted Computing, Privacy, and Securing Multimedia: A Perspective from the National Science Foundation (NSF) M Prabha (Balakrishnan), Program Director in the Human Centered Computing (HCC) program of Information and Intelligent Systems (IIS) Division of the CISE (Computer and Information Science and Engineering) Directorate of the National Science Foundation (NSF)
2:00p Tutorial: Approaches for Multi-modal Synthetic Media Detection Zigfried Hampel-Arias (IQT Labs), Michael Lomnitz (IQT Labs)
3:30p Special Session: Future of Healthcare Policy and Regulation in the Digital Age Monica Weldon (Moderator), Candace Lerman, Esq. (Panelist), Andrea Downing (Panelist), Michael Mittelman (Panelist)
21 2020 Applied Imagery Pattern Recognition Workshop
Abstracts
22 2019 Applied Imagery Pattern Recognition Workshop
T1 Adaptive Data Generation and Bidirectional Mapping for Polyp Images HAOQI GAO (Wakayama University)
Abstract: Polyps are one of the main reasons for colorectal cancer. Early screening of diseases has great value for clinical diagnosis. However, the diagnosis of polyps in colonoscopy is a challenging task due to variations in the size and shape of polyps. Traditional methods depend on the doctor’s experience. Eye fatigue and lack of attentiveness during long and back-to-back procedures may lead to a decrease in detection rate, and the estimated average missed diagnosis rate is about 4%-12%. This is time-consuming, labor-intensive, and inefficient. And another challenging task is that medical images are scarce and fraught with legal concerns regarding patient privacy. Inspired by this limitation, we propose a new simulated transfer model based on Cycle Generative Adversarial Nets. It learns a mapping from medical images to segmented images and then trains another learning model to generate synthetic medical images. We use the CVC-ClinicDB datasets to train our model. 196 ETISLaribPolypDB test datasets were used to evaluate our proposed method. For generated segmented images, our method achieves a dice coefficient of 0.763 and AUC of 0.914. For generated synthetic polyp images, we use ”Toy Experiment” to verify the performance of our method. To get more information, you can visit our website: https://github.com/qiqi7788/Adaptive-Data-Generation-and-Bidirectional-Mappingfor-Polyp-Images.
T2 Tamper-Proofing Imagery from Distributed Sensors using Learned Blockchain Consensus Alexander D Wissner-Gross (Harvard University)*; Jared Willard (U.S. Army Research Laboratory); Noah Weston (US ARMY CERDEC)
Abstract: Area monitoring using wireless sensor networks that collect imagery and multimodal data from multiple vantage points, while requiring only limited local bandwidth and compute resources, promises improved resilience and scalability over single-camera imagery. However, the distributed nature of such networks can also increase their relative vulnerability to subversion via physical tampering. Here we address that nascent vulnerability by introducing a blockchain application that (1) learns correlations between low-dimensional projections of observed sequences captured by pairs of sensors, and (2) uses those correlations as a baseline for a soft consensus mechanism that identifies potentially compromised sensors with the strongest pairwise statistical anomalies. We then demonstrate our approach in a simulated environment in which a network of virtual cameras are aimed at a common dynamical scene from different vantage points, and show that after a training period of observing baseline behavior followed by the subversion of various numbers of the cameras, our application can correctly identify the cameras that have been compromised. Finally, we explore automated responses to such compromised sensors, including denying them shared resources and services.
P4 Detection of unknown maneuverability hazards in low-altitude UAS color imagery using linear features Jeffrey J Dale (University of Missouri)*; Grant Scott (University of Missouri); James Keller (University of Missouri, Columbia, USA)
Abstract: Deep learning approaches have very quickly become the most popular framework for both semantic segmentation and object detection/recognition tasks. Especially in object detection, however, supervised models like deep neural networks are inherently prone to find only classes from the training data in the testing set. In domains where the safety and security of operators are entrusted to machine learning algorithms, it is often infeasible or impossible to train supervised models on all possible classes; thus, a supplementary unsupervised approach is needed. For the specific problem of detecting potential maneuverability hazards within road segmentation networks, we propose an unsupervised solution using linear features with a voting scheme at each pixel within a pre-supplied road segmentation map, yielding a consensus-based confidence of how unlike a pixel is to surrounding road pixels. This approach is verified on UAS imagery collected by the U.S. Army ERDC.
23 2020 Applied Imagery Pattern Recognition Workshop
P5 Deep Convolutional Neural Network Ensembles for Improved Malaria Parasite Detection Hussin K Ragb (Christian Brothers University)*; Ian Dover (Christian Brothers University); Redha Ali (University of Dayton)
Abstract: Malaria prognosis, performed through the identification of parasites using microscopy, is a vital step in the early initiation of treatment. Malaria inducing parasites such as plasmodium falciparum are difficult to identify and thus have a high mortality rate. For these reasons, a deep convolutional neural network algorithm is proposed in this paper to aid in accurately identifying parasitic cells from red blood smears. By using a mixture of machine learning techniques such as transfer learning, stochastic gradient descent with restarts, and ensemble methods, we have developed a model capable of accurately identifying parasitic cells within red blood smears. 15 networks pre-trained from the ImageNet database are trained again with the fully connected layers replaced. Stochastic gradient descent with restarts is used to traverse local minima in each network. The output of each trained neural network is representing a single vote that is used in the classification process. 3-to-2 voting criteria are applied in the final classification decision between the candidate malaria cells. Several experiments were conducted to evaluate the performance of the proposed model. The NIH Malaria Dataset from the National Institute of Health, a dataset of 27,558 images formed from microscopic patches of red blood smears, is used in these experiments. The dataset is segmented into 80% training set, 10% validation set, and 10% test set. The validation set is used as the decision metric for choosing ensemble network architectures and the test set is used as the evaluation metric for each model. Different ensemble network architectures are experimented with and promising performance is observed on the test dataset with the best models achieving a test accuracy better than several state-of-the-art methodologies.
T6 Spatiotemporal Maneuverability Hazard Analytics from Low-Altitude UAS Sensors Trevor M Bajkowski (University of Missouri - Columbia); David Huangal (University of Missouri - Columbia); J. Alex Hurt (University of Missouri - Columbia); Jeffrey J Dale (University of Missouri); James Keller (University of Missouri, Columbia, USA); Grant Scott (University of Missouri)*; Stanton Price (U.S. Army Engineer Research and Development Center)
Abstract: Low-altitude unmanned aerial systems (UAS) have a rapidly changing field of view for most sensor payloads, especially downward-looking high-resolution visual imagery. In this research, we explore visible-spectrum derived spatiotemporal
awareness applied to companion sensor phenomenologies and computationally derived information, such as computer vision based road extraction and maneuverability hazard localization. Specifically, short-span visual-temporal analysis is performed to understand the platform motion and therefore geospatial context of measurements, whether from sensors or analytical processes. The mathematical model of frame-to-frame platform motion is extracted as homographies and affine transformations that describe how one image “fits into” the next. By coupling these computed motion models with derived data, such as road segmentation and object localization, we show how this technique allows the development of temporal-confidence to aid scene understanding of a complex environment and results in more robust feature matching process. The resulting context-aware geospatial platform facilitates increasing complex spatiotemporal analytics for enhanced maneuverability and route planning for ground units from assistive low-altitude airborne platforms. The tools proposed show promise for increasing user security through spatial awareness in regions that are GPS-denied or unseen. In particular, these capabilities are demonstrated on a collection of sequential aerial images captured from a region with mixed use.
P7 A Review on Image Dehazing Algorithms for Vision based Applications in Outdoor Environment Teena Sharma (Indian Institute of Technology Kanpur, India)*; Tejashwani Shah (Indian Institute of Technology Kanpur, India); Dr. Nishchal K Verma (Indian Institute of Technology Kanpur, India); Shantaram Vasikarla (California State University, Northridge, CA)
Abstract: Vision-based applications deal with the degraded quality of images due to bad environment. During the bad environment, particles such as haze, fog, and mist, etc. diminish the clarity of the scene. 24 2019 Applied Imagery Pattern Recognition Workshop
Therefore, the information can be lost from such images. The haze removal algorithms play a vital role to improve the quality as well as remove the haze from the image. The objective of this paper is to present a comprehensive study and implementation of various existing dehazing algorithms and their evaluation using realistic single-image dehazing dataset exploited in many vision-based applications. Further, the quantitative and qualitative comparisons of the bench-marked dehazing algorithms are also presented in this paper. For evaluation, various performance measuring criteria including subjective comparison, quantitative comparison, full reference metrics such as peak signal-to-noise ratio and the structural similarity index, no-reference metrics such as spatial-spectral entropy-based quality, and blind image integrity notator using discrete cosine transform statistic has been used. The experimental results highlight the divergence in the various performance metrics used. Furthermore, the comparison among various existing image dehazing and their limitation is highlighted and suggested for future work in this direction.
T8 Mixed Fuzzy Pooling in Convolutional Neural Networks for Image Classification Shahrukh Masood (Indian Institute of Technology Kanpur, India); Teena Sharma (Indian Institute of Technology Kanpur, India)*; Dr. Nishchal K Verma (Indian Institute of Technology Kanpur, India); Shantaram Vasikarla (California State University, Northridge, CA)
Abstract: Convolutional neural networks are being widely practiced in computer vision applications, where pooling indicates as an integral part of the deep CNN. Pooling operation significantly reduces the training time and computational cost of any network. However, the pooling operation used is either max, average, and stochastic, but performance can be increased by using more than one pooling function. Herein, a mixed pooling is proposed for image classification in CNN architecture. In the proposed mixed pooling, max, and fuzzy pooling are combined together to boost the classification accuracy. For this purpose, the proposed mixed pooling is designed as a function of max pooling, fuzzy pooling, and $\alpha$ parameter where the learning of the proposed function takes place by using the value of $\alpha$. Basically, fuzzy pooling is based on fuzzy logic-based approach to deal uncertainties and ambiguities faced in real-time applications. Firstly, the Type-2 fuzzy set is used to get the convolved dominant features. This has been obtained by the estimation of a threshold for image region to be pooled. Thereafter, the Type-1 fuzzy set is used to get the pooling output by a weighted average of convolved dominant features. The results show better accuracy than standard pooling operations. The performance comparison of various pooling strategies on MNIST dataset for handwritten digits classification and CIFAR-10 dataset for RGB images classification is analyzed and presented with various deep neural network architectures.
T9 Ensemble Lung Segmentation System Using Deep Neural Networks Redha Ali (University of Dayton)*; Russell Hardie (University of Dayton); Hussin K Ragb (Christian Brothers University)
Abstract: Lung segmentation is a significant step in developing computer-aided diagnosis (CAD) using Chest Radiographs (CRs). CRs are used for diagnosis of the 2019 novel coronavirus disease (COVID-19), lung cancer, tuberculosis, and pneumonia. Hence, developing a Computer-Aided Detection (CAD) system would provide a second opinion to help radiologists in the reading process, increase objectivity, and reduce the workload. In this paper, we present the implementation of our ensemble deep learning model for lung segmentation. This model is based on the original DeeplabV3+, which is the extended model of DeeplabV3. Our model utilizes various architectures as a backbone of DeeplabV3+, such as ResNet18, ResNet50, Mobilenetv2, xception, and inceptionresnetv2. We improved the encoder module of DeeplabV3+ by adjusting the receptive field of the Spatial Pyramid Pooling (ASPP). We also studied our algorithm's performance on a publicly available dataset provided by Shenzhen Hospital, which contains 566 CRs with manually segmented lungs (ground truth). The experimental result proves the effectiveness of the proposed model on the dataset, achieving an Intersection-Over-Union (IoU, Jaccard Index) score of 0.96 on the test set.
P11 Shape and Texture Aware Graph Processing 25 2020 Applied Imagery Pattern Recognition Workshop
Michael J Reale (SUNY Polytechnic Institute)*; MARIA CORNACCHIA (AFRL); Micah Church (SUNY Polytechnic Institute); Nicholas Goutermout (SUNY Polytechnic Institute); John Rubado (SUNY Polytechnic Institute); Matthew Ochrym (SUNY Polytechnic Institute)
Abstract: Neural network training is time-consuming and often application specific. While pre-trained convolutional neural networks can be fine-tuned for other applications, the lower levels are often affected by changes in rotation and scale, since the approach is purely appearance-based. However, hybrid approaches that take both shape and appearance into account (e.g., AAM) have shown success in general computer vision, so in this work we propose a graph network architecture that captures shape and texture information. This separates shape and texture within a graph neural network in a manner analogous to an active appearance model (AAM), offering potential robustness to shape and scale. The training of this graph neural network is done using an unsupervised image data and node position reconstruction approach. This will allow flexible adaptation/fine-tuning to different applications. In addition, we extend the spectral graph convolution approach GCN by automatically generating weights based on a local and global representation in an effort to make the convolutions more flexible and less application or data specific. We call this new approach Generative GCN (GenGCN). Reconstruction experiments in terms of image data and node positions are performed on the CIFAR and MNIST datasets, and a cross database experiment is also conducted.
T12 Enhancing Network-edge Connectivity Security in Drone Video Analytics Alicia Esquivel (University of Missouri); Prasad Calyam (University of Missouri-Columbia)*; Deniz Kavzak Ufuktepe (University of Missouri-Columbia); Chengyi Qu (University of Missouri-Columbia)
Abstract: Multi-drone systems with video analytics tasks are an emerging need mainly being used in many applications including smart agriculture, disaster scene scenarios, surveillance and package delivery. However, building such systems involves overcoming challenges in terms of security and privacy involving involving malicious attacks in performing computation offloading (CO) and control networking (CoNet). Moreover, drone operations in Flying Ad Hoc Networks (FANETS) are inherently insecure and need effective security schemes to be capable of defending against attacks. In this paper, we propose a light- weight intelligent security framework viz., ``DroneCOCoNet-Sec'' that provides network-edge connectivity security in drone video analytics. The proposed framework has three main modules: (i) an intelligent and dynamic decision algorithm to detect anomaly events in the system without decreasing the performance in a real-time FANET deployment, (ii) a hybrid testbed management module that synergizes simulation and emulation via NS-3, an open-source network simulator and POWDER, a research platform for mobile wireless networks, and (iii) a web-based experiment control module that features a graphical user interface (GUI) to assist users in the execution/visualization of repeatable and high-scale experiments. Our performance evaluation experiments of DroneCOCoNet-Sec in a holistic hybrid testbed show that our proposed security framework successfully detects anomaly events on drones and securely operates containers for video analytics.
T13 Occupancy Detection for Smart HVAC Efficiency in Building Energy: A Deep Learning Neural Network Framework using Thermal Imagery Yaa T. Acquaah (North Carolina Agricultural and Technical State University)*; Raymond Tesiero (North Carolina Agricultural and Technical State University); Balakrishna Gokaraju (North Carolina Agricultural and Technical State University); Jonathan Steele (North Carolina Agricultural and Technical State University); Gregory Monty (North Carolina A&T state University)
Abstract: Heating, ventilation, and air conditioning (HVAC) is responsible for a significant portion of total building energy consumption and measuring occupant comfort and satisfaction perceptions is significant to building efficiency. Continuous detection of occupancy numbers in a large conference room can be used to automate the HVAC system of buildings, for energy efficiency and human comfort as well. In this paper, two techniques are investigated to estimate the number of people in a large room by using 26 2019 Applied Imagery Pattern Recognition Workshop
Thermal Camera Imagery. Firstly, the performance of a pretrained ResNet-50 with Support Vector Machine (SVM) multi classification is investigated. Additionally, the performance of pre-trained AlexNet transfer learning is also investigated to classify the number of people present in a room using thermal images. The ResNet-50 with multi class SVM and AlexNet models obtained accuracies of 96% and over 98% respectively. The experiment data was collected with an Optrix PI connect software and a temperature and humidity sensor for calibration.
T15 Understanding the ability of deep neural networks to detect connected components in images SHUYUE GUAN (George Washington University); Murray H Loew (George Washington University)*
Abstract: Humans detect the number of objects by a spontaneous “number sense” (called “subitizing”) if the number is small, and by deliberately counting and memorizing for a large number. Humans can count very fast by subitizing, but slow substantially as the number of objects increases. In deep learning, previous studies have successfully applied deep neural networks (DNN) to object counting. A trained DNN detector can count the number of objects in an amount of time that increases slowly with the number of objects. Such a phenomenon suggests the subitizing ability of DNNs, and unlike humans, it works equally well for large numbers. Many existing studies have examined the applications of DNNs to object counting, but few studies have studied the subitizing ability of a DNN and its interpretation. In a DNN, subitizing does not first detect and then count objects (as in traditional image processing), but it recognizes the number of objects in an image directly.
The DNN’s ability to transform images to binary images through segmentation has been verified by numerous studies. For a DNN’s subitizing, however, its essential ability is to detect connected components in binary images because object counting depends on their connectedness. In this study, we have created binary images containing various numbers of connected components for DNN detectors. Each component has its size, shape, and location selected at random. Our experiment of a DNN pixel-wise counter indicates that a DNN can learn how to subitize instead of memorizing patterns from the training set. In addition, we interpret this subitizing ability by comparing different structures and analyzing weights and data flow within neural networks. Understanding this ability is an important step toward explaining and trusting object counting with deep learning.
P16 DeepFakes Detection in Videos using Feature Engineering Techniques in Deep Learning Convolution Neural Network Frameworks Balakrishna Gokaraju (University of West Alabama)*; Balakrishna Gokaraju (North Carolina Agricultural and Technical State University); Kaushik Roy (North Carolina A&T State University); Sonya Burroughs (North Carolina A&T State 27 2020 Applied Imagery Pattern Recognition Workshop
Univ.)
Abstract: In this paper, we discuss the intermediate results of our on-going study of DeepFakes detection in videos. Our main focus is in exploitation of feature engineering as a precursor filtering technique, to the deep learning based convolution neural network (CNN) classification frameworks. We derived (i) Scale- Invariant Feature Transform (SIFT) features in spatial domain and (ii) Discrete Fourier Transform (DFT) features in frequency domain for comparison of detection performances. We focus on the standard deviation of the points of interest from SIFT, to detect the possibility, whether visual media has been compromised with misinformation. We use key point matching technique to analyze the video in frame by frame. SIFT clusters the unique key points in sub-images and especially the facial features (Fig.1). In our recent work, we used SIFT features, followed by CNN binary classification technique to detect DeepFakes against original videos, [1]. In this case, we obtained an accuracy of 93% using SIFT+CNN deep learning approach using high quality resolution and simple background videos. The detection accuracies were very low for poor quality resolution and complex background videos.
Hence, in this new approach we rely on classical frequency analysis of images that reveals different behaviors at higher frequencies. We noticed in literature review that there lies a distinct contrast of range of frequency components, in power spectrum densities in artificial images [2], and [3]. We plan to use the DFT based feature engineering, and anticipate improvement of detection accuracies in complex and poor quality images. The DFT features will be input to the CNN binary classification, to compare the detection performance against the SIFT features. The experimentation results, and findings will be included further in the full paper submission.
P17 Improving Anomaly Based Intrusion Detection System for IoT by Exploring Multi-Class Machine Learning Techniques Balakrishna Gokaraju (University of West Alabama)*; Balakrishna Gokaraju (North Carolina Agricultural and Technical State University); Kaushik Roy (North Carolina A&T State University); Xiaohong Yuan (North Carolina A&T State University); addison shaver ( North Carolina A&T State University); Zhipeng Liu ( North Carolina A&T State University)
Abstract: The Internet of Things (IoT) is the network that connects smart devices over the In-ternet. These devices are increasingly found in every facet of life, providing distrib-uted data computing power and improving the accessibility of everyday routines in many households. However, these connected devices expand and so does the risk that they become valuable targets for malicious threats. This is because, IoT devices have lower power and computation management, meaning that traditional methods of security like encryption or firewalls tend to be unworkable to secure these devices. Therefore, Intrusion Detection Systems (IDS) provide an alternative for securing IoT devices, by classifying with anomaly detection, whether a network communica-tion is a potential attack. Enhancing existing IDS by integrating various common machine learning models could provide a logical solution to this issue. In this study, we contribute by reviewing and comparing various machine learning (ML) models with intrusion detection. In this comparative analysis, the experimental results from the integrated ML models were promising with an achieved 99% accuracy rates in both binary and multiclass classifier for intrusion detection.
T18 Multimodal Approach for the DeepFake Detection Challenge Michael Lomnitz (IQT Labs)*; Zigfried Hampel-Arias (In-Q-Tel); Vishal Sandesara (IQT Labs)
Abstract: Generative Adversarial Networks (GANs) have become increasingly popular in machine learning due to their ability to mimic any distribution of data. Though GANs can be leveraged for legitimate purposes, they have increasingly been used to create manipulative and misleading synthetic media, known as deepfakes, designed to be used for nefarious purposes. In this submission we discuss a multimodal deepfake detection solution submitted against the Facebook DeepFake Detection Challenge (DFDC), a state-of-the-art benchmark dataset and competition released at the end of 2019. We built individual models to infer on single frames, their power spectrum and the temporal information from video and audio. Finally, we combined these models using a multilayer perceptron to provide a single answer using the information from all modalities, which was ultimately ranked in the top 28% of submissions to the competition. In this presentation we will cover the data preprocessing, model 28 2019 Applied Imagery Pattern Recognition Workshop
architectures and training. The performance of individual modalities, and the combined system is reported using our own validation dataset as well as the competition’s official, private test-set.
T19 A Framework for Public Blockchain-based Trustworthy Federated Learning Muhammad Habib ur Rehman (Khalifa University of Science and Technology)*; Khaled Salah (Khalifa University of Science and Technology ); Ernesto Damiani (Khalifa University); Davor Svetinović (Khalifa University of Science and Technology)
Abstract: Federated learning (FL) is the rapidly developing machine learning technique which is used to perform collaborative model training over decentralized datasets. FL enables privacy preserving model development whereby the datasets are scattered over a large set of data producers (i.e. devices, systems) which 1) train their own model, 2) encapsulate the model updates with differential privacy techniques, and 3) share them to centralized systems for global aggregation. However, these centralized models are always prone to adversarial attacks (such as data-poisoning and model poisoning) due to the large- number of data producers. Hence, FL methods need to ensure fairness, trustworthiness, and high quality model availability across all the participants in the underlying AI systems. In this paper, we propose a public blockchain integrated FL framework to meet the trust, fairness, and high quality data requirements. The proposed framework enables multiple reputation and incentive models to compel the participants in bringing the active, responsible, and honest contributions. We use python-simulated FL framework to enable large-scale training over MNIST and synthetic datasets. In addition, we integrate the public blockchain-based Ethereum smart contract technologies for reputation calculation and incentive mechanisms. We thoroughly tested the proposed framework against various attack models and compared our initial results with traditional centralized approaches. Our research produce significantly better results in multiple aspects.
P20 Interactive Graphical Visualization of Deep Learning Models Akhil Vyas (University of Illinois Urbana-Champaign); Prasad Calyam (University of Missouri-Columbia)*
Abstract: Deep Learning models such as Convolutional Neural Nets (CNNs) and Recurrent Neural Nets (RNNs) have become routine with their continuous utilization and successes in various practical applications such as natural language processing and image recognition. However, despite these rapid developments, a continued weakness of using these models is the opaqueness of their internal workings. The current black box approach to deep learning makes models difficult to understand and fine-tune. This can in turn serve as a major drawback since the lack of information directly influences productivity and elongates the process to come up with reliable and consistent models. In this paper, we present a novel graphical approach to visualizing CNNs and RNNs. The approach involves an interactive graphical interface, where the user can control the width and height of the network and view the resulting model performance, data flow, and the decision boundaries of each neuron. We show the effectiveness of our visualization techniques on the MNIST dataset corresponding to an image recognition application. Having effective graphical visualizations for complex neural networks enable a deeper understanding of their behavior that makes it easier to manage, manipulate, and increase the performance of these networks.
T21 Deepfake Video Detection Based on Spatial, Spectral, and Temporal Inconsistencies Using Multimodal Deep Learning John K Lewis (University of Missouri-Columbia)*; Helen Chen (University of Missouri); Imad Eddine Toubal (University of Missouri); Vishal Sandesara (IQT Labs); Michael Lomnitz (IQT Labs); Zigfried Hampel-Arias (In-Q-Tel); Prasad Calyam (University of Missouri-Columbia); Kannappan Palaniappan (University of Missouri)
Abstract: Authenticity of digital media has become an ever-pressing necessity for modern society. Since the introduction of Generative Adversarial Networks (GANs), synthetic media has become increasingly difficult to identify. Synthetic videos that contain altered faces and/or voices of a person are known as deepfakes and threaten trust and privacy in digital media. Deepfakes can be weaponized for political advantage, slander, and to undermine the reputation of public figures. Despite imperfections of 29 2020 Applied Imagery Pattern Recognition Workshop
deepfakes, people struggle to distinguish between authentic and manipulated images and videos. Consequently, it is important to have automated systems that accurately and efficiently classify the validity of digital content. Many recent deepfake detection methods use single frames of video and focus on the spatial information in the image to infer the authenticity of the video. Some promising approaches exploit the temporal inconsistencies of manipulated videos; however, research primarily focuses on spatial features. We propose a hybrid deep learning approach that uses spatial, spectral, and temporal content that is naturally coupled in a consistent way to differentiate real and fake videos. In this work, we build a computationally efficient cloud-ready multimodal system to detect deepfake videos. We evaluate the performance of our proposed system compared to recent approaches, in terms of accuracy and speed, on the Facebook Deepfake Detection Challenge and FaceForensics++ video datasets.
T23 SMVNet: Deep Learning Architectures for Accurate and Robust Multi-View Stereopsis Shizeng Yao (University of Missouri - Columbia)*
Abstract: We describe Spatial Voxel-Net (SVNet) and Multi-View Voxel-Net (MVNet), a cascade of two novel deep learning architectures for calibrated multi-view stereopsis that reconstructs complicated outdoor 3D models accurately. Both networks use a sequence of RGB images based on ordered camera poses in a coarse-to-fine fashion. SVNet extracts summarized features and analyzes the spatial relationship among a block of 3D voxels using 3D convolutions, then predicts block-level occupancy information. MVNet then receives the occupancy information together with RGB images to predict the final voxel-level occupancy information. SMVNet is an end-to-end trainable network, which can reconstruct complex outdoor 3D models and be applied to large-scale datasets in a parallel fashion without the need of estimating or fusing multiple depth maps, typical of other approaches. We evaluated SMVNet on the complex outdoor Tanks and Temples dataset, in which outperformed two well-known state-of-the-art MVS algorithms.
T25 Mitigating Security Risk of IoT with A Blockchain-Based Solution SOUNDARARAJAN EZEKIEL (IUP)*
Abstract: Internet of Things (IoT) devices are currently being widely implemented in various applications, in addition to blockchain (BC) utilizations growing in popularity. The number of IoT appliances being utilized on a daily basis is predicted to nearly double within several years. The ever-growing number of devices demands an effective management system, which in turn needs to successfully manage the data exchange between devices and overall network while maintaining efficiency, security, and reliability. BC- based systems have been recently starting to be practiced as more than distributed ledgers for cryptocurrencies. They are being applied more frequently to various systems in order to compensate for the weaknesses found in traditional networks. Some examples include healthcare for secure data sharing, supply chain logistics for improved monitoring, and real-time IoT analysis. In this paper, we propose new applications of BC technologies that help alleviate numerous security threats found in traditional centralized IoT systems in addition to overcoming various issues found in applying BCs to IoT devices. These threats consist of the following but are not limited to botnets, distributed denial of service (DDoS) attacks, man in the middle (MITM), etc. Testing and evaluation studies consist of running a small scale, a real-life scenario involving the testing of the BCs resistance against potential attacks in a simulation environment. The results of the testing and evaluation confirm that the application of BC technologies to allow for improved security in IoT systems with more appliances and devices implemented.
P30 Analyzing Information Flow in the IoT Network for Threat Detection Using DCNN and Clustering Techniques SOUNDARARAJAN EZEKIEL (IUP)*; Laurent Njilla (Air Force Research Lab)
Abstract: Devices and sensors typically associated with the Internet of Things(IoT) have seen a significant proliferation in recent years. Many of these devices are resource-constrained and do not have the capabilities of traditional security methods. To improve IoT security, it is important to detect potential 30 2019 Applied Imagery Pattern Recognition Workshop
threats that manifest as anomalies in the data. Anomalies are defined as readings that are possibly indicative of malicious behavior, malfunctions, or other abnormalities. An effective approach for detecting these anomalies is the use of deep learning techniques for advanced threat detection. Convolutional neural networks (CNNs) have proven to be highly successful deep learning models, especially in image classification. In this paper, we investigated the effectiveness of CNNs for anomaly detection in IoT time- series data. A novel ensemble network architecture consisting of three different CNNs is implemented. Furthermore, this investigation leverages multiple convolutional neural network models through heterogeneous and homogeneous network fusion. In homogeneous DCNN fusion, network models of the same architecture but with differing parameters are combined into a single fused feature vector. Conversely, heterogeneous fusion leverages multiple network models with differing architectures as a means of utilizing them simultaneously. In each case, the penultimate fully-connected layer produces a feature vector that is then used for classification using k-means clustering. Several datasets consisting of time-series data collected by IoT sensors with artificially injected anomalies are tested on the model to assess its performance.
P31 A Framework for Hierarchical Detection in IoT Networks SOUNDARARAJAN EZEKIEL (IUP)*; Laurent Njilla (Air Force Research Lab)
Abstract: Devices and sensors typically associated with Internet of Things (IoT) are resource-constrained and therefore are not capable of leveraging traditional security techniques. Hierarchical detection is a security paradigm in which compromised nodes in a large network can be isolated and remedied without affecting the rest of the network. Due to the inherent security weaknesses in IoT devices, existing security methods must be improved by other means. Deep learning methods are a cutting-edge technique that can be employed to improve on these security flaws. In this paper, we investigate utilizing deep learning techniques to improve the detection of security threats in IoT devices within different levels of the hierarchical network. A Generative Adversarial Network (GAN) is trained to perform anomaly detection on IoT sensor data. The GAN is given input consisting of ground truth data with the addition of synthetic anomalies. The discriminator component of the GAN acts to remove any anomalies from this input by training the generator to match non-anomalous data. This result is then subtracted from the input data, which leaves only the anomalies. This neural network model can be used to classify threats in real-time and alert the network when an anomaly is detected at a certain location.
P34 Real-Time Traffic Management System using an Object-detection based Signal Logic Murthy V Jonnalagedda (Jawaharlal Nehru Technological University Kakinada)*; Sashank Taduri (Kakinada Smart City Corporation Ltd.,); Rachana Reddy Dereddy (VIT Vellore )
Abstract: Traffic management is a critical issue in densely populated cities. Automated object detection and counting technologies can provide benefits to develop the next-generation of AI-supported real-time traffic management systems. In this paper, we present a novel green time estimation algorithm that provides the green signal time in each direction (North, West, South, and East) based on the traffic density and use AI and ML techniques for predicting the green time in a short interval. Inputs include: number of directions, maximum green time for a direction, maximum green cycle time, and arrival and departure rates in various directions. Two dictionaries are maintained to store vehicle counts and green time is provided in all the directions. A signal logic scheme is detailed that uses directional threads to get vehicle counts every second by implementing Yolo algorithm for object detection. The cycle starts with North followed by West, South and East. We conclude by providing a case study of our object-detection based signal logic scheme using real world data sets in the Kakinada Smart City test bed.
T35 IC-ChipNet: Deep Embedding Learning for Fine-grained Retrieval, Recognition, and Verification of Microelectronic Images Md Alimoor Reza (Indiana University)*; David Crandall (Indiana University))
Abstract: Modern electronic devices are comprised of a wide range of integrated circuits (ICs) from 31 2020 Applied Imagery Pattern Recognition Workshop
various manufacturers. Ensuring that a device functions as expected requires verifying that its ICs and other parts are correct and legitimate. Towards this goal, we propose a deep metric learning approach to learn a feature embedding to capture important visual features of the external packages of ICs. We explore several variations of Siamese networks for this task and learn an embedding using a joint loss function. To evaluate our approach, we collected and manually annotated a large dataset of 6,387 IC images, and tested our embedding on three challenging tasks: 1) fine-grained IC recognition, 2) fine- grained retrieval, and 3) verification. We believe this to be among the first papers targeting the novel application of fine-grained IC visual recognition, and establishes baselines to advance research in this area.
T36 Video streaming in ad hoc networks Andrei Sukhov (National Research University Higher School of Economics)*; Evgeny Sagatov (Samara National Research University); Aleksei Gladkii (Samara National Research University)
Abstract: The methods for delivering multimedia content to viewers have undergone significant changes in recent decades. This research proposes using ad hoc networks, including networks based on user devices (e.g., smartphones), as a transport. Currently, all existing solutions for various types of self- organizing wireless networks (WSN, mesh, MANET) can transmit only text messages and small files attached to these messages, because starting from the second section of long routes, numerous packet losses occur making video transmission impossible. The main difference between this work and other techniques for improving communication in self-organizing wireless networks is that it aims to improve the network component of video transmission, e.g., the structure of the video stream. Thus, the focus is on enhancing communication protocols instead of video compression algorithms. Approaches for making video transmission stable include partial duplication of transmitted data (e.g., key frames), optimization of Wi-Fi subchannels usage and search for specific types of packets for marking routes and transmitting video data. Another problem addressed in this research is the security of the proposed prototype. Special attention is given to video transmission protection against interceptions. For this purpose, the structure of the video stream is modified. We also try to protect the streaming video server from unauthorized requests that can overload the network. These studies are carried out in parallel with research on stream modernization to increase the stability of video transmission.
T37 Deep Fakes for Histopathology Images: Myth or Reality? Nouf Alrasheed (University of Missouri-Kansas City); Arun Zachariah (University of Missouri-Columbia); Shivika Prasanna (UNIVERSITY OF MISSOURI-COLUMBIA); Deepthi Rao (UNIVERSITY OF MISSOURI-COLUMBIA); Praveen Rao (University of Missouri-Columbia)*
Abstract: Deepfakes have become a major public concern as the generated images or videos could be used to spread misleading information and damage people’s reputations. In this paper, we explore the possibility of generating deepfakes for whole slide images (WSIs). WSIs are gigapixel images of histopathology glass slides scanned at near-optical resolution. As of this year, the Food and Drug Administration (FDA) has approved a few companies to market their digital pathology systems for primary diagnosis and consultation in the United States. Therefore, we investigate how well the state-of-art generative adversarial networks (GANs) can produce fake histopathology images and whether an expert pathologist can be tricked. We used WSIs of colon cancer patients available in The Cancer Genome Atlas (TCGA) and produced 50,000 patches of 256x256 pixels. We trained 4 GANs and constructed a set of images containing 20 fake and 20 real patches. An expert pathologist reviewed these images and marked them as either real or fake. The pathologist marked 8 fake patches as real; 25 patches (fake or real) were correctly identified; and 7 patches were marked as fake although they were real. The pathologist remarked that fake patches had missing morphological features, abrupt background change, pleomorphism, etc. As a pathologist must review an entire WSI before calling the diagnosis, our results show that to generate a completely fake WSI (and not just patches), we must improve the quality of images produced by GANs and have an even larger training dataset.
32 2019 Applied Imagery Pattern Recognition Workshop
T38 Aiding Material Design Through Machine Learning Stanton Price (U.S. Army Engineer Research and Development Center)*; Christina Young ( U.S. Army Engineer Research and Development Center); Matthew Maschmann (University of Missouri-Columbia); Steven Price (Mississippi College)
Abstract: Machine learning is a field that has been around for decades whose impact and presence continues to increase across scientific and commercial communities. However, until recently, machine learning has not been thought of as a viable
methodology that could significantly aid novel material discovery and design. That is, machine learning-aided material design and/or discovery is an emerging research topic, but one that holds immense potential. Such a system could, theoretically, be used to discover novel materials or surfaces that possess desirable properties across the electromagnetic spectrum under specific conditions. Herein, we present our preliminary machine learning based framework for novel material design and discovery.We emphasize that our proposed framework is in its infancy; however, it is laying the foundation for the discovery of fundamental theories and knowledge for this novel technology. Baseline elementary experiments are presented as a proof-of-concept to show the feasibility of our proposed framework for the task of material design. Specifically, we put forth a multi- stage machine learning framework for new material discovery considering material surface geometries for predicting target signatures in the X band. Our proposed multi-stage framework is structured as follows: 1) a deep neural network (NN) is trained for predicting the time response scattered from an object based upon surface geometries (micro-feature spacing, height, etc.); and 2) a genetic algorithm is used to search for the optimal surface geometry configuration whose predicted scattered response (closely) matches that of a desired target response in the X band.
T39 Autonomous Navigation Assurance with Explainable AI and Security Monitoring Denzel Hamilton (Morgan State University)*; Lanier Watkins (Johns Hopkins University); Kevin Kornegay (Morgan State Universiry); Denzel Hamilton (Morgan State University)
Abstract: With the rise of disruptive artificial intelligence (AI) technology, assuring autonomous systems is of the utmost importance. The drive to spread autonomous vehicles to all facets of society, whether military or commercial space, makes assurance a must for autonomous navigation. In this paper, we introduce the concept of using explainable AI to assure the operational autonomy of a vehicle while also monitoring the security of the system. We demonstrate our paradigm by developing a system monitor using AI (i.e., Random Forest Tree algorithm) to assure the native autonomy of an autonomous Turtlebot3, and also monitor the security status of the Turtlebot3 computer system and sensor data. Then we dig into the decision trees of the AI algorithm and extract information to help explain the navigation autonomy and decisions made by our monitor. The security monitor is useful for helping the autonomous navigation make decisions in the instance of attack. Further, we demonstrate the efficacy of the monitor by subjecting the Turtlebot3 to a "Sharp Turn Maze," which was designed to be difficult for the native autonomy to solve, and a barrage of security attacks. These experiments allow the AI-Monitor to demonstrate its ability to identify and explain unproductive or wrong decisions made by the native autonomy, and identify cyber-security related faults to trigger navigation contingencies.
T40 Deep-Learning Semantic Segmentation for High-Resolution Medical Volumes Imad Eddine Toubal (University of Missouri)*; Ye Duan (University of Missouri - Columbia); Deshan Yang (Washington School of Medicine St. Louis)
Abstract: Automated semantic segmentation in the domain of medical imaging can enable a faster, more reliable, and more affordable clinical workflow. Fully convolutional networks (FCNs) have been heavily used in this area due to the level of success that they have achieved. In this work, we first leverage recent architectural innovations to make an initial segmentation: (i) spatial and channel-wise squeeze and excitation mechanism; (ii) a 3D U-Net++ network and deep supervision. Second, we use classical methods for refining the initial segmentation: (i) spatial normalization and (ii) local 3D refinement network applied to patches. Finally, we put our methods together in a novel segmentation pipeline. We train and evaluate 33 2020 Applied Imagery Pattern Recognition Workshop
our models and pipelines on a dataset of a 120 abdominal magnetic resonance – volumetric – images (MRIs). The goal is to segment five different organs of interest (ORI): liver, kidneys, stomach, duodenum, and large bowel. Our experiments show that we can generate full resolution segmentation of comparable quality to the state-of-the-art methods without adding computational cost.
T41 MACHINE LEARNING ON SMALL UAVS Steven Israel (Draper Laboratory)*; Alison Cleary (Draper Laboratory)
Abstract: Commonly, machine learning (ML) workflows show training and inferencing occurring in resource rich environments. Draper Laboratories are pushing ML to the edge. This paper shows the concept of operations (CONOPs), design parameters, and constraints the team faced for the workflow design. The overarching requirement is to build a workflow to integrate any ML model into small unmanned aerial vehicles (UAV) or drones. Given the limited payload capacity and power available on small UAVs, integration of computing resources sufficient for hosting both ML and Autonomy functions is a challenge. Past efforts have relied on an Intel NUC as the primary processing unit. However, modern GPU-based systems offer better compatibility with ML algorithms while still providing sufficient CPU resources to host Autonomy algorithms. In addition, more recently developed processing units designed specifically for ML applications at the edge enable scaled down variants of standard ML algorithms to be integrated onto significantly smaller platforms. In this paper, we identify a common software architecting strategy that supports a micro UAV ~ 150 grams configured with a traditional CPU and a small UAV 3 kg configured with a GPU. Draper’s automation strategy leverages open-source Robotic Operating System (ROS). The ML models were built using open-source Python-PyTorch libraries and packages. The initial models focused on vehicle detection. Future applications are expected to include visual navigation and target tracking.
P44 Determining the Impact of Cybersecurity Failures During and Attributable to Pandemics and other Emergency Situations Maria Clare Lusardi (University of Missouri); Isaac Dubovoy (North Dakota State University); Jeremy Straub (Department of Computer Science North Dakota State University)*
Abstract: Emergency response presents organizations with a challenge that they have limited resources and time to respond to. During the COVID-19 pandemic response, businesses, governments and other organizations have rushed to find methods for sustaining income, maintaining operations, and restructuring their operations model for remote work. This occurred rapidly – in some cases in a week or less. Rapid changes like this, during an emergency, can cause cybersecurity to be treated as a concern secondary to the emergency. Cyber-criminals and others with nefarious motives are demonstrably aware of this and have shown that they can take advantage of emergency situations that create cybersecurity vulnerabilities in myriad ways. Phishing scams, ransomware and fraudulent websites impersonating the Centers for Disease Control, the World Health Organization, and key responders have been launched during the COVID-19 response. Nefarious advertisements for fake vaccines, masks, and even ‘anti- coronavirus’ computer software have circulated. Attackers have also targeted the tools used for remote work including video conference and email systems and virtual private networks. Perhaps most opportunistically and appallingly, attackers have targeted organizations under strain due to the pandemic, such hospitals and governments.
P47 A Semi-automatic System for Moving Objects Annotation using Deep Detections and Multi-object Tracking Techniques NOOR M AL-SHAKARJI (University of Missouri Columbia)*; Filiz Bunyak (University of Missouri-Columbia); Hadi Aliakbarpour (University of Missouri-Columbia); Guna Seetharaman (Naval Research Laboratory); Kannappan Palaniappan (University of Missouri)
Abstract: With the recent advances in sensor technologies for both cameras and platforms. Computer vision problems have become more complicated and more in demand. Moving object detection and 34 2019 Applied Imagery Pattern Recognition Workshop
tracking are one of the most popular computer vision problem. Researchers attempt to explorer different methodologies and algorithms on different challenging datasets. So that, it is important to have benchmark information of the datasets either for training or evaluation purposes for different tasks (i.e. single object tracking, multi-object tracking, object detection and recognition). In this paper, a fast semi- automatic system for individual object annotation is proposed. First, a deep learning-based method for object detection; Second, multi-object tracking for high-quality tracks production are used. YOLOv3 [1], an object detection, is used to generate bounding boxes for the objects in the scenes. SCTrack [2], a multi- object tracking algorithm, is used to generate trajectory for each object individually followed by a number of post-processing steps for high performance. Our proposed system was tested on FPSS dataset [3] and shows promising results compared to the other system annotations methods.
P49 Monitoring and Detection Time Optimization of Man in the Middle Attacks using Machine Learning Otily Toutsop (Morgan State University)*; Paige Harvey (Morgan State University); Kevin Kornegay (Morgan State University) Abstract: The internet of things (IoT) is growing with the advancement of technology. Many vendors are creating IoT devices to leverage the quality of life of consumers. These devices include smart grid, smart homes, smart health care systems, smart transportation, and many other device applications. IoT devices interact with the environment and each other using sensors and actuators. However, the widespread proliferation of IoT devices poses many cybersecurity threats. The interconnection of the IoT devices opens the door to attackers who try to gain unauthorized access to the devices. For many IT networks, establishing trust and security during the devices' regular operation is a very challenging endeavor. Further, devices may leak vital information, which is a huge concern in cybersecurity. Prior research has shown that security breaches have increased by 67% in the past five years, and 95% of HTTPs servers are vulnerable to Man In The Middle (MIM) attacks. Furthermore, it is incredibly complex to differentiate between the MIM attack and other types of attacks and minimize the risk related to this type of attack when it is happening. In this work, we analyze MIM attacks using an AI model to optimize the monitoring and the response time. We then train the model using machine learning algorithms to detect MIM-type security vulnerabilities automatically.
T50 Using models of cortical development based on sparse coding to discriminate between real and synthetically generated faces Nga T Nguyen (Los Alamos National Laboratory)*; Juston Moore (Los Alamos National Laboratory); Garrett Kenyon (Los Alamos National Laboratory)
Abstract: We compare classifiers based on state-of-the-art Deep Neural Networks (DNNs) with classifiers based on a model of cortical development using a single layer of sparse coding to which is appended a linear support vector machine (SVM). The comparison is based on the ability of the two distinct types of classifiers to distinguish between faces of celebrities culled from the CelebA database and synthetic faces created using multi-scale GANs trained on the same CelebA database. In particular, we examine the robustness of DNNs compared to classifiers based on sparse coding after the addition of universal adversarial perturbations (UAPs), which fool most or all of the DNN classifiers we examined. Our results show that classifiers based on sparse coding are robust to UAPs that elicit substantial performance gaps across a wide range of DNN architectures. We hypothesize that sparse latent representations, which correspond to fixed points of a dynamical attractor, or Hopfield network, are naturally denoising and remove small perturbations, and observe similar robustness is conferred by deep denoising autoencoders. We further report that the same DNNs that perfectly discriminate between real and synthetic faces are unable to distinguish between sparse reconstructions of those same real and synthetic faces, further suggesting that DNN-based classifiers may be relying on non-robust features that are not present in the sparse reconstructions but are still present in deep denoising autoencoders.
T52 Broad Area Damage Assessment Zachary J DeSantis (Lockheed Martin Space)*; Matthew Reisman (Lockheed Martin Space); Latisha Konz (Lockheed Martin Space); Shabab Siddiq (Lockheed Martin Space) 35 2020 Applied Imagery Pattern Recognition Workshop
Abstract: We present Lockheed Martin’s deep learning solution for broad area damage/change assessment (BADA) for monitoring environmental change—water levels, deforestation, crop disease, etc.—and anthropogenic activities like construction, land clearing, or battlefield damage. BADA was developed against the xView2 challenge dataset. The purpose of the xView2 challenge was to provide a capability that could aid humanitarian relief efforts in response to a natural disaster. Given a pre-disaster and a post-disaster image (satellite EO), the goal of the challenge was to locate structures in the imagery and, on a per-structure basis, apply a damage level classification: no damage, minor damage, major damage, or destroyed.
We pursued a more general solution by extending and adapting the dataset. While providing damage assessment to building structures is valuable, it is also important to know the change to the environment, independent of the presence of a structure. To facilitate training, we extended the provided truth labels to all pixels in the image using a nearest neighbor interpolation. This is valid for regions right next to a structure, but increasingly less valid as the distance to the nearest structure grows. To account for this, we included a weighting in our network to emphasize getting the structures and nearby pixels correct. In addition to providing a continuous-scale broad area damage assessment, our solution also localized buildings. The product of the broad area damage with the localization map still produces a per-structure label to assess accuracy. The localization was performed with a custom U-net, while the damage assessment was performed using the Siamese version of the same custom U-net.
P53 Autonomous Navigation Under Adversarial Attack Md Tanvir Arafin (Morgan State University)*; Kevin Kornegay (Morgan State University)
Abstract: With the progress in deep neural network design and hardware accelerators, deep-learning models outperform conventional machine learning techniques in different sub-domains of artificial intelligence research, such as object recognition, object detection, image classification, reinforcement learning, natural language processing, and audio synthesis. As a result, within years of its inception, deep learning techniques and models have been integrated into consumer software and real-world applications in biometric authentication, facial recognition, malware discovery, anomaly detection etc. This rapid development has encouraged researchers to engage in developing novel solutions and products to solve intricate and fundamentally challenging problems. For example, in recent years, researchers are exploring the application of deep learning techniques in different autonomous navigation and driving tasks [1]–[3]. Although vision-based end-to-end autonomous navigation exists, a complete driving solution usually incorporates both visual data and data from other sensors in the vehicle, such as a global positioning system (GPS), gyroscope, and accelerometer. inertial measurement unit(IMU), collision avoidance infrared sensors etc. Recent works by Tian et al. [4] have demonstrated that top-performing deep learning models used for autonomous driving show erroneous behavior in different (simulated) environmental conditions. Moreover, deep learning models are vulnerable to adversarial attacks that can fool, evade, or misled a model by adding small perturbation in the sample [5]. To address these issues, in this work, we demonstrate the application of classical sensor fusion techniques in detecting spoofing attacks on the sensor data that feeds a deep learning engine. We also discuss hardware-dependent attack survival mechanisms that support an autonomous system during an adversarial attack.
P54 Investigating Global Mosaic Generation from Mesentery Video Sequences using Deep Learning Rumana Aktar (University of Missouri-Columbia)*; Jaired Collins (University of Missouri-Columbia); Giovanna Guidoboni (University of Missouri); Virginia Huxley (University of Missouri-Columbia); Kannappan Palaniappan (University of Missouri)
Abstract: Mosaicing or large scale panoramic image generation is a crucial task in medical imaging especially examining cellular features or microstructures. Earlier we have proposed an algorithm Deformable Normalized Cross Correlation (DNCC) to mosaic feature-poor, motion-blurred Frog Mesentery video sequences. DNCC algorithm generates a single mosaic from each sequence establishing broad structure morphology. Unfortunately, a single mosaic cannot incorporate all information captured from 36 2019 Applied Imagery Pattern Recognition Workshop
all the sequences together. In another word, we aim to generate a global mosaic combining or fusing all the single mosaics from Mesentery sequences. Global mosaic is desirable for comprehensive observation and exploration of the target region in microscopic resolution. Unfortunately, generating a global mosaic is challenging due to the change in size, perspective, and illumination of single mosaics from individual frames. We propose to use a deep learning based feature detector (for example, SuperPoint) to learn features, building a dictionary and use vocabulary tree to index all frames in all video sequences to find matching frames across different sequences. Using these matched frames across all sequences, a global mosaic or panorama can be build to ensure broad structure morphology. In addition, we would like to investigate the mini-mosaics generated from DNCC (instead of individual frames) and construct a global mosaic by finding the matching or overlapping region among the mini-mosaics. Finally, a thorough comparison between these two methods of global mosaic generation will be presented for Mesentery dataset.
T55 Evaluation of Different Data and Decision Fusion Mechanisms for Robust Moving Object Detection Gani Rahmon (University of Missouri-Columbia)*; Filiz Bunyak (University of Missouri-Columbia); Kannappan Palaniappan (University of Missouri); Guna Seetharaman (US Naval Research Laboratory)
Abstract: An important processing step in computer vision and video processing applications is detection of moving objects in the camera view. The task becomes very challenging when various difficult scenarios happen, such as environmental conditions, shadows, changes in the illumination, etc. Recently, deep learning methods started to have an impressive impact on the field of computer vision particularly for tasks such as detection, recognition, and semantic image segmentation. In this paper, we evaluate different data and decision fusion strategies for moving object detection within a hybrid system composed of deep learning and classical computer vision approaches. The proposed system uses appearance, motion, and change cues by integrating change through background subtraction module, motion through flux tensor module, and appearance through unsupervised deep semantic segmentation network. These multi-cue information sources are fused together to achieve robust performance under challenging real-world conditions. The proposed system was tested and evaluated on the Change Detection challenge datasets (CDnet-2014) and compared to state-of-the-art methods.
T56 Transductive Learning with Graphical Models: Characterizing Adversarial Data in the Wild Diane Oyen (LANL)*
Abstract: Adversarial information is intended to fool humans or machine classifiers or both. One common way of producing adversarial data is by perturbing or modifying an existing datapoint, whether produced by a human or an artificial intelligence-based adversarial generator. Either way, we expect that the data is not independent and identically distributed (IID), as most machine learning algorithms assume. Our hypothesis is that a generative model that reveals the patterns of similarities among samples will be harder to fool with adversarial examples that are designed to exploit the discriminative boundaries learned by discriminative classifiers. Furthermore, detection of such adversarial examples is often only the first step in analyzing and/or mitigating the adversarial attack. The next step is to analyze the source of the example or demonstrate in a convincing way to humans that the example is adversarial. We use a generative model, in this case, a Gaussian graphical model, to identify the relationships among data samples; then predict the label of each new sample based on its relationship to the labelled training data. The approach of mixing the data with known labels along with data of unknown labels is known as transductive learning in which we assume that that there is no real separation between training and testing data, only that we have data with labels and data without labels and we must leverage all of this information to infer labels on the unlabeled data. The power of our transductive learning approach is demonstrated in two different applications: malware characterization and fake news image tracking.
T57 A Cell Augmentation Tool for Blood Smear Analysis Yasmin M. Kassim (National Institutes of Health )*; Stefan Jaeger (National Institutes of Health)
37 2020 Applied Imagery Pattern Recognition Workshop
Abstract: Most object detection networks rely on image tiles for training. However, the number of different objects in those tiles are often unbalanced, which leads to biased training. To solve this problem, we propose a tool to add new instances of pre-defined objects into an image. For example, we have used our tool to add parasite-infected red blood cells to blood smear images for malaria screening based on a Mask R-CNN network. The number of infected and uninfected red blood cells is typically unbalanced. Our proposed tool extracts cells, augments them by rotation, and then pastes the augmented cells back into empty spaces between existing cells. To paste augmented cells back into an image, the underlying algorithm computes a set of candidate locations that are far away from existing cells by applying a distance transform to a cell mask. Among these candidate locations, the final locations are selected by choosing the locations that are far from each other and distributed across the image to avoid cell clusters and touching cells. Parameters of the algorithm control the number of times a cell is augmented and pasted, and the minimum distance of candidate locations from existing cells, which can depend on general cell density and other criteria. We augmented cells by rotation only to avoid changing the shape of infected cells, although our tool could easily accommodate other ways of augmentation. For evaluation, we augmented 165 blood smear images from 33 patients and ran tests on 800 blood smear images from 160 patients. In the training set, the number of infected red blood cells is much smaller than the number of the uninfected cells, 901 vs. 34,766. The tool added 8,208 rotated copies of the infected cells across the images, increasing the total number of infected cells to 9,109. We then used Mask R-CNN to detect infected cells, before and after augmentation, and found that our tool improved the recall, precision, and F1 measure by around 6 %, respectively.
T58 Methods of Exploiting Multispectral Imagery for the Monitoring of Illicit Coca Fields Emily E Berkson (Lockheed Martin)*; Charlene Cuellar Vite (Lockheed Martin); Austen Groener (Lockheed Martin); Michael Harner (Lockheed Martin); Stephen O'Neill (Lockheed Martin); Tyler Bartelmo (Lockheed Martin)
Abstract: State-of-the-art deep learning (DL) algorithms for automatic target recognition (ATR) typically rely on transfer learning from networks pre-trained on RGB imagery. In this work, we investigate the benefits of using additional spectral information contained in multispectral satellite imagery, with the targeted development of a suite of ATR algorithms to identify illegal coca fields in Colombia. The automated recognition of coca fields is challenging for several reasons: they are generally small and non- uniform, there is limited ground truth available, and the crop turnover can occur up to four times per year. This application of counter narcotics is especially interesting because coca is typically planted with the intent to evade detection, amongst legal crops and along forest lines. In support of this effort, the team curated a custom multispectral dataset of coca fields in WorldView-3 imagery, starting with annotations provided by expert image analysts. We begin by presenting baseline landcover classification results, using non-DL methods such as maximum likelihood classification and support-vector machines. These methods, which focus purely on pixel spectra, are not ideal to extract the unique texture and shape of coca fields. We compare these baseline results to detection metrics obtained with Faster-RCNN run on classic RGB imagery and false color imagery, which exploits the near-infrared image content. Mean average precision metrics indicate that the false color imagery outperforms standard RGB imagery, showing the benefit of spectral content beyond the classic RGB. Deep learning-based semantic segmentation algorithms are also investigated, with a comparison between backbones trained on RGB and false color imagery. Finally, we include a discussion of ongoing research to modify state-of-the-art CNNs to accept multispectral inputs of more than three channels, while still benefitting from pretrained RGB weights.
T59 Efficient Training of Object Detection Models for Automated Target Recognition Charlene Cuellar Vite (Lockheed Martin)*; Gary Chern (Lockheed Martin); Austen Groener (Lockheed Martin); Mark Pritt (Lockheed Martin)
Abstract: GATR (Globally-scalable Automated Target Recognition) is a software system for object detection and classification on a worldwide basis developed by Lockheed Martin. One of the targets it detects are oil/gas fracking wells. In this presentation we will explore the following techniques for 38 2019 Applied Imagery Pattern Recognition Workshop
efficiently training a fracking well detector in WorldView satellite imagery: determining the optimal ground sample distance (GSD) of data to use for training/inference, leveraging the use of auto-generated training labels, training on different sized subsets of the training data, and using active learning to determine the best subsets of data to train with. In our case efficient training means reduction in human labeling time and model training time as well as improvement in area coverage rate during inference. We compare the mean average precision (mAP) for models trained at different GSDs from 0.3 to 12 m and find optimal performance at 1.2 m GSD (mAP = 0.91), but acceptable performance all the way to 6 m GSD (mAP = 0.80). We also show that using auto-generated training labels from fracking well permits results in a model with mAP reduced by less than 4% (IoU threshold = 0.2) compared to a model trained on the human annotated dataset. We show that training on just 20% of the full dataset results in a mAP drop of less than 6% (mAP = 0.86). Finally, we compare multiple active learning selection methods.
T60 Optimizing Deep Learning Based Intrusion Detection Systems Defense Against White-Box and Backdoor Adversarial Attacks Through a Genetic Algorithm KHALED ALRAWASHDEH (Uinversity of Cincinnati)*
Abstract: Recent years have witnessed rapid progress and significant success in the use of deep learning neural networks (DLNNs) in a wide range of applications. Recently, DLNN has been integrated with intrusion detection system (IDS) to enhance network security to detect zero-day attacks. However, DLNNs themselves have been recently found vulnerable to attacks called adversarial examples and backdoor attacks for image recognition applications. In this work, we present an effective defense method for DLNN based IDS by using Genetic Algorithm (GA) to optimize the generation of triggers neurons selected based on their response to the features to produce the output. We embed the GA-Trigger-Detection neurons within the model to detect and prevent white-box advertorial examples and backdoor attacks against two DLNNs based IDS: Deep Belief Network (DBN) and Stacked Sparse AutoEncoder Based Extreme Learning Machine (SSAELM). We implement two white-box adversarial examples and backdoor attacks from prior work and use them to investigate the proposed defense method. We show that the defense method is sufficient to defend against sophisticated attackers with 99% success rate and only 1% degrade in accuracy. We then show that it successfully weakens backdoor attacks on the two DNN architectures using two benchmark datasets: KDDCUP’99 and Kyoto. Our work provides an important step toward defenses against white-box advertorial examples and backdoor attacks in DLNNs based IDS.
T61 Improving Robustness to Audio Perturbations Using Biological Receptive Fields Ben Migliori (Los Alamos National Laboratory); Derek Zhang (University of Maryland)*
Abstract: In the early stages of artificial intelligence, machine learning scientists took inspiration from neurons to develop artificial neural networks (ANNs). It is quite clear that much of deep learning has now diverged from modeling the brain. Machine learning scientists, inspired by the robustness of living neural systems, believe that ML models that mimic the brain more closely may be the solution to achieving robustness against adversarial examples. For instance, neuronal data from primate’s visual cortex (V1) has been shown to help create models that are robust to adversarial examples [1]. This project studies trustworthy machine learning systems “in the wild”. Thus, we performed a course of experiments in an increasing order of realism with respect to data. Our data gets closer and closer to that of naturally occurring data (in the wild). First, we investigated the effects of neurally-inspired preprocessing on classical ML techniques. For audio, the cochlear nucleus is the first stop in the preprocessing part of the brain. One transform we apply is the gammatone filter bank. Next, our experiments move to applying realistic acoustic distortions such as echoes, fades, etc. on standard adversarially-perturbed examples. Finally, we extend the results of Elhilali et al. [2] by showing the effect of applying A1-like receptive fields to the first layer of neural network acoustic classifiers. Our hypothesis is that biological receptive fields will increase the robustness of our acoustic classifier.[1] Joel Dapello, Tiago Marques, Martin Schrimpf, Franziska Geiger, David D. Cox, and James J. DiCarlo. Simulating a primary visual cortex at the front of CNNs improves robustness to image perturbations. bioRxiv, doi/10.1101/2020.06.16.154542, 2020.[2] Mounya Elhilali, Jonathan B. Fritz, Tai-Shih Chi and Shihab A. Shamma. Auditory Cortical Receptive Fields: 39 2020 Applied Imagery Pattern Recognition Workshop
Stable Entities with Plastic Abilities. Journal of Neuroscience, doi/10.1523/JNEUROSCI.1462-07.2007, 26 September 2007.
T62 Person-Deidentification Versus Reidentification: A Novel Approach Towards Performance Evaluation of Privacy Enhancements in Images Ankur Chattopadhyay (Northern Kentucky University)*; Levi Pfantz (University of Wisconsin -White Water); Robert Ruska (Northern Kentucky University)
Abstract: Existing literature show that there has been considerable work done in the last decade to innovate smart surveillance-cameras for privacy-protection. However, even though privacy-preserving computer vision has seen many research advances, there has been relatively limited work on the evaluation techniques for these privacy-enhancing image-deidentification (DeID) methods. Most of the prior work on evaluating image-DeID methods utilize human perception-based user-studies, which have been driven by the viewer-experience of the privacy-enhanced camera-image contents. These include studies, where the efficacy of the person-DeID processes have been computed by the accuracy with which human-subjects recognize identities within the privacy-enhanced images. Other works study how the privacy-enhancing image filters affect the utility and aesthetics of the images, that is, how DeID impacts image utility. The effectiveness of privacy enhancements in these studies have been determined by the level of trade-off between privacy and utility. In this paper, we present a novel research-study that demonstrates how person-reidentification (ReID), can be utilized as an ally in the process of evaluating the effectiveness of person-DeID techniques. We describe a maiden investigation, where a popular ReID algorithm is pitted against multiple DeID algorithms. We conduct these unique experiments in an effort to show a new way for validating DeID techniques by determining how effectively they can deter successful ReID in the privacy-enhanced images. Our experiments involve a faceoff between our selected ReID algorithm and multiple DeID techniques. We compare the performances of our chosen DeID methods by the extents to which they are able to successfully reduce the ReID percentage-rates. Our selected DeID methods exhibit contrasting performances in the form of different resistance levels offered to the chosen ReID technique.
P64 A Hybrid Approach for Human Activity Recognition with Support Vector Machine and 1D Convolutional Neural Network Md Maruf Hossain Shuvo (University of Missouri-Columbia)*; Nafis Ahmed (University of Missouri-Columbia); Rohit Nouduri (University of Missouri-Columbia); Kannappan Palaniappan (University of Missouri)
Abstract: The Human Activity Recognition (HAR) is a pattern recognition task that learns to identify human physical activities recorded by different sensor modalities. The application areas include human behavior analysis, ambient assistive living, surveillance-based security, gesture recognition, and context-aware computing. The HAR remains challenging as the sensor data is noisy in nature and the activity signal varies from person to person. To recognize different types of activity with a single classifier is error-prone. To mitigate this problem, we introduced an adaptive human activity recognition model by breaking the multi-class problem into simpler multiple classification tasks. We present a two-stage learning process to recognize human activity recorded using a waist-mounted smartphone’s accelerometer and gyroscope sensor. In the first step, we classify activity into static and moving, using a Random Forest (RF) binary classifier. In the second step, we adopt a Support Vector Machine (SVM) to identify individual static activity and 1D Convolutional Neural Network (CNN)-based deep learning model for individual moving activity recognition. This makes our approach more robust and adaptive. The static activity has less frequency variation in features compared to dynamic activity waveforms for CNN to learn. On the other hand, SVM demonstrated superior performance to recognize static activities but performs poorly on moving, complex, and uncertain activity recognition. Our method is similarly robust to different motion intensity and can also capture the variation of the same activity effectively. In our hybrid model, the CNN captures local dependencies of activity signals as well as preserves the scale invariance. We achieved 97.71% overall accuracy on six activity classes of widely accepted benchmark UCI-HAR dataset. Our results are also comparable to a different state of the art deep learning method which confirms the effectiveness 40 2019 Applied Imagery Pattern Recognition Workshop
of our proposed approach.
P65 Benchmark and Evaluation of Low Resource Object Detection in Biomedical Images Sanjoy Kundu (Oklahoma State University); Nikhil Gunti (Oklahoma State University); Bailey Hendrickson (Oklahoma State University); Sunil More (Oklahoma State University); Sathya Narasimhan (OK State)*
Abstract: Object detection has seen tremendous progress in recent times. Their performance is limited to domains with large amounts of annotated training data. A carefully curated, balanced data set is necessary for the successful training and deployment of current models. Their ability to adapt to novel domains, particularly to those with significant visual variations such as biomedical domains, remains limited. Existing work on biomedical image analysis has focused on recognition with large amounts of training data. To address these limitations, we focus on the creation of a benchmark and evaluation platform for low-resource object detection, primarily for biomedical images. We introduce a novel data set for the evaluation of object detection models under resource-constrained environments such as biomedical images. We show that current models do not generalize to such significant domain change and provide a thorough analysis of failure modes to offer a way forward in this difficult task.
T66 Black-box Adversarial Attacks in Autonomous Vehicles K. Naveen Kumar (Indian Institute of Technology Hyderabad); Reshmi Mitra (Southeast Missouri State University)*; C. Krishna Mohan (Indian Institute of Technology Hyderabad)
Abstract: Despite the high accuracy of deep neural networks, they are susceptible to minor perturbations of black-box adversarial attacks which are occasionally undetected to human vision [1]. The impact of such attacks become extremely determinantal in autonomous vehicles with real-time “safety” concerns. Within a self-driving car, a deep learning system processes the incoming data from the car’s array of sensors to provide scene perception [2]. It identifies objects around the car, such as pedestrians, cyclists, traffic signs, etc., to safely navigate and follow traffic rules. The black-box adversarial attack introduces minor but systematic perturbations in key model layers such as filters and input datasets. By adding a meticulously selected layer of noise to an image, it is possible to cause drastic misclassification in critical scene elements such as road signs and traffic lights. This can ultimately cause the autonomous vehicle to crash into other vehicles or pedestrians. In this research, we are proposing a tool for conducting multi-level attack in our deep neural networks based traffic scene perception model. It can also be applied to other research domains such as aerial view traffic state classification [3]. A thorough investigation with this attack tool can bring about a significant improvement in the robustness and reliability of model predictions. References:[1] L. Jiang, X. Ma, S. Chen, J. Bailey, Y. Jiang, “Black-box Adversarial Attacks on Video Recognition Models”, ACM International Conference on Multimedia, pp. 864-872, 2019.[2] X. Song, P. Wang, D. Zhou, R. Zhu, C. Guan, Y. Dai, H. Su, H. Li, R. Yang, “Apollo-Car3D: A Large 3D Car Instance Understanding Benchmark for Autonomous Driving”,IEEE Conference on Computer Vision and Pattern Recognition, pp. 5452-5462. 2019.[3] D. Roy, K. N. Kumar, C. K. Mohan, “Defining Traffic States using Spatio-Temporal Traffic Graphs”, IEEE Intelligent Transport Systems Conference (ITSC), 2020.
T68 MACHINE LEARNING ARCHITECTURE AND IMAGE QUALITY Steven Israel (Draper Laboratory)*; John Irvine (); Steven Tate (Draper)
Abstract: The current drive toward putting machine learning (ML) models within automated closed-loop workflows renews existing issues for image quality prediction; specifically, maintaining the operator-in- the-loop’s trust in system operation. This paper delves into how image quality changes through the ML workflow, specifically for generative models. Generative models include promising ML techniques such as Generative Adversarial Networks (GANs) and Autoencoders. Generative strategies are commonly used for semi-supervised ML workflow and reconstruction of compressed information. This paper reviews methods to assess information content in the regeneration process to maintain trust in closed-loop workflows. For manual exploitation, the interpretability of an image indicates the potential intelligence value of the data. Historically, the National Imagery Interpretability Rating Scale (NIIRS) has been the 41 2020 Applied Imagery Pattern Recognition Workshop
standard for quantifying the intelligence potential based on image analysis by human observers. Empirical studies have demonstrated that spatial resolution is the dominant predictor of the NIIRS level of an image and that compression to 1 bit per pixel can maintain that NIIRS level. However, with modern ML that digests images rather than extracted features, what is the corresponding size of the latent space required to maintain the NIIRS levels? Our research uses autoencoders, image quality metrics, and human observations to answer these questions. To those ends, we will operate on moderate size images, 480x480 pixels, to provide realistic generalizable estimates over those experiments against similar sprite size (< 100x100 pixels) images.
T69 Adversarial Examples in Deep Learning Regression Models for Multivariate Time Series Forecasting Gautam Raj mode (University of Missouri); Khaza Anuarul Hoque (University of Missouri)*
Abstract: Multivariate time series (MTS) regression tasks are common in many real-world data mining applications including finance, cybersecurity, energy, healthcare, prognostics, and many others. Due to the tremendous success of deep learning (DL) algorithms in various domains including image recognition and computer vision, researchers started adopting these techniques for solving MTS data mining problems, many of which are targeted for safety-critical and cost-critical applications. Unfortunately, DL algorithms are known for their susceptibility to adversarial examples which also makes the DL regression models for MTS forecasting also vulnerable to those attacks. To the best of our knowledge, no previous work has explored the vulnerability of DL MTS regression models to adversarial time series examples, which is an important step, specifically when the forecasting from such models is used in safety-critical and cost-critical applications. In this work, we leverage existing adversarial attack generation techniques from the image classification domain and craft adversarial multivariate time series examples for three state-of-the-art deep learning regression models, specifically Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU). We evaluate our study using Google stock and household power consumption dataset. The obtained results show that all the evaluated DL regression models are vulnerable to adversarial attacks, transferable, and thus can lead to catastrophic consequences in safety-critical and cost-critical domains, such as energy and finance.
P70 RefinedFed: A Refining Algorithm for Federated Learning Mohamed Gharibi (University of Missouri-Kansas City)*; Praveen Rao (University of Missouri-Columbia)
Abstract: Federated learning (FL) is a machine learning approach where the goal is to train a centralized model using a large number of distributed datasets without accessing the actual data. FL trains a smaller version of the model at each dataset site and then aggregates all the models at the server. In practice, clients (i.e., storing the datasets) that participate in the learning process might possess corrupted or noisy datasets leading to low-accuracy models. On top of that, some clients might have malicious goals, such as poisoning the data or carry out model discovery attacks. In this paper, we propose a refining algorithm, called RefinedFed, to eliminate corrupted, low-accuracy, and noisy models that can negatively affect to the centralized model by reducing its accuracy or other malicious activities. Furthermore, our algorithm reduces the uplink communication cost with the centralized server, which in return produces a faster aggregation process on the server-side. RefinedFed shows an improvement in the centralized model accuracy from 90% to 96% with a faster training time based on a preliminary evaluation using the MNIST dataset.
T71 Multi-modal Data Analysis and Fusion for Robust Object Detection in 2D/3D Sensing Jonathan Schierl (University of Dayton)*; Theus H Aspiras (University of Dayton); Vijayan K Asari (University of Dayton); Quinn Graehling (University of Dayton); Andre Van Rynbach (Air Force Research Laboratory); David Rabb (Air Force Research Laboratory)
Abstract: Multi-modal data is useful for complex imaging scenarios due to the exclusivity of information found in each modality, but there is a lack of meaningful comparisons of different modalities for object detection. In our work, we propose three contributions: (1) Release of a multi-modal, ground-based small 42 2019 Applied Imagery Pattern Recognition Workshop object detection dataset, (2) A performance comparison of 2D and 3D imaging modalities using state-of- the-art algorithms, and (3) a multi-modal fusion framework for 2D/3D sensing. The new dataset encompasses various small objects for detection in EO, IR, and LiDAR modalities. The labeled data has comparable resolutions across each modality for better performance analysis. The modality comparison conducted in this work uses advanced deep learning algorithms, such as Mask R-CNN for 2D imaging and PointNet++ for 3D imaging. The comparisons are conducted with similar parameter sizes and the results are analyzed for specific instances where each modality performed the best. To complement the effectiveness of different data modalities, we developed a fusion strategy to combine the region proposals of one modality with the classification strength of a different modality for accurate detection and region segmentation. We investigated the functionality of the You Only Look Once (YOLO) algorithm, which computes partitioned image classification and region proposals in parallel for detection. Our fusion strategy learns the optimum features of different modality combinations for appropriate candidate selection for classification. The effectiveness of the proposed fusion method is being evaluated on the multi-modal dataset for object detection and segmentation and we observe superior performance when compared to single-modality algorithms.