Ensemble Algorithms and Their Applications Their and Algorithms Ensemble • Panagiotis Pintelas and • Panagiotis Ioannis Pintelas E. Livieris Ensemble Algorithms and Their Applications Edited by Panagiotis Pintelas and Ioannis E. Livieris Printed Edition of the Special Issue Published in Algorithms www.mdpi.com/journal/algorithms Ensemble Algorithms and Their Applications Ensemble Algorithms and Their Applications Editors Panagiotis Pintelas Ioannis E. Livieris MDPI • Basel • Beijing • Wuhan • Barcelona • Belgrade • Manchester • Tokyo • Cluj • Tianjin Editors Panagiotis Pintelas Ioannis E. Livieris University of Patras University of Patras Greece Greece Editorial Office MDPI St. Alban-Anlage 66 4052 Basel, Switzerland This is a reprint of articles from the Special Issue published online in the open access journal Algorithms (ISSN 1999-4893) (available at: https://www.mdpi.com/journal/algorithms/special issues/Ensemble Algorithms). For citation purposes, cite each article independently as indicated on the article page online and as indicated below: LastName, A.A.; LastName, B.B.; LastName, C.C. Article Title. Journal Name Year, Article Number, Page Range. ISBN 978-3-03936-958-4 (Hbk) ISBN 978-3-03936-959-1 (PDF) c 2020 by the authors. Articles in this book are Open Access and distributed under the Creative Commons Attribution (CC BY) license, which allows users to download, copy and build upon published articles, as long as the author and publisher are properly credited, which ensures maximum dissemination and a wider impact of our publications. The book as a whole is distributed by MDPI under the terms and conditions of the Creative Commons license CC BY-NC-ND. Contents About the Editors .............................................. vii Panagiotis Pintelas and Ioannis E. Livieris Special Issue on Ensemble Learning and Applications Reprinted from: Algorithms 2020, 13, 140, doi:10.3390/a13060140 .................. 1 Ioannis E. Livieris, Andreas Kanavos, Vassilis Tampakas, Panagiotis Pintelas A Weighted Voting Ensemble Self-Labeled Algorithm for the Detection of Lung Abnormalities from X-Rays Reprinted from: Algorithms 2019, 12, 64, doi:10.3390/a12030064 ................... 5 Konstantinos I. Papageorgiou, Katarzyna Poczeta, Elpiniki Papageorgiou, Vassilis C. Gerogiannis and George Stamoulis Exploring an Ensemble of Methods that Combines Fuzzy Cognitive Maps and Neural Networks in Solving the Time Series Prediction Problem of Gas Consumption in Greece Reprinted from: Algorithms 2019, 12, 235, doi:10.3390/a12110235 .................. 21 Emmanuel Pintelas, Ioannis E. Livieris and Panagiotis Pintelas A Grey-Box Ensemble Model Exploiting Black-Box Accuracy and White-Box Intrinsic Interpretability Reprinted from: Algorithms 2020, 13, 17, doi:10.3390/a13010017 ................... 49 Stamatis Karlos, Georgios Kostopoulos and Sotiris Kotsiantis A Soft-Voting Ensemble Based Co-Training Scheme Using Static Selection for Binary Classification Problems Reprinted from: Algorithms 2020, 13, 26, doi:10.3390/a13010026 ................... 67 Konstantinos Demertzis and Lazaros Iliadis GeoAI: A Model-Agnostic Meta-Ensemble Zero-Shot Learning Method for Hyperspectral Image Analysis and Classification Reprinted from: Algorithms 2020, 13, 61, doi:10.3390/a13030061 ................... 87 Kudakwashe Zvarevashe and Oludayo Olugbara Ensemble Learning of Hybrid Acoustic Features for Speech Emotion Recognition Reprinted from: Algorithms 2020, 13, 70, doi:10.3390/a13030070 ...................113 Giannis Haralabopoulos, Ioannis Anagnostopoulos and Derek McAuley Ensemble Deep Learning for Multilabel Binary Classification of User-Generated Content Reprinted from: Algorithms 2020, 13, 83, doi:10.3390/a13040083 ...................137 Ioannis E. Livieris, Emmanuel Pintelas, Stavros Stavroyiannis and Panagiotis Pintelas Ensemble Deep Learning Models for Forecasting Cryptocurrency Time-Series Reprinted from: Algorithms 2020, 13, 121, doi:10.3390/a13050121 ..................151 v About the Editors Panagiotis Pintelas, Ph.D. professor of Computer Science with the Informatics Division of Department of Mathematics at University of Patras, Greece. His research interests include software engineering, AI and ICT in education, machine learning and data mining. He was involved in and directed several dozens of National and European research and development projects. Scholar link: https://scholar.google.gr/citations?user=6UFNG84AAAAJ&hl=el&oi=ao. Ioannis E. Livieris, Ph.D. holds a Bachelor degree in Mathematics and a Ph.D in Computational Mathematics and Neural Networks from University of Patras. His research includes more than 40 publications in high-level, peer-reviewed journals, 13 volumes in books and 11 peer-reviewed conferences. His scientific interests include neural networks, deep learning, data mining and computational intelligence. Scholar link: https://scholar.google.gr/citations?hl=el&user=0h3 4goAAAAJ. vii algorithms Editorial Special Issue on Ensemble Learning and Applications Panagiotis Pintelas * and Ioannis E. Livieris Department of Mathematics, University of Patras, 265-00 GR Patras, Greece; [email protected] * Correspondence: [email protected] Received: 5 June 2020; Accepted: 9 June 2020; Published: 11 June 2020 Abstract: During the last decades, in the area of machine learning and data mining, the development of ensemble methods has gained a significant attention from the scientific community. Machine learning ensemble methods combine multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone. Combining multiple learning models has been theoretically and experimentally shown to provide significantly better performance than their single base learners. In the literature, ensemble learning algorithms constitute a dominant and state-of-the-art approach for obtaining maximum performance, thus they have been applied in a variety of real-world problems ranging from face and emotion recognition through text classification and medical diagnosis to financial forecasting. Keywords: ensemble learning; homogeneous and heterogeneous ensembles; fusion strategies; voting schemes; model combination; black, white and gray box models; incremental and evolving learning 1. Introduction This article is the editorial of the “Ensemble Learning and Their Applications”(https://www.mdpi. com/journal/algorithms/special_issues/Ensemble_Algorithms) Special Issue of the Algorithms journal. The main aim of this Special Issue is to present the recent advances related to all kinds of ensemble learning algorithms, frameworks, methodologies and investigate the impact of their application in a diversity of real-world problems. The response of the scientific community has been significant, as many original research papers have been submitted for consideration. In total, eight (8) papers were accepted, after going through a careful peer-review process based on quality and novelty criteria. All accepted papers possess significant elements of novelty, cover a diversity of application domains and introduce interesting ensemble-based approaches, which provide readers with a glimpse of the state-of-the-art research in the domain. During the last decades, the development of ensemble learning methodologies and techniques has gained a significant attention from the scientific and industrial community [1–3]. The basic idea behind these methods is the combination of a set of diverse prediction models for obtaining a composite global model which produces reliable and accurate estimates or predictions. Theoretical and experimental evidence proved that ensemble models provide considerably better prediction performance than single models [4]. Along this line, a variety of ensemble learning algorithms and techniques have been proposed and found their application in various classification and regression real-word problems. 2. Ensemble Learning and Applications The first paper is entitled “A Weighted Voting Ensemble Self-Labeled Algorithm for the Detection of Lung Abnormalities from X-Rays” and it is authored by Livieris et al. [5]. The authors presented a new ensemble-based semi-supervised learning algorithm for the classification of lung abnormalities from chest X-rays. The proposed algorithm exploits a new weighted voting scheme which assigns a vector of weights on each component learner of the ensemble based on its accuracy on each class. The proposed algorithm was extensively evaluated on three famous real-world benchmarks, namely the Pneumonia Algorithms 2020, 13, 140; doi:10.3390/a130601401 www.mdpi.com/journal/algorithms Algorithms 2020, 13, 140 chest X-rays dataset from Guangzhou Women and Children’s Medical Center, the Tuberculosis dataset from Shenzhen Hospital and the cancer CT-medical images dataset. The presented numerical experiments showed the efficiency of the proposed ensemble methodology against simple voting strategy and other traditional semi-supervised methods. The second paper is authored by Papageorgiou et al. [6] entitled “Exploring an Ensemble of Methods that Combines Fuzzy Cognitive Maps and Neural Networks in Solving the Time Series Prediction Problem of Gas Consumption in Greece”. This paper presents an innovative ensemble time-series forecasting model for the prediction of gas consumption demand in Greece. The model is based on an ensemble learning technique which exploits evolutionary Fuzzy Cognitive Maps (FCMs), Artificial Neural Networks (ANNs) and their hybrid structure, named FCM-ANN, for time-series