Portland State University PDXScholar Dissertations and Theses Dissertations and Theses 1-1-2011 Prestructuring Multilayer Perceptrons based on Information-Theoretic Modeling of a Partido-Alto- based Grammar for Afro-Brazilian Music: Enhanced Generalization and Principles of Parsimony, including an Investigation of Statistical Paradigms Mehmet Vurkaç Portland State University Follow this and additional works at: https://pdxscholar.library.pdx.edu/open_access_etds Let us know how access to this document benefits ou.y Recommended Citation Vurkaç, Mehmet, "Prestructuring Multilayer Perceptrons based on Information-Theoretic Modeling of a Partido-Alto-based Grammar for Afro-Brazilian Music: Enhanced Generalization and Principles of Parsimony, including an Investigation of Statistical Paradigms" (2011). Dissertations and Theses. Paper 384. https://doi.org/10.15760/etd.384 This Dissertation is brought to you for free and open access. It has been accepted for inclusion in Dissertations and Theses by an authorized administrator of PDXScholar. Please contact us if we can make this document more accessible: [email protected]. Prestructuring Multilayer Perceptrons based on Information-Theoretic Modeling of a Partido-Alto -based Grammar for Afro-Brazilian Music: Enhanced Generalization and Principles of Parsimony, including an Investigation of Statistical Paradigms by Mehmet Vurkaç A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical and Computer Engineering Dissertation Committee: George G. Lendaris, Chair Douglas V. Hall Dan Hammerstrom Marek Perkowski Brad Hansen Portland State University ©2011 ABSTRACT The present study shows that prestructuring based on domain knowledge leads to statistically significant generalization-performance improvement in artificial neural networks (NNs) of the multilayer perceptron (MLP) type, specifically in the case of a noisy real-world problem with numerous interacting variables. The prestructuring of MLPs based on knowledge of the structure of a problem domain has previously been shown to improve generalization performance. However, the problem domains for those demonstrations suffered from significant shortcomings: 1) They were purely logical problems, and 2) they contained small numbers of variables in comparison to most data-mining applications today. Two implications of the former were a) the underlying structure of the problem was completely known to the network designer by virtue of having been conceived for the problem at hand, and b) noise was not a significant concern in contrast with real-world conditions. As for the size of the problem, neither computational resources nor mathematical modeling techniques were advanced enough to handle complex relationships among more than a few variables until recently, so such problems were left out of the mainstream of prestructuring investigations. In the present work, domain knowledge is built into the solution through Reconstructability Analysis, a form of information-theoretic modeling, which is used to identify mathematical models that can be transformed into a graphic representation of the problem domain’s underlying structure. Employing the latter as a pattern allows the researcher to prestructure the MLP, for instance, by disallowing certain connections in i the network. Prestructuring reduces the set of all possible maps (SAPM) that are realizable by the NN. The reduced SAPM—according to the Lendaris–Stanley conjecture, conditional probability, and Occam’s razor—enables better generalization performance than with a fully connected MLP that has learned the same I/O mapping to the same extent. In addition to showing statistically significant improvement over the generalization performance of fully connected networks, the prestructured networks in the present study also compared favorably to both the performance of qualified human agents and the generalization rates in classification through Reconstructability Analysis alone, which serves as the alternative algorithm for comparison. ii Dedicated to my mother, Sabiha Tuğcu Vurkaç iii ACKNOWLEDGMENTS I am grateful to many people for their help while I worked on this dissertation. First, I would like to thank my mother, Sabiha Tuğcu Vurkaç, for more than twenty years of emotional and financial sacrifice and support as I pursued my higher education away from home, and for a lifetime of love, friendship, teaching, guidance, and music. This dissertation would not have been possible without the insights, encouragement, mentoring, wisdom, inspiration, confidence and optimism provided by (and the support and open-mindedness of) my adviser, teacher, mentor, and ally Dr. George G. Lendaris. Dr. Lendaris has shown a level of care and engagement in both my general academic, scientific, and intellectual growth, and specifically my research and dissertation well beyond what anyone would expect. He is a true teacher and mentor. This dissertation also would not have been possible without generous and timely gifts of housing and computing resources from George Karagatchliev and Lisa Brandt Heckman (respectively) that made my continued work possible at the most difficult times of my student career. I’m similarly indebted to Eric Egalite and Joe Reid of OIT for automation and analysis assistance, respectively, and the valuable insights gathered through years of discussions and music-making with Derek Reith. I am also grateful to Woods Stricklin, Patryk Lech, Travis Henderson, Anita Rodgers, Hank Failing, Gary Beaver, Michelle Thayer, Cory Troup, Dr. Bruce Barnes, Andrew Zvibleman, Dr. Xin “Ryan” Wang, and my aunt Semra Bastıyalı for critical assistance at crucial points in the PhD process. iv In addition, I would like to specially mention the vital role played by Christina Luther, friend and SEVIS adviser, in keeping me legal and on track. Also as a primary contributor, I’d like to thank the emergent complexity that resulted from the billions of years of evolution that enabled my wet neural network to engage in such a pursuit as this dissertation. The presentation of this dissertation and associated work was greatly improved thanks to input from Woods Stricklin, Sabiha Vurkaç, Dr. Eve Klopf, Dan Craver, Dr. Ahmet Müfit Ferman, and the PSU Writing Center, and due to the technical assistance of Dr. Serap Emil (literature review), Tamara Turner and Đhsan Tunç Çakır (notation), and Buğra Giritlioğlu (music computing). I would also like to thank Dr. Melanie Mitchell for inspiring me to go into Computational Intelligence; Brian Davis, Andrew Hartzell and Derek Reith of Lions of Batucada for providing the environment for samba to become an integral part of my life; and once again, Dr. Lendaris for bringing the two realms together in my research. I would also like to thank Dr. Brad Hansen (for his all-around knowledge of the topics, and the efficiency and legitimacy he has brought to my work), Dr. Douglas V. Hall (for being a true teacher and role model), Dr. Dan Hammerstrom (for guidance at various stages, and for introducing me to Jeff Hawkins’ work), Dr. Marek Perkowski (for his excitement, encouragement, and many research sources), Dr. Fu Li (for peppering his lectures with invaluable insights), Mestre Jorge Alabê (for all his encouragement, musicality, private lessons, workshops, brain dumps, and repique solos), v “long-lost brother” Mark Lamson (for the fourth clave-direction category and for verifying my conceptualization of rhythm), Michael Spiro (for asking the tough questions, and for the IMD factor), Dr. Rob Daasch (for directing me to Dr. Lendaris), Dr. David Glenn of Whitman College (for believing in and supporting me, and giving me time and resources with which to learn to play), Dean/Professor/Cellist Robert Sylvester (for helping me recognize the breadth of my musical experience), Dr. James Morris (for recognizing my teaching ability, which in turn helped finance my studies and prepared me for a teaching career), Dr. Malgorzata Chrzanowska-Jeske (for support and guidance), Professor and Chair Harold Gray (for being a resource and friend), Dr. Martha Balshem (for helping me begin to learn how to conduct research in Ethnomusicology), Derek Reith (first tamborim teacher), Brian Davis (first pandeiro teacher), Andrew Hartzell (first surdo and repique teacher), Chris Perry (first reviewer of my clave tutorial), Tobias Manthey (for clave vigilance), Andy Sterling, David Huerta & Jesse Brooke (for clues to 12/8 directionality), Dr. Michael Cummings & Dr. Bill Becker (for early thesis-writing advice), Đhsan Tunç Çakır (for the proper way to write standard European music notation), Emily Brown, Steve White and Matthew Stanbro (for “getting it”), Sue Firpo (for substantial assistance with the paperwork and logistics of financing the early years of my PhD), and the following for their work on the golden- ear and benchmark sessions: Tobi Lehman, Ron Scroggin, Lisa Brandt Heckman, Krasi Nikolov, Chaz Mortimer, Tobias Manthey, Rafael Otto, Jake Pegg, John Jenness, Tofer Towe, Gary Beaver, Rachel Sandy, Cory Troup, Michelle Becka and Renata Secco. vi For the information-theoretic aspect of this work, I would like to thank Dr. Martin Zwick and Joe Fusion, along with Better World roadside service, and Duane at Farwell Towing. Furthermore, since this dissertation is truly my life’s integrated work, I would like to express my gratitude to Professors David Weber, Don Moor, and Peter Nicholls (for changing my brain for the better); Michelle Thayer; Cingöz Canavar