University of Vermont ScholarWorks @ UVM Graduate College Dissertations and Theses Dissertations and Theses 2016 Enabling Machine Science through Distributed Human Computing Mark David Wagy University of Vermont Follow this and additional works at: https://scholarworks.uvm.edu/graddis Part of the Artificial Intelligence and Robotics Commons Recommended Citation Wagy, Mark David, "Enabling Machine Science through Distributed Human Computing" (2016). Graduate College Dissertations and Theses. 618. https://scholarworks.uvm.edu/graddis/618 This Dissertation is brought to you for free and open access by the Dissertations and Theses at ScholarWorks @ UVM. It has been accepted for inclusion in Graduate College Dissertations and Theses by an authorized administrator of ScholarWorks @ UVM. For more information, please contact [email protected]. Enabling Machine Science Through Distributed Human Computing A Dissertation Presented by Mark Wagy to The Faculty of the Graduate College of The University of Vermont In Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Specializing in Computer Science October, 2016 Defense Date: June 2nd, 2016 Dissertation Examination Committee: Josh Bongard, Ph.D., Advisor Paul Hines, Ph.D. Robert Snapp, Ph.D. Elizabeth Pope, Ph.D., Chairperson Cynthia J. Forehand, Ph.D., Dean of Graduate College Abstract Distributed human computing techniques have been shown to be effective ways of accessing the problem-solving capabilities of a large group of anonymous individuals over the World Wide Web. They have been successfully applied to such diverse domains as computer security, biology and astronomy. The success of distributed human computing in various domains suggests that it can be utilized for complex collaborative problem solving. Thus it could be used for “machine science”: utilizing machines to facilitate the vetting of disparate human hypotheses for solving scientific and engineering problems. In this thesis, we show that machine science is possible through distributed human computing methods for some tasks. By enabling anonymous individuals to collaborate in a way that parallels the scientific method – suggesting hypotheses, testing and then communicating them for vetting by other participants – we demonstrate that a crowd can together define robot control strategies, design robot morphologies capable of fast- forward locomotion and contribute features to machine learning models for residential electric energy usage. We also introduce a new methodology for empowering a fully automated robot design system by seeding it with intuitions distilled from the crowd. Our findings suggest that increasingly large, diverse and complex collaborations that combine people and machines in the right way may enable problem solving in a wide range of fields. Citations Material from this dissertation has been published in the following form: Wagy, Mark, and Bongard, Josh C.. (2014). "Collective design of robot loco- motion." ALIFE 14: The Fourteenth Conference on the Synthesis and Simulation of Living Systems. Vol. 14. AND Wagy, Mark D., and Bongard, Josh C.. (2015). "Combining Computational and Social Effort for Collaborative Problem Solving." PloS one 10.11: e0142524. AND Wagy, Mark D., and Bongard, Josh C.. (2015). "Crowdseeding robot design." Proceedings of the Companion Publication of the 2015 on Genetic and Evolutionary Computation Conference. ACM. (Short paper / extended abstract format) AND Wagy, Mark D., and Bongard, Josh C.. (2015). "Crowdseeding: a novel approach for designing bioinspired machines." Biomimetic and Biohybrid Systems. Springer International Publishing. 293-303. Material from this thesis has been accepted for publication in “ALIFE 16: The Sixteenth Conference on the Synthesis and Simulation of Living Systems. Vol. 16.” in the following form: Wagy, Mark D., and Bongard, Josh C.. (2016). "Social contribution in the design of Adaptive Machines on the Web." ALIFE 16: The Sixteenth Conference on the Synthesis and Simulation of Living Systems. Vol. 16. Material from this dissertation has been submitted for publication to IEEE Trans- actions on Smart Grid on May 9, 2015 in the following form: Wagy, Mark D., Bongard, Josh C., Bagrow, James P., Hines, Paul D.H.. (2016) “Crowdsourcing Predictors of Residential Electric Energy Usage.” IEEE Transactions on Smart Grid. ii Acknowledgements First and foremost, I want to express deep thanks to my Ph.D. adviser, Josh Bongard, for his mentorship and guidance. I would also like to thank the members of my com- mittee: Robert Snapp, Lizzy Pope and in particular Paul Hines for his mentorship, instruction and collaboration throughout all years of my Ph.D. Additionally, I would like to thank Maggie Eppstein for helpful discussion and guidance throughout the first year of my Ph.D. I would like to extend a special thanks to my parents for their encouragement and guidance. I am also very grateful to the support of my parents-in-law, Giuseppe and Giuseppa Porcelli – in particular Giuseppa, who came to the United States to live with us and take care of our son for many of the months after he was born so that I could work on research. Grazie mille. These past few years have been enlivened and enrichened by fellow Ph.D. students- turned-friends. In particular (alphabetical), Christopher (or is it Chris?), Curtis, Dan, Emily and Tom. Oh, and Bob. And finally to my wife, Emanuela, who has been patient and supportive through- out the Ph.D experience. iii Table of Contents Citations..................................... ii Acknowledgements............................... iii List of Figures.................................. xiii List of Tables.................................. xv 1 Introduction1 2 Crowd-Contributed Robot Control Strategies 11 2.1 Introduction................................ 12 2.1.1 Interactive Evolutionary Algorithms............... 13 2.1.2 Visual Design Languages..................... 14 2.1.3 Collaborative Problem Solving.................. 15 2.2 Methodology............................... 16 2.2.1 Robot Form............................ 18 2.2.2 User Interaction.......................... 19 2.2.3 Collaborative and Partitioned Groups.............. 21 2.2.4 Evolutionary Algorithm..................... 22 2.2.5 User Incentivization....................... 22 2.3 Results................................... 23 2.4 Discussion................................. 25 2.5 Conclusion and Future Work....................... 30 3 Combining computational and social effort for collaborative problem solving 32 3.1 Abstract.................................. 32 3.2 Introduction................................ 33 3.3 Methods.................................. 35 3.3.1 Robot design by the crowd/machine and individual/machine teams............................... 36 3.3.2 Behavior optimization for the crowd/machine team and indi- vidual/machine team....................... 41 3.3.3 Communicating the results of design and optimization.... 43 3.3.4 Robot design by the machine team............... 44 3.3.5 Controller generation by the machine team........... 46 3.3.6 Behavior optimization for the machine team.......... 47 3.4 Results................................... 48 3.5 Discussion................................. 50 3.6 Conclusion................................. 56 iv 4 Social Contribution in the Design of Artifacts on the Web 60 4.1 Introduction................................ 62 4.2 Methods.................................. 64 4.3 Results................................... 68 4.4 Discussion................................. 71 4.5 Conclusion and Future Work....................... 79 5 Crowdsourcing Features in a Predictive Model 81 5.1 Introduction................................ 83 5.2 Methods.................................. 84 5.2.1 Differences from standard survey research........... 87 5.2.2 Ex post data analysis....................... 89 5.3 Results................................... 92 5.4 Discussion................................. 93 5.4.1 Contributed Questions and Participation............ 93 5.4.2 Predictive Model......................... 97 5.5 Conclusion and Future Work....................... 101 6 Crowdseeding with Observable Features 104 6.1 Introduction................................ 106 6.2 Related Work............................... 107 6.3 Methods.................................. 108 6.3.1 Stage 1............................... 109 6.3.2 Stage 2............................... 111 6.4 Results................................... 114 6.5 Discussion and Conclusions....................... 115 6.6 Future Work................................ 121 7 Crowdseeding using Automated Feature Discovery 123 7.1 Introduction................................ 124 7.2 Related Work............................... 125 7.3 Methods.................................. 126 7.3.1 First Stage: Crowdsourcing................... 126 7.3.2 Second Stage: Mining Latent Features............. 128 7.3.3 Third Stage: Seeding the Objective............... 130 7.4 Results................................... 132 7.5 Discussion................................. 132 7.6 Conclusion and Future Work....................... 135 8 Conclusion 137 v A Appendix 147 vi List of Figures 2.1 Screenshot of web-based design tool. A description of how to use the tool is on the top with an arrow indicating that the ordering of designs is from best to worst. The red arrow below the designs indicates in real-time how the current robot compares to the historical designs displayed. The GO button runs a robot simulation