Design and Implementation of Multi-Sensory Fabric As Deformable Musical Interface
Total Page:16
File Type:pdf, Size:1020Kb
Design and Implementation of Multi-sensory Fabric as Deformable Musical Interface by Irmandy Wicaksono B.Eng. Electronic Engineering, University of Southampton (2014) Submitted to the Department of Electrical Engineering and Information Technology, in partial fulfilment of the requirements for the degree of Master of Science in Electrical Engineering and Information Technology Areas of Specialisation: Electronics and Photonics at the EIDGENÖSSISCHE TECHNISCHE HOCHSCHULE ZÜRICH SWISS FEDERAL INSTITUTE OF TECHNOLOGY ZURICH November 2016 © MIT, ETH Zürich, 2016. All Rights Reserved. 1 signature redacted signature redacted signature redacted Abstract This work presents StretchyKeyboard: a multi-modal fabric sensate surface as a deformable musical interface. Multi-layer fabric sensors that detect touch, proximity, electric field, pressure, and stretch were machine-sewn in a keyboard pattern on a stretchable substrate. The result is a fabric-based musical controller that combines both the discrete control of a keyboard and various continuous controls from the fabric sensors. This enables new tactile experiences and novel interactions both with physical and non-contact gestures: physical by pressing, pulling, stretching, and twisting the fabric and non-physical by hovering and waving towards/against the keyboard and a field source. We also developed other fabric interfaces such as ribbon-controller and trackpad allowing performer to add more expressive, position sensing controls. The multi-modal fabric sensate surface demonstrates an effort towards seamless, self-aware, and washable media. Supervisor Prof. Joseph A Paradiso, Responsive Environments, MIT Media Lab Co-supervisor Prof. Gerhard Tröster, Wearable Computing Lab, ETH Zürich Thesis Reader Dr. Nan-Wei Gong, Responsive Environments, MIT Media Lab 3 4 Acknowledgments In this opportunity, I would like to sincerely thank Prof. Joe Paradiso for giving me the opportunity to be a part of Responsive Environments, for his continuous support, guidance, and patience. Electronics and Music have always been the two, separate, big chunks of my life; I could barely find the connections between them, until I met Prof. Joe. His admirable attitude and his work have inspired me and changed my perspectives about what’s possible and the future. Media Lab is such an interesting place. These past six months, have definitely been the most challenging and enjoyable time throughout my academic journey -- I’ve never thought that I’d be able to make my own musical controller! I would also like to express my gratitude to Prof. Gerhard Tröster. For inspiring me with his visions, motivating me to pursue them, and shaping me with his teachings. I gained a lot of knowledge and insights from his lectures and my time at the Wearable Computing Lab, and I am very thankful for that. Thank you to Dr. Nan-Wei Gong for spending her time in reading my thesis and for the initial discussions of this project, to John DiFrancesco and Sam Calisch from Centre of Bits and Atoms for the help with laser cutting and Instron, to Amna and Keira for making everything possible, and to the Zeno Karl Schindler Foundation for supporting me throughout my time here at MIT. To Responsive Environments crew, thank you for being the superstars, for being kind and supportive from the first day, for all of the deep conversations, for making it feels right, for everything. To all my new and old friends that I’ve met these past two years of my Master study, either at ETH, KAUST, or MIT, either in Zurich, Jeddah, Boston, or back home, you guys have made me realize how tiny this world is, encouraging me to further explore it, and that there is no such thing as goodbye. Finally, to my family and my girlfriend, thank you for being patience, for always understanding me, and believing in me. I miss you and I’ll see you soon. I would like to end my thesis by quoting my favourite writer, who reminds me to always look forward to the future, to always imagine all of the possibilities. Looking back, I’d never have thought that the textiles I was seeing back then could be the textiles that I am seeing today. “And above all, watch with glittering eyes the whole world around you because the greatest secrets are always hidden in the most unlikely places. Those who don't believe in magic will never find it.” Roald Dahl (1991) The Minpins Irmandy Wicaksono Cambridge, MA 27/11/2016 5 Contents Abstract ................................................................................................................................................. 3 Acknowledgements ............................................................................................................................ 5 Contents ................................................................................................................................................ 6 List of Figures .................................................................................................................................... 10 List of Tables ...................................................................................................................................... 13 List of Abbreviations ........................................................................................................................ 14 Chapter 1: Introduction .................................................................................................................... 15 Chapter 2: Background Research and Related Work .................................................................. 17 2.1 Keyboard Instruments: from Acoustical to Digital .................................................... 17 2.1.1 Early Electronic Musical Controllers ............................................................ 18 2.1.2 Integration of Discrete and Continuous Controls ...................................... 20 2.1.3 Flexible and Deformable Musical Interface ................................................. 21 2.1.4 Parameter Mapping in Electronic Instrument Design ............................... 22 2.2 Smart Textiles: from Manufacturing to Applications ................................................ 23 2.2.1 Textile Fibers and Manufacturing Techniques ........................................... 24 2.2.2 Textile-based Electronics ................................................................................ 24 2.2.2.1 Conductive Threads and Fabrics .................................................. 24 2.2.2.2 Textile-based Sensors ..................................................................... 25 2.2.2.2.1 Muscle Actuated ............................................................. 25 2.2.2.2.1.1 Touch ................................................................ 25 2.2.2.2.1.2 Pressure ............................................................ 26 2.2.2.2.1.3 Position ............................................................. 27 6 2.2.2.2.1.4 Strain ................................................................. 27 2.2.2.2.1.5 Stroke and Bend .............................................. 28 2.2.2.2.2 Other types ...................................................................... 29 2.2.2.2.2.1 Temperature .................................................... 29 2.2.2.2.2.2 Humidity .......................................................... 29 2.2.2.2.2.3 pH ......................................................................... 30 2.2.2.2.2.4 Chemical ........................................................... 30 2.2.3 System-On-Textile ........................................................................................... 31 2.2.4 Applications of Textile-based Computing ................................................... 33 2.3 Multi-modal Electronic skin and Sensate Surface ...................................................... 34 2.4 Project Goals and Contribution ..................................................................................... 35 Chapter 3: Multi-sensory Fabric Keyboard .................................................................................. 37 3.1 Structure and Construction ........................................................................................... 37 3.2 E-threads and Stretchable Interconnects...................................................................... 40 3.3 Electro-active Fabrics ...................................................................................................... 45 3.4 Soft-Hard Connections ................................................................................................... 47 Chapter 4: Sensor Design and Characterization.......................................................................... 49 4.1 Sensing Elements ............................................................................................................ 49 4.1.1 Touch and Proximity ...................................................................................... 49 4.1.2 Electric Field .................................................................................................... 52 4.1.3 Pressure ............................................................................................................ 54 4.1.4 Stretch ............................................................................................................... 57 4.2 Additional Fabric Interfaces .......................................................................................... 62 4.2.1 Fabric Ribbon Controller ...............................................................................