Neuromedia Combinations of Artistic Interpretation with Neuroscience Research for the Public : Tactile Interfaces : the Visual System : Cognition and Behaviour
Total Page:16
File Type:pdf, Size:1020Kb
Neuromedia combinations of artistic interpretation with neuroscience research for the public : tactile interfaces : the visual system : cognition and behaviour Prof. Dr. Jill Scott Institute Cultural Studies ZHDK University of the Arts, Zurich, Switzerland Neuromedia My Biological Research - PHD since 1993 Digital Body Automata Premise: The idea of the human body has changed to incorporate virtual (digital), organic (material) and mechanical (artificial) concepts Issue: Researching the ethical implications of human bio- manipulation. Need: Understanding neurological development and perceptive impairment Inspirations: Neuroscience Brainport by Paul Bach-y-Rita. Neuroscientist: USA NEUROM- Neurons Moving on a Polymer Substrate, ETH Zurich “Embodied HCI interaction involves ubiquity, tangibility, shared awareness, intimacy and emotion” Paul Dourish Aims of Neuromedia Neuromedia -To produce novel media art interpretations which attempt to raise public awareness about sensory impairment -especially the per ceptions of touch and vision (sensory and motor neurons) -To feed this interest into my long-term experience of designing sensory interfaces, which help people learn about their own perception -To transfer what I have learnt about neuroscience not only to the public, but to include members of the public who are impaired in the formation of ideas and in an appreciation of neuroscience research. The University of Zurich 2003-2005 Artificial Intelligence Lab 2007-8 Neurobiology Lab. Institute of Zoology 2009-2010 Institute for Molecular Biology e-skin@Artifi cial Intelligence Lab Neuromedia Aims: -to incorporate a clearer unde rstanding of neural behaviour underneath the skin itself -to contribute to a more creative sensory technology for the disabled or visually impaired -to explore the cognitive potentials of cross modal interaction Stage 1: AI- Building to Understand Prototype 1: Controlling the mediated stage with 3 interfaces based on 4 skin modalities, 2003 STAGE 1: AI Building to Understand 1. Temperature - surgical sensors (shifted volume 6Ch surround sound) 2. Pressure - pressure pads (triggered sound and graphics) 3. Vibration - piazzo vibration sensors (speed of animations 4. Proprioception -infra-red (tracks user)- tilt sensors -track interface movement control the 2D plane visual layers on the screen 5. System (real time) -PICS (microcontrollers) to Linux server and 3 client macs STAGE 2: Workshops- Visually Impaired People e-skin@Artificial Intelligence Lab User tests: Blindenheim Zurich -Cross-modal interaction-acoustic tactile/feedback- navigation Test Example: Cognitive Mapping- tactile information-orientation Electro Stimulation on the skin and pattern recognition-Braille patterns/2cm apart Results: Translation problems between the floor and the stimulation on the arm-motor abilities based on memory-but learnt new codes easily e-skin@Artificial Intelligence Lab Neuromedia e-skin@Artificial Intelligence Lab Neuromedia -provide a wireless and ergonomic technical infrastructure which can be explored by visually impaired as actors or dancers -an embodied system (stage and e-skin interface) -use these experiments to create further developments in content for the stage -allow visually impaired people to create their own cultural mediated theatre What did I learn? Conclusions:e-skin -That for the visually impaired technology needs to be humanized -They need to be consulted during the process of technical development -That HCI Designers and Artists need a deeper overview from scientific research about Eyes Disease, genetics and neural degeneration. -That the variety of problems-differences in impairment need to be considered - 5 million blind people and 135 million visually impaired people worldwide, 1.4 million children are blind, 320,000 of whom live in sub-Saharan Africa - 70% of childhood blindness may be preventable - 90% of people who are blind live in developing countries, - Cataract is the leading cause of blindness in the world: in 1998, an estimated 20 million people The Electric Retina@ The Neuromorphology Lab -invent experiment to measure behaviour and understand human impairments through Retinal Zebra Fish Research -use results from Histologies, DNA analysis, Electromagnetic response, Opto Kinetic Response as evidence The Optokinetic Response (OKR) Corinne and Collette-Neurobiologists in zebra fish larvae mutants Brain Inst. Uni Zurich-Stephan Neuhaus Group The Electric Retina@ The Neuromorphology Lab THE ELECTRIC RETINA Neuro-morphology: Visual Impairment -The zebra fish - diurnal animals/ neuro-retinal activity = the human eye Aims of lab researchers: -to gain a deeper insight into the wt genetic control of visual system development and function. Phenotype: zebra fish -here wild types are breed to produce mutants with knockouts genes which mimic human disease The Electric Retina@ The Neuromorphology Lab Distribution of Photoreceptors in the Human Eye - Cone-daylight responders: Sculpture-used as a touch sensitive interface to show different vision impairments The Electric Retina Position Viewer stands "inside the brain" looking out through the retinal photoreceptors Animated Scientific evidence in Cones -Genetic Deficiency cognitive or retinal impairments congenital nyastagmus (Belladonna) glaucoma -Light Adaptation -Protein deficiency (Fish Noir) Projections -display the symptoms Exhibition -Parcour des Wissens" (2008 Brain Fair, Zurich) -Life Science Zurich (2008 Central Station, Zurich) -Super Human (2009 RMIT Gallery, Melbourne ) The Electric Retina@The Neuromorphology Lab DISPLAY IN CONES PROJECTED IMPAIRMENT Macular Degeneration - Degeneration Fish Noir-Mutant - Protein Deficiency Low Pressure Glaucoma - Optic Nerve Disease The Electric Retina@ The Neuromorphology Lab Neuromedia - a catalyst for the public to learn : conclusion - about how the neural system works helping to provide more sensitive interface s for the visual and tactile impaired in a "visually dominant world" - how perception is affected by genetics, disease and degeneration - about "their own bodies" cognitive and neural development and behaviour (interactive learning) results also: - promote a dialogue between artists/clinical researchers/and scientific researchers (Knowhow transfer-neurobiology/the semiotics of communication) Comments from Scientists and Clinical Researchers -"Working alongside an artist allowed us access to different approaches and points of view about our own research and how to bring it to the public." -"It gave us the ability to see an experiment or problem from another perspective." -"We gained a lot of training in answering all those great "why" from the artist." - "It was interesting for us to watch the interpretative process unfold-from conception to production and presentation of the media art work." -"We realized that media art could be a catalyst for the opening up of more discourses and scientific debates in the future." New Contexts: Artists in Labs What kinds of cultural programs, current debates and discourses could help art and science to gain a closer understanding of each other? Science Disciplines: life sciences, physics, engineering and computing Art Disciplines: art researchers - film, video, new media, sound art, sculpture, architecture, theatre and dance. Started in 2003: 12 artists in 9 labs. Shared potentials: Funded by the KTI-Innovation and Technology Group discovery, systematic methods of critical Since 2006: Funded by the Zurich analysis, human behaviour, empirical University for the Arts and the observations, criticism of science / Ministry for Culture. technology, combinations TOTAL: 28 Artists in 21 Labs Aims of the Artists-in-Labs program The AIL is educational and experiential -9 month full time actual “hands on” access and immersion inside the culture of scientific research (develop and inspire their content) -the attendance of relevant lectures and conferences held by the scientists themselves and delivery of lectures about contemporary art by artists for scientists -be open to discourses about relation between science and the public leading to further collaboration Thank you for listening Neuromedia 1. Artificial Intelligence Lab Dr. Rolf Pfeifer. Dr. Daniel Bisig 2. Neurobiology Lab. Institute of Zoology Uni Zurich Dr. Stephan Neuhauss, Corrine Hodel 3. Institute of Molecular Life Sciences, Developmental aspects of the vertebrate central nervous system Dr. Esther Stockli AIL funded by BAK, Ministry for Culture, Switzerland Prof. Dr. Jill Scott, Institute Cultural Studies ZHDK, University of Applied Science and Art Switzerland, Zurich: www.artistsinlabs.ch und www.z-node.net . .