Neuromedia combinations of artistic interpretation with neuroscience research for the public : tactile interfaces : the visual system : cognition and behaviour
Prof. Dr. Jill Scott Institute Cultural Studies ZHDK University of the Arts, Zurich, Switzerland
Neuromedia My Biological Research - PHD since 1993
Digital Body Automata
Premise: The idea of the human body has changed to incorporate virtual (digital), organic (material) and mechanical (artificial) concepts
Issue: Researching the ethical implications of human bio- manipulation.
Need: Understanding neurological development and perceptive impairment
Inspirations: Neuroscience
Brainport by Paul Bach-y-Rita. Neuroscientist: USA
NEUROM- Neurons Moving on a Polymer Substrate, ETH Zurich
“Embodied HCI interaction involves ubiquity, tangibility, shared awareness, intimacy and emotion” Paul Dourish
Aims of Neuromedia Neuromedia
-To produce novel media art interpretations which attempt to raise public awareness about sensory impairment -especially the per ceptions of touch and vision (sensory and motor neurons)
-To feed this interest into my long-term experience of designing sensory interfaces, which help people learn about their own perception
-To transfer what I have learnt about neuroscience not only to the public, but to include members of the public who are impaired in the formation of ideas and in an appreciation of neuroscience research.
The University of Zurich 2003-2005 Artificial Intelligence Lab 2007-8 Neurobiology Lab. Institute of Zoology 2009-2010 Institute for Molecular Biology
e-skin@Artifi cial Intelligence Lab Neuromedia
Aims:
-to incorporate a clearer unde rstanding of neural behaviour underneath the skin itself
-to contribute to a more creative sensory technology for the disabled or visually impaired
-to explore the cognitive potentials of cross modal interaction
Stage 1: AI- Building to Understand
Prototype 1: Controlling the mediated stage with 3 interfaces based on 4 skin modalities, 2003 STAGE 1: AI Building to Understand
1. Temperature - surgical sensors (shifted volume 6Ch surround sound)
2. Pressure - pressure pads (triggered sound and graphics)
3. Vibration - piazzo vibration sensors (speed of animations
4. Proprioception -infra-red (tracks user)- tilt sensors -track interface movement control the 2D plane visual layers on the screen
5. System (real time) -PICS (microcontrollers) to Linux server and 3 client macs
STAGE 2: Workshops- Visually Impaired People
e-skin@Artificial Intelligence Lab
User tests: Blindenheim Zurich -Cross-modal interaction-acoustic tactile/feedback- navigation
Test Example: Cognitive Mapping- tactile information-orientation Electro Stimulation on the skin and pattern recognition-Braille patterns/2cm apart
Results: Translation problems between the floor and the stimulation on the arm-motor abilities based on memory-but learnt new codes easily e-skin@Artificial Intelligence Lab Neuromedia
e-skin@Artificial Intelligence Lab Neuromedia
-provide a wireless and ergonomic technical infrastructure which can be explored by visually impaired as actors or dancers
-an embodied system (stage and e-skin interface)
-use these experiments to create further developments in content for the stage
-allow visually impaired people to create their own cultural mediated theatre
What did I learn? Conclusions:e-skin
-That for the visually impaired technology needs to be humanized -They need to be consulted during the process of technical development -That HCI Designers and Artists need a deeper overview from scientific research about Eyes Disease, genetics and neural degeneration. -That the variety of problems-differences in impairment need to be considered
- 5 million blind people and 135 million visually impaired people worldwide, 1.4 million children are blind, 320,000 of whom live in sub-Saharan Africa - 70% of childhood blindness may be preventable - 90% of people who are blind live in developing countries, - Cataract is the leading cause of blindness in the world: in 1998, an estimated 20 million people The Electric Retina@ The Neuromorphology Lab
-invent experiment to measure behaviour and understand human impairments through Retinal Zebra Fish Research -use results from Histologies, DNA analysis, Electromagnetic response, Opto Kinetic Response as evidence
The Optokinetic Response (OKR) Corinne and Collette-Neurobiologists in zebra fish larvae mutants Brain Inst. Uni Zurich-Stephan Neuhaus Group
The Electric Retina@ The Neuromorphology Lab
THE ELECTRIC RETINA Neuro-morphology: Visual Impairment
-The zebra fish - diurnal animals/ neuro-retinal activity = the human eye
Aims of lab researchers: -to gain a deeper insight into the wt genetic control of visual system development and function.
Phenotype: zebra fish -here wild types are breed to produce mutants with knockouts genes which mimic human disease
The Electric Retina@ The Neuromorphology Lab
Distribution of Photoreceptors in the Human Eye - Cone-daylight responders: Sculpture-used as a touch sensitive interface to show different vision impairments
The Electric Retina
Position Viewer stands "inside the brain" looking out through the retinal photoreceptors
Animated Scientific evidence in Cones -Genetic Deficiency cognitive or retinal impairments congenital nyastagmus (Belladonna) glaucoma -Light Adaptation -Protein deficiency (Fish Noir)
Projections -display the symptoms
Exhibition -Parcour des Wissens" (2008 Brain Fair, Zurich) -Life Science Zurich (2008 Central Station, Zurich) -Super Human (2009 RMIT Gallery, Melbourne )
The Electric Retina@The Neuromorphology Lab
DISPLAY IN CONES PROJECTED IMPAIRMENT
Macular Degeneration - Degeneration
Fish Noir-Mutant - Protein Deficiency
Low Pressure Glaucoma - Optic Nerve Disease
The Electric Retina@ The Neuromorphology Lab
Neuromedia - a catalyst for the public to learn : conclusion
- about how the neural system works helping to provide more sensitive interface s for the visual and tactile impaired in a "visually dominant world"
- how perception is affected by genetics, disease and degeneration
- about "their own bodies" cognitive and neural development and behaviour (interactive learning)
results also: - promote a dialogue between artists/clinical researchers/and scientific researchers (Knowhow transfer-neurobiology/the semiotics of communication)
Comments from Scientists and Clinical Researchers
-"Working alongside an artist allowed us access to different approaches and points of view about our own research and how to bring it to the public."
-"It gave us the ability to see an experiment or problem from another perspective."
-"We gained a lot of training in answering all those great "why" from the artist."
- "It was interesting for us to watch the interpretative process unfold-from conception to production and presentation of the media art work."
-"We realized that media art could be a catalyst for the opening up of more discourses and scientific debates in the future." New Contexts: Artists in Labs
What kinds of cultural programs, current debates and discourses could help art and science to gain a closer understanding of each other?
Science Disciplines: life sciences, physics, engineering and computing
Art Disciplines: art researchers - film, video, new media, sound art, sculpture, architecture, theatre and dance. Started in 2003: 12 artists in 9 labs. Shared potentials: Funded by the KTI-Innovation and Technology Group discovery, systematic methods of critical Since 2006: Funded by the Zurich analysis, human behaviour, empirical University for the Arts and the observations, criticism of science / Ministry for Culture. technology, combinations TOTAL: 28 Artists in 21 Labs
Aims of the Artists-in-Labs program
The AIL is educational and experiential
-9 month full time actual “hands on” access and immersion inside the culture of scientific research (develop and inspire their content)
-the attendance of relevant lectures and conferences held by the scientists themselves and delivery of lectures about contemporary art by artists for scientists
-be open to discourses about relation between science and the public leading to further collaboration
Thank you for listening Neuromedia
1. Artificial Intelligence Lab Dr. Rolf Pfeifer. Dr. Daniel Bisig 2. Neurobiology Lab. Institute of Zoology Uni Zurich Dr. Stephan Neuhauss, Corrine Hodel 3. Institute of Molecular Life Sciences, Developmental aspects of the vertebrate central nervous system Dr. Esther Stockli
AIL funded by BAK, Ministry for Culture, Switzerland
Prof. Dr. Jill Scott, Institute Cultural Studies ZHDK, University of Applied Science and Art Switzerland, Zurich: www.artistsinlabs.ch und www.z-node.net .