Mark F. Bocko | Professor Department: Electrical and Computer Engineering Focus: Spatial audio
Pilot Project: “Development of a quantitative framework for spatial audio characterization”
Project Goals • Develop quantitative methods to assess spatial audio rendering systems • Incorporate quantitative binaural hearing models into audio system design tools • Predict what listeners will report hearing (locations, spatial extent of sources, diffusiveness)
October 1, 2018 Geunyoung Yoon | Professor
Department: Ophthalmology, The Institute of Optics, Center for Visual Science, Biomedical Engineering Focus: Physiological Optics, Vision Correction, Visual Psychophysics, Optical Imaging, Biomechanics, Eye Diseases Lab website: http://www.cvs.rochester.edu/yoonlab/
RESEARCH TOPICS:
OCULAR OPTICS & CUSTOMIZED ACOOMMODATION & PRESBYOPIA VISION CORRECTION • Vergence-Accommodation conflict • Eye’s aberration and visual quality under VR/AR environments • Ocular wavefront sensing • Extended depth of focus technology • Advanced ophthalmic lenses • Accommodating intraocular lens • Sport vision • Peripheral vision and optics • Optical metrology • Binocular accommodation • Emmetropization / Refractive error OCULAR OPTICS and VISION ANTERIOR SEGMENT IMAGING • Adaptive optics vision simulator • Mechanisms of pathologic cornea • Adaptation to habitual optics diseases • Neural processing and perception • Ocular surface diseases and dry eye • Binocular integration • Corneal biomechanics • Neural plasticity • Multimodal high-resolution ocular • Stereopsis imaging • Advanced cataract surgery
October 2018 Zhiyao Duan | Assistant Professor Department: Electrical and Computer Engineering Focus: Computer Audition, Music Information Retrieval, Audio-Visual Analysis, Audio for AR/VR Lab: Audio Information Research (AIR) Lab
RESEARCH TOPICS:
MUSIC INFORMATION RETRIEVAL SPEECH PROCESSING • Music transcription • Speech emotion classification • Audio-score alignment • Speaker recognition and diarization • Music source separation • Speech enhancement • Music generation
ENVIRONMENTAL SOUND AUDIO-VISUAL PROCESSING UNDERSTANDING • AV analysis of music performances • Sound search by vocal imitation • Visually informed source separation • Sound event detection • Music performance generation • Source localization and tracking • Talking face generation from speech
October 2018 Andrew White| Assistant Professor Department: Chemical Engineering Focus: Role of AR/VR in higher education, computational chemistry Courses: CHE 116 – Numerical Methods & Statistics | CHE 477 – Advanced Numerical Methods
RESEARCH TOPICS:
Role of AR and VR in Higher Education Computational Chemistry • Collaborative project exploring the use of • Computer simulation of dynamics of AR and VR in STEM curriculum molecules at the level of atoms • • Emphasis on tactile, collaborative, Provides insight at a length-scale inaccessible interactive replacement for traditional to experiments hands-on labs • Requires careful design of scientifically • Intended for topics that are abstract or accurate, highly-parallel, algorithms and code impossible to have labs, e.g. quantum • mechanics or solving ODEs Use techniques like multi-scale modeling to study complex phenomena like protein • Provides a new tool for outreach to adsorption generate enthusiasm for STEM careers • High accuracy through novel methods to incorporate data derived from experiments in simulations
October 2018 Yuhao Zhu | Assistant Professor Department: Computer Science, Goergen Institute for Data Science Focus: Architecting next-generation computer hardware for an AR/VR-driven future!
RESEARCH TOPICS: Co-Design Computing Systems with Non-computing Systems for AR/VR • Optical system + image sensor + imaging + computer vision (w/ Jannick Rolland) • Does the optimal design of optical system change with the specific vision task? • Does the optimal design of vision hardware change with optical systems? • How to design an end-to-end system with specific tasks and quality metrics in mind? • How to dynamically reconfigure both optical systems and computer systems on the fly?
October 2018 Michele Rucci | Professor Department: Brain & Cognitive Sciences Focus: Action & perception, visual perception in humans and machines, human behavior, sensory processing
RESEARCH TOPICS:
COMPUTATIONAL MECHANISMS ACTIVE VISION • Computational goals of visual • Vision as sensorimotor integrated processing process • Establishment of spatial • Dependence of visual functions on representations eye movements • Multimodal integration • Disruption of oculomotor cycle via gaze-contingent control
HUMAN BEHAVIOR VISUAL DEVELOPMENT • Action properties • Consequences of eye movements in • Identification of visuomotor visual maturation strategies • Abnormal eye movements • Limits of oculomotor control • Myopia • Natural head-eye coordination
October 2018 Building Virtual Concert Halls with Spatial Audio MingMing-Lun-LunLee,Lee, Matthew Matthew Brown, Brown, ZhiyaoDuanDuan, ,and and Steven Steven Philbert Philbert Department of Electrical and Computer Engineering Department of Electrical and Computer Engineering Eastman School of Music Eastman School of Music Zhen Bai| Assistant Professor Department: Computer Science Focus: Human-Computer Interaction, Augmented Reality, Tangible User Interface, Embodied Conversational Agent, Education Technology, Computer-Supported Collaboration, Design for Diversity
RESEARCH TOPICS: Augmented Reality - Theory of Mind, Symbolic Play, Children with Autism Spectrum Condition
[email protected] http://zhenbai.io October 2018 UR AR/VR Pilot Project: Real-time synthesis of a virtual talking face from acoustic speech
• Chenliang Xu, Assistant Professor of Computer Science • Collaborators: • Ross Maddox, Assistant Professor of Biomedical Engineering • Zhiyao Duan, Assistant Professor of Electrical and Computer Engineering
[Chen, Li, Maddox, Duan, and Xu, ECCV 2018] Ross Maddox | Assistant Professor Department: Neuroscience & Biomedical Engineering Focus: Audio-visual integration, selective attention, sound localization
RESEARCH TOPICS:
AUDIO-VISUAL INTEGRATION GAZE EFFECTS ON SPATIAL HEARING • Multisensory binding and object • Interaction between eye formation movements and auditory spatial • Impact of “uninformative” visual acuity stimuli on auditory perception • Benefits of directed eye gaze on speech-in-noise understanding
VISUAL HEARING AID BRAINSTEM CODING OF SPEECH • Generate an artificial talking face • Use electroencephalography and from speech audio in real-time novel signal processing schemes to • Improve listening abilities for study how brainstem codes speech people with hearing impairments, • Investigate subcortical effects of attention disorders attention and cognition
October 2018 Martina Poletti| Assistant Professor Department: Neuroscience Focus: Visual perception, eye movements, attention, eyetracking.
RESEARCH TOPICS:
VISUOSPATIAL ATTENTION SPATIAL REPRESENTATIONS • Resolution of attention in the fovea • Multimodal integration • Attention contribution to fine • Spatial updating across saccades spatial vision • Spatial updating during fixation • Pre-microsaccadic enhancements of foveal vision
FOVEAL PRIORITY MAPS HIGH ACUITY VISION • Driving factors • Fine control of eye movements • Perceptual benefits during high acuity tasks • Spatiotemporal dynamics • Distribution of high acuity • Visual exploration at the foveal capabilities across the fovea scale • Oculomotor strategies in fine spatial vision
October 2018 River Campus Libraries Presenter: Lauren Di Monte, Director of Research Initiatives Focus: Support AR/VR research, teaching, and learning
AR/VR Creation and Exploration Space
Enhance access and support Provide on-ramps Grow a community of practice
October 2018 Cognitive Behavior Therapy Mobile App with Embedded Virtual Reality
UR Medicine Health Lab (Hasselberg, Mitten, Dasilva) • Expertise in technology innovation to improve delivery of care
Department of Psychiatry (Cross, Hasselberg) • Expertise in cognitive behavior therapy, and implementation science
Eastman School of Music (Brown, Winders) • Expertise in visual and audio therapeutic functions
Art, Science, & Engineering (Luo) • Expertise in computer science and smartphone sensors
13 Edmund Lalor | Associate Professor Department: Biomedical Engineering and Neuroscience Focus: Human neuroscience, sensory processing, perception, cognition
RESEARCH TOPICS:
HUMAN SENSORY PROCESSING ATTENTION • Hierarchical processing of natural • Visual spatial attention audio and visual stimuli • Auditory selective attention – • The effects of knowledge and particularly to speech (i.e., the prediction on early sensory cocktail party problem) processing
MULTISENSORY INTEGRATION NEURAL SIGNAL DECODING • Audiovisual speech processing • Methods for decoding multivariate • The effect of visual input on neural data auditory scene segregation • Decoding representations of acoustic space in the cortex • Decoding selective attention in real- time
October 2018 Ania Busza | Assistant Professor Department: Neurology (Stroke division) Focus: Stroke Rehabilitation
RESEARCH TOPICS:
KEY ISSUES IN STROKE REHABILITATION AND RECOVERY • What factors predict stroke recovery? • What is the best timing/dose for rehabilitation therapies? • How can we use new technologies to create more effective therapeutics?
(1) Surface-EMG controlled (2) Using superficial sensors to (3) EMG analysis of Motor Virtual Arm quantify rehab “dose” System fatigue during learning
October 2018 Michael Jarvis| Associate Professor Director, Digital Media Studies Program Departments: History & ATHS
RESEARCH: VR, Visualization & Analysis of Cultural Heritage Sites; VR and Public History
UR/University of Ghana Digital Archaeology & Structural Analysis Field Research, 2017-present: VR applications
October 2018 Jessica M. Keith| Doctoral Candidate Department: Clinical & Social Sciences in Psychology; Research Lab: Dev. Neuropsychology Lab (PI: Loisa Bennetto, PhD) Self-regulation processes underlying social motivation in ASD: The influence of social context Aims 1 & 2: Self-regulation (executive functioning & autonomic regulation) à social motivation Executive Functioning Tasks (done with and without social noise): Social Motivation: Collecting 2. Inhibitory 3. Cognitive HRV & EDA 1. Decision during tasks Making Control Flexibility
Aim 3: Self-regulation (autonomic & behavioral reactivity/regulation) during immersive, 360 social filmà social motivation
Measuring: autonomic • Custom-made reactivity/regulation & head orientation video (~6 min) • Spatial audio • Progressively more directive social overtures made toward participant Benjamin T. Crane| Associate Professor Department: Otolaryngology, Biomedical Engineering and Neuroscience Focus: Human visual-vestibular multisensory integration, motion perception, rehabilitation of vestibular lesions
RESEARCH TOPICS:
VISUAL-VESTIBULAR INTEGRATION PATHOLOGICAL MOTION PERCEPTION • Common causality between visual • Rotation perception and inertial stimuli • Heading offset • Disambiguation of extern vs. self • Integration with visual motion motion • Effect of unilateral lesions • Migraine Acute Chronic
VISUAL-VESTIBULAR ADAPTATION NEW METHODS OF VESTIBULAR • Use of visual-inertial heading offsets REHABILITATION to adapt heading • Develop method for rehabilitation V V I • Exposure to a rotating environment of heading deviation for heading adaptation • Decoding representations of acoustic space in the cortex • Decoding selective attention in real- I time
October 2018 Duje Tadin | Professor
Department: Brain & Cognitive Science, Center for Visual Science, Neuroscience, Ophthalmology Co-Director, Center for Augmented and Virtual Reality (Neuroscience) Focus: Visual perception, Multisensory processing, Brain plasticity Conflict of interest declaration: Senior Scientist, NeuroTrainer
RESEARCH TOPICS:
VISUAL PERCEPTION & COGNITION MULTISENSORY PROCESSING • Motion perception • Audiovisual interactions • Visual attention & awareness • Vision & proprioception • Binocular vision • Synesthesia • Visual adaptation • Cognitive assessment in VR
BRAIN PLASTICITY METHODS • Perceptual learning • Psychophysics • Cognitive training • Computational modeling • Brain training in VR • Eye tracking • Used in heathy adults, dementia, • Brain stimulation stroke, corneal disease, concussion, • Neuroimaging schizophrenia.
October 2018