Human-computer interaction in 3D object manipulation in virtual environments: A cognitive ergonomics contribution Sarwan Abbasi To cite this version: Sarwan Abbasi. Human-computer interaction in 3D object manipulation in virtual environments: A cognitive ergonomics contribution. Computer Science [cs]. Université Paris Sud - Paris XI, 2010. English. tel-00603331 HAL Id: tel-00603331 https://tel.archives-ouvertes.fr/tel-00603331 Submitted on 24 Jun 2011 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Human‐computer interaction in 3D object manipulation in virtual environments: A cognitive ergonomics contribution Doctoral Thesis submitted to the Université de Paris‐Sud 11, Orsay Ecole Doctorale d'Informatique Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur (LIMSI‐CNRS) 26 November 2010 By Sarwan Abbasi Jury Michel Denis (Directeur de thèse) Jean‐Marie Burkhardt (Co‐directeur) Françoise Détienne (Rapporteur) Stéphane Donikian (Rapporteur) Philippe Tarroux (Examinateur) Indira Thouvenin (Examinatrice) Object Manipulation in Virtual Environments Thesis Report by Sarwan ABBASI Acknowledgements One of my former professors, M. Ali YUSUF had once told me that if I learnt a new language in a culture different than mine, it would enable me to look at the world in an entirely new way and in fact would open up for me a whole new world for me. I have found what he had said to be entirely true. So this was one of the things which had motivated me to look for a scholarship in a country, where the primary language of communication was not English (or for that matter, any of the other local South‐Asian languages that I learnt as a child), and hence I am here today writing my thesis in France. More concretely, this research was made possible through a contract between Higher Education Commission (HEC) and the French Embassy in Pakistan. While HEC funded most of my research for which I am grateful; the French Embassy arranged for us (me and certain other selected scholars) to learn and discover the French language and its culture by means of immersive language courses as well as arranging visits to different parts of the country before the commencement of academics. My interactions with the different people that I have met in France as well as the exploration of its different cities and regions has enabled me to enrich myself personally, academically, and culturally. This research came into being as a result of a meeting that was held between me and Michel DENIS, who has become my thesis director at LIMSI‐CNRS at the Université Paris‐Sud 11 (Orsay). Since that time and up until now, it would not have been possible to carry on, had it not been for the help provided to me by him on a continual basis. He has always been there whenever I have needed him for his intervention in the capacity of my aid, be it administrative or academic. He has gone beyond the normal limits, in helping me out with progressing farther, even when sometimes I had thought that it was beyond my limits, though his aid was not limited just to encouraging me only. He responded to my emails most promptly, and always gave invaluable, timely, and relevant feedback whenever I had needed it. This was indeed extraordinary. Michel, I really thank you for that. ii Object Manipulation in Virtual Environments Thesis Report by Sarwan ABBASI I was lucky enough to have gotten a chance to work under the direct supervision of Jean‐ Marie BURKHARDT, my co‐director, and with whom I also shared the same work‐place at the Université Paris Descartes. Whether at his office desk or at lunch table, he provided me with invaluable academic feedback on a continual basis and helped me learn and generate new ideas whenever I had needed them, and made me see things from a fresh point of view. I cannot thank him enough, for other than giving me useful advises throughout my research, he also ensured that I had enough resources for my personal sustenance especially towards the end of my thesis. No research takes place in isolation and without the interaction and/or help from a vast variety of people and my research is not an exception to that. The most critical portion of the technical facilities and assistance was provided by the VENISE team at the LIMSI‐CNRS lab at Université Paris‐Sud 11 (Orsay). I would primarily like to thank Patrick BOURDOT, who is also the director of VENISE, who kindly accepted to let his virtual reality room and other facilities at his lab be used for my research. I cannot possibly thank him enough for accepting to help me, as a major part of my research would not have been possible without it. In addition to just the place and equipment, I was also helped by various other people of the VENISE team, including Pierre MARTIN, Damien TOURAINE and Jean‐Marc VEZIEN, without whose help also, the experiments in the virtual setting would not have materialised. In particular, Pierre MARTIN helped me extraordinarily, to make my underlying ideas about the virtual puzzle (especially the interactions part) come to life. I cannot thank him enough for the devoted and untiring effort that he put in for me. I am grateful to Françoise DETIENNE and Stéphane DONIKAN for agreeing to be reviewers of my thesis. They provided me feedback and suggestions which proved very valuable for me and which are reflected in the current version of the thesis. I am also thankful to Philippe TARROUX and Indira THOUVENIN who graciously accepted to be the examiners of my thesis. There are many other people that I would like to thank, who have helped me in one way or the other, and I will probably not be able to name them all here. They appear here in no iii Object Manipulation in Virtual Environments Thesis Report by Sarwan ABBASI particular order: my former colleague Rami AJAJ (researcher in graphics), a former student and an internee at Université Paris Descartes, Gosia TARASEWICZ (ergonomist), internee Adrien SCHWARTZ (ergonomics student) and Karin HEIDLMAYR (psychology student). I am thankful also to many friends and family members, particularly Safdar Abbas KHAN, Sohail IQBAL, Issa ABBASI, Tjaša NABERGOJ, Marie‐Laure LE GUEN, Asim ARIF and Taj KHAN, who helped me at different times and in different ways, whether their help was in the form of brain‐storming, typing, proof‐reading, giving ideas when I got stuck, or in the form of giving helpful advices and encouragement. I thank also all the people who graciously accepted to participate in my experiments and without whom this research would also not have been possible. Last but not the least, I would like to thank my colleagues Julien NELSON, Cécile GIRARD, Amel Ben AYED, Sophie CAPO, Jeannine BOURDEAUX, Katarina BOHACOVA and Stanislas COUIX, who helped me at different times and in different ways with my research work, as well as with my personal development. It is perhaps impossible to name everyone, and I am sure there would still be a lot of people whose names I have forgotten, and for which I sincerely apologise in advance. iv Object Manipulation in Virtual Environments Thesis Report by Sarwan ABBASI Executive Summary It is proposed to investigate the cognitive processes involved in assembly/disassembly tasks, and then to apply the findings to the design of 3D virtual environments (VEs). Virtual Environments are interactive systems that enable one or more users to interact with the simulation of objects and scenes usually in three dimensions, in a realistic fashion, by means of a set of computational techniques covering one or more sensory modalities (vision, touch, haptic, hearing, etc.). Often described as the ultimate direct manipulation interface, this technology seeks to make the interface eventually ‘disappear’ in order to provide users with a ‘natural’ mode of interaction. Virtual reality (VR) is the experience of being within a VE. One objective of the VR technology is indeed to exploit natural human behaviour without requiring any learning from their users [Fuchs2003], [Bowman2005]. Moreover, VEs are a stimulating field of research because they involve perceptually and cognitively novel situations [Burkhardt2003]. VEs also offer a large potential of innovative solutions to existing application problems. Among others, assembly tasks are a major focus for VEs [Boud2000], [Brooks1999], [Lok2003‐a], [Lok2003‐b], due to their numerous potential applications, such as assembly/disassembly of objects, scientific research (e.g., molecular docking [Ferey2009] etc.). The common feature in VEs is the use of representations and devices to support the users in handling and arranging several distinct elements in a three dimensional (3D) space under specific constraints. Most of the current devices and interaction techniques have focused on providing users with high‐fidelity sensory stimulations, rather than targeting real‐life or task‐centred functions associated with the corresponding interfaces. While many contributions have been made to the field of VR, there are only few empirical data that have been published. We believe that it is very unlikely that more adapted VEs and assistance to users’ task – in the specific context of assembly tasks – will follow either just by chance [Brooks1999], by making repeated trials, by tuning what we already have at hand, or v Object Manipulation in Virtual Environments Thesis Report by Sarwan ABBASI by more realistic sensory renderings, without any reference to the ‘specific properties of the tasks’ including its cognitive dimension.