• Abstract and Figures
• Public Full-text

Special Focus Paper—An Augmented Reality U-Academy Module: From Basic Principles to Connected… An Augmented Reality U-Academy Module: From Basic Principles to Connected Subjects https://doi.org/10.3991/ijim.v11i5.7074 Paulo Menezes University of Coimbra, Coimbra, Portugal [email protected] Abstract—A module for learning about virtual and augmented reality is be- ing developed under the U-Academy project. The module is composed of three parts. The first part is an introduction to the basic concepts of virtual and aug- mented reality with the help of illustrative examples. The second part presents some of the current uses of augmented reality and its prospective use in several areas that range from industry to medicine. The final part aims at those students interested in the insights of this technology by presenting the underlying con- cepts such as: camera models, computer graphics, pattern detection and pose es- timation from inertial sensors or camera images. Keywords—Augmented Reality, Cognitive Processes, Direct Manipulation, Hand Tracking 1 Introduction Augmented reality has recently received an enormous amount of attention from both the general public and companies. Naturally, the game industry has been quite attentive to the long promised technologies to support it. There are indeed several companies doing important investments in the development of products for support- ing augmented reality (AR), like Microsoft Hololens, Vufuria, Magic Leap, and Meta 2, or for virtual reality (VR) such as HTC Vibe and Oculus Rift. As a matter of fact, some technical difficulties have limited the achievable quality of visualization in AR, and for this reason its inclusion in the games offered by the major players in this industry has been postponed until recently. But, with the new visualization devices available, and the high computational power of current game consoles and personal computers, we can say that the principal barriers to AR adop- tion have been removed. Moreover, the current awareness of the general public to this type of technology makes it impossible to ignore at the risk of losing visibility to other competing companies. While on the users side it is mainly the novelty that attracts attention, in particular amongst the younger generations, on the commercial side several companies have perceived the possibilities that this new concept was creating. As a result, we have seen the promotion of products via AR, for example by adding markers to their en- iJIM ‒ Vol. 11, No. 5, 2017 105 Special Focus Paper—An Augmented Reality U-Academy Module: From Basic Principles to Connected… closing boxes that can be used with some AR-enabled applications, typically down- loadable to smartphones and tablets. Beyond the promotional use, there are indeed several areas where augmented reali- ty may create new opportunities and added value. Fashion selling stores can use it for enabling people to try on clothes without having to put them on and off. We can also expect that it will bring important benefits to several industrial areas, and in particular to manufacturers, that have the opportunity to include it as a helping tool in assembly, inspection, or maintenance tasks. Among the foreseen uses, we can mention the use of AR for providing guidance about the sequencing of operations to be executed during the inspection of aircrafts, complex assembly procedures, or maintenance tasks. Be- yond guidance, it may support the visualization of quantities being measured at a given instant, or related with some functioning parameters of a particular machine. For all these reasons, it becomes clear that engineering students should be intro- duced to the AR concepts, as it is most likely that they will encounter this type of technology in their future workplace. Beyond the question of what AR is, how it dif- fers from VR [1,2,3], and how it can be used, the question of what is it built upon may also be explored, either by the curious student, or in the context of specific courses like computer vision (CV) or human-machine interaction (HMI). The use of AR as a motivation for computer vision can be employed to give practi- cal examples of the use of various subjects that may range from pattern recognition to projective geometry. In the case of HMI, it opens the possibility to use AR as a basis for the creation of new interaction mechanisms. These new mechanisms in turn may be applied to support activities like: AR-guided minimally invasive surgery, immer- sive teleoperation of micro or remote robots, tele-surgery and tele-diagnosis, to name but a few. The remainder of this paper will present some of the subjects that will be progres- sively integrated on this U-Academy module. The next section discusses some con- cepts that are needed to understand the difference between AR and showing infor- mation or graphics on top of images, how is reality perceived and what are the ingre- dients for creating systems capable of inducing augmented reality perceptions. Sec- tion III provides an analysis of the main types of interaction used nowadays, their limitations and the need to develop direct manipulation mechanisms. Section IV is about the development of AR applications and how it can be explored for motivating students to subjects like computer vision, signal processing, filtering and estimation, graphics programming, or even electronics. Related with the two latter subjects ex- amples are presented of inertial-based hands, and object trackers that can be used to explain both the electronics, the signal filtering and estimation processes or even computer graphics. Section V summarizes and concludes the article, by leaving point- ers for the interested reader to access the material of the module that is already availa- ble. 106 http://www.i-jim.org Special Focus Paper—An Augmented Reality U-Academy Module: From Basic Principles to Connected… 2 Augmented Reality Concepts There are indeed several misconceptions about augmented reality (AR), especially among programmers and companies willing to use the current hype to promote their products. The most common one is the notion that for creating AR, one needs to get some nice 3D model and just superimpose it on live video. In fact, that can be part of it, but it is not enough to create "augmented reality. This is similar to the subtitles that frequently appear superimposed on a movie or TV show, but which are not (perceived as) part of the scene (or “reality”) being shown. On the other hand, it has become common in sport transmissions to have virtual field marks displayed on the field, e.g. to help spectators understand why a referee has taken some decision, or why someone claims that it was a wrong decision. In these cases, those marks can be perceived as lying on the field, so they "augment" the per- ceived scenario. For this reason, we can say that this case corresponds to an example of augmented reality. 2.1 So, what is reality augmentation? To know how to augment reality we need first to understand what is reality. Is it some absolute truth or is it the result of a set of cognitive processes that involve learned concepts, mental models and perception mechanisms? As human beings, we can only verify (and accept as true) what we see, touch, hear, smell, or taste, and compare it with memories of previous experiences or with ac- quired concepts. We can say that it is the combination of what is acquired through the senses, its processing, and matching against pre-learned models that results in the perception of reality. In fact, it also involves the use of pre-acquired models and con- cepts, that may completely change the interpretation of any sensed (acquired) infor- mation. An example of how knowledge may affect reality perception can be when an adult and a child walk on a field and encounter a strawberry poison-dart frog (Oophaga pumilio). The child will probably become excited with the beauty of the frog and will want to try to catch it, while the adult will be terrified and will stop the child from doing the probably mortal move. Here the two persons will have completely different notions of reality for exactly the same situation. 2.2 Cognition and perception of the reality Our senses and cognitive processes being limited both in acquisition and pro- cessing terms, we have developed impressive capabilities of inference, recognition and reasoning, even in face of incomplete data. This is probably the result of our evo- lution as to what concerns anticipating dangers or survival advantages. The capability of using partial data has made possible the development of our visual system, which is based on 2D projections of the 3D world, and, from these 2D representations, is able to infer about the 3D structures and deal with them. But the 2D nature of this percep- iJIM ‒ Vol. 11, No. 5, 2017 107 Special Focus Paper—An Augmented Reality U-Academy Module: From Basic Principles to Connected… tual system leads to the appearance of illusions, that are just the result of some model fitting process upon incomplete or ambiguous data. Although the two-eye configuration has an important role in the perception of 3D structures, the great capability of our brain for integrating sensory information along time enables us to use self-motion to get more information about the neighbouring 3D structures, in particular when the stereo-based vision is not enough for that purpose. These movements, which are frequently done in an automatic and unconscious way, have the purpose of removing ambiguities or breaking up misinterpretations. In other words, this is the way we check how realistic is what we perceive. This can be seen as a geometry-related consistency verification, where we move to check if the 2D struc- ture we are perceiving respects some 3D mental model that was selected as hypothe- sis.

Load more

  13

Copy Link