Visualization Optimization: Application to the RoboCup Rescue Domain

2 3 Pedro Miguel Moreira1,4, Luis Paulo Reis ,4 and A. Augusto de Sousa ,4

1ESTG-IPVC - Escola Superior de Tecnologia e Gesrao de Viana do Castelo, 2LlACC - Laboratorio de Inteligencia Artificial e Ciencia de Computadores da Univ. Porto, Portugal 3INESC-Porto - Instituto de Engenharia de Sistemas e Computadores do Porto, Portugal 4FEUP - Faculdade de Engenharia da Universidade do Porto, Portugal

Abstract In this paper we demonstrate the use of intelligent optimization methodologies on the visualization optimization of virtual/simulated environments. The problem of automatic selection of an optimized set of views, which better describes an on-going simulation over a virtual environment is addressed in the context of the RoboCup Rescue Simulation domain. A generic architecture for optimization is proposed and described. We outline the possible extensions of this architecture and argue on how several problems within the fields of Interactive Rendering and Visualization can benefitfrom it.

Categories and Subject Descriptors (according to ACM CCS): 1.3.7 [Computer Graphics]: 1.3.7 Virtual reality 1.2.8 []: Problem Solving, Control Methods, and Search

Additional Info corresponding author: [email protected]

Submitted to: SIACG 2006 Ibero-American Symposium on Computer Graphics 2006, Santiago de Compostela, , July 2006. Visualization Optimization: Application to the RoboCup Rescue Domain

l 2 3 Pedro Miguel Moreira ,4t, Luis Paulo Reis ,4:j: and A. Augusto de Sousa ,4§

I ESTG-IPVC - Escola Superior de Tecnologia e Gestao de Viana do Castelo, Portugal 2LIACC - Laborat6rio de Inteligencia Artificial e Ciencia de Computadores da Univ. Porto, Portugal 3INESC-Porto - Instituto de Engenharia de Sistemas e Computadores do Porto, Portugal 4FEUP - Faculdade de Engenharia da Universidade do Porto, Portugal

Abstract In this paper we demonstrate the use ofintelligent optimization methodologies on the visualization optimization ofvirtual/simulated environments. The problem ofautomatic selection ofan optimized set ofviews, which better describes an on-going simulation over a virtual environment is addressed in the context ofthe RoboCup Rescue Simulation domain. A generic architecture for optimization is proposed and described. We outline the possible extensions ofthis architecture and argue on how several problems within the fields ofInteractive Rendering and Visualization can benefit from it.

Categories and Subject Descriptors (according to ACM CCS): 1.3.7 [Computer Graphics): 1.3.7 Virtual reality 1.2.8 [Artificial Intelligence): Problem Solving, Control Methods, and Search

1. Introd uction ally feasible due to the inherent problem complexity and to time constraints. Thus, meta-heuristics are used tollnd a best In this paper we address the problem of automatically find (suboptimal) solution. a fixed set of views (multi-view) over a three dimensional dynamical and evolving simulated environment that gives to the user a good representation. We report the application In our approach we propose an optimization architecture to the RoboCup Rescue Domain where we aim at obtain­ relying on an optimization agent that works autonomously ing, at each moment, the set of views which better describe from the main visualization / rendering application. Our ob­ the emergency situations and rescue operations. These multi­ jective is to develop the proposed architecture in order to view should provide the user with useful information leading apply it on other problems with minor effort. to a correct understanding of the whole environment.

The problem offinding the best set of views over a scene The optimization agent provides a set use of intelligent can be stated as an optimization problem. This is an in­ optimization techniques [PKOO], such as (but not restricted teresting and useful problem which relates to other prob­ to): genetic algorithms, tabu search, simulated annealing, or lems such as : automatic object [VS03] and scene explo­ neural networks to efficiently find optimized solutions. ration [AVF04], virtual camera motion [MCOO], virtual cinematography [DZ95] and automatic selection of images The rest of the paper is organized as follows. Following to Image-Based Modeling and Rendering (IBM&R) sys­ this introduction, Section 2 describes the aplication domain tems [VFSH03]. Finding the optimal solution is not usu- - Robocup Rescue, where reported experiments were con­ ducted. Next, in Section 3, we detail and formalize the ad­ dressed problem. Our proposed methodology is presented in t [email protected] Section 4. In Section 5 experimental results are presented :j: [email protected] and discussed. Finally, in Section 6, conclusions and outlines § [email protected] ofour on going and future work are presented. o P. M. Moreira, L. P. Reis & A, A, de Sousa / Visualization Optimization Application to the RoboCup Rescue Domain

2. RoboCup rescue domain in this area, but the need for a more comprehensive tool was only made more visible [AriOS]. The Freiburg team, RoboCup was created as an international research and ed­ amongst others, has developed its own viewer, releasing it to ucation initiative, aiming to foster artificial intelligence and the rescue community. Freiburg's 3D viewer [KGOS] is one research, by providing a standard problem, where of the most used by the community, second only to Mori­ a wide range of technologies can be examined and inte­ moto's 2D viewer [Mor02], which is included in the official grated [Ano06], simulator package. The huge success ofthe RoboCupSoccer international re­ search and education initiative, led the RoboCup Federation Our purpose was to develop a visualization tool to the to create the RoboCupRescue project focussing on Urban Robocup Rescue Domain, that features a multi-view over Search and Rescue (USAR) operations [Ano06]. the simulated environment. Camera positions are restricted to existing rescue agents or entities (such as buildings, po­ The RoboCupRescue Simulation League consists of a lice, fire brigades, etc). Aerial views are planned but not yet simulated city in which heterogeneous simulated , act­ implemented. Users monitoring the rescue simulation should ing in a dynamic environment, coordinate efforts to save benefit from such tool since they are provided with a fixed people and property. Heterogeneous robots in a multi- (and small, e.g. four) number ofviews selected based on cri­ system share a common goal, but have different abilities teria that tries to optimize the relevance ofthe virtually cap­ and specializations, adding further complexity and strategic tured imagery to the understanding of the evolving simula­ options. These systems can manifest self-organization and tion. Our viewer is partially based on [KGOS]. complex behaviors even when the individual strategies ofall the robots are simple. The team-programmed robots are of three different types: Fire Brigades, Police Forces and Am­ 3.1. Problem description bulance Teams. Fire Brigades are responsible for extinguish­ ing fires; Police Forces open up blocked routes; and Ambu­ The problem can be informally stated as: In an urban res­ lance Teams unbury Civilians trapped under debris. Each of cue setting there are m visualization agents that can obtain these types of robots is coordinated by an intelligent cen­ views over the scene. The objective is to find an optimal set tre responsible for communication and strategies. In order to of k views that better describe the simulation for each mo­ obtain a good score, all these robots work together to explore ment. These k views can have a different purpose, but for the city, extinguish fires, and unbury Civilians. the sake of simplicity, we restrict the discussion to the mon­ itoring ofemergency situations. The visualization agents are controllable in the sense that one can affect their viewing parameters. The optimization problem can be fonnalized as follows:

v = {VI,'" VOl}

v;=!(Pas;,VD"VUP"FoV,) iE {l, ... ,m}

Figure 1: RoboCup Rescue Simulated Environment MV = {mvi , .. . ,mvd whereMV c V and mv; ¥-mvj Vi¥-J

MAXIMIZE: Q(MV) = The simulated environment, Fig. I is composed ofseveral sub-simulator modules such as the fire simulator and block­ L)=I L7=1 Vis(e{).Red(e;IMV)(WI.Rel(e;) + W2. Ecc(e{)) ade simulator. This structure allows for independent module development, permitting the addition of new modules and making the simulator system more realistic. where, E is the set of n entities that have relevance in the scene (buildings, agents, etc). V is the set ofdifferent views (equals the number of agents/entities with viewing capabil­ 3. Visualization of RoboCup Rescue Simulations ities). Each view is characterized by common camera pa­ There are several tools for visualizing RoboCup Rescue rameters, as the position Pas" view direction vb" relative Simulations. Log viewers are used to track the evolution and camera orientation VUP; and field of view FoV,. Aspect ra­ result ofrescue simulations. Some viewers have been written tio is not being considered as it remains unchaged. MV is so far, enabling different viewing perspectives of the simu­ a multiview setting consisting of k distinct views from V. lation, but all of them lack the functionality of a good de­ The problem is to find the optimal MV set, with appropri­ bugging viewer [AriOS]. Some teams have perfonned work ate view parameters, that describes the rescue scenario with

o P. M. Moreira, L. P. Reis & A, A. de Sousa / Visualization Optimization Application to the RoboCup Rescue Domain better quality. Quality Q is computed using the following cri­ • Small but easily extensible; teria (note that e{ denotes the visual properties of an entity • Generality of its format and possible incorporation ofdif­ ei in a image obtained by the view j) ferent abstraction levels; • Robustness, with simple data validation and override of Visibility: Vis(ej) Only visible objects are considered to usual errors. the solution. For those they can be displayed with more or less pixels on the image due to their visible projected The main advantages ofthis architecture are: area. The visible projected area relates to the object's dis­ Generality The architecture is designed to be easily applied tance to the viewer, to its size and orientation, and also to to a wide variety of problems. how much partial occlusion it suffers from other objects. Flexibility Its behavior should adapt to the demands of the visualization application. Relevance: Rel(e,) A measure ofhow relevant is that object Modularity The design of the supporting architecture en­ for the purpose ofthe visualization. For example, iftrack­ ables its extension by modular pieces. ing emergency situations, a building on fire will have a Portability Operating at a higher level and being indepen­ greater relevance than an unaffected building. The intrin­ dent from the implementation details ofthe visualization sic importance of an object is also considered, e.g. hos­ application and the rendering engine it is expected to be pitals, fireman headquarters, schools have more relevance easily portable or used by many different systems. than ordinary buildings. As a main drawback there is some possible loss of perfor­ Redundancy: Red(eiIMV) It is expected that the multiple mance comparatively to "proprietary" approaches were op­ set of views describe as much as possible distinct situ­ timization is hardly embedded within the visualization pro­ ations occurring during the simulation. Thus, redundant cesses or the rendering pipeline. This loss of performance views over the same objects are penalized. is fundamentally due to the necessary communication over­ Eccentricity: Ecc(ej) A measure on how distant to the cen­ head introduced by the detachment ofsuch processes. ter ofthe image an object is displayed. There is a penalty for more eccentric objects based on the fact that users pay 5. Experimental results more attention to central objects in the image. In order to evaluate usefulness of our approach, we are im­ plementing the proposed architecture and conducting our 4. Optimization Methodology first experiments. A scenario with 1035 relevant entities and 50 agents with viewing capabilities was used. We propose an architecture relying on an optimization agent that operates detached from the main application. As showed As it is expected that the quality function has several local in Fig. 2, communication is achieved by means of a sim­ optima, simulated annealing [KGV83] (SA) was chosen as ple protocol (in the figure a simplified version is depicted). the meta heuristic as it has the ability to continue the search The optimization agent (OA) informs the visualization ap­ even if a local optimum is found. Another contributing rea­ plication (VA) that it is available with a connect message. son is its computational efficiency compared to other meta­ The VA acknowledges the connection and sends a problem heuristics(e.g. genetic algorithms). At each iteration SA con­ description. The OA requests data relevant to the optimiza­ siders a neighbour of the current state, and probabilistically tion process. As optimized solutions are computed, they are decides on moving to it or staying in the current state. The communicated to the VA which sets up and operates appro­ probabilities are chosen so that the system ultimately tends priately. to move to states with better quality. Our neighbour states are obtained, by exchanging one view from the multiview set or by changing one ofthe view parameters. In this experiment we are not changing the view position, but using the agent's position during the rescue simulation. We are also exploring the concept of adaptive neighborhood by adaptively defining the range of change in view parameters as a function of the evolution of quality. Figure 2: The optimization architecture and a simplified Note that for problems where the conditions vary with some version ofthe communication protocol kind of continuity, the optimization step (as well as visibil­ ity determination) can benefit from the exploration ofspatial and temporal coherence. Further communication between the two processes should In Fig 3 the evolution of quality can be observed for an be well established by means of a proper language. Some optimization ofa multivew consisting in four views. The so­ desirable features ofsuch a language are: lution converged to a stable maximum on approximately 500 • Independency ofimplementation details; iterations. o P. M. Moreira, L. P. Reis & A, A, de Sousa / Visualization Optimization Application to the RoboCup Rescue Domain

which information should be transmitted to the client and on

0.8 +------how to balance the rendering load between the server and

.. 0.8 t-F-=------­ the client is a difficult problem that we think can substan­ ': tially benefit from intelligent optimization strategies. <7 0.• -/------

0.2 +------'

o-l--~-_- __-_-~ Acknowledgements o 200 .00 600 600 1000 l'fOr,uiOll The first author acknowledges the support by European Social Fund program, public contest lIS.3/PRODEP/2003, financing re­ Figure 3: Evolution ofthe quality ofthe views along the quest no. 1012.012, medida 5/ac9ao S.3 - Fomla9ao Avan9ada de optimization process. Docentes do Ensino Superior, submitted by Escola Superior de Tecnologia e Gestao do Instituto Politecnico de Viana do Castelo. This work was partially supported by the project Rescue: Coordi­ nation of Heterogeneous Teams in Search and Rescue Scenarios ­ The exact visible set is necessary to detennine whether an FCT/POSIIEIN63240/2004. object is actually visible. We are using pre-processed from­ region visibility and then using an item-buffer technique for References exact visibility. Image analysis reveals the visible objects as well as coverage statistics. Histogram utilities from OpenGL [Ano06] ANONYMOUS: Robocup international project home­ can be used to improve perfonnance. Further acceleration page, 2006. available at: http://www . robocup. org/. can be achieved ifconsidered that objects with small visible [AriOS] ARIAN TEAM: Official arian team home page, 200S. areas are not relevant. In such case the above process can be Sharif University of Technology, http: // ce. sharif. edul done at smaller image resolutions. We are also investigating -arianl Visited in Nov, 10, 200S. the use ofOpenGL hardware occlusion queries. Eccentricity [AVF04] ANDUJAR C., VAZQUEZ P., FAIREN M.: Way-finder: is estimated by projecting the center ofthe objects bounding Guided tours through complex walkthrough models. Comput. boxes. Graph. Forum 23, 3 (2004), 499-S08. [DZ9S] DRUCKER S. M., ZELTZER D.: Camdroid: a system for 6. Conclusions and Future Work implementing intelligent camera control. In S!3D '95 Proceed­ ings of the 1995 symposium on Interactive 3D graphics (New We have presented and described an optimization architec­ York, NY, USA, 1995), ACM Press, pp. 139-144. ture, suitable to be applied to several visualization and ren­ [KGOS] KLEINER A., GOBELBECKER M.: Rescue3D Mak­ dering problems. We pointed several advantages of our ap­ ing rescue simulation attractive to the public. Tech. rep., proach. Institut fur Informatik, Universitat Freiburg, 2005. Avail­ In our first experiments, we have applied this architecture able at http://kaspar.informatik.uni-freiburg.de/ -rescue3D/3dview.pdf. to a visualization problem in the Robocup rescue domain. Our first results are promising and demonstrate its useful­ [KGV83] KIRKPATRICK S., GELATT C. D., VECCHI M. P.: Op­ ness. timization by simulated annealing. Science 220, 4S98 (1983), 671-680. As future work we plan to define a framework based [MCOO] MARCHAND E., COURTY N.: Image-based virtual cam­ on the proposed architecture including several optimization era motion strategies. In Graphics Interface (2000), pp. 69-76. methods. Also planned is the fonnal specification of a lan­ guage for efficient communication between the optimization [Mor02] MORIMOTO T.: How to Develop a RoboCupRescue Agent for RoboCupRescue Simulation System version O. The agent and the rendering /visualization application. RoboCup Rescue Technical Committee, Nov. 2002. Another guideline ofour research concerns to path advice [PKOO] PHAM D., KARABOGA D.: Intelligent Optimization leading to assisted exploration ofscenes. Due to high model Techniques. Springer Verlag, London, 2000. complexity or scene extension, unguided exploration by the [TLOI] TELER E., LlSCHINSKI D.: Streaming of complex 3D user can result on poor explorations, namely by missing rele­ scenes for remote walkthroughs. Computer Graphics Forum vant parts ofthe model. The expected result will be a system (Proceeding of Eurographics '01) 20, 3 (2001), 17-2S. ISSN that generates and suggests to the user optimized paths ac­ 1067-70SS. cording to the users interests, advertisements, etc. [VFSH03] VAZQUEZ P.-P., FEIXAS M., SBERT M., HEIDRICH The proposed architecture could be also extended and W.: Automatic view selection using viewpoint entropy and its adapted to be used in other areas, namely in remote ren­ applications to image-based modelling. Comput. Graph. Forum dering [TLO I]. Optimizing the quality of the visualization 22,4 (2003), 689-700. in remote rendering architectures depends on many factors. [VS03] VAZQUEZ P.-P., SBERT M.: Lecture Notes in Computer Examples of such factors are the rendering potential of the Science. No. 2669. Springer Verlag, Jan 2003, ch. Fast Adaptive client and the available bandwidth. Deciding on when and Selection ofBest Views, pp. 29S - 30S. (Proc. of ICCSA'2003).

o