Apply Or Die: on the Role and Assessment of Application Papers in Visualization

Lawrence Berkeley National Laboratory Recent Work Title Apply or Die: On the Role and Assessment of Application Papers in Visualization. Permalink https://escholarship.org/uc/item/9bd62631 Journal IEEE computer graphics and applications, 37(3) ISSN 0272-1716 Authors Weber, Gunther H Carpendale, Sheelagh Ebert, David et al. Publication Date 2017 DOI 10.1109/mcg.2017.51 Peer reviewed eScholarship.org Powered by the California Digital Library University of California This is the author’s version submitted to IEEE Computer Graphics and Applications before final editing and layouting. The definitive version of this paper will appear in IEEE Computer Graphics and Applications Volume 37, Number 3, May/June 2017. DISCLAIMER: This document was prepared as an account of work sponsored by the United States Government. While this document is believed to contain correct information, neither the United States Government nor any agency thereof, nor the Regents of the University of California, nor any of their employees, makes any warranty, express or implied, or assumes any legal responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by its trade name, trademark, manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof, or the Regents of the University of California. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof or the Regents of the University of California. Apply or Die: On the Role and Assessment of Application Papers in Visualization Gunther H. Weber1, Sheelagh Carpendale2, David Ebert3, Brian Fisher4, Hans Hagen5, Ben Shneiderman6 and Anders Ynnerman7 1 Lawrence Berkeley National Laboratory & University of California, Davis, USA 2 University of Calgary, Canada 3 Purdue University, USA 4 Simon Fraser University & University of British Columbia, Canada 5 Technische Universität Kaiserlautern, Germany 6 7 University of Maryland, College Park, USA Linköping Universitet, Sweden In his controversial 2004 paper “On the Death of consistently. This Visualization Viewpoint summarizes Visualization,”1 Bill Lorensen asked “can it survive without points that arose in this panel to encourage continued customers?” Concerns about the need for documented discussion and derives a first set of conclusions from them. impact on applications has inspired outreach to application stakeholders and efforts to increase the presence and Current Trends Favoring Application Papers visibility of application-driven research in VIS conferences. Despite a general agreement that application-oriented Three major trends that favor an increased emphasis on papers are of value—application papers have won several application papers emerged from the discussion: 1) the need honorable mentions, e.g., at the 2016 SciVis and VAST for bigger teams with an increased emphasis on integration conferences—it remains an open question as to what and infrastructure in order to realize the ambitious visions criteria should be applied to judge whether an application outlined in proposals; 2) changing research policies; and 3) paper merits acceptance. This is evidenced by the wide the increasing number of researchers involved in range of evaluations that this paper type receives in interdisciplinary research. conference reviews and in the diversity of opinions voiced in the discussion between reviewers. These discussions Realizing the “grand vision” of overall data frequently focus on how best to achieve the goals of understanding requires significant infrastructure- relevance to application problems as well as the more building and integration efforts. One of the main drivers familiar criteria of technical excellence and methodological of VIS research is to achieve the grand vision of enabling rigor that are expected for IEEE venues (conferences and data understanding in science, engineering and society. journals). Technical VIS papers play an important role in accomplishing this mission by advancing the state of the art The panel “Application Papers: What are they and how in algorithms and methods. These ultimately serve as should they be evaluated?” at IEEE VIS 2016 in Baltimore, building blocks or “dots” for achieving this grander vision. Maryland brought together leading researchers in all three Realizing the overall vision also requires significant VIS main areas—InfoVis (Carpendale and Shneiderman), infrastructure-building and integration work to “connect the SciVis (Hagen and Ynnerman) and VAST (Ebert and dots” to build a complete large-scale system (Figure 1). Fisher). Our goal was to gain a better understanding of From this perspective, application papers are success stories prevalent views in the VIS community and to start a that demonstrate that we are achieving our goal and discussion on how to evaluate application papers more accomplishing this grander vision. Multi-touch Boundary Aware Multi-Resolution Table System Reconstruction 2014 Volume Rendering 2011 2004, 2006 TF Based Semantics 2010 Historygrams Ambient Occlusion 2012 2008 Image Plane Sweeping 2011 Figure 1: The visualization table developed at the Norrköping Visualization Center C, Linköping University, Sweden, is based on a progression of visualization papers over the past decade2-9 (left hand side), integrated into an application that has impact both in the medical domain and is used in science communication. The image on the right hand side shows visitors to the Mediterranean Museum in Stockholm interacting with a combined surface and volumetric visualization of the mummy Neswaiu. A fuller account of this project has recently been published as a contributed article in the Communications of the ACM.10 Papers of this kind provide valuable success stories to the visualization community showing how to combine and tailor the contributions of many technical VIS papers into a widely used system. Integrating visualization techniques into a system— To remain competitive, groups funded by these including work with application professionals, the institutions must publish papers that focus on the combination and refinement of individual methods and combination, adaptation and refinement of existing detail work to create usable systems—requires large teams techniques to address a particular problem. More with dedicated researchers working at the interface between importantly, application papers also provide the “success visualization and application. These researchers bridge the stories” necessary to show the potential of visualization gap between fundamental visualization technology research across a wide range of application domains and obtain and the grander vision of data understanding. Application funding in the first place. If we “shun” application papers papers document these efforts, providing valuable from VIS venues, we rob ourselves and future generations knowledge to peers on how to build these large scale of young visualization researchers of important funding systems. Furthermore, researchers at the interface between opportunities. This trend is likely to be exacerbated as even visualization and application science—who invest an funding agencies focusing on basic research are starting to immense effort in infrastructure-building and integration— encourage or even require collaborations with application need a clear career path, and in the traditional evaluation scientists and professionals, e.g., applications are one model this requires publications. Application-oriented method to show “Broader Impact” for NSF proposals. papers are important for these researchers to earn recognition and to prosper in their careers. The success of The Emergence of Interdisciplinary Researchers. these researchers in turn supports the field, demonstrating Computation is playing an increasingly important role in that innovation in newly developed methods can be used many sciences and the data deluge from simulations and and will have an impact on real-world applications. experiments requires increasingly sophisticated computational analysis methods. As a result, a new Current research policies favor application impact and generation of interdisciplinary students and researchers is societal relevance. Many sources of funding for emerging: as researchers in many disciplines become better visualization—e.g., from the National Science Foundation versed in computational methods and computer science (NSF)—focus on basic research. However, many programs create new interdisciplinary degrees in visualization research groups also rely on funding sources computational science and engineering—such as with an application-driven mission. Government computational chemistry, computational neuroscience, etc. agencies—such as the U.S. Department of Energy (DOE) Many schools are also creating new interdisciplinary and the National Institutes for Health (NIH)—as well as programs in data science. This trend has grown to the point private foundations—like the Stiftung für Innovation and that some students even pursue doctoral degrees in two the Keck Foundation—have requirements that visualization disciplines. These researchers are looking for appropriate can help to accomplish, but do not propose the development venues for publishing

Load more