Live Object Exploration: Observing and Manipulating Behavior and State of Java Objects

Benjamin Biegel, Benedikt Lesch, and Stephan Diehl Department of Computer Science University of Trier Trier, Germany Email: {biegel,diehl}@uni-trier.de

Abstract—In this paper we introduce a visual representation of It is not surprising that developers try to avoid compre- Java objects that can be used for observing and manipulating be- hending the program behavior by solely reading the source havior and state of currently developed classes. It runs separately, code. In about 40% of the time spent for understanding a e.g., on a tablet, beside an integrated development environment. Within the visualization, developers are able to arbitrarily change certain code fragment, developers execute the program and the object state, then invoke any method with custom parameters investigate the code rational at runtime [5]. This includes a and observe how the object state changes. When changing the frequent use of debuggers [6], testing the program behavior source code of the related class, the visualization holds the by using the user interface as end-users will do [7], or simply previous object state and adapts the new behavior defined by using loggers or console outputs. Nevertheless, developers the underlying source code. This instantly enables developers to observe functionalities objects of a certain class have and are forced to permanently switch between the structural and how they manipulate their state, and especially, how source behavioral view on a program, which still is tedious and code changes influence their behavior. We implemented a first cognitively demanding. Furthermore, since the program state prototype as a touch-enabled web application that is connected is only changed temporarily, for subsequent executions, all to a conventional integrated development environment. In order changes to the program state must be re-applied manually. to gain first practical insights, we evaluated our approach in a pilot user study. A promising paradigm that addresses the previously men- tioned issues is live programming [8]–[10]. In general, live I.INTRODUCTION programming environments provide an infrastructure, that Current tools and research methods introduce many tech- enables developers to immediately recognize how state and niques that help developers to comprehend and manage their behavior of a running program are affected, when changing source code [1]. When syntax or compiler errors occur, for the underlying source code [11], [12]. Live programming example, modern integrated development environments pro- occurs in several application scenarios, e.g., creating user vide built-in features that immediately notify developers. What interfaces [13], in continuous testing [14], or when using an in- is more, syntax highlighting, displaying structural context in- situ debugger [15]. Recent studies show that live programming formation, automatic source code formatting, links for source significantly helps developers in finding and fixing bugs [16], code navigation, specialized visualizations like UML [2] and [17]. In particular, since live programming narrows the gap other features support understanding the overall structure of between editing and debugging, developers become aware of a software project. Nevertheless, the behavior (or semantics) newly introduced bugs more early. Nevertheless, when using of a certain code fragment is not observable. While creating object-oriented languages like Java, current live programming or changing source code, developers have to put themselves approaches have some limitations. Some only allow to execute into the role of a computer [3]. As for that, they are forced to the entire program, which makes it hard to investigate smaller execute code fragments mentally and guess how a code change parts in isolation. Others use a fine-grained view on runtime would affect state and behavior of the program at runtime [4]. data by putting it directly into or beside source code, e.g., This issue was nicely summarized by Chris Granger, who is line by line, which makes it hard to keep track of bigger mainly known for developing an interactive open-source IDE parts of the program. Beside program and source code level, named ‘Light Table’1: however, object-oriented languages also require observing on the object level, which is not explicitly supported by current “We are quite literally throwing darts in the dark live programming approaches. and praying that we at least hit the board. We simply This paper fills this gap by introducing an additional visual- cannot see what our programs do and that’s a huge ization representing Java objects. Such objects are instantiated problem whether you’re just starting out or have directly from classes developed within an IDE and can be written millions of lines of beautiful code.” explored by changing their states and invoking their methods. —Chris Granger, 20142 Changes in the underlying class will be injected into the 1http://lighttable.com/ “living objects” and can immediately be observed in the 2http://www.chris-granger.com/2014/03/27/toward-a-better-programming/ visualization. Fig. 1. A Screenshot from our prototype implementation displaying a living object of a class named “Motorcycle”.

II.MAIN CONCEPTAND PROTOTYPE IMPLEMENTATION settings for showing and hiding several types of members, like The main idea of this paper is to bring live execution inherited members or constructors. Another way for filtering feedback to the development of Java classes. An advantage specific members is pinning. By clicking on the star symbol of live programming is that effects on the program execution in the upper right corner of any box, the related box can be can be observed continuously while changing the source pinned or unpinned. Changing the mode from “Show all” to code in the editor. As for that, we propose a visualization “Show pinned” (right side of the toolbar) hides all unpinned that (1) is permanently connected to an IDE, (2) represents boxes and shows only the selected ones. instances of currently developed classes, (3) ensures updating 2) Fields: In the head of a field box (green part) the name object behavior that comes with source code changes while of the field is displayed. In this first concept, we refrain from keeping the current object state, and (4) enables to observe and displaying type information to keep the layout more tidy. manipulate behavior and state of currently represented objects The value of the field is displayed in the lower white area. within the visualization. Depending on the type, the value is represented differently, e.g., a toggle button is used for boolean values, whereas for A. Live Object Exploration string values an input field is used. Arrays and objects are not As can be seen in Figure 1, the entire view is used to displayed directly on the main screen. They are only indicated visualize an object. Its fields (green), constructors (orange) by buttons that lead to a special dialog showing their array and methods (red) are represented as floating boxes. These elements and members respectively. If any number is saved in boxes can be rearranged by simple drag and drop gestures a field, developers can chose between a regular representation, with mouse or touch input. In the toolbar at the top, there are that is only displaying the current value, and a line chart tabs for switching between currently living objects. In order representation, that makes it possible to observe how the to distinguish between multiple objects of the same class, it is underlying value changed over time. With the button next to possible to rename the tabs. The + symbol on the right allows the star symbol one can toggle between both representations. to create new instances of classes from the currently opened 3) Changing Values: The object state can be changed project within the IDE. directly within the visualization by entering new field values. 1) Filtering: Because all members of an object, including In order to support an explorative interaction style, we also the inherited members, are shown per default, representing provide, besides entering new values solely with the keyboard, larger objects can be very complex. To solve this problem, we touch-optimized visual components like sliders and buttons to use two different filtering mechanisms. First, by clicking on the change the values interactively. Furthermore, instead of typing cog symbol in the upper right of the toolbar, a menu opens with a new value, the developer is also able to choose from a list CodePad [18] and our previous work [19] we suggest using a conventional desktop environment with an additional touch device. Furthermore, we wrote a plug-in for the IntelliJ IDE4 that handles the communication between IDE and web ap- plication. By using the reflection API of Java, this plug-in instantiates new objects, manipulates their state and invokes their methods. After changing the source code, a new object is being created based on previously chosen parameter values for the constructor. Then, based on the saved object state in the web application, the newly created object in the IDE will be updated immediately. It is worth mentioning that living objects in the web application and objects within the IDE are only loosely connected. That means, we synchronize structure and state manually. In contrast to standard Java techniques like serializing objects, this enables us to also change the structure of a living object without losing its state.

C. Application Scenarios While elaborating our approach, we discussed several ap- Fig. 2. A dialog for creating a new object. plication scenarios. A classical live programming scenario is using the tool next to an IDE by a single developer. But also a pair programming setting is conceivable, as we have seen of predefined values. Field boxes that contain changed values in our study. The driver could concentrate on writing code are temporarily highlighted by changing its color. and the navigator could use our tool for testing the expected 4) Method Invocation: In order to explore the behavior of behavior. Since our prototype was implemented with web a living object, all its methods can be invoked. Parameter technologies, this approach can also be applied in teaching. values can be entered as described in Section II-A2. If a field While the lecturer presents source code of a Java class, for value is changed after invoking a method, the field box is example, students could use our tool simultaneously on tablets highlighted temporarily. Furthermore, if only pinned boxes are to explore its structure and behavior. shown and a hidden box was affected by the invocation, it is then automatically highlighted for a certain time. By invoking III.EVALUATION constructors, a new instance will be created and the current living object will be replaced. In order to gain first practical insights, we asked 9 devel- 5) Creating Arrays and Objects: Since primitive values opers to participate in a think-aloud user study. Each session can be entered directly, arrays and objects must be created in lasted about 45 minutes in a pair programming setup (one special dialogs. In the array dialog, elements of an array can session with three participants, other three in pairs). During be added and removed. Developers are also able to navigate the study, we observed the sessions and took notes. After the through multi-dimensional arrays. In order to keep track of study, we performed a semi-structured interview and asked inner array elements, they can also be pinned, and are then each participant to fill in a questionnaire. be shown on the main screen. In contrast to arrays, objects can only be instantiated by invoking one of its constructors. A. Questionnaire Figure 2 shows a dialog for creating objects. On the left side, developers can chose the preferred constructor. Then, on the Before filling in the questionnaire, we informed the par- right side, they have to enter values for its parameters. If a ticipants that their answers will be evaluated and published parameter requires an object or array, another dialog can be anonymously. In the first part of the questionnaire, the par- opened in order to create one of these. ticipants were asked to provide some demographic data, their prior knowledge and skills. The rest of the questionnaire is B. Prototype Implementation and Infrastructure subdivided in three parts, addressing the proposed approach in For our prototype implementation we strongly rely on state- general, and the used visualization and interaction in particular. of-the-art web technologies like HTML5, JavaScript, Poly- In order to assess our approach, we provided both free-text mer3, and web sockets. This enables us to independently forms (5 in total) as well as statements (25 in total), that can run our prototype visualization on diverse devices, e.g., on be rated by a 5-point Likert scale ranging from ‘Disagree’ a tablet used with touch gestures as well as in a classical (rating 0) to ‘Agree’ (rating 4). The scale value ‘2’ refers to desktop environment with mouse and keyboard. Inspired by a neutral position.

3http://www.polymer-project.org 4http://www.jetbrains.com/idea/ B. Participants some participants had problems with the proposed object The participants of the study were 9 male developers, three and array visualizations. One participant was not completely master and two PhD students from the University of Trier, convinced (rating 1) that objects were visualized in a good Germany, three bachelor students from the HTW Saarbrucken,¨ way. 2 participants partly disagreed that arrays were easy to Germany, and one professional software developer working in use. Instead of using modal dialogs, most participants wanted the industry. They were between 23 and 31 years old. All to expand objects and arrays directly on the main screen, e.g., participants rated themselves as ‘intermediates’ in Java. In as a tree map. Furthermore, modal dialogs made it difficult to object-oriented programming, two see themselves as ‘experts’ keep the current context. Nevertheless, 8 participants (rating (rest as ‘intermediates’), and two specify only an ‘average’ 3 or 4) confirmed that changes in the object state were very experience in debugging, whereas the rest are ‘intermediates’. well highlighted. Still 7 participants thought that the line charts In summary, all participants are quite experienced in Java, helped them to keep track of those changes. Remarks were object-oriented programming, and debugging. Experiences in missing labels of the axis, access to the complete history in a visualizations and JUnit are more diverse. 4 participants had scrollable graph, and fixing several bugs, like vanishing values. only little experience with program or object visualizations (2 In general, the line chart view was preferred to the regular ‘average’, 2 ‘advanced’). The same distribution also apply for view. the experience with JUnit. 2) Interaction: In general, all participant found that method invocation was easy and helped them to understand the func- C. Experimental Design tional behavior. Most noted that they got no visual feedback if The participants had to solve ten exercises by using our an method invocation caused no change in the object state or prototype implementation. We used a classical desktop envi- if an exception was thrown. As setting up primitive or string ronment running IntelliJ. Next to this, we used an additional values was consistently considered as an easy task, only one touch-monitor running our visualization connected to the IDE. participant found it challenging to create an array and an entire The overall study was divided into three parts, focusing on (1) object. Using predefined values were only questionable for visualization, (2) interaction, (3) integration. In the first two one participant (rating 1), whereas 6 participants found this parts, we evaluated the external view without using the IDE feature sufficient for exploration purposes. Half of them said only. In the third part, the participants used both IDE and that the given examples were not very useful and seemed to be external view to investigate how our approach integrates in randomly chosen. All participants thought that the pin mode their usual work flow. helped well in focusing on a specific set of elements. But they suggested to expand this feature, e.g., to allow hiding D. Results elements, or pin all children of an object. Another helpful The overall impression was quite positive. Although we filtering feature would be a text search for elements. did not introduced our prototype to the participants on pur- 3) Integration: Using our visualization together with the pose, no additional help was needed or requested. Since our IDE was considered differently. Only 4 participants thought observations suggest that using our tool was intuitive and that this combination supported them in the development easy, one participant slightly disagreed (rating 1), whereas process, and two participants were skeptical (rating 1) about 6 participants (rating 3 or 4) confirmed our observations it. Nevertheless, many of them mentioned that they liked and 2 took a neutral position (rating 2). Most participants, to quickly test new implemented or changed functionality, however, agreed that using our prototype was beneficial. Also without executing the program or starting the debugger. Us- the following statements were mostly rated by 3 or 4, whereas ing an additional touch-enabled device for displaying the 2 (‘neutral’) was the worst rating. They confirmed that our visualization was also considered very differently. Only 3 approach offered sufficient support for object exploration, participants (rating 3 or 4) liked the idea of a touch input, helped in understanding the related class structure, supported whereas 3 took a neutral position, one was doubtful (rating 1), in exploring the functional behavior of the object, and made and two gave no rating at all. One half of the participants the object state easy to explore. 6 participants said they would preferred using a second monitor, while the other half thought use our prototype more often in the future. that touch interaction fits more in an exploration scenario like 1) Visualization: In contrast to the general impression, debugging. the opinions concerning our visualization, however, diverged. 8 participants (rating 3 or 4) found the visualization well IV. RELATED WORK structured and clear, one participant had a ‘neutral’ position Besides the related work in the field of live program- and another partly disagreed (rating 1). In order to get an ming already mentioned in the introduction, there are several even clearer structure, half of the participants suggested to use other approaches, that visualize the program state at runtime. a table layout instead of floating boxes. Moreover, in order Algorithm animation systems like Jeliot [20] or LIVE [21] to get a better understanding of the structure of an object, provide automatically generated animations for visualizing some participants also requested displaying type information the execution of an application step-by-step. In contrast to of fields and parameters. Since 7 participants still thought our approach, they are strongly focused on source code lines that the visualization of primitives and strings was clear, instead of a more abstract view on object level. Jive [22] is a visual debugger that visualizes the runtime state, displayed [5] T. D. LaToza, G. Venolia, and R. DeLine, “Maintaining mental models: as an UML object diagram [2], and the call history of an a study of developer work habits,” in 28th International Conference on Software Engineering (ICSE 2006), Shanghai, China, May 20-28, 2006, application, displayed as an UML sequence diagram [2]. A pp. 492–501. similar strategy was used in jGRASP [23], but instead of using [6] G. C. Murphy, M. Kersten, and L. Findlater, “How are java software general visualizations, they use specialized dynamic object developers using the eclipse ide?” IEEE Software, vol. 23, no. 4, pp. 76–83, 2006. viewers that consider the underlying data structure of an ob- [7] T. Roehm, R. Tiarks, R. Koschke, and W. Maalej, “How do professional ject. Since both approaches offer more detailed visualizations, developers comprehend software?” in 34th International Conference on they can only visualize data provided by a debugger, thus Software Engineering, ICSE 2012, June 2-9, 2012, Zurich, Switzerland, 2012, pp. 255–265. they are not able to provide live execution feedback as our [8] S. L. Tanimoto, “VIVA: A visual language for image processing,” J. Vis. approach does. With BlueJ [24] it is possible to invoke code Lang. Comput., vol. 1, no. 2, pp. 127–139, 1990. without starting the program manually. The result can then be [9] C. M. Hancock, “Real-time programming and the big ideas of compu- tational literacy,” Ph.D. dissertation, Massachusetts Institute of Technol- inspected or used for subsequent invocations. BlueJ also allows ogy, 2003. to create class instances that can be explored. In contrast to [10] S. McDirmid, “Living it up with a live programming language,” in our approach, after changing the underlying code, interactively Proceedings of the 22nd Annual ACM SIGPLAN Conference on Object- Oriented Programming, Systems, Languages, and Applications, OOP- created objects are dropped. Thus, BlueJ does not support live SLA, October 21-25, Montreal, Quebec, Canada, 2007, pp. 623–638. programming. Finally, all above mentioned approaches have [11] B. Victor, “Inventing on principle,” Invited talk at the Canadian Univer- integrated visualizations that cannot be run on other devices sity Software Engineering Conference (CUSEC), January 2012. [12] S. L. Tanimoto, “A perspective on the evolution of live programming,” separated from the IDE . That is why they are less suitable in Proceedings of the 1st International Workshop on Live Programming, for the application scenarios described in Section II-C. LIVE 2013, San Francisco, California, USA, May 19, 2013, pp. 31–34. [13] S. Burckhardt, M. Fahndrich,¨ P. de Halleux, S. McDirmid, M. Moskal, V. CONCLUSION N. Tillmann, and J. Kato, “It’s alive! continuous feedback in UI pro- gramming,” in ACM SIGPLAN Conference on Programming Language In this paper, we introduced live object exploration, a live Design and Implementation, PLDI ’13, Seattle, WA, USA, June 16-19, programming approach for representing instances of currently 2013, pp. 95–104. developed Java classes. The visualization enables developers [14] D. Saff and M. D. Ernst, “An experimental evaluation of continuous testing during development,” in Proceedings of the ACM/SIGSOFT to interactively observe and manipulate behavior and state of International Symposium on Software Testing and Analysis, ISSTA 2004, such “living objects”. After changing the underlying source Boston, Massachusetts, USA, July 11-14, 2004, pp. 76–85. code, the behavior and structure of living objects also change, [15] S. McDirmid, “Usable live programming,” in ACM Symposium on New Ideas in Programming and Reflections on Software, Onward! 2013, part while their current states still remain. of SPLASH ’13, Indianapolis, IN, USA, October 26-31, 2013, pp. 53–62. The user study shows that the approach does work in [16] J. Kramer,¨ J. Kurz, T. Karrer, and J. O. Borchers, “How live coding general and helps to understand structure and behavior of Java affects developers’ coding behavior,” in IEEE Symposium on Visual Languages and Human-Centric Computing, VL/HCC 2014, Melbourne, objects, especially, to observe impacts of source code changes VIC, Australia, July 28 - August 1, 2014, pp. 5–8. on runtime. The study results, gathered from questionnaires, [17] E. M. Wilcox, J. W. Atwood, M. M. Burnett, J. J. Cadiz, and C. R. Cook, interviews and observations, also provide important informa- “Does continuous visual feedback aid debugging in direct-manipulation programming systems?” in Human Factors in Computing Systems, CHI tion for further improvements. More specialized visualizations, ’97 Conference Proceedings, Atlanta, Georgia, USA, March 22-27, for example, were requested for different data structures as 1997, pp. 258–265. used in previous work (cf. Section IV). Partly it was tedious [18] C. Parnin, C. Gorg,¨ and S. Rugaber, “Codepad: interactive spaces for maintaining concentration in programming environments,” in Proceed- to navigate through highly nested data structures, especially, ings of the ACM 2010 Symposium on Software Visualization, Salt Lake because the current visualization also uses modal dialogs that City, UT, USA, October 25-26, 2010, pp. 15–24. sometimes distract attention from the main context. Another [19] B. Biegel, S. Baltes, I. Scarpellini, and S. Diehl, “Codebasket: Making developers’ mental model visible and explorable,” in 2nd Workshop on drawback is creating new instances of complex classes or Context for Software Development (CSD), Florence, Italy, May 19, 2015. arrays. Some participants suggest to provide also techniques [20] A. Moreno, N. Myller, E. Sutinen, and M. Ben-Ari, “Visualizing for defining objects and arrays within the IDE. programs with jeliot 3,” in Proceedings of the working conference on Advanced visual interfaces, AVI 2004, Gallipoli, Italy, May 25-28, 2004, pp. 373–376. EFERENCES R [21] A. E. R. Campbell, G. L. Catto, and E. E. Hansen, “Language- [1] M. D. Storey, “Theories, tools and research methods in program com- independent interactive data visualization,” in Proceedings of the 34th prehension: past, present and future,” Software Quality Journal, vol. 14, SIGCSE Technical Symposium on Computer Science Education, 2003, no. 3, pp. 187–208, 2006. Reno, Nevada, USA, February 19-23, 2003, pp. 215–219. [2] G. Booch, J. E. Rumbaugh, and I. Jacobson, The unified modeling [22] P. V. Gestwicki and B. Jayaraman, “JIVE: java interactive visualiza- language user guide - the ultimate tutorial to the UML from the original tion environment,” in Companion to the 19th Annual ACM SIGPLAN designers, ser. Addison-Wesley object technology series. Addison- Conference on Object-Oriented Programming, Systems, Languages, and Wesley-Longman, 1999. Applications, OOPSLA 2004, October 24-28, Vancouver, BC, Canada, [3] H. Lieberman and C. Fry, “Bridging the gulf between code and behavior 2004, pp. 226–228. in programming,” in Human Factors in Computing Systems, CHI ’95 [23] J. H. C. II and T. D. Hendrix, “jgrasp: a lightweight IDE with dynamic Conference Proceedings, Denver, Colorado, USA, May 7-11, 1995, pp. object viewers for CS1 and CS2,” in Proceedings of the 11th Annual 480–486. SIGCSE Conference on Innovation and Technology in Computer Science [4] J. L. Snell, “Ahead-of-time debugging, or programming not in the dark,” Education, ITiCSE 2006, Bologna, Italy, June 26-28, 2006, p. 356. in Software Technology and Engineering Practice, 1997. Proceedings., [24]M.K olling,¨ B. Quig, A. Patterson, and J. Rosenberg, “The bluej system Eighth IEEE International Workshop on [incorporating Computer Aided and its pedagogy,” Computer Science Education, vol. 13, no. 4, pp. 249– Software Engineering]. IEEE, 1997, pp. 288–293. 268, 2003.