JOURNAL OF SOFTWARE, VOL. 8, NO. 10, OCTOBER 2013 2425 How Much Integrated Development Environments (IDEs) Improve Productivity? Iyad Zayour, PhD Computer Science Lebanese University, Lebanon [email protected] Hassan Hajjdiab College of Engineering and Computer Science Abu Dhabi University, Abu Dhabi [email protected] Abstract—Modern integrated development environments and its massive libraries and ready-made components like (IDE) such as Microsoft Visual Studio and Eclipse, and UI controls. languages, such as Java and .Net, represent a far step ahead from the legacy development environments such as grep and A. Back then Emacs. One would assume that the productivity of programmers has been dramatically improved. To get a In our original investigation of the legacy system, the more realistic assessment of improvements, after more than goal was to improve the productivity of programmers ten years on an investigation on programmer productivity, particularly those doing software maintenance, as this we recently embarked into an investigation of a group of activity was considered to be highly inefficient by the programmers in a software company using all the modern target company. It was our conjecture that by studying features of Visual Studio. We found that indeed there were and observing programmers doing software maintenance, significant improvements. But, at the same time, other side we can identify the “inefficient” tasks. And then, we effects made the overall improvement not so clear. The reduce inefficiency by developing a tool that addresses complexity of the development environment and its associated libraries and ready-made components represent these tasks thus hoping to improve the overall a significant new source of loss of productivity that productivity of the programmers. Among other things, manifests the most in the cost of debugging and learning. A the study revealed that programmers spend significant better understanding of the challenges associated with time doing keyword searches (see Table 1 for the list of adopting new development technologies may rescue some of most common tasks). the gain in productivity. A more profound analysis, however, showed that the search problem was only a facade for a more complex Index Terms—IDE, Productivity, empirical software and fundamental task: program comprehension. We can engineering. show this by exploring the typical scenario for solving a maintenance request. In order to solve the request, a I. INTRODUCTION programmer must: More than ten years ago, we embarked into an 1. Locate the code related to the maintenance empirical investigation to help a telecommunication request that is often described in term of company reduce the cost of maintaining a large legacy program behavior description. The programmers software [1]. The development environment for that must search for any clue (user interface strings software was based on classical tools such as grep and and routine names) that relates the external Emacs and a Pascal like proprietary language. Today, program behavior to the code relevant to the with modern IDE tools such as Microsoft Visual Studio maintenance problem. and Eclipse, and languages such as Java and .Net, one 2. Understand the code that generates the behavior would assume that the productivity of programmers has so changes can be done. This code, however, is been dramatically improved. not typically stored in one contiguous passage of To get a more realistic assessment of improvements, code, but rather in delocalized pieces of code we recently embarked into a similar investigation of a that only gets connected during run time by group of programmers in a software company using all branching and routine calls. This necessitate: the modern features of Visual Studio 2010 to develop a a. Locating these delocalized pieces often hospital information system. It is important to note that by doing more searches. by IDE we do not reference only to the software system b. Understanding the code dynamics at run that integrates the text editor, the compiler and other tools, time by creating a mental simulation of but to the whole environment that includes the language execution synchronizing program © 2013 ACADEMY PUBLISHER doi:10.4304/jsw.8.10.2425-2431 2426 JOURNAL OF SOFTWARE, VOL. 8, NO. 10, OCTOBER 2013 events with code to find what went order of magnitude improvement in productivity, in wrong. reliability, in simplicity.” The difficulties in program comprehension by The essence of difficulties in such scenario does not programmers have been studied for long, and different reside in the effectiveness of searching mechanism, but cognitive models for program comprehension have been rather in the inherent complexity of the code space that suggested. Von Mayrhauser and Vans surveyed this area need to be understood. The mental reconstruction of in [3] and compared six cognitive models execution paths can overload human cognitive abilities The empirical study of the programmers work particularly in large systems with deeply nested relations practices was one step ahead after cognitive models that such as routine call trees that often reach more than ten mostly used controlled experiments. It seeks to levels of nesting. understand how work occurs and suggest appropriate technologies for the workplace [Singer 97]. This can be TABLE 1 done by “following and recording the work that people do” in their industrial setting. TOP 5 ACTIVITIES IDENTIFIED DURING SOFTWARE MAINTENANCE OF THE More recently Latoza et. al. [6] used a series of LEGACY SYSTEM surveys and interviews to capture the work habits of 1.1 Finding the next piece of code in a mentally programmers. They found that programmers spend about simulated control flow (e.g. call flow/ call tree) half their time debugging, and the rest is spent writing 1.2 Finding the definition of a routine new features and making their code maintainable. 1.3 Finding a starting point in code The study of the effect of modern IDE on 1.4 Finding where an external behaviour is programming has not been so frequent. Ko et.al [5] claim implemented in code to be the first to do so. They conducted an “in vitro” 1.5 Finding the definition of a variable or a variable experiment designed to assess the impact of modern IDEs type (Eclipse) on maintenance tasks. They found that In short, although on the surface programmers spent programmers spent most of their time “reading and significant time doing searching, yet improving this navigating” code. It wasn’t clear how their experiment mechanism only removed “accidental” difficulties [9]. captured the effect of IDE given that predefined tasks The “essential” difficulties faced by the programmers were given to the subjects. We argue that controlled were due to the complexity of program comprehension. experiments are incapable of capturing the diversity of the development ecosystem. B. More than 10 Years Later Murphy et. al. [13] used a tool that generates traces With modern IDEs and modern programming representing the interaction history of programmers with languages, one can trace little similarity, at the surface, to IDE to study how Java programmers use the Eclipse IDE. the development experience of the legacy system. Yet, The data collected represents the frequency of use in our thesis remains that, at a more profound level, the general of the different features, views and commands of fundamental activities such as program comprehension Eclipse but without relation to the mental model or the tasks should be comparable. high level intention of use by the programmers. They To report on what really changed, what has been found, for example, that copy and paste are among the 10 solved and improved and what has not, we investigated a top most used commands. We, on the other hand, ignored company developing a hospital information system of copy and paste as we considered them too low level medium size (about 320 forms/screens). The team perhaps at the same level with typing (programmers may developing this system is composed of 13 programmers copy and paste to save typing) and has no relevance to the of varying expertise levels, from six months to four years essential difficulties experienced by programmers. of experience in the development environment (IDE, Recently, the “Debugging Canvas” [10], an add-on to languages, and class libraries). Activities included both Visual Studio that extends its debugger and provides developing new features and maintaining existing ones. visual representations of call relations among other The programmers use all modern visual features of dynamic information, represents a serious attempt to Visual Studio 2010, such as forms, reports, and database address the program comprehension difficulties during schema design. We were “embedded” with the programming. So far, it is still a research tool and did not development team repeating the same techniques used to make it into official release of Visual Studio. investigate the legacy system such as shadowing, observing and interviewing the team. The study spanned III. CHALLENGES OF THE STUDY three months in total. Like in any empirical study, we wanted to measure and compare specific variables, something that can be best II. RELATED WORK done in a lab-like setting under controlled experiments. Brooks [9] in his classical paper “No silver bullet” But a software development environment is more like an argues against high expectation from new technologies: ecosystem with an infinite number of interdependent “there is no single development, in either technology or variables. The tools, the expertise and skill of in management technique that by itself promises even one programmers, the nature of projects, the development © 2013 ACADEMY PUBLISHER JOURNAL OF SOFTWARE, VOL. 8, NO. 10, OCTOBER 2013 2427 process and many more variables, all affect the and drop, positioning and resizing of controls productivity of programmers. almost entirely visually using the mouse. A lab-like (“in vitro”) experiment is too limited to 2. The business logic tier was developed using the capture the versatility of the development ecosystem.