DOCSLIB.ORG

Software Engineering for Machine-Learning Applications

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Software Engineering for Machine-Learning Applications Editor: Giuliano Antoniol Polytechnique Montréal INVITED CONTENT [email protected] Editor: Steve Counsell Brunel University Editor: Phillip Laplante [email protected] Pennsylvania State University [email protected] Software Engineering for Machine-Learning Applications The Road Ahead Foutse Khomh, Bram Adams, Jinghui Cheng, Marios Fokaefs, and Giuliano Antoniol THE NEED AND desire for more auto- to address these challenges. In fact, experts could come together to dis- mation and intelligence have led to the learned behavior of an ML-based cuss challenges, new insights, and breakthroughs in machine learning system might be incorrect, even if practical ideas regarding the engi- (ML) and artifi cial intelligence (AI), the learning algorithm is imple- neering of ML- and AI-based sys- yet we still experience failures and mented correctly, a situation in tems. The program included talks shortcomings in the resulting soft- which traditional testing techniques and panels presented by renowned ware systems. The main reason is the are ineffective. A critical problem is academic researchers and indus- shift in the development paradigm in- how to effectively develop, test, and trial practitioners, including keynote duced by ML and AI. Traditionally, evolve such systems, given that they speakers David Parnas, Lionel Briand, software systems are constructed don’t have (complete) specifi cations and Yoshua Bengio. The full pro- deductively, by writing down the or even source code corresponding gram is at http://semla.polymtl.ca. rules that govern the system behav- to some of their critical behaviors. Here, we summarize some key chal- iors as program code. However, Motivated by these challenges, we lenges these experts identifi ed. with ML techniques, these rules are organized the First Symposium on inferred from training data (from Software Engineering for Machine System Accuracy which the requirements are gener- Learning Applications (SEMLA) at The fi rst topic concerned the accu- ated inductively). This paradigm Polytechnique Montréal on 12 and racy of systems built using ML and shift makes reasoning about the be- 13 June 2018, with the kind support AI models, and the responsibilities of havior of software systems with ML of Polytechnique Montréal’s Depart- engineers building them. For exam- components diffi cult, resulting in ment of Computer Engineering and ple, one keynote speaker mentioned software systems that are intrinsi- Software Engineering, the Institute three categories of AI research: cally challenging to test and verify. for Data Valorization (IVADO), SAP, Given the critical and increasing and Red Hat. The event attracted • building programs that imitate role of ML- and AI-based systems around 160 participants from all over human behavior to better under- in our society, it’s imperative for the world, including students, aca- stand human thinking (used in both the software engineering (SE) demics, and industrial practitioners. psychology research), and ML communities to research SEMLA’s main objective was to • building programs that play games and develop innovative approaches create a space in which SE and ML well (challenging and fun), and 2469-7087/19/$33.000740-7459/18/$33.00 © 2019 IEEE © 2018 IEEE Published by the IEEE Computer SocietySEPTEMBER/OCTOBER 2018 February | IEEE 2019SOFTWARE 8121 INVITED CONTENT • demonstrating that practical other domains (such as requirement systems, since an AI system’s behav- computerized products can use elicitation) are more challenging. ior might be incorrect even if the the same methods that humans Overall, AI’s full impact on SE is learning algorithms are implemented use (risky and often naive). still unclear. correctly. One keynote speaker Because of AI and ML systems’ explained how in complex cyber- He stressed that researchers should intrinsic imperfection, one panelist physical systems (CPSs), when no be very concerned about AI systems argued, only harmless AI technology clear specifications of the intended in the third category because they or applications should be released to systems exist (that is, humans have a can’t guarantee 100 percent accu- the public, since the responsibility of lot of knowledge but can’t formalize racy or correct answers in all cases. every engineer is to protect the pub- it), only AI can approximate the sys- He also raised concerns that people lic. He also mentioned that the pub- tem’s intended behavior by learning are using the Turing test to falsely lic should be informed accurately of models from the available data. claim intelligence in systems. He the AI technology it’s being exposed This is a clear improvement over commented, “Turing did not claim to. For example, instead of touting a the manual design of models and that his test was a test for artificial “100 percent self-driving car,” auto- controllers. However, it pushes most intelligence!” motive companies should advertise of the risk toward the trained models’ In response, a leading AI expert their products as “AI-assisted cars,” quality. So, how can we perform stated that AI’s goal is not to achieve with a clear list of the ways in which adequate quality assurance (QA) of 100 percent accuracy because AI is assisting. AI models, given that the number Another panelist emphasized that of environments in which the mod- • humans are also far from 100 AI isn’t a panacea. He illustrated els will be deployed is unlimited and percent accuracy in their daily how simple techniques could give that the human operator will re- tasks, and the illusion of AI, or how the blind quire a detailed explanation of any • AI technology’s strength comes application of AI wouldn’t improve failures? from the ability to abstract up the workflow of workers. For ex- Fortunately, we can use AI tech- from different factors of varia- ample, in principle, an intelligent nology to reduce the search space of tion between environments, to robot could easily replace a human the environments to be tested, nudg- obtain models that can general- worker to hand another worker the ing QA techniques to those environ- ize and transfer to situations that right tool for a given job, but not if ments most likely to have failures or weren’t encountered before. the worker afterward throws the violate important safety constraints. tool back on a pile. (The robot will Such an approach could even work He further explained that AI tech- have a hard time retrieving the right in the system-of-systems context of nologies’ main challenge is the curse tool from an unordered pile.) How- CPSs, where each sensor and actua- of dimensionality—that is, the need ever, using an intelligent robot to tor must be validated not only in iso- for sufficient, labeled data to cover return tools in an ordered fashion lation but also in close integration all important factors (features) of (which is a different problem) could with each other. a given problem. AI, in fact, needs allow other robots later on to be de- However, this QA doesn’t guard more training data than humans do! ployed to hand over tools to work- against hardware failure. So, hard- Whereas the key properties of ers. If a traditional computer science ware systems should incorporate techniques such as deep learning algorithm can solve a problem, we fault-tolerance mechanisms to cope (for example, compositionality, en- should just use that. with such failures. One audience par- coding into a simpler domain, and ticipant also observed that hardware conditional computation) aim to re- System Testing could incorporate fault-tolerance duce dimensionality’s impact, appli- The second hot topic our experts dis- mechanisms to mitigate the effect of cations of AI still risk being limited cussed was the difficulty of testing AI model errors, improving AI sys- to domains in which labeled data ML and AI systems. Our panelists tems’ robustness. is cheap. Although labeled data is debated whether we should tackle Another major challenge is that somehow abundant in some SE do- the testing of those systems the same humans, once they’ve started trust- mains (such as defect prediction), way we do the testing of traditional ing AI in their daily tasks, could 2282 IEEE SOFTWAREComputingEdge | WWW.COMPUTER.ORG/SOFTWARE | @IEEESOFTWARE February 2019 INVITED CONTENT INVITED CONTENT • demonstrating that practical other domains (such as requirement systems, since an AI system’s behav- computerized products can use elicitation) are more challenging. ior might be incorrect even if the the same methods that humans Overall, AI’s full impact on SE is learning algorithms are implemented FOUTSE KHOMH is an associate professor MARIOS FOKAEFS is an assistant profes- use (risky and often naive). still unclear. correctly. One keynote speaker at Polytechnique Montréal, where he leads the sor in Polytechnique Montréal’s Department of Because of AI and ML systems’ explained how in complex cyber- SWAT (Software Analytics and Technology) Lab. Computer Engineering and Software Engineer- He stressed that researchers should intrinsic imperfection, one panelist physical systems (CPSs), when no Contact him at [email protected]. ing. Contact him at [email protected]. be very concerned about AI systems argued, only harmless AI technology clear specifications of the intended in the third category because they or applications should be released to systems exist (that is, humans have a can’t guarantee 100 percent accu- the public, since the responsibility of lot of knowledge but can’t formalize racy or correct answers in all cases. every engineer is to protect the pub- it), only AI can approximate the sys- He also raised concerns that people lic. He also mentioned that the pub- tem’s intended behavior by learning BRAM ADAMS is an associate professor at GIULIANO ANTONIOL is a professor of soft- are using the Turing test to falsely lic should be informed accurately of models from the available data. Polytechnique Montréal, where he leads the ware engineering in Polytechnique Montréal’s claim intelligence in systems.
Load more

Recommended publications
  • Revealing the Secrets of David Parnas
    Revealing the Secrets of David Parnas H. Conrad Cunningham Department of Computer and Information Science University of Mississippi March 7, 2014 Those of us in the fast-changing field of computing often dismiss anything writ- ten more than five years ago as obsolete. Yet several decades-old papers by David L. Parnas [1, 4, 5, 6, 7, 8] are as timely as those published in recent issues of the top journals. Parnas articulates the timeless software design concepts known as information hiding and abstract interfaces. Most programmers would describe a module as a unit of code such as a sub- routine or class. Parnas focuses on the programmers rather than the programs. He defines a module as \a work assignment given to a programmer or group of programmers" as a part of a larger software development project [7]. His goals are to enable programmers to develop each module independently, change one module without affecting other modules, and comprehend the overall system by examining one module at a time [5]. Programmers often design a software system by breaking the required pro- cessing into steps and making each step a module. Instead, Parnas uses in- formation hiding to decompose the system into modules that satisfy his goals (2); each module keeps its own secreta design decision about some aspect of the system (e.g., choice of a data structure). A modules design decision can change but none of the other modules should be affected. If some aspect is unlikely to change, the design can distribute this knowledge across several modules and the interfaces among them.
    [Show full text]
  • Keeping Secrets Within a Family: Rediscovering Parnas
    Keeping Secrets within a Family: Rediscovering Parnas H. Conrad Cunningham Cuihua Zhang Yi Liu Computer Science Computer & Information Systems Computer Science University of Mississippi Northwest Vista College University of Mississippi University, MS, 38677 San Antonio, TX 78251 University, MS 38677 Abstract of related programs. The motivation for product lines and frameworks is to take advantage of the David Parnas wrote several papers in the 1970’s commonalities among the members of the product line and 1980’s that are now considered classics. The to lower the overall cost of producing and maintaining concepts he advocated such as information hiding and a group of related software systems. use of abstract interfaces are generally accepted as Since the foundation of software product lines and the appropriate way to design nontrivial software frameworks is what Parnas proposed in his papers, an systems. However, not all of what he proposed has examination of the concepts in these papers (collected been fully appreciated and assimilated into our in [5]) can still reveal much of value to current-day practices. Many of his simple, elegant ideas have software developers and researchers. Many of the been lost amongst the hype surrounding the lessons taught in these works should also be technologies and methods that have arisen in the past incorporated into our college-level teaching. two decades. This paper examines Parnas’s ideas, This paper examines several of the lessons on the especially his emphasis on program families, and design of program families taught by Parnas that are proposes that college-level computing science and still important for contemporary students to learn.
    [Show full text]
  • Stephan Goericke Editor the Future of Software Quality Assurance the Future of Software Quality Assurance Stephan Goericke Editor
    Stephan Goericke Editor The Future of Software Quality Assurance The Future of Software Quality Assurance Stephan Goericke Editor The Future of Software Quality Assurance Editor Stephan Goericke iSQI GmbH Potsdam Germany Translated from the Dutch Original book: ‘AGILE’, © 2018, Rini van Solingen & Manage- ment Impact – translation by tolingo GmbH, © 2019, Rini van Solingen ISBN 978-3-030-29508-0 ISBN 978-3-030-29509-7 (eBook) https://doi.org/10.1007/978-3-030-29509-7 This book is an open access publication. © The Editor(s) (if applicable) and the Author(s) 2020 Open Access This book is licensed under the terms of the Creative Commons Attribution 4.0 Inter- national License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made. The images or other third party material in this book are included in the book’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the book’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.
    [Show full text]
  • Systems Development Life Cycle
    Systems Development Life Cycle From Wikipedia, the free encyclopedia Jump to: navigation, search For other uses, see SDLC (disambiguation). Model of the Systems Development Life Cycle with the Maintenance bubble highlighted. The Systems Development Life Cycle (SDLC), or Software Development Life Cycle in systems engineering, information systems and software engineering, is the process of creating or altering systems, and the models and methodologies that people use to develop these systems. The concept generally refers to computer or information systems. In software engineering the SDLC concept underpins many kinds of software development methodologies. These methodologies form the framework for planning and controlling the creation of an information system[1]: the software development process. Contents y 1 Overview y 2 History y 3 Systems development phases o 3.1 Requirements gathering and analysis o 3.2 Design o 3.3 Build or coding o 3.4 Testing o 3.5 Operations and maintenance y 4 Systems development life cycle topics o 4.1 Management and control o 4.2 Work breakdown structured organization o 4.3 Baselines in the SDLC o 4.4 Complementary to SDLC y 5 Strengths and weaknesses y 6 See also y 7 References y 8 Further reading y 9 External links [edit] Overview Systems and Development Life Cycle (SDLC) is a process used by a systems analyst to develop an information system, including requirements, validation, training, and user (stakeholder) ownership. Any SDLC should result in a high quality system that meets or exceeds customer expectations, reaches completion within time and cost estimates, works effectively and efficiently in the current and planned Information Technology infrastructure, and is inexpensive to maintain and cost-effective to enhance.[2] Computer systems are complex and often (especially with the recent rise of Service-Oriented Architecture) link multiple traditional systems potentially supplied by different software vendors.
    [Show full text]
  • Inspection's Role in Software Quality Assurance
    focusguest editors’ introduction Inspection’s Role in Software Quality Assurance David L. Parnas, University of Limerick Mark Lawford, McMaster University espite more than 30 years’ effort to improve software quality, companies still release programs containing numerous errors. Many major products have thousands of bugs. It’s not for lack of D trying; all major software developers stress software quality as- surance and try to remove bugs before release. The problem is the code’s complexity. It’s easy to review code but fail to notice significant errors. Researchers have responded to these prob- lems by studying methods of formal correct- ness verification for programs. In theory, we now know how to prove programs correct with the same degree of rigor that we apply to mathematical theorems. In reality, this is rarely practical and even more rarely done. Most research papers on verification make simplifying assumptions (for example, a 1:1 correspondence between variables and vari- able names) that aren’t valid for real pro- grams. Proofs of realistic programs involve long, complex expressions and require pa- tience, time, and diligence that developers don’t think they have. (Interestingly enough, they never have time to verify a program be- fore release, but they must take time to re- spond to complaints after release.) Inspection methods can be more effective than informal reviews and require less effort than formal 16 IEEE SOFTWARE Published by the IEEE Computer Society 0740-7459/03/$17.00 © 2003 IEEE proofs, but their success depends on having a I The TSE papers do communicate the re- sound, systematic procedure.
    [Show full text]
  • Devops E Continuous Delivery Release-Automation Di Software Mediante Implementazione Di Una Pipeline
    POLITECNICO DI TORINO Corso di Laurea Magistrale in Ingegneria Informatica (Computer Engineering) Tesi di Laurea Magistrale DevOps e Continuous Delivery Release-Automation di software mediante implementazione di una pipeline Relatore Candidato prof. Marco Mezzalama Marco Punzi Supervisore aziendale Consoft Sistemi dott. Marco Casu Anno accademico 2017 – 2018 Ai complici della realizzazione di un sogno Abstract «L’avvento della Application Economy impone sempre più l’idea del soft- ware come strumento per generare direttamente valore di business. Occorre quindi garantire servizi di deployment rapidi e in alta affidabil- ità, che permettano il rilascio di software testato e di qualità in qualsi- asi istante. La Continuous Delivery quale estensione della Continuous Integration prevede un approccio strutturato ed automatizzato ai test funzionali, di non regressione e di integrazione, ottimizzando i processi ripetibili di Lifecycle e Deploy. Per assicurare soluzioni flessibili ed in- tegrabili all’interno dei processi aziendali, si necessita di un approccio metodologico di gestione del ciclo del software agendo sulla comunicazione e collaborazione tra gli sviluppatori e gli operatori IT.» (Consoft Sistemi SpA, DevOps Solution, 2017) iii Indice Abstract .................................... iii Introduzione Generale 1 1 Modelli e filosofie a confronto 5 1.1 Introduzione . 5 1.2 Nascita e decadenza del modello Waterfall ............... 6 1.2.1 Le fasi del modello . 6 1.2.2 Gli svantaggi del modello . 7 1.3 Lo sviluppo Agile ............................. 8 1.4 Software release: Anti-pattern e Soluzioni . 12 1.5 L’etica DevOps .............................. 13 1.6 DevOps: il valore di business . 16 1.6.1 Il Return of Investment del DevOps . 18 1.7 Conclusione .
    [Show full text]
  • A Debate on Teaching Computing Science
    Teaching Computing Science t the ACM Computer Science Conference last Strategic Defense Initiative. William Scherlis is February, Edsger Dijkstra gave an invited talk known for his articulate advocacy of formal methods called “On the Cruelty of Really Teaching in computer science. M. H. van Emden is known for Computing Science.” He challenged some of his contributions in programming languages and the basic assumptions on which our curricula philosophical insights into science. Jacques Cohen Aare based and provoked a lot of discussion. The edi- is known for his work with programming languages tors of Comwunications received several recommenda- and logic programming and is a member of the Edi- tions to publish his talk in these pages. His comments torial Panel of this magazine. Richard Hamming brought into the foreground some of the background received the Turing Award in 1968 and is well known of controversy that surrounds the issue of what be- for his work in communications and coding theory. longs in the core of a computer science curriculum. Richard M. Karp received the Turing Award in 1985 To give full airing to the controversy, we invited and is known for his contributions in the design of Dijkstra to engage in a debate with selected col- algorithms. Terry Winograd is well known for his leagues, each of whom would contribute a short early work in artificial intelligence and recent work critique of his position, with Dijkstra himself making in the principles of design. a closing statement. He graciously accepted this offer. I am grateful to these people for participating in We invited people from a variety of specialties, this debate and to Professor Dijkstra for creating the backgrounds, and interpretations to provide their opening.
    [Show full text]
  • Dr. Dobb's | Software Engineering ≠ Computer Science | June 4, 2009
    Software Engineering ≠ Computer Science Software engineering seems different, in a frustrating way, from other disciplines of computer science By Chuck Connell, Dr. Dobb's Journal Jun 04, 2009 URL:http://www.ddj.com/architect/217701907 Chuck Connell is a software consultant. He can be reached at www.chc-3.com. A few years ago, I studied algorithms and complexity. The field is wonderfully clean, with each concept clearly defined, and each result building on earlier proofs. When you learn a fact in this area, you can take it to the bank, since mathematics would have to be inconsistent to overturn what you just learned. Even the imperfect results, such as approximation and probabilistic algorithms, have rigorous analyses about their imperfections. Other disciplines of computer science, such as network topology and cryptography also enjoy similar satisfying status. Now I work on software engineering, and this area is maddeningly slippery. No concept is precisely defined. Results are qualified with "usually" or "in general". Today's research may, or may not, help tomorrow's work. New approaches often overturn earlier methods, with the new approaches burning brightly for a while and then falling out of fashion as their limitations emerge. We believed that structured programming was the answer. Then we put faith in fourth-generation languages, then object-oriented methods, then extreme programming, and now maybe open source. But software engineering is where the rubber meets the road. Few people care whether P equals NP just for the beauty of the question. The computer field is about doing things with computers. This means writing software to solve human problems, and running that software on real machines.
    [Show full text]
  • Revealing the Secrets of David Parnas
    Revealing the Secrets of David Parnas H. Conrad Cunningham Department of Computer and Information Science University of Mississippi March 7, 2014 Those of us in the fast-changing field of computing often dismiss anything writ- ten more than five years ago as obsolete. Yet several decades-old papers by David L. Parnas [1, 4, 5, 6, 7, 8] are as timely as those published in recent issues of the top journals. Parnas articulates the timeless software design concepts known as information hiding and abstract interfaces. Most programmers would describe a module as a unit of code such as a sub- routine or class. Parnas focuses on the programmers rather than the programs. He defines a module as \a work assignment given to a programmer or group of programmers" as a part of a larger software development project [7]. His goals are to enable programmers to develop each module independently, change one module without affecting other modules, and comprehend the overall system by examining one module at a time [5]. Programmers often design a software system by breaking the required pro- cessing into steps and making each step a module. Instead, Parnas uses in- formation hiding to decompose the system into modules that satisfy his goals (2); each module keeps its own secreta design decision about some aspect of the system (e.g., choice of a data structure). A modules design decision can change but none of the other modules should be affected. If some aspect is unlikely to change, the design can distribute this knowledge across several modules and the interfaces among them.
    [Show full text]
  • From Play-In Scenarios to Code: an Achievable Dream
    COVER FEATURE From Play-In Scenarios to Code: An Achievable Dream A development scheme for complex reactive systems leads from a user- friendly requirements capture method, called play-in scenarios, to full behavioral descriptions of system parts, and from there to final implementation. 1 David Harel n a 1992 Computer article, I tried to present an system model, most notably its structure and behav- The Weizmann optimistic view of the future of development ior. The linking of structure and behavior is crucial Institute of methods for complex systems. Research since and by no means a straightforward issue. In SA/SD, Science then only supports this optimism, as I will for example, each system function or activity is asso- I attempt to show. ciated with a state machine or a statechart2 that This article presents a general, rather sweeping describes its behavior. In OOAD, as evident in the development scheme, combining ideas that have been Unified Modeling Language (UML)3 and its exe- known for a long time with more recent ones. The cutable basis, the XUML,4 each class is associated scheme makes it possible to go from a high-level user- with a statechart, which describes the behavior of friendly requirements capture method—which I call every instance object. The “Structured Analysis and play-in scenarios—via a rich language for describing Structured Design” and “Object-Oriented Analysis message sequencing to a full model of the system, and and Design” sidebars give some background on these from there to final implementation. modeling approaches. A cyclic process of verifying the system against An indispensable part of any serious modeling requirements and synthesizing system parts from the approach is a rigorous semantical basis for the model requirements is central to the proposal.
    [Show full text]
  • Software Engineering
    Software Engineering ACM Fellow David Parnas, 2004 What advice do you have for computer science/software engineering students? Most students who are studying computer science really want to study software engineering but they don't have that choice. There are very few programs that are designed as engineering programs but specialize in software. I would advise students to pay more attention to the fundamental ideas rather than the latest technology. The technology will be out-of-date before they graduate. Fundamental ideas never get out of date. However, what worries me about what I just said is that some people would think of Turing machines and Goedel's theorem as fundamentals. I think those things are fundamental but they are also nearly irrelevant. I think there are fundamental design principles, for example structured programming principles, the good ideas in "Object Oriented" programming, etc. What is the most often-overlooked risk in software engineering? Incompetent programmers. There are estimates that the number of programmers needed in the U.S. exceeds 200,000. This is entirely misleading. It is not a quantity problem; we have a quality problem. One bad programmer can easily create two new jobs a year. Hiring more bad programmers will just increase our perceived need for them. If we had more good programmers, and could easily identify them, we would need fewer, not more. What is the most-repeated mistake in software engineering? People tend to underestimate the difficulty of the task. Overconfidence explains most of the poor software that I see. Doing it right is hard work.
    [Show full text]
  • Integrative Educational Approach Oriented Towards Software And
    PAPER INTEGRATIVE EDUCATIONAL APPROACH ORIENTED TOWARDS SOFTWARE AND SYSTEMS DEVELOPMENT Integrative Educational Approach Oriented Towards Software and Systems Development http://dx.doi.org/10.3991/ijep.v3i1.2345 A.J. Stoica1 and S. Islam2 1 Uppsala University, Uppsala, Sweden 2 University of East London, London, United Kingdom Abstract—The paper is based on our academic teaching and between conceptual and operational aspects in software research work in software and system engineering to effec- engineering education. tively develop modern, complex real-life Web application In our educational work, we are guided by: i) lifelong systems. It bridges the gap between academic education and learning preparing our students for applications-oriented industry needs and illustrates how such collaboration can be careers, working in all levels of computer systems engi- successfully developed in the IT area where technology neering in particular software and systems engineering development is rapid. Its scope covers the processes, models, domain; ii) contributions Harlan D. Mills award recipi- technologies, people, and knowledge that have the capability ents: Bertrand Meyer for practical and fundamental con- to contribute to developing such systems. The paper also tributions to object-oriented software engineering, soft- relates to contributions of some of Harlan D. Mills award ware reuse, and the integration of formal methods into the recipients for software engineering achievement, to address above; Barry Boehm for developing empirical software the needs to: i) improve the engineering education in an engineering models that consider cost, schedule, and academic setting, and ii) develop real-life software and quality, as well as software process spiral model, Theory system projects.
    [Show full text]