DOCSLIB.ORG

DESMET: a Method for Evaluating Software Engineering Methods and Tools

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
DESMET: a Method for Evaluating Software Engineering Methods and Tools ISSN:1353-7776 DESMET: A method for evaluating Software Engineering methods and tools Barbara Kitchenham Technical Report TR96-09 August 1996 Department of Computer Science University of Keele Keele Staffordshire ST5 5BG U.K. TEL: 01782 583075 FAX: 01782 713082 Abstract This report describes the results of the DESMET project. The DESMET project was a collaborative project part funded by the U.K. DTI. Its goal was develop and validate a method for evaluating software engineering methods and tools. This report describes the guidelines developed by DESMET for • selecting an appropriate evaluation method; • performing a feature analysis; • performing a quantitative case study. It also describes some of the management and sociological issues that can influence evaluation exercise. Finally, the report describes briefly the attempt the project made to evaluate the DESMET method itself. CONTENTS 1. INTRODUCTION..............................................................................................................................................1 2. THE DESMET METHOD: SCOPE, TERMINOLOGY AND LIMITATIONS..........................................2 2.1 DESMET USERS.................................................................................................................................................2 2.2 EVALUATION CONTEXT.......................................................................................................................................2 2.3 EVALUATION OBJECTS........................................................................................................................................2 2.4 EVALUATION TYPES ............................................................................................................................................2 2.5 EVALUATION PROCEDURE...................................................................................................................................3 2.6 LIMITATIONS OF THE DESMET METHOD............................................................................................................3 3. EVALUATION METHODS .............................................................................................................................4 3.1 QUANTITATIVE EVALUATION METHODS..............................................................................................................4 3.1.1 Case Studies ...............................................................................................................................................4 3.1.1.1 Advantages of case studies....................................................................................................................................4 3.1.1.2 Disadvantages of case studies ...............................................................................................................................4 3.1.2 Surveys .......................................................................................................................................................5 3.1.2.1 Advantages of surveys ..........................................................................................................................................5 3.1.2.2 Disadvantages of surveys......................................................................................................................................5 3.2 QUALITATIVE METHODS.....................................................................................................................................5 3.3 HYBRID EVALUATION METHODS ........................................................................................................................6 3.3.1 Collated expert opinion - Qualitative Effects Analysis...............................................................................6 3.3.2 Benchmarking.............................................................................................................................................6 3.4 SUMMARY OF EVALUATION METHODS ................................................................................................................6 4. SELECTING AN APPROPRIATE EVALUATION METHOD...................................................................7 4.1 TECHNICAL SELECTION CRITERIA........................................................................................................................7 4.1.1 Introduction................................................................................................................................................7 4.1.2 The evaluation context................................................................................................................................7 4.1.3 The nature of the impact.............................................................................................................................7 4.1.4 Nature of evaluation object ........................................................................................................................8 4.1.5 The scope of impact....................................................................................................................................9 4.1.5.1 Product granularity................................................................................................................................................9 4.1.5.2 Extent of impact....................................................................................................................................................9 4.1.6 The maturity of the item............................................................................................................................10 4.1.7 Learning time ...........................................................................................................................................10 4.1.8 Evaluation maturity of an organisation....................................................................................................10 4.1.9 Summary of Evaluation criteria ...............................................................................................................11 4.2 PRACTICAL ISSUES ............................................................................................................................................12 4.2.1 Evaluation Timescales..............................................................................................................................12 4.2.2 Confidence in results ................................................................................................................................13 4.2.3 Costs of an evaluation ..............................................................................................................................14 4.2.3.1 Costs of Feature Analysis....................................................................................................................................14 4.2.3.2 Costs of Qualitative Effects Analysis..................................................................................................................15 4.2.3.3 Costs of Benchmarking.......................................................................................................................................15 4.2.3.4 Costs of a Quantitative Case Study .....................................................................................................................15 4.2.3.5 Costs of a Quantitative Formal Experiment ........................................................................................................15 4.2.3.6 Costs of a Quantitative Survey............................................................................................................................16 4.2.4 External Evaluation..................................................................................................................................16 5. THE INFLUENCE OF HUMAN FACTORS................................................................................................16 5.1 INTRODUCTION .................................................................................................................................................16 5.2 IMPACT OF HUMAN FACTORS.............................................................................................................................16 5.3 SOCIOLOGICAL EFFECTS....................................................................................................................................17 5.3.1 Novelty Effects..........................................................................................................................................17 5.3.1.1 Learning curve effect ..........................................................................................................................................17 5.3.1.2 The Hawthorne effect..........................................................................................................................................17 5.3.2 Expectation effects....................................................................................................................................18 5.3.2.1 The placebo effect...............................................................................................................................................18 5.3.2.2 The doctor effect.................................................................................................................................................18 5.3.3 Summary...................................................................................................................................................19
Load more

Recommended publications
  • From Grain to Pixel: the Archival Life of Film in Transition Fossati, G
    UvA-DARE (Digital Academic Repository) From Grain to Pixel: The Archival Life of Film in Transition Fossati, G. Link to publication Citation for published version (APA): Fossati, G. (2009). From Grain to Pixel: The Archival Life of Film in Transition. (Framing film). Amsterdam: Amsterdam University Press. General rights It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons). Disclaimer/Complaints regulations If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: http://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible. UvA-DARE is a service provided by the library of the University of Amsterdam (http://dare.uva.nl) Download date: 10 Feb 2019 FROM GRAIN TO PIXEL Fossati Herdruk DEF.indd 1 20-07-11 14:22 FRAMING FILM FRAMING FILM is a new book series dedicated to theoretical and analytical studies in restoration, collection, archival, and exhibition practices, in line with the existing archive of EYE Film Institute. With this series, Amsterdam University Press and EYE aim to support the academic research community, as well as practitioners in archive and restoration.
    [Show full text]
  • ﾀﾜdigital Desmet￢ﾀﾝ
    “DigitAL DESMET” BARBARA FLUECKIGER, FRANZISKA HELLER, CLAUDY OP DEN KAMP, AND DAVID PFLUGER Translating Early Applied Colors Despite advances in the digitization of archival fi lms, the translation of early applied colors into the digital domain has remained a critical issue for many reasons. Among the objectives of the Swiss project DIASTOR1 was the development of new approaches for the digitization and restoration of fi lm colors, employing scientifi c analysis in conjunction with fi lm historical research and software development. The project’s research areas pertained to early applied col- ors and their digitization, including the aesthetic and historiographic consequences of the technical processes involved. Based on an early fi lm example from the 1910s, this article presents several areas of that particular focus, which resonate with contemporary archival debates. flUECKIGER, ET AL. 108 The aim of this article is to contextualize the translation of historic aesthetic objects to the digital domain on several levels: • to raise awareness of the film materials’ origin and of the interrelated network of technological as well as institutional frameworks affecting the digitization process • to point out the potentials, as well as the contingencies, when dealing with digital technologies within workflows2 • to highlight the potential role of research to mediate between disciplines, such as computer science, engineering, physical chemistry, film history and aesthetics, restoration ethics, and philological principles An exhaustive documentation of all the factors at work in the historiographic process of transferring early films to the digital realm is impossible. Nonetheless, the awareness of certain specific factors that originate in digital technologies, as well as in the hetero- geneous interdisciplinarity of the fields involved, can help to point out the importance of and the need for more open documentation and communication.
    [Show full text]
  • Curating Research Free
    FREE CURATING RESEARCH PDF Hyunjoo Byeon,Carson Chan,Olga Fernandez-Lopez,Kate Fowle,Maja Fowkes,Reuben Fowkes,Liam Gillick,Georgina Jackson,Sidsel Nelund,Simon Sheikh | 262 pages | 24 Feb 2015 | Open Editions | 9780949004031 | English | London, United Kingdom Curating for Research | JAR A curator from Latin : curameaning "to take care" [1] is a manager or overseer. Traditionally, a curator or keeper of a cultural heritage institution e. A traditional curator's concern necessarily involves tangible objects of some sort — artwork, collectibles, historic items or scientific collections. More recently, new kinds of curators have started to emerge: curators of digital data objects and biocurators. In smaller organizations, a curator may have sole responsibility for acquisitions and even for collections care. The curator makes decisions regarding what objects to select, oversees their potential and documentation, conducts research based on the collection and its history, provides proper packaging of art for transportation, and shares research with the public and community through exhibitions and publications. In very small, volunteer-based museums such as those of local historical societies, a curator may be the only paid staff-member. In larger institutions, the curator's primary function is that of a subject specialist, with the expectation that he or she will conduct original research on objects and Curating Research the organization in Curating Research collecting. Such institutions can have multiple curators, each assigned to a specific collecting area e. In such organizations, the physical care of the Curating Research may be overseen by museum collections-managers or by museum conservators, with documentation and administrative matters such as personnel, insurance, and loans handled by a museum registrar.
    [Show full text]
  • Japanese Art Free
    FREE JAPANESE ART PDF Joan Stanley-Baker | 240 pages | 28 Oct 2014 | Thames & Hudson Ltd | 9780500204252 | English | London, United Kingdom Kintsugi - Wikipedia Lacquerware is a longstanding tradition in Japan[6] [7] and at some point kintsugi may have been combined with maki-e as a replacement for other ceramic repair techniques. While the process is associated with Japanese craftsmen, the technique was also applied to ceramic pieces of other origins including China, Vietnam, and Korea. Kintsugi became closely associated with ceramic vessels used for chanoyu Japanese tea ceremony. When it was returned, Japanese Art with ugly metal staples, it may have prompted Japanese Art craftsmen to look for a more aesthetic means Japanese Art repair. As a philosophy, kintsugi can be seen to have similarities to the Japanese philosophy of wabi-sabi Japanese Art, an embracing of the flawed or imperfect. This can be seen as a rationale for keeping an object around even after it has broken and as a justification of kintsugi itself, highlighting the cracks and repairs as simply an event in the life of an object rather Japanese Art allowing its service to end at the time of its damage or breakage, and can be seen as a variant of the adage "Waste not, want not". Not only is there no attempt to hide the damage, but the repair is literally illuminated Mushin is often literally translated as "no mind," but carries connotations of fully existing within the moment, of non-attachment, of equanimity amid changing conditions. The Japanese Art of existence over time, to which all humans are susceptible, could not be clearer than in the breaks, the knocks, and the shattering to which ceramic ware too is subject.
    [Show full text]
  • 163448 369986.Pdf
    UvA-DARE (Digital Academic Repository) From Grain to Pixel: The Archival Life of Film in Transition Fossati, G. Publication date 2009 Document Version Final published version License CC BY-NC Link to publication Citation for published version (APA): Fossati, G. (2009). From Grain to Pixel: The Archival Life of Film in Transition. (Framing film). Amsterdam University Press. http://oapen.org/search?identifier=369986 General rights It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons). Disclaimer/Complaints regulations If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible. UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl) Download date:08 Oct 2021 FROM GRAIN TO PIXEL Fossati Herdruk DEF.indd 1 20-07-11 14:22 FRAMING FILM FRAMING FILM is a new book series dedicated to theoretical and analytical studies in restoration, collection, archival, and exhibition practices, in line with the existing archive of EYE Film Institute.
    [Show full text]
  • The Colour Fantastic Framing Film
    THE COLOUR FANTASTIC FRAMING FILM FRAMING FILM is a book series dedicated to theoretical and analytical studies in restoration, collection, archival, and exhibition practices in line with the existing archive of EYE Filmmuseum. With this series, Amsterdam University Press and EYE aim to support the academic research community, as well as practitioners in archive and restoration. SERIES EDITORS Giovanna Fossati, EYE Filmmuseum & University of Amsterdam, the Netherlands Leo van Hee, EYE Filmmuseum Frank Kessler, Utrecht University, the Netherlands Patricia Pisters, University of Amsterdam, the Netherlands Dan Streible, New York University, United States Nanna Verhoeff, Utrecht University, the Netherlands EDITORIAL BOARD Richard Abel, University of Michigan, United States Jane Gaines, Columbia University, United States Tom Gunning, University of Chicago, United States Vinzenz Hediger, Goethe University Frankfurt, Germany Martin Koerber, Deutsche Kinemathek, Germany Ann-Sophie Lehmann, University of Groningen, the Netherlands Charles Musser, Yale University, United States Julia Noordegraaf, University of Amsterdam, the Netherlands William Uricchio, Massachusetts Institute of Technology, United States Linda Williams, University of California at Berkeley, United States GIOVANNA FOSSATI, VICTORIA JACKSON, BREGT LAMERIS, ELIF RONGEN-KAYNAKÇI, SARAH STREET, JOSHUA YUMIBE THE COLOUR FANTASTIC Chromatic Worlds of Silent Cinema AMSTERDAM UNIVERSITY PRESS Published by EYE Filmmuseum / Amsterdam University Press Cover illustration: Cover design and lay-out: Magenta Ontwerpers, Bussum Amsterdam University Press English-language titles are distributed in the US and Canada by the University of Chicago Press. isbn 978 94 6298 301 4 e-isbn 978 90 4853 298 8 doi 10.5117/9789462983014 nur 670 © All authors / Amsterdam University Press B.V., Amsterdam 2018 All rights reserved.
    [Show full text]
  • Amsterdam University Press
    Zurich Open Repository and Archive University of Zurich Main Library Strickhofstrasse 39 CH-8057 Zurich www.zora.uzh.ch Year: 2017 Film museum practice and film historiography : the case of the Nederlands filmmuseum (1946-2000) Lameris, Bregt Abstract: Rich in detail, this is a study of the interrelationships between film historical discourse and archival practices. Exploring the history of several important collections from the EYE Film Museum in Amsterdam, Bregt Lameris shows how archival films and collections always carry the historical traces of selection policies, restoration philosophies, and exhibition strategies. The result is a compelling argument that film archives can never be viewed simply as innocent or neutral sources of film history. DOI: https://doi.org/10.5117/9789089648266 Posted at the Zurich Open Repository and Archive, University of Zurich ZORA URL: https://doi.org/10.5167/uzh-148766 Monograph Published Version The following work is licensed under a Creative Commons: Attribution-NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0) License. Originally published at: Lameris, Bregt (2017). Film museum practice and film historiography : the case of the Nederlands filmmuseum (1946-2000). Amsterdam: Amsterdam University Press. DOI: https://doi.org/10.5117/9789089648266 FILM MUSEUM PRACTICE AND FILM HISTORIOGRAPHY Amsterdam University Press FRAMING FILM FRAMING FILM is a book series dedicated to theoretical and analytical studies in restoration, collection, archival, and exhibition practices in line with the existing archive
    [Show full text]
  • RECENT ACQUISITIONS BOOKS to Order from List 128: Call Toll-Free 1-800-441-0076 400 Summit Avenue Outside the United States Call 1-651-290-0700 St
    RULON~ List 128 MILLER RECENT ACQUISITIONS BOOKS To Order from List 128: Call toll-free 1-800-441-0076 400 Summit Avenue Outside the United States call 1-651-290-0700 St. Paul, Minnesota E-mail: [email protected] 55102-2662 Other catalogues available at our website:Rulon.com USA Member ABAA/ILAB ~ R A R E & F I N E B O O K S VISA, MASTERCARD, DISCOVER, and AMERICAN EXPRESS accepted. IN MANY FIELDS If you have any questions regarding billing, methods of payment, shipping, or foreign currencies, please do not hesitate to ask. M ANUSC R IPTS 1. Ali Bey, Tauri Saleem Malik. Amazing thoughts. and soiled; occasional underlinings, annotations, n.p.: [by the author], 1974. $250 and calculations in an unidentified hand. Contained First edition, 4to, pp. [36]; printed from typescript; original in a new quarter red morocco clamshell box. With pictorial stapled wrappers inscribed at the top: “To Poetry the bookplate of A.W. Greely, and a 1-page note in Outloud, best wishes, Tauri Saleem Malik Ali Bey.” Try his hand attached to the front free endpaper reading: as we did, we can find “This volume - Nordenski- nothing about this poet. No old’s Arctic Voyages - was copies appear to be recorded. one of the few books in our library at Camp Clay, One of the most Grinnell Land, during the poignant books we’ve winter of 1883-84. It shows handled marks of usage in keeping 2. [Arctic.] A.W. with the vicissitudes expe- Greely’s copy of Nordenski- rienced by the loyal soldiers old’s Arctic Voyages, 1858- of the American Army, and 1879.
    [Show full text]
  • Ideals of the East: the Spirit of Japanese Art Free
    FREE IDEALS OF THE EAST: THE SPIRIT OF JAPANESE ART PDF Kakuzo Okakura,Sister Nivedita | 128 pages | 11 Mar 2005 | Dover Publications Inc. | 9780486440248 | English | New York, United States The Ideals of the East Index Lacquerware is a longstanding tradition in Japan[6] [7] and at some point kintsugi may have been combined with maki-e as a replacement for other ceramic repair techniques. While the process is associated with Japanese craftsmen, the technique was also applied to ceramic pieces of other origins including China, Vietnam, and Korea. Kintsugi became closely associated with ceramic vessels used for chanoyu Japanese tea ceremony. When it was returned, repaired with ugly metal staples, it may have prompted Japanese craftsmen to look for a more aesthetic means of repair. As a philosophy, kintsugi can be seen to have similarities to the Japanese philosophy of wabi-sabian embracing of the flawed or imperfect. This can be seen as a rationale for keeping an object around even after it has broken and as a justification of kintsugi itself, highlighting the cracks and repairs as simply an event in the life of an object rather than allowing its service to end at the time of its damage or breakage, and can be seen as a variant of the adage "Waste not, want not". Not only is there no attempt to hide the damage, but the repair is literally illuminated Mushin is often literally translated as "no mind," but carries connotations of fully existing within the moment, of non-attachment, of equanimity amid changing conditions. The vicissitudes of existence over time, to which all humans are susceptible, could not be clearer than in the breaks, the knocks, and the shattering to which ceramic ware too is subject.
    [Show full text]
  • BOOK ARTS NEWSLETTER ISSN 1754-9086 No
    BOOK ARTS NEWSLETTER ISSN 1754-9086 No. 90 June - July 2014 Published by Impact Press at the Centre for Fine Print Research, UWE Bristol, UK ARTIST’S COVER PAGE: CFPR BOOK ARTS SUMMER INSTITUTE 2014 In this issue: National and International Artists’ Books Exhibitions Pages 1 - 23 Announcements Pages 23 - 24 Courses, Lectures & Workshops Pages 24 - 36 Opportunities Pages 37 - 39 Artist’s Book Fairs & Conferences Pages 39 - 41 Internet News Pages 41 - 43 New Artists’ Publications Pages 43 - 52 Reports & Reviews Pages 52 - 61 Stop Press! Pages 61 - 62 Artists’ Books Exhibition, UWE, Bristol, UK Please note - due to library building works, all exhibitions will be on display in room OC4 at Bower Ashton until October 2014. Artist’s Book Club ABC - Open Call: Summer Show Monday 2nd June - Wednesday 30th July 2014 ABC is an independent club working with book arts. We are based at the University of the West of England (UWE), Bristol, UK. The club, founded by Lilla Duignan in 2009, has developed from the MA Multidisciplinary Print course. It is a student-organised, autonomous group that includes both current students and alumni, many from Folded Books was the Spring project. We had an extended MA Printmaking but also from other undergraduate and discussion about what constituted a ‘folded’ book, after postgraduate courses in the faculty. initial preconceptions depending on difference in practice. There were some assumptions that the result would be a concertina format but the conclusion was that a folded book is a book that features folds. The results of this project formed the basis of the ABC stand at Leeds Artist’s Book Fair at the Tetley in March 2014.
    [Show full text]
  • Journal of Film Preservation Offers a Forum for Both General and Specialised Discussion on All Theoretical and Technical Aspects of Moving Image Archival Activities
    Journal of Film Preservation Contributors to this issue... Ont collaboré à ce numéro... Han participado en este número... Journal ANTTI ALANEN 84 4/2011 Head of the FIAF Programming and Access to Collections of Film Commission; Film Programmer at the National Audiovisual Archive of Finland (Helsinki) PAOLO CHERCHI USAI Director, Haghefilm Foundation (Amsterdam) Preservation ROBERT DAUDELIN Rédacteur en chef du Journal of Film Preservation (Montréal) CHRISTIAN DIMITRIU Éditeur et membre de la rédaction du Journal of Film ANTONI KIRCHNER Preservation (Bruxelles) Exdelegado de Cinematografía y Vídeo de la Generalitat de Catalunya (Barcelona) COLIN DUPRÉ Étudiant en Master d’histoire, Université de Toulouse le ÉRIC LE ROY Mirail (Toulouse) Chef du Service accès, valorisation et enrichissement des collections, Archives françaises du film du CNC (Paris- FRANCISCO GAYTÁN FERNÁNDEZ Bois d’Arcy) Revista de la Federación Internacional de Archivos Fílmicos Preservation Officer, Filmoteca de la UNAM (México) MICHAEL LOEBENSTEIN LUCA GIULIANI Curator and Researcher, Oesterreichisches Filmmuseum Head of Film Archive, Museo Nazionale del Cinema- and Ludwig Boltzmann Institute for History & Society Fondazione Maria Adriana Prolo (Torino) (Wien) HELMUT HERBST DONALD McWILLIAMS Filmmaker; founder of cinegrafik-Film-Studio; formerly Filmmaker and teacher (Montréal) teacher at the DFFB (German Film & Television Academy), Berlin, and Professor at the College of Art and SABRINA NEGRI Design, Offenbach (Brombachtal-Birkert) Film archivist, Museo Nazionale del
    [Show full text]
  • Giovanna Fossati
    FRAMING GIOVANNA FOSSATI FILM GIOVANNA FOSSATI GIOVANNA FROM GRAIN TO PIXEL In From Grain to Pixel, Giovanna Fossati analyzes the transition from analog to digital film and its profound effects on filmmaking and film archiving. By reflecting on the theoretical conceptualiza- tion of the medium itself and posing significant questions about the status of physical film and the practice of its archival preservation, restoration, and presentation, Fossati proposes a novel theorization The Archival Life of Film of film archival practice. From Grain to Pixel attempts to bridge the fields of film archiving in Transition and academic research by addressing the discourse on film’s ontology and analyzing how different interpretations of what film is Giovanna Fossati is the chief PIXEL FROM GRAIN TO affect the role and practices of film archives. curator of Eye Film museum THIRD REVISED EDITION Almost a decade after its first publication, this revised edition and professor of Film covers the latest developments in the field. Besides a new general Heritage and Digital Film introduction, a new conclusion, and extensive updates to each Culture at the University of chapter, a novel theoretical framework and an additional case study Amsterdam. have been included. Giovanna Fossati’s From Grain to Pixel has been an immensely valuable source for theories and practices of film digitization and restoration since its first publication in 2009. The new updated edition considers the changing requirements of this extremely dynamic, evolving field. Thanks to her double expertise as a film scholar and a curator, Fossati is able not only to provide deep insights into current approaches and advanced tools, but also to integrate them into her highly reflected theoretical framework, thereby providing a state-of-the-art reference for research and teaching.
    [Show full text]