DOCSLIB.ORG

Corner Cases and Possible Improvements of Automated Testing

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Corner Cases and Possible Improvements of Automated Testing MASARYK UNIVERSITY FACULTY OF INFORMATICS Corner cases and possible improvements of automated testing MASTER'S THESIS Petra Mikova Brno, Spring 2020 MASARYK UNIVERSITY FACULTY OF INFORMATICS Corner cases and possible improvements of automated testing MASTER'S THESIS Petra Mikova Brno, Spring 2020 This is where a copy of the official signed thesis assignment and a copy of the Statement of an Author is located in the printed version of the document. Declaration Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Petra Mikova Advisor: RNDr. Adam Rambousek, Ph.D. i Acknowledgements I would like to thank my advisors Mgr. Jiri Vanek and RNDr. Adam Rambousek, Ph.D. for their help, valuable advice and support, and the members of OpenJDK QE team in Red Hat for their expertise and input. ii Abstract The aim of this thesis is analysis of current state of OpenJDK QE in Red Hat, identification of biggest issues and their resolution. That is achieved both by outlining new and reusing existing tools and software. Most of the problematic corner cases mentioned in this thesis are not product specific, making the solutions applicable for other QE teams. The last part of the thesis is dedicated to improving automated results processing and reporting. iii Keywords OpenJDK, Jenkins, quality engineering, test automation, Java iv Contents Introduction 1 1 Automation in testing 3 2 OpenJDK 4 2.1 What is OpenJDK 4 2.2 OpenJDK and Red Hat 4 2.3 OpenJDK QE 5 3 Test automation tooling 6 3.1 Jenkins 6 3.1.1 Architecture 6 3.2 Beater 7 3.2.1 Beaker usage 8 3.2.2 Beaker machine pool 9 3.2.3 Downsides 10 4 Major issues of OpenJDK QE 11 4.1 Mixing results of various projects 11 4.1.1 Sorting and comparing results 12 4.1.2 Separating jobs 14 4.2 Incomplete matrix of current jobs 14 4.2.1 Problems caused by manual cloning 15 4.3 Template job generation 16 5 O-tool 18 5.1 The O-tool idea 18 5.2 Build and test matrix 19 5.2.1 Matrix declaration 20 5.3 Job generation 25 5.4 Implementation 29 6 Common issues and handling 30 6.1 Hanging Jenkins jobs 30 6.1.1 Aborting of long-running jobs 32 6.1.2 Implied consequences 34 v 6.2 Product download issues 34 6.2.1 Inaccessible remote server 34 6.2.2 Insufficient executor storage 35 6.3 Known test failures 41 6.3.1 Old tests with compatibility issues 41 6.3.2 Testcase timeouts 42 6.3.3 Dependency issues 44 6.3.4 False negatives of unmaintained test suites ... 44 7 Test result evaluation and processing 48 7.1 New available result reporting 48 7.2 Result comparison 49 7.2.1 Narrowing the case 49 7.2.2 Results comparing 50 7.2.3 Jenkins report plugin 52 7.3 Test fail processing possible improvements 55 7.3.1 Type of the fail recording 55 7.3.2 Broader history discovery 55 7.3.3 Automatic reproducer extraction 55 7.3.4 Test author search 56 7.3.5 Blame a commit 56 8 Conclusion 60 Bibliography 61 vi List of Tables vii List of Figures 3.1 Master node with several additional machines, that are used as slave nodes, split into several executor slots. 7 3.2 Artificial node that gathers all Jenkins jobs that are running in Beaker. 9 4.1 Different builds originating from different forests mixed in one test job. 13 5.1 JDK project representation and configuration. 27 5.2 JDK test project representation and configuration. 28 6.1 A stable test suite build trend 31 6.2 Unstable test suite with manually shut down hanging jobs. 32 6.3 Current build step that starts the executor and runs the test suite. 33 6.4 Build Timeout plugin version of execute shell. 33 6.5 Example configuration for node monitoring 37 7.1 Results of jtreg test suite displayed in the Jenkins Report JCK plugin view 53 7.2 Comparison of previous and current run of jtreg test suite shows a new failing test. 54 Introduction These days, information technology and software are an integral part of our lives. It is in the cars we drive, the planes that fly above our heads, the phones in our hand and in the banks that control flow of the money Technology is everywhere and it is so intertwined with our lives, that the costs of a single failure can be unimaginable. Failures don't have to always cost us only money; it can cost us lives, knowledge or it can cause instability of our society. That makes technology reliability a key asset. Quality engineering is an inseparable part of software develop• ment. Each day, thousands of quality engineers and testers find mis• takes and errors in the software that could cause, if released, serious issues to people, property and break customer trust. They find new ways, how the software can be misused, broken or exploited, but also check that already found issues do not appear again. Programming languages and their libraries are like any other soft• ware product. However, their impact can be much wider than most single purpose applications, because the developers use it to create their software. A single bug can affect hundreds of applications and expose unexpected vulnerabilities. The Java platform has a special place in the software world; it is one of the most favored programming languages, it is heavily used in financial services, data centres and other important industries, and it is the official language for Android development, that currently dominates the world of mobile operating systems. That is also because of the fact that Java is considered a secure and reliable platform. The expectations are high from the community and the developers working on the implementations are working hard to make sure they are fulfilled. There are many implementations of Java platform. The open source implementation is called OpenJDK and it has a wide range of contributors from many companies like Oracle, Red Hat, IBM, Amazon, Azul and more. The goal of this thesis is to analyze the current state of automated testing of OpenJDK in the Red Hat OpenJDK QE team, to identify the biggest issues that are being struggled with and to outline possible solutions to these problems. The problems are not usually product or 1 company specific, so the designed solutions are applicable to other software and can be used by other engineers for similar purposes. First, we will put the thesis into context - explain why it is important to introduce automated solutions and briefly acquaint the reader with OpenJDK. Then we will introduce the most important tools currently used by the team, that are either directly related to the issues or a key part of the testing process. Then we will analyze the major problems in the testing process and test suites and either try to find an existing solution for these problems, or propose some. At last, we will outline possible changes in the results processing that could have significantly benefiting impact on the workflow, as it seems to be one of the bigger bottlenecks in the testing process. Those changes could be both implemented as a result processing tool, or be used to extend existing programs created by the members of the team. 2 1 Automation in testing "A major problem of the software industry is its inability to develop error- free software." [1] The quote above pretty much sums up the current situation in software development. Today's software products are becoming increasingly complex, but the need for reliability, stability, performance and functionality persists. That makes quality engineer• ing and software testing an integral part of software development, because with growing code base and functionality, it gets harder to ensure the quality of the product. The increasing requirements for testing are then reflected in higher need of manpower in quality assurance and engineering. Even though manual testing has its perks and is useful especially for cases that are difficult to automate, the automatic testing is usually more cost- effective, less error prone and more accurate, especially in the regres• sion testing 1, because these test suites tend to grow with every bug found. These days, automation is a key factor in quality engineering. Once created, the automated tests are easy to repeat and reproduce, fast in execution and if configured correctly, much more reliable. It enables us to maximize usage of resources and in general, it increases produc• tivity, especially in the time, when manpower is much more expensive than machine resources. However, the test automation is not just about writing and main• taining the tests. It is also about the scripts that execute them, mecha• nisms that record and collect the results and also the interpretation of the results. In general we can say, the more tests and results we have, the more we want and need to automate the process. 1. Regression testing is repetitive re-rerunning of tests, that ensures that the devel• oped and tested software performs the same way after a change. 3 2 OpenJDK 2.1 What is OpenJDK OpenJDK is an open source implementation of Java Standard Edition. It is licensed under the GNU General Public License v2 with classpath exception[2].
Load more

Recommended publications
  • Java™ Technology Test Suite Development Guide
    Java™ Technology Test Suite Development Guide 1.2 For Java Compatibility Test Suite Developers Sun Microsystems, Inc. 901 San Antonio Road Palo Alto, CA 94303 U.S.A. 650-960-1300 November 2003 Copyright © 2003 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved. THIS SOFTWARE CONTAINS CONFIDENTIAL INFORMATION AND TRADE SECRETS OF SUN MICROSYSTEMS, INC. USE, DISCLOSURE OR REPRODUCTION IS PROHIBITED WITHOUT THE PRIOR EXPRESS WRITTEN PERMISSION OF SUN MICROSYSTEMS, INC. U.S. Government Rights - Commercial software. Government users are subject to the Sun Microsystems, Inc. standard license agreement and applicable provisions of the FAR and its supplements. Sun, the Sun logo, Sun Microsystems, Java, the Java Coffee Cup logo, JavaTest, Java Community Process, JCP,J2SE, Solaris and Javadoc are trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. The Adobe®logo is a registered trademark of Adobe Systems, Incorporated. This distribution may include materials developed by third parties. Third-party software, including font technology, is copyrighted and licensed from Sun suppliers. UNIX is a registered trademark in the U.S. and other countries, exclusively licensed through X/Open Company, Ltd. The Adobe® logo is a registered trademark of Adobe Systems, Incorporated. Products covered by and information contained in this service manual are controlled by U.S. Export Control laws and may be subject to the export or import laws in other countries. Nuclear, missile, chemical biological weapons or nuclear maritime end uses or end users, whether direct or indirect, are strictly prohibited. Export or reexport to countries subject to U.S.
    [Show full text]
  • Efficient Dependency Detection for Safe Java Test Acceleration
    Efficient Dependency Detection for Safe Java Test Acceleration Jonathan Bell, Gail Kaiser Eric Melski, Mohan Dattatreya Columbia University Electric Cloud, Inc 500 West 120th St 35 S Market Street New York, NY USA San Jose, CA USA {jbell,kaiser}@cs.columbia.edu {ericm,mohan}@electric-cloud.com ABSTRACT Our study of popular open source Java programs echoes Slow builds remain a plague for software developers. The fre- these results, finding projects that take hours to build, with quency with which code can be built (compiled, tested and most of that time spent testing. Even in cases of projects packaged) directly impacts the productivity of developers: that build in a more manageable amount of time | for ex- longer build times mean a longer wait before determining if ample, five to ten minutes | faster builds can result in a a change to the application being built was successful. We significant increase in productivity due to less lag-time for have discovered that in the case of some languages, such as test results. Java, the majority of build time is spent running tests, where To make testing faster, developers may turn to techniques dependencies between individual tests are complicated to such as Test Suite Minimization (which reduce the size of discover, making many existing test acceleration techniques a test suite, for instance by removing tests that duplicate unsound to deploy in practice. Without knowledge of which others) [11, 12,23,24,27,28,37,39], Test Suite Prioritization tests are dependent on others, we cannot safely parallelize (which reorders tests to run those most relevant to recent the execution of the tests, nor can we perform incremen- changes first) [14, 15, 35, 36, 38], or Test Selection [17, 25, 32] tal testing (i.e., execute only a subset of an application's (which selects tests to execute that are impacted by recent tests for each build).
    [Show full text]
  • Javatest Harness Architect’S Guide, Javatest Harness 4.4.1 for the Java Platform E20663-02
    JavaTest Harness Architect’s Guide, JavaTest Harness 4.4.1 for the Java Platform E20663-02 December 2012 This Architect’s Guide is intended for those who design JavaTest harness test suites. You should be familiar with the JavaTest harness. JavaTest Architect's Guide, JavaTest Harness 4.4.1 for the Java Platform E20663-02 Copyright © 2002, 2011, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable: U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007).
    [Show full text]
  • Java™ Technology Compatibility Kit User's Guide Template
    Java™ Technology Compatibility Kit User’s Guide Template For Technology Licensees Release [VersionNumber] Sun Microsystems, Inc. 4150 Network Circle Santa Clara, California 95054 U.S.A. 1-800-555-9SUN or 1-650-960-1300 May, 2003 Sun Microsystems, Inc. 901 San Antonio Road Palo Alto, CA 94303 U.S.A. 1-800-555-9SUN or 1-650-960-1300 Sun Microsystems, Inc. 4150 Network Circle Santa Clara, California 95054 U.S.A. 1-800-555-9SUN or 1-650-960-1300 <NOTE this copyright page states Sun’s copyrights regarding this TCK User’s Guide Template. It is not intended for use as a template or model for TCK User’s Guide copyright statements.> COPYRIGHT © 2003 SUN MICROSYSTEMS, INC. ALL RIGHTS RESERVED. SUN MICROSYSTEMS, INC. HEREBY GRANTS A NON-EXCLUSIVE, NON-TRANSFERABLE, WORLDWIDE LICENSE TO JAVA COMMUNITY PROCESS (JCP) MEMBERS TO USE, REPRODUCE, AND CREATE DERIVATIVE WORKS FROM THIS DOCUMENT, SOLELY FOR THE PURPOSE OF CREATING THEIR OWN JAVA TECHNOLOGY COMPATIBILITY KIT USER GUIDES, AND TO DISTRIBUTE, PUBLICLY PERFORM, OR PUBLICLY DISPLAY SUCH USER GUIDES, IN WHOLE OR IN PART, AND IN ANY MEDIA OR FORMAT. LICENSEE AGREES THAT IT MAY NOT MODIFY OR CLAIM ANY LEGAL RIGHTS IN ANY SUN TRADEMARK OR LOGO. LICENSEE MAY NOT USE ANY SUN TRADEMARK OR LOGO EXCEPT IN CONFORMANCE WITH SUN’S TRADEMARK AND LOGO USAGE REQUIREMENTS (WWW.SUN.COM/POLICIES/TRADEMARKS/). THIS LICENSE IS SUBJECT TO AND CONDITIONED UPON LICENSEE’S COMPLIANCE WITH THE TERMS AND CONDITIONS OF THIS LICENSE, AND LICENSEE’S RETENTION, ON ALL REDISTRIBUTIONS, IN WHOLE OR IN PART, OF THE ABOVE COPYRIGHT NOTICE, THIS PERMISSION NOTICE, AND ALL DISCLAIMERS.
    [Show full text]
  • 2007 Javaonesm Conference Word “BENEFIT” Is in Green Instead of Orange
    there are 3 cover versions: Prospect 1 (Java) It should say “... Save $200!” on the front and back cover. The first early bird pricing on the IFC and IBC should be “$2,495”, and the word “BENEFIT” is in orange. ADVANCE CONFERENCE GUIDE Prospect 2 (Non-Java) The front cover photo and text is different from Prospect 1. The text of the introduction Last Chance to Save $200! Register by April 4, 2007, at java.sun.com/javaone paragraphs on the IFC is also different, the 2007 JavaOneSM Conference word “BENEFIT” is in green instead of orange. Features Java Technology, Open Source, Web 2.0, Emerging Technologies, and More Don’t miss this year’s newly expanded content. Advance your development skills with hundreds of expert-led, in-depth technical sessions in nine tracks over four days: The back cover and the IBC are the same as Consumer Technologies | Java™ SE | Desktop | Java EE | Java ME Prospect 1. The Next-Generation Web | Open Source | Services and Integration | Tools and Languages How to navigate this brochure and easily find what you need... Alumni For other information for Home Conference Overview JavaOnePavilion this year’s Conference, visit java.sun.com/javaone. It should say “... Save $300!” on the front Registration Conference-at-a-Glance Special Programs and back cover. The first early bird pricing on Hyperlinks Bookmark Buttons Search Click on any of the underlined Use the bookmark tab to Click on the buttons at the top Pull down from the Edit menu and the IFC and IBC should be “$2,395”, and the links to visit specific web sites.
    [Show full text]
  • Joseph D. Darcy Oracle #Include "Std Orcl Disclaimer.H"
    The State of JDK and Joseph D. Darcy Oracle #include "std_orcl_disclaimer.h" The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Outline . The JDK and OpenJDK . The JCP . Compatibility . JDK 7 Features . JDK 8 Features . Q & A What are JDK and OpenJDK? . The JDK is an implementation of a Java SE specification (amongst other things) . Bundled Webstart / plugin are not part of Java SE . OpenJDK is • A community • A set of projects • A set of code bases . Oracle and others develop the reference implementation of Java SE specifications in OpenJDK. Much of this code is reused in Oracle’s JDK product . Related projects, including experimental projects, also occur as part of OpenJDK . More on OpenJDK at OSCON: OpenJDK – When And How To Contribute To The Java SE Reference Implementation Dalibor Topic (Oracle, Corp.) 3:30pm Tuesday http://hg.openjdk.java.net/jdk7/jdk7 Note: log scale on x-axis. jaxp jax-ws corba JDBC javac Core libraries Fork/join Much of HotSpot Gervill etc., etc. IcedTea — http://icedtea.classpath.org Note: figure not drawn to scale. jaxp port.html jax-ws Build harness - openjdk javac corba - shark JDBC - and - Core libraries Netx/plugin Fork/join Much of HotSpot Gervill etc., etc.
    [Show full text]
  • Javatest™ Harness 4.5
    JavaTest™ Harness 4.5 JavaTest Agent User’s Guide December 2013 Copyright © 2002, 2013, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government. This software or hardware is developed for general use in a variety of information management applications.
    [Show full text]
  • Making Software More Reliable by Uncovering Hidden Dependencies
    Making Software More Reliable by Uncovering Hidden Dependencies Jonathan Bell Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY 2016 ©2016 Jonathan Bell All Rights Reserved ABSTRACT Making Software More Reliable by Uncovering Hidden Dependencies Jonathan Bell As software grows in size and complexity, it also becomes more interdependent. Multiple internal components often share state and data. Whether these dependencies are intentional or not, we have found that their mismanagement often poses several challenges to testing. This thesis seeks to make it easier to create reliable software by making testing more efficient and more effective through explicit knowledge of these hidden dependencies. The first problem that this thesis addresses, reducing testing time, directly impacts the day-to- day work of every software developer. The frequency with which code can be built (compiled, tested, and package) directly impacts the productivity of developers: longer build times mean a longer wait before determining if a change to the application being build was successful. We have discovered that in the case of some languages, such as Java, the vast majority of build time is spent running tests.Therefore, it’s incredibly important to focus on approaches to accelerating testing, while simultaneously making sure that we do not inadvertently cause tests to erratically fail (i.e. become flaky). Typical techniques for accelerating tests (like running only a subset of them, or running them in parallel) often can’t be applied soundly, since there may be hidden dependencies between tests. While we might think that each test should be independent (i.e.
    [Show full text]
  • Javatest™ Harness 4.3
    JavaTest™ Harness 4.3 Graphical User Interface User’s Guide June 2010 Copyright © 2005, 2010, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related software documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable: U.S. GOVERNMENT RIGHTS. Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle America, Inc., 500 Oracle Parkway, Redwood City, CA 94065.
    [Show full text]
  • Javatest™ Harness (The Harness) to Run Tests of the Test Suite, Browse Results, and Write Reports Test Results
    JavaTest ™ Harness 4.5 Graphical User Interface User’s Guide December 2013 Copyright © 2002, 2013, Oracle and/or its affiliates. All rights reserved. This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government. This software or hardware is developed for general use in a variety of information management applications.
    [Show full text]
  • Automatizzazione E Controllo Delle Soluzioni Per I Test Dello Strumento TCK Di JSR331
    UNIVERSITÀ DEGLI STUDI DI PARMA Facoltà di Scienze Matematiche, Fisiche e Naturali Corso di Laurea in Informatica Automatizzazione e controllo delle soluzioni per i test dello strumento TCK di JSR331 Relatore: Chiar.mo Prof. Gianfranco Rossi Candidato: Riccardo Zangrandi Anno Accademico 2012/2013 1 A mia mamma Maria, mio papà Domenico e mia nonna Carla. 2 Ringraziamenti Prima di addentrarsi negli argomenti che sono stati sviluppati nel lavoro di tesi è doveroso fare un ringraziamento ad alcune persone. Un grazie particolare va al prof. Gianfranco Rossi che oltre ad avere avuto il ruolo di relatore è sempre stato disponibile e i suoi consigli sono stati preziosi per portare a termine questo lavoro. Un ulteriore ringraziamento deve andare a Fabio Biselli che si è sempre prestato a aiutarmi con il materiale necessario a terminare il lavoro di tesi con disponibilità e serietà. Inoltre intendo ringraziare tutti i professori che in questi anni mi hanno aiutato nel percorso di studi intrapreso facendo nascere in me un interesse sempre più crescente per le materie del corso di laurea. Un altro grazie intendo rivolgerlo a tutti i miei compagni di corso che mi sono sempre stati vicino negli anni di studi. Indice 1 Introduzione 5 2 Lo strumento TCK: struttura e funzioni 12 2.1 Introduzione ............................ 12 2.2 Caratteristiche principali ..................... 13 2.3 Componenti di un TCK ..................... 14 2.3.1 Documento di pianificazione del TCK . 15 2.3.2 Norme di conformità ................... 15 2.3.3 Collezione di test (test suite) . 16 2.3.4 Processo d’appello .................... 17 2.3.5 Exclude List .......................
    [Show full text]
  • Java Technology Compatibility Kit (TCK) Project Planning And
    TCK Project Planning and Development Guide Version 1.2 Sun Microsystems, Inc. 4150 Network Circle Santa Clara, California 95054 U.S.A. 1-800-555-9SUN or 1-650-960-1300 August, 2003 Copyright © 2003 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, U.S.A. All rights reserved. Sun Microsystems, Inc. has intellectual property rights relating to technology embodied in this product. In particular, and without limitation, these intellectual property rights may include one or more of the U.S. patents listed at http://www.sun.com/patents and one or more additional patents or pending patent applications in the U.S. and other countries. This product is distributed under licenses restricting its use, copying distribution, and decompilation. No part of this product may be reproduced in any form by any means without prior written authorization of Sun and its licensors, if any. Third-party software, including font technology, is copyrighted and licensed from Sun suppliers. Sun, Sun Microsystems, the Sun logo, J2ME, Java, JavaTest, JavaDoc, and the Java Coffee Cup logo trademarks or registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. Federal Acquisitions: Commercial Software - Government Users Subject to Standard License Terms and Conditions. DOCUMENTATION IS PROVIDED “AS IS” AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID. Copyright © 2003 Sun Microsystems, Inc., 4150 Network Circle, Santa Clara, California 95054, Etats-Unis.
    [Show full text]