AUTOMATIC TEST ENVIRONMENT SETUP a Project
Total Page:16
File Type:pdf, Size:1020Kb
AUTOMATIC TEST ENVIRONMENT SETUP A Project Presented to the faculty of the Department of Computer Engineering California State University, Sacramento Submitted in partial satisfaction of the requirements for the degree of MASTER OF SCIENCE in Computer Engineering by Trupti Deepak Naik SPRING 2015 AUTOMATIC TEST ENVIRONMENT SETUP A Project by Trupti Deepak Naik Approved by: __________________________________, Committee Chair Fethi Belkhouche, Ph.D. __________________________________, Second Reader Preetham Kumar, Ph.D. ____________________________ Date ii Student: Trupti Deepak Naik I certify that this student has met the requirements for format contained in the University format manual, and that this project is suitable for shelving in the Library and credit is to be awarded for the Project. __________________________, Graduate Coordinator ________________ Nikrouz Faroughi, Ph.D. Date Department of Computer Engineering iii Abstract of AUTOMATIC TEST ENVIRONMENT SETUP by Trupti Deepak Naik Manual verification in industry testing is a very complex and tedious process. Verification test engineers need to carefully go through various application screens, make the correct oscilloscope setups to get the expected results, try various input combinations and compare the results with the expected behavior. Furthermore, this type of testing is repetitive in nature. With the introduction of the new chips and/or new firmware releases, Automatic Test Environment (ATE) is becoming widely used to address the limitations of the traditional verification testing methods. This project focuses on creating an automated test environment for Solid State Devices (SSD) hardware verification testing. The project will contribute in making the tedious process of manual testing simpler and more time efficient for verification engineers. The project is implemented using Python Scripting Language and Test Environment Suite libraries from Agilent Command Expert tool. It sends Standard iv Commands for Programmable Instruments (SCPI) commands to instruments like Oscilloscope, power supplies, etc. The project automates both the test setup and the documentation of the test results. ______________________, Committee Chair Fethi Belkhouche, Ph.D. _______________________ Date v ACKNOWLEDGEMENTS I offer my sincere gratitude to my project advisor Dr. Fethi Belkhouche for helping me choose my project topic and for the time he spent with me during the entire project implementation. I would also like to thank him for guiding and correcting my project report. A special thanks to Dr. Preetham Kumar for reviewing my project report. I express my thanks to the Computer Engineering department and all the faculty members without whom this project would have been a distant reality. I also extend my heartiest thanks to my family and my well wishers. vi TABLE OF CONTENTS Page Acknowledgements…………………………………………………………….……......vi List of Tables.....………………………………………………………………………...ix List of Figures...……………………………………………………………………….....x Chapter 1. INTRODUCTION………………………………………………………………..1 2. BACKGROUND…………………….……………………...………..…………..5 2.1 Overview of Manual Testing Procedure…….………………………..……5 2.2 Disadvantages of Manual Testing…………….…..….......…...………........7 2.3 General Purpose Interface Bus (GPIB) History and Challenges...…...…….8 3. AUTOMATIC TEST SETUP ENVIRONMENT………..……………………..12 3.1 Architecture of Automatic Test Environment Setup…………..………….12 3.2 Agilent Command Expert Tool……………..…….……………...……….15 3.3 Advantages of Automatic Test Environment Setup…………….………...20 4. DEMONSTRATION OF AUTOMATION SETUP FOR TESTS……………...21 4.1 Power Consumption Measurement Test……….….......………………......21 4.2 Inrush Current Measurement Test…….……..……..…….…….………....30 4.3 Rail Switching Frequency Test…………..…….…..…………..…….........40 4.4 Voltage Rail and Noise Test………….…….……………….....……….....46 4.5 Capacitor Hold Up Time Measurement Test….…….………...……....…..54 vii 5. CONCLUSION ……………………………………………………….………63 Appendix A Results…………………………………………………..……………64 Appendix B Source Code……...……………………...…...……………………....82 References…..……….…………………………………………….…………………96 viii LIST OF TABLES Tables Page 1. Initial Expenditure in Automation Test Environment ………………………..………….2 2. Cost Benefit with Test Automation ………………………….…….………………….....2 ix LIST OF FIGURES Figures Page 1. Manual Test Process Flowchart………...….…….…………….………………………….5 2. GPIB Measurement Setup …………………………………………………………….…..9 3. Architecture of Automatic Test Environment ………………………………………..….12 4. Instrument Communication Block Diagram ………………………………….………….14 5. Agilent Command Expert: Choose Connection Type………...………………….………15 6. Agilent Command Expert: Choose command set...………..…………………….……….16 7. Agilent Command Expert: Connect and simulate Option…..…………..….…………….17 8. Agilent Command Expert: SCPI Commands...…………………...…..………………….19 9. Power Consumption Measurement Test Setup …………………………………………..22 10. Automatic test environment (ATE) main window (GUI)………………………………..26 11. ‘Power Consumption Test’ input parameters..…………………..……………………….27 12. ‘Power Consumption Test’ default input parameters..…………………..……………….28 13. ‘Power Consumption Test’ HTML Report …………………………………..…………..29 14. Current waveform when SSD’s are powered up…………………..……………………..30 15. Inrush Current Measurement Test Setup.…………………………..…………………….32 16. ‘Inrush Current Test’ input parameters..……………………………….…….…………..34 17. ‘Inrush Current Test’ default input parameters..……………….…………..…………….35 18. ‘Inrush Current Test’ HTML Report ……………………………………...……………..37 19. ‘Rail Switching Frequency Test’ Test Setup ……………………………..……………...41 20. ‘Rail Switching Frequency Test’ input parameters..……………………….....………….43 x 21. ‘Rail Switching Frequency Test’ default input parameters..……………..…...………….44 22. ‘Rail Switching Frequency Test’ HTML Report ………………………..…..….………..45 23. ‘Voltage Rail and Noise Test’ Test Setup.………...………………….…….……...........48 24. ‘Voltage Rail and Noise Test’ input parameters..……………….…………..…………...50 25. ‘Voltage Rail and Noise Test’ default input parameters.………..……………….………51 26. ‘Voltage Rail and Noise Test’ HTML Report …………………………..…….…………52 27. Block Diagram of SSD Power Back up System…………………….………………...….54 28. ‘Hold Up Time Test’ Test Setup.…………………...……..………..….……………..….56 29. ‘Hold Up Time Test’ input parameters..………………………….……….…..……..…..59 30. ‘Hold Up Time Test’ default input parameters.……….……………………….……...…60 31. ‘Hold Up Time’ HTML Report ………………………….…….……………………...…61 32. ATE: Power Consumption Test Result..…………………….…..…………..…………...65 33. ATE: Inrush Current Test Result: PASS.…………………………...…………..………..66 34. ATE: Inrush Current Test Result: FAIL ……………………………………….……...…69 35. ATE: Rail Switching Frequency Test Result: PASS…….……………...........…………..72 36. ATE: Rail Switching Frequency Test Result: FAIL…...……………………………...…74 37. ATE: Voltage Rail and Noise Test Result: PASS….…………………...………………..76 38. ATE: Voltage Rail and Noise Test Result: FAIL……………………………………...…78 39. ATE: Hold Up Time Test Result: PASS.……………….……………...……….………..80 40. ATE: Hold Up Time Test Result: FAIL ………………………………………………...81 xi 1 Chapter 1 INTRODUCTION Verification and validation are the key factors that play a vital role in any hardware product lifecycle. These two techniques are required in the design, construction, operation and maintenance stages of any hardware development activity. Verification and validation improve hardware quality and help deliver the best possible hardware to the users. In the field of Computer Engineering, verification engineers establish validation environments and test suites. They are typically responsible for creating validation standards, developing test protocols, implementing test protocols, analyzing test results and documenting the results for further analysis. Their duties also include maintaining the equipment used for testing and optimizing test procedures. Studies done on the benefit analysis of automation [Ref. 1], confirmed that automation benefits are substantial on the long run even when the initial expenditure is high. Table 1, illustrates cost analysis of automation during the initial setup and table 2 shows the benefits of automation. Even though the initial setup for test automation is costly, it can be observed that after the initial period, return on investment (ROI) on test automation is very high. Test engineers are not required to spend their valuable 2 time on repetitive testing. Automation will make significant amount of gain in test execution expenditure. Table 1: Initial Expenditure in Automation Test Environment [Ref. 1] Test Parameters Test Parameters Test Type Automation Manual Tool Cost $134,000 Zero Cost of scripting/test design $30,000 $12500 Cost of executing/running test Zero $102000 Cost of maintenance $11,520 (24 weeks) Zero Total $175,520 $114,500 Return On Investment -45.53% Table 2: Cost Benefit with Test Automation [Ref. 1] Test Parameters 2nd Year(2 cycles) Test Type Automation Manual Test Tool Cost Zero Zero Cost of scripting/test design Zero Zero Cost of executing/running test Zero $204,000 Cost of maintenance $23 040 Zero Total $23 040 $204,000 Return On Investment 135% 3 Automatic test environment also increases the depth and coverage of testing and helps in improving reliability by reducing human errors. It contributes in building efficient product by providing fast, repetitive and reusable test environment and hence reduces the overall cost of the product. Typically in any hardware related testing; verification engineers spend the majority of their time on preparing test setups