AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Building Automated Test Systems
Microsoft Windows version 2.0
Power consists in one’s capacity to link his will with the purpose of others, to lead by reason and a gift of cooperation. Woodrow Wilson
Bringing Intelligence Into Test Automation
[email protected] Author’s Name: Abhinav Vaid Free Software Foundation (FSF) www.indyacellular.in
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Copyright (C) <2005>
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
DECLARATION OF NON-DISCLOSURE OF COMPANY CONFIDENTIAL INFORMATION
This is to declare that whatever has been produced in the book has entirely been my own effort and research.1 And there has been no disclosure of Confidential Information from my present/previous employers.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Dedication:
Parents, who made me competent enough to reach this stage.
Raadhu, Meenakshi whose innocence keeps strengthening me to bounce back irrespective of blows/crisis.
Vipul Kocher, who stands with me today to make this happen.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Foreword: The topic has a vast coverage on Test Automation, taking up real code examples, so that the engineer/tester himself has a privilege to decide as to what is good and makes sense as per his/her Test Development/Automation requirements. Test Automation in itself is extremely vast field, and Abhinav in has book has made a serious attempt to touch each and every aspect and has succeeded. Abhinav Vaid is professionally as well as personally known to me and has good command over the subject. He was involved extensively in Test Automation related activities when we worked together for a couple of years as a part of Motorola India Design Center. Rajesh Tibrewala Technology Lead, .Net, Microsoft India [email protected]
Abhinav Vaid is professionally known to me, and we have worked together in the past with a Semiconductor R & D Engineering Center, where Abhinav was heading QA and Product Release activities. He has an excellent command over the subject. Dr Vivek Mudgil Senior Product Manager, [email protected] Alcatel India
About the Author:
Its been almost 12 years since the time I got associated to IT and I have been doing documentation/ testing/ product release/ dev(mostly scripting related). I have been leading Release and Test teams from past 5 years. I have worked with RiverRun Software/Motorola India/McAfee India. Some of the key projects that I have tested and released: Arm architecture based wireless chipsets with various OS/Interface flavors targeted for Client(embedded, mobile, & handset) and Gateway markets · CodeWarrior IDE Pro 6 development Tools for X86 · CodeWarrior IDE Pro 8 development Tools for X86 · CodeWarrior IDE development Tools for ARM based Architectures · CodeWarrior IDE for eLinux
5 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • · BSP's for ARM 9 Line of processors on all the embedded OS'es SP, WinCE, Symbian 7 to 8.0b · Middleware Apps for ARM line of processors (include video and audio codecs) · McAfee GroupShield 5.0 · McAfee GroupShield 5.2 · McAfee GroupShield 6.0 followed by patch release · CDMA Based Wireless PCO Terminal, first live implementation in India
Technical Background: B.Tech. in Electronics and Communications Microsoft Certified Professional (MCP) Microsoft Internet Specialist Microsoft Certified Software Engineer Certified Software Test Engineer ISTQB Certified Test Engineer—Foundation level
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Assumptions:
I have assumed that the end user (a Test Engineer or an Automation Engineer) has some level of OS knowledge and has done some programming/testing in the past.
1. Target Audiences:
This book is not for: · Teaching Programming/Scripting
This book is for: 1. Test Engineers 2. Engineers, who aspire to build Automated Test Systems. Irrespective of whether they are Dev engineers/or Test engineers. 3. Intermediate to Advanced Software Engineers/Testers.
This book can be used by: 1. Beginners: who aspire to take up Software Testing/Software Test Automation as a career path. 2. Developers: who aspire to get into Test Automation or know the insights of the domain.
Feedback and Support: Although I have tried my level best to cover the subject and tried to trim down to as short as possible to have a focused approach. as much as I possibly could and have tried to make it as generic as possible. I would like to hear as to what you liked and what is it you did not like (all the more important). Any questions/feedback etc is more than welcome. I will address all the queries, it might take some time to reply but I will definitely reply.
Contact me: [email protected]
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Building Automated Test Systems -- Microsoft Windows version 2.0:
Changes Incorporated in Version 2.0
1. There were bugs in code as well as language, which have been fixed in this version. 2. There were various suggestions, inputs from various sources which I have tried to incorporate (which made sense to me and gave me time to do so). Most of it is the latest Windows Flavors in the market today.
Summary Snapshot of the Book: Building Automated Test Systems-- Microsoft Windows starts with giving a background on Test Automation. Technical as well as process perspective. Briefs about the Technical background required to build Test Automated Systems and recommended automation methodologies. The book has taken some of most popular 3rd Party Test Automation Tools available in the market, practically implemented/automated a test suite/plan using all the tools, one by one. The Test Application/Plan picked up to automate is one of the most common among computing guys( for easier understanding) and has taken Test Plan of 1 particular Functionality, and the Automation is done using them. Using some of the most popular Test Automation Tools, and why do they fail,? Misleading promises by the Tool Vendors have always costed a lot to the Organizations, and still does not deliver what it is supposed to. Why do Organizations buy the expensive test automation tools and later regret,when they are not been able to justify the ROI. The Book has gone into Operating System Internals(only from the Testing and Automation perspective), and the inherent engines that support Test Automation. Explains the background insights as to what happens when you automate an application, how do the test automation tools handle it and what is the suggested workarounds to make them generic enough so that it works across across platforms removing the dependencies.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
The book has taken a couple of scripting tools\languages most of them under open source licensing and automated the same Test Suite(which we did using Test Automation tools using 10 different technologies\languages etc using OS internals which even Test Automation Tools also use.
Conventions Used in the Book:
There are a couple of conventions that I have tried to use to have a better clarity in terms of the important points and their associated interpretations. The same could be used as a handy Reference.
Conventions Used Intended Interpretations Food For Thought R & D Related Stuff, and tips/scope to dig further if you look forward towards to dig further
Tips Important tips for successful and handy Reference Beware about Running the Tests as there Precautions and Suggested could be side effects to the Lets say for workarounds Example being quoted, tips to take precautions, and suggested workarounds in case it happens. Tools and Associated Support Tools which are around for a while and have penetrated into the space to the extent of contributions to the Test Engineering Automation Bringing Intelligence in Test The highest level aspiration every good tester looks forward to; to be a perfectionist. Automation
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Table of Contents
Chapter1: The Evolution of Software Testing to Test Automation...... 14 Preface and Introduction...... 14 Setting up the Stage for Test Automation...... 14 Test Automation in Embedded Systems...... 17 Test Automation Linux...... 18 Organizing the Grey Matter 18 Some Higher Level abstracts on Test Automation Tools...... 19 Getting into Insights of Test Automation...... 20 Chapter 2 : Using 3rd Party Tools for Test Automation...... 20 Using Winrunner for Test Automation...... 20 Using LoadRunner for Test Automation...... 33 Using QuickMacros for Test Automation...... 38 Implementation using QuickMacros...... 48 How Quick Macros Works: Insights...... 57 Using Quick Test Professional (QTP) for Test Automation...... 67 Chapter 3: Touching the OS Internals, insights to make a Test Automation Project a Success.. 76 Architecture Of Microsoft Windows...... 77 Bridgeing The Logical Divide: Test Automation Checks & mapping/interpretation to the Conventional Microsoft Windows insights...... 79 Microsoft Windows Kernel - A Brief Primer...... 80 The Changing Trends of Partitioning in Windows:...... 82 What is COM: And how is it associated with Test Automation ...... 92 Chapter 4: Making a Business Case to Build Test Automation Suites/Frameworks: Using 10 different technologies/languages to do the same Test Automation done using Commercial Test Tools...... 94 What are Object Models: ...... 105 Capturing and Using Objects:...... 106 Automating Office Applications ...... 115 Using WSH for Test Automation...... 119
10 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
A brief primer on Application Programming Interfaces (API)...... 165 OLE for Test Automation for Automation of Microsoft Access...... 165 OLE for Test Automation with Microsoft Word...... 166 Using Perl for Software Testing ...... 202 Using Perl with OOPS...... 202 Using Java Script for Test Automation...... 221 Testing with/For Databases ...... 223 Perl/ Active Perl For Test Automation...... 231 Untold Truths of Winbatch/CLI/DOS BASED Programming for Test Automation...... 235 Chapter 5: Technological Advancements/diversification and Automation Challenges: Suggested workarounds and Best Practices...... 243 Chapter 6: Touching the Extremes in Test Automation...... 243 Chapter 7: Stepping up for Vertical Growth in Test Automation/associated technologies..243 Effective Reporting Diagnostics tools in windows:...... 243 Configuring to Capture Kernel Dumps...... 245 Using Dr Watson for Effective Reporting...... 246 Event Viewer and tricks to effective Usages: ...... 248 Some Third Party Automation Tools: Comments and Technical details: ...... 253 Log Analysis Tools ...... 263 Software QA and Testing-related Organizations and Certifications ...... 265 Chapter 8: Conclusions and Summary...... 267 Some Clarifications...... 269 A couple of un-answered and debate-able questions in Software Testing...... 269 Some Myths about Testing and Test Automation...... 271 Live Examples & Proofs: ...... 272 Knowledge base Test Certifications, climbing up the Technical Ladder...... 272 Lessons Learnt from the Book...... 272 Where to Go from Here:...... 272 Resources and References...... 274 Touching the Extremes/Cherish towards Intelligence Perfection...... 274 Gearing up for the Future: What lies ahead, and What are going to be the Testing Challenges...... 275
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Security Challenges: ...... 275 Other References/ Technical Articles by Author (The published ones)...... 284
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Table of Figures
Figure 7.1.: Navigating to WinRunner...... 20 Figure 7.2.: Winrunner Home Page...... 21 Figure 7.3.: Object handling by the Test Automation Tool ...... 28 Figure 7.4.: Winrunner Test Code for Word.application...... 29 Figure 7.5.: UsingWinrunner for Test Automation...... 30 Figure 7.6.: ...... 32 Figure 7.7.: LoadRunner Installation Screen 2...... 32 Figure 7.8.: LoadRunner Installation Screen 1...... 32 Figure 7.9.: Destination Folder Selection Dialog Box...... 33 Figure 7.10.: Loadrunner Installation Screen 2...... 34 Figure 7.11.: Loadrunner Installation Screen Sample Prompt...... 36 Figure 7.12.: First time Invocation Screen gives the error message ...... 37 Figure 7.13.: Copying Files Dialog Box...... 38 Figure 7.14.: Available Disk Screen...... 39 Figure 7.15.: Param Configuration Screen...... 39 Figure 7.16.: Registration Confirmation Screen ...... 40 Figure 7.17.: Navigating to QuickMacros...... 54 Figure 7.18: Error in Test Automation - Office A...... 57 Figure 7.19.: Sticky Tools and Supportive/Intuitive Interface of The Tool ...... 69 Figure 7.1.: Installers Tracking and information...... 187 Figure 7.2.: Task Manager Support by Microsoft Windows...... 190 Figure 7.3.: Log Generation Support by Microsoft Windows ...... 192
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Chapter1: The Evolution of Software Testing to Test Automation
PREFACE AND INTRODUCTION
Since the time I started my career, during the old programming days, and I was more inclined towards Testing more than development. So the background activity that was always happening was self study of Testing\QA\Automation related books.
An effort to share insights required to Building Automated Test Systems: Cost Effective, and generic enough to work across platforms in 1st go. Principles behind to make Automation Project a Success.
There was lot of stuff available on Testing , but not on Test Automation. And whatever I read with respect to the subject, most of it gave abstract views about how to implement Test Automation. None of them really talked about the real time practical implementation. I must have referred to hundreds of urls/technical information sites/technical journals etc., but after going through tonnes of pages, realized that the real concrete part was of less than10 pages, which confused me all the more.
Setting up the Stage for Test Automation Necessity is the mother of Invention. To fill in the gaps that existed became the main source of inspiration to go ahead and write this book.
The book starts with the way the Test Automation Tools available in the market work, their benefits and pitfalls. Then goes to the level of explaining as how to do automation in Windows using various programming/scripting languages, removing all the dependencies. Displays the real code examples (in various languages for doing the same task), so that the Test/Automa-
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
tion engineer can himself decide as to what can be best for him.
Bringing Intelligence in Test Automation
The story seems to go to the same old time quote that I had heard, “Are computers intelligent”. Unfortunately not, it is we who make them intelligent.
When the work was nearing the completion, it was covering everything including generation of automation for all the automation related projects right from OS Windows to Linux, and then Embedded systems also. The idea was to address most of the audiences that we have (which even had a some small implementation AI in “Building Automated Test Systems-- Embedded Systems”.
There were lots of reviews from peers, a couple of Industry Wizards, and of course Publishers, and there was 1 feedback from all the fronts, i.e., Focus on 1st and do it in phases.
So had to converted it into 3 books and this happens to be the 1st one. The other 2 will/might come up as a separate books sometime down the line (again in case I get sufficient time to do so).
The idea was to have a more focused approach, which made a lot of sense to me also. I have tried to be precise and crisp. With absolutely no abstract views about technical aspects. Real code examples have been given which have been tested, and successfully implemented.
Why Automation: As they say “Necessity is the Mother of Invention”. The same fact holds true for Software Test Automation also. The computing technologies have been moving at such a rapid pace which result in diverse and complex systems targeted towards various industry needs.
In such a scenario, Software Testing becomes all the more critical as 1 code change at 1 place might result in breaking the system at “n” number of places. Eventually what that means is
15 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • that even if there is 1 code change, you need to execute the whole test cycle associated with product. Now 1 test cycle might take any number of days depending on the coverage that needs to be achieved. Here automation definitely comes handy, and if it works effectively, the physical presence of the Test Engineer is not even required. Unfortunately that is not so simple as it sounds. Yes, doing intelligent automation/scripting is definitely complicated. But when it works, it is a different fun altogether. I have tried to cover each and every part of Test Automation for every type of environment (hw/sw/OS specifics etc).
What's the challenge?
To make the scripts generic enough so that they work without manual intervention required to do so. In case it turns the outer way round, it can be really disastrous.
I remember once an application targetted for the web, which was supposed to handle auto-mailers, and was about to go for a realtime testing. The guy had tested simulating and automating, and using the same script, he used to test the application in the realtime environment and before he could realize, there
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
were already 40 thousand mails sent. Imagine, if he had run it before leaving for the day.
Typical Test Automation Requirements and Technical Challenges and the Suggested Methodologies Before moving ahead, thought to share a couple of examples (small subsets automation tasks), so that the audiences get just a hang of it & maybe make it little more interesting. I have personally very used and have known the Industry wide acceptance and implementation it. 1. Microsoft Office Test Automation 2. Web Related Test Automation 3. Registry Related Test Automation 4. Mailing Solutions Related Test Automation 5. Installer Test Automation 6. Smoke/Sanity Test Automation 7. Automating Client Server related Testing 8. Building Auto-build and Release Infrastructure 9. Interoperability Test Automation 10.Networks Related Test Automation 11.Wireless Test Automation 12.Testing for Help Systems 13.Building Career In Automation & Associated Technologies 14.Testing Methodologies- Contradictions and Clarifications 15.Unanswered Questions in Software Testing/Test Automation 16.Microsoft's own Problems of Memory Management and Recommended Solutions 17.Touching the Extremes to make an impact 18.Measuring Quality and Testing Effectiveness 19.Bug Tracking Related Test Automation 20.Usability and Help Systems Related Testing
6.4.2 Test Automation in Embedded Systems
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • The generic principles or the basic principles of which that make the make the Embedded Automation Systems to work. I have tried to make it as generic as I possibly could and tested across all the Embedded Operating systems Symbian 7.0 to Symbian 8.0s, Windows CE, eLinux although could not work on Microsoft Smartphone in 1 go, for which I have given the workaround.
As I mentioned earlier, this would be coming in as the 3rd volume of the book. There is a different world out there that in Embedded needs to be addressed/explained. Also these are the inputs that I received from the various industry experts to make it very specific, targeted for specific audiences. I am very sure that embedded guys will like it. Personally, to me also embedded technologies and testing/automation has always been more fun
6.4.3 Test Automation Linux I remember the time when we first heard Linux, and were kind of amazed as to what’s going on. Frankly speaking, when I started my career, I knew only UNIX and nothing else not even windows. And had the least idea as to what’s Linux is all about. Today I am the biggest fan of Linux, though in my career never really got much chance to do deep into it and explore. Its got to do a lot in the way it has been built, everything is opensource kind of and you have the privilege to compile kernel itself. So one can Imagine, the kind of automation and the level and maturity of automation that can be achieved in Linux.
Organizing the Grey Matter
How: Precisely, I have taken a couple of Test Cases, and then automated all of them using: 1. Test Automation Tools that are commercially available 2. Different Scripting languages/technologies (using the same/different engine) So what you are going to achieve is Automating the same stuff(Test Cases) using 10 different technologies( Test Tools and/or Scripting languages).
Why: The objective remains to explore/make the Software Engineers realize, what goes behind those fancy Test Tools and to what extent that helps/doesn’t help. Workarounds if it does not help.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
SOME HIGHER LEVEL ABSTRACTS ON TEST AUTOMATION TOOLS
Explains as to how most of the commercially available Test Tools in the market work, what happens right from clicking the Record button to Compiled Script generation. What happens in the background when the test cases are being recorded. How does the Test Tool interface with the Windows. What are the components of the Test Tool interacting with Windows(and which component of windows) and how. What are the benefits of Test Tools and what are the pitfalls to checkout for. Why do the Organizations end up buying the Test Tools and later regret because of non justification of money spent and also the time lags, R&D efforts spent by the Test Engineers. Later the Test System itself becoming a different project in itself, making the real objective itself going for a toss. Although this is a worst case scenario but I have personally seen happening so many times in some of the most respected brand names in the Industry.
Whatever technical evaluations I have mentioned in the book has come out as a result of my own experience/research and comes out with my own understanding of the Test Tools and Scripting Languages associated. It can no way be taken as a technical claims or otherwise. Technical Redundancy, if any would be purely co-incidental, as there is no book/journal etc that I have referred to while writing this book. I have myself tried a lot to find a similar book/material but could not find any. Yes, there are a couple of url's which could be handy for the engineers which I have included
Making a business Case for Test Automation In the recent past around 6 months back or something, my current manager Dr Ravi Shankar had made a statement that he hates using the Word “Monkey Testing”, but he really does not need monkeys working as laborers, he just needs brains. And make scripts work like monkeys instead.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • It was an interview, and my reply was how soon can I join. There really is a lot of fun but it comes with a heavy price associated, not monetary but the time to do R & D to make the project a success. If you aspire to do this, this book is for you.
Getting into Insights of Test Automation Whenever I open any book, I generally see the Index and then dig inside whether the book is providing the information I am looking forward to have. Ironically Software Test Automation is a field which is maturing with time, but again there is some background that is essentially required to understand the core depths associated with the subject. So in this book, here we go.
Chapter 2 : Using 3rd Party Tools for Test Automation
1.USING WINRUNNER FOR TEST AUTOMATION
There might be various copywrite issues associated by putting the insights to the internal engine and architectural details of the test tools. So I really am not going to that stage, and from the testers perspective, its not required also. We do not intend to do reverse engineering, right !!!
· A Declaration by Author: Whatever is mentioned here has been the result of my own research and R & D. and sharing this with a intention of contribution to the Industry. Any technical claims\litigations etc do not apply in this case as it a self sponsored & non-commercial contribution to the Industry. Also I have avoided to talk about inside architectures of the test tools, (but any queries related to this can be mailed to me and I will definitely disclose on need basis)
Procedure/Test Steps/Test Case:
The basic steps in Test Development/Automation are more or less the Requirements Specifications which are used for development Team.
20 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Further, the pre requisite before any automation task to do is to execute it once manually so that the requirement/expectation of the Task is absolutely crystal clear.
Test Plan/Test Steps:
1. Invoke WinRunner. 2. The main screen will be displayed as shown in the figure.
Figure Number range Illustration.: Navigating to WinRunner
1. Click the Record button.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure : Winrunner Home Page
2. will get minimized (only if specified; but a lot of Automation Tools do it as a default behavior). Go to Start->Programs-->Microsoft Word (the actual/default path might not be same on all windows flavors. 3. Invoke Microsoft Word. 4. Type a couple of words. 5. Save the Document as C:\Test-Report\TC-1.doc 6. Exit and observe the Code Window
The following window will be displayed:
# Author: Abhinav Vaid
#Building Automated Test Systems version 2.0
set_window ("", 7);
button_press ("Start");
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
set_window ("Start Menu", 2);
list_select_item ("Abhinav Vaid", "Microsoft Office Word 2003"); # Item Number 4;
# Document1 - Microsoft Word
set_window ("Document1 - Microsoft Word", 2);
obj_type ("Microsoft Word Document_1","rgua ua vKSM rjSAJLjdh
obj_mouse_click ("Menu Bar", 28, 15, LEFT);
# File_1
win_mouse_click ("File_1", 29, 30);
# Save As_1
set_window ("Save As_1", 3);
obj_mouse_click ("MSO Generic Control Container_1", 18, 211, LEFT);
list_activate_item ("SysListView32", "Local Disk (D:)"); # Item Number 1;
obj_mouse_dbl_click ("RichEdit20W", 32, 2, LEFT);
obj_mouse_click ("RichEdit20W", 32, 2, LEFT);
obj_type ("RichEdit20W","bRAEWAYKRA
# bRAEWAYKRA.doc - Microsoft Word
set_window ("bRAEWAYKRA.doc - Microsoft Word", 0);
obj_type ("Microsoft Word Document_1","
7. Close Microsoft Word
Implementation And Evaluation of the Automated Test
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
1. Maximize the Test Tool. 2. Click the button Run, as shown in the Figure 3. Observe the Test Results. 4. Record the Test Results. 5. Save the Test Results.
Test Results: 1. Winrunner writes the code for you. 2. A sample code written(by the Test Automation Tool) is written below:
# Author: Abhinav Vaid
#Building Automated Test Systems # Winrunner for Microsoft Word Automation
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
set_window ("", 3);
button_press ("Start");
# BaseBar
set_window ("BaseBar", 15);
toolbar_select_item ("ToolbarWindow32", "Microsoft Office;Microsoft Office Word
2003");
# Document4 - Microsoft Word
set_window ("Document4 - Microsoft Word", 3);
obj_type ("Microsoft Word Document","building automated Test Systems
# Microsoft Office Word
set_window ("Microsoft Office Word", 0);
button_press ("No");
Now What? Run the Script. Observe the results, wow it works. Is that really so simple? Unfortunately it is not? Read on to know why.
Attention to Detail: go to the folder where Winrunner has recorded and saved the results. Observe the directory structure.
Pitfalls: Take the script to another machine and run, will this work. Unfortunately no. But Why?
25 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • The test tool is not installed on the target machine. What do you do, buy another license and install. Now run the script. Will this work? Well it depends (on various dependencies).
Processor Configuration (this one might vary and might not be true for all the test automation tools out there.
Software configuration needs to exactly match with the machine on which the automation script was generated (this includes right from the OS version to where the test program has been generated to where the targeted Test program (that needs to be tested) has been installed. But why?
Importance of Checks:
What is a check, anyway ???
· A check can be defined something similar to Objects, which is identified by the Test Automation Tool.
Technical Jargon associated with Test Developers and Tool Vendors as well as the Users and/or Test Engineers like us. · The most popular Test Automation Tools available work on the principle of checks. · While you keep clicking buttons for the test steps. · What the test tool does is captures all the events and saves them as checks. · There are some standard checks that most of the Automation Tools use are : 1. BMP Checks 2. UI/ Checks 3. Text Checks 4. Co-ordinate Checks · and so on…. · Please refer to the screen capture as displayed in figure : TSL Online Help File for Checks:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
So as you keep hitting the Test Steps, the associated Test Automation Tool keeps recording the steps as checks and writes the code for the test steps.
TSL is the Scripting language associated with the Test Tool being evaluated. A couple of Proofs: Allows you to select another application's objects that you want available in your code by setting a Reference to that application's Obj Library.
Dialog Box Options/ Available References
Lists the references available to your project.
1. After you set a Reference to an object library by selecting the check box next to its name, you can find a specific object and its methods and properties in the Object Browser.
2. If you are not using any objects in a referenced library, you should clear the check box for that Reference to minimize the number of object references Visual Basic must resolve, thus reducing the time it takes your project to compile. You can't remove a Reference for an item that is used in your project.
3. If you remove a Reference to an object that your are currently using in your project, you will receive an error the next time you refer to that object.
4. References not in use are listed alphabetically.
Note You can't remove the "Visual Basic For Applications" and "Visual Basic objects and procedures" references, because they are necessary for running Visual Basic.
Priority Buttons
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Moves references up and down,on the list. When you refer to an object in code, Visual Basic searches each referenced library selected in the References dialog box in the order the libraries are displayed. If two referenced libraries contain objects with the same name, Visual Basic uses the definition provided by the library listed higher in the Available References box.
Result: Displays the name and path of the Reference selected in the Available References box, as well as the language version.
Browse: Displays the Add Reference dialog box so that you can search other directories for and add references to the Available Resources box for the following types:
1. Setting References to Type Libs lets say for example *.olb, *.tlb 2. Exe Files – For Example *.exe, *.dll
3. ActiveX Controls – For Example *.ocx
4. All Files *.*
The Add References dialog box is the Open common dialog box
6.4.4 How do The Automation Script Engines Work?
So based on the checks it keeps writing scripts in the background and it is the inherent support that Microsoft Windows supports to make it work. What happens even if there is just 1 pixel mismatch in the target matching
What happens if the Operating System is installed on C drive and is xyz drive.
What happens if the software under test does not have icon under the Program Files Folder.
· Note: Unfortunately in all the above cases, the test script would fail.
So how do we make it work ?
Make it generic enough so that it works everywhere in 1 go.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
How do we do that? We as Testers have a critical approach and a mindset towards more or less everything. So we tend to ignore the good things at times.
The same is true for the Tool we are evaluating. Positive Points: Yes there are a lot of them. Any layman, xyz without any knowledge of scripting should be able to do it in one go. Amazing isn't it and it works( Of course with limitations mentioned earlier)
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure .: Object handling by the Test Automation Tool
For a broader level understanding, lets do 1 more implementation Prerequisite: Invoke the Winrunner and click the record button.
Test Plan: Test Case 1: Invoke the Calculator App via Start-->All Programs-->Accessories-->Calculator. Expected behavior: The app should get invoked.
Test Case 2: Click the Scientific view. Expected behavior: Test Case 1: Click 12345679*8 and hit the return key. Expected Expected Results: should display 98765432 Test Case: Exit Calculator Expected Behavior Test Results: Pass\Fail
30 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Stop Recording Run the test using the Winrunner Record button. Observe the results. Result: It works. Problem 1: Keep the Calc Application Open, and Run the test. Test Results: it will display the following error
Figure WinRunner-Object Error
Figure .: Winrunner Test Code for Word.application
The Problem: its not been able to recognize the object. Please refer to the screen capture. Workaround: As the conventional Test tools are based on Object and checks based
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • concepts, we need to ensure that we bring in the kind of generic-ness that is required to ensure that it runs successfully in 1 go, no matter what. Even if one instance of any app is open which was not there when the Test was recorded, it will fail. If the machine has different OS/Display settings and so on, it will still fail. There are tonnes of other similar issues, and the list is endless. Permanent Solution: Winrunner has got its own scripting language associated with it. Which is pretty intuitive and backed up with good amount of support from the vendor It has all the debugger support available, and it is very intuitive and user friendly. A screen cap is attached below. It has highlighted the debugger controls.
Figure : UsingWinrunner for Test Automation
Then where is the problem/ Identification/Isolation the Problem 1. Were we prepared to debug the Test App, or just record and play? 2. Were we prepared to learn the scripting language associated with Winrunner? 3. Were we ever told the cost of the tool that we have sourced. 4. Were we ever told that in case you really want to learn the points 1 and 2, there are so many opensource languages available with little or no cost implication.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Winrunner also uses the Objects concepts withing itself.. Please refer to the screen capture
pasted below.
Figure Object Winrunner
Another Tool by the same Vendor: Load Runner:
USING LOADRUNNER FOR TEST AUTOMATION
LoadRunner, as the name suggests is primarily for Load Testing and commands a fair amount of respect among testers. Personally I have used it and find it a good tool to use. While working on this book, I was involved in evaluating so many Test Tools, testing as well as automating. There were a lot of very interesting findings which I have shared in this book at various places. One of the most interesting finding was when I found that the most popular test automation tool with maximum no of bugs in the installation wizard. So can we bank on this. I do not
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • intend to criticize any Organization/ Tool, so have not specifically mentioned the bugs but just pasted the screen captures for a handy Reference.
Figure : Destination Folder Selection Dialog Box
Figure Number range FigureLoad Runner Installation Wizard:Destination Path Selection Dialog box
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure : Loadrunner Installation Screen 2
There really are more than 25 bugs in in its installer itself. Please refer to the Screen captures I have included in the current section.
35 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure Load Runner Installation Wizard
Figure .: Available Disk Screen
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure .: Param Configuration Screen
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure : Registration Confirmation Screen
Another Classic Example from Microsoft itself: I was trying to reproduce a crash and trying to use Dr Watson to trace the dump. And came across this dialog box. I must have seen this 100 times, but still wondering as to what it means.
There is a whole world out there in the said tool because of which I will not be able to do justification by just taking a project with 1 scenario.
But a couple of points that I discovered during evaluating Load Runner for Testing. Its more or less same (with some inbuilt and some freeware tools) when it comes to giving the values it provides. Most of the tools in the market are not been marketed, and thus most of them are not known to the end users/testers.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
My take: Do not use it for Performance Related Testing.com
Useful: When the product is very stable, the stress/load testing is extremely important and the turn around time is pressed for results.
USING QUICKMACROS FOR TEST AUTOMATION
What is QuickMacros: An Introduction Its also one of the popular test automation tool thats been extensively being used for test automation. Its known to be more cost effective, and delivers as per the prices go. What is a macro; work; Quick Macros window Creating new macro, menu or toolbar Adding macro commands Recording A macro is a list of commands that are executed when the macro runs. The commands can perform the same actions as you do manually each day: type text, click menus, run files, etc. To launch a macro, you can assign it a trigger (e.g., hot key), or place it in a toolbar, or use some other way. That is, instead of manually doing the same sequence of actions again and again, you can place it in a macro, and launch it with just a single click or keystroke. Macros are easy to create. In many cases, you can simply record your actions (click the Record button on the toolbar, ...). In the Quick Macros window, you can create macros, as well as items of other types (functions, toolbars, etc). You create a macro for each task you want to automate. You can assign it a trigger (e.g., hot key), or place it in a toolbar or menu. Quick Macros runs continuously (although normally it is hidden most of the time). It watches for triggers that are assigned to macros. When a trigger occurs (e.g., hot key pressed), is launched the macro to which the trigger is assigned. When the macro runs, are executed commands it contains. Quick Macros also manages your toolbars and menus. Quick Macros window
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • To show the Quick Macros window, click icon in the system tray or in the Start menu. Window parts: Left side - list of QM items - macros, functions, menus, toolbars, TS menus. Right side - text (list of commands) of the currently selected item. Bottom-right - QM output or tips. In the output are displayed error descriptions and other messages. Bottom - status bar. Top - menu, toolbar, floating toolbar. Creating new macro, menu or toolbar You create a macro for each task you want to automate. To create new macro, click the New Macro button on the toolbar, and type a name in the small field that appears at the top of the list. To create an item of other type (function, toolbar, etc), use the popup menu that appears when you click the small arrow beside the New Macro button. FAQ's on QuickMacros
Adding macro commands Macros are stored as simple text. Usually, each command is stored in a separate line. To add macro commands, you can use dialogs from the floating toolbar. However, QM does not provide dialogs for all commands, and therefore you have to type other commands directly. You can find all commands listed in the Reference topic. You can also type a period (.) to view and insert available commands, functions, etc. The simplest way to add macro commands - record. Recording You can record keystrokes, mouse, and some other commands. You can record complete macro, or only some parts. To start recording, click the Record button on the toolbar. When recording is finished, click Insert or other button in the QM Recording dialog. Recorded macro often is not perfect. You have to review and possibly edit it. For example, recorded macro may run too fast, and you have to insert delays or change the wait time in wait commands. Recorded window names often contain document name, which causes error when running the macro when there is open other document. In this case, remove document name.
40 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
When recording, it is recommended to use the keyboard instead of the mouse (where possible), especially to select menu items (use Alt and underlined letters). It is because menus and other objects next time may be in another place, even when recorded mouse coordinates are relative to window (default). Adding menu items and toolbar buttons Menu text is simply a list of macro commands. Each command is preceded by some text (label), space and colon. The same with toolbars. Each line creates menu item or toolbar button, which will execute the command. Example:
Macro18 :mac "Macro18" Notepad :run "notepad.exe" Email :key "[email protected]" * text.ico Login :key "abcd"; key T; key "1234"; key Y * text.ico To quickly add a macro, drag and drop it from the list to the menu/toolbar text. To add a file, use the Run File dialog from the floating toolbar, or drag the file from the desktop or Windows Explorer. To add other commands, use other dialogs from the floating toolbar. To add a separator, type - or |. To add icons, use the Icons dialog.
Launching macros, menus and toolbars You can use various ways to launch a macro: You can use the Run button on the QM toolbar. You can assign it a hot-key, mouse, window, or other trigger. To assign a trigger click the Properties button on the toolbar. You can place it in a toolbar. Open the toolbar and drag the macro onto the toolbar text. Similarly, you can place it in a popup menu. A toolbars can be attached to a window. You can schedule it to run at a certain time or on some other event. Click the Schedule button in the Properties dialog. You can create a shortcut on the desktop. Click the Shortcut button in the Properties dialog. Other programs can launch QM macros using command line. You can use the mac command to launch it from another macro. T.S. menu commands are executed when you type a text.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Menus, toolbars, T.S. menus and functions can be launched using the same ways as macros. How to know that a macro is running When a macro is running, the QM tray icon is red . When items of other types (functions, menus, etc) are running, the tray icon does not change. You can see running functions in the Threads dialog (menu Tools -> Threads). How to end a running macro When a macro is running, the QM tray icon is red. To end running macro manually, press Pause key (you can change this in Options). If input is blocked (BlockInput is used in the macro), at first press Ctrl+Alt+Delete. If it is a function, you cannot end it with Pause. Right click the QM tray icon, click End Function, select the function in the Threads dialog, and click End Thread. Or, if you use the AddTrayIcon function to add a separate icon, you can Ctrl+click it. Or, you can place special code in the macro or function. Example: rep ifk(F12) ret ;;exit if F12 is pressed ... How to assign a toolbar to a window Assign it a window trigger. To assign a window trigger, select Window in the Properties dialog, drag the "Drag" picture and drop onto the window. You may have to edit or remove window name, e.g., remove the name of the currently open document. How to create an "auto-hide" toolbar Click menu File -> New -> Templates -> Toolbar Top or Bottom, or Toolbar Left or Right. To launch it, you for example can assign a mouse trigger. Or, launch it at startup. Managing toolbars You can drag toolbars with the right mouse button. To drag or delete buttons, Shift+Drag. You can drag macros, files and Internet links and drop onto a toolbar. To change toolbar style, or quickly open to edit, use the right-click menu. If you have lost a toolbar (it is running but invisible), use the Toolbars dialog (menu Tools -> Toolbars). How to create text-replacement ("autotext") macros In QM, instead of creating separate macro for each replacement, you create one or several T.S. menus, and place the replacements there.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Can macros run in any program? Macros can run in any Windows program. You can make a macro to run only in a certain program(s). Some triggers may not work in DOS and console programs, especially in full screen mode and on Windows 98/Me/Vista. Try low level hooks (in Options). Also, macros may not run properly in some games. Can two macros run simultaneously? Several macros cannot run simultaneously. If a macro (macro2) is launched while a macro is already running, macro2 will not run. Similarly, several instances of the same macro cannot run simultaneously. Several functions can run simultaneously. If a function is launched while a macro or function is already running, they will run simultaneously.
Can macros run in the backround? Macros that use keyboard, mouse and some other commands, cannot run in the background. However, it is often possible (but not so easy, and will not always succeed) to replace keyboard and mouse commands to other commands that usually can work in the background. To insert such commands, use dialogs from the floating toolbar's "Windows, Controls" menu. Can I run a macro without running QM? Yes. You can create executable programs from macros and functions. Can macro run when an error occurs? When a macro is launched, at first it is compiled. This includes error checking. If the macro contains errors, it is not executed. Error description is displayed in the QM output, and error place is highlighted. When macro runs, some commands may fail (e.g., due to a missing file or window). Then macro ends. To continue on run-time error, you can use err. Example: run "abc.exe" ;;macro would end if file "abc.exe" does not exist err ;;allows macro to continue when an error occurs in the preceding command What is the System folder and other default items in the list? Many QM features are written in QM language. It includes the floating toolbar, all dialogs from it, the Dialog Editor, and many useful functions that you can use in macros. All this code is placed in the System folder. That is why you should add System to each file you use (by
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • default, it is added automatically). The System folder actually is loaded from separate file System.qml. Macros in the Default Macros folder aren't necessary but can be useful to you. The Samples folder is absolutely not necessary, and is provided only for learning. What is the list that pops up when I type a period (.)? The list contains functions and other identifiers that you can use in macro. Double click an item in the list to insert it in macro. At the top of the list, are listed categories - collections of related functions. When you double click a category in the list, pops up another list containing functions from that category. Disabling commands, adding comments Lines that begin with a space are disabled. This also can be used to add comments. To quickly disable/enable single line, right click the selection bar (thin gray bar at the left of the macro text). To disable/enable several lines, select them and right click the selection bar. To add comments at the end of line, use two semicolons. Example: comments key Cv ;;press Ctrl+V Using strings; inserting newlines and quotes Strings (text) must be enclosed in double quotes. To insert a double quote in a string, use '' (two apostrophes). To insert a newline, use []. All this is true when you enter strings directly. When you enter strings through dialogs (except the Keys dialog), all this is done automatically.
Example: out "This is[]multiline string[]with ''double quotes''" Repeating Assume you want to repeatedly execute two commands: lef 100 200 key Y Insert rep before. Then select the commands and press Tab key. This tab-indents the selected lines: rep lef 100 200
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
key Y Repeat 10 times:
rep 10 lef 100 200 key Y Programming in QM Using variables You can use variables almost everywhere in code. To store numeric integer values, use variables of type int. To store floating-point values - double. To store text - str. Example: without variables lef 100 200 "Notepad" use variables str s="Notepad" ;;declare str variable s, and store "Notepad" int x y ;;declare int variables x and y x=100 y=x*2 lef x y s To share a variable between macros, declare it with +: int+ global_var You can use variables in dialogs (floating toolbar). To use a variable in a text field, enclose it in parentheses. Parentheses are not used in fields that accept numeric values. Creating and understanding macros is easier if you know the following 20 commands. Some of them can be entered using dialogs, or recorded, but often it is quicker to write them directly. lef, rig Click mouse left or right button. The easiest way to enter these commands is recording. Or, use the Mouse dialog. Examples: lef 100 200 ;;left click at 100x200 pixels lef 90 40 "Notepad" ;;left click at 90x40 pixels in Notepad window key Press keyboard keys. Type text. To type text, enclose the text into quotes. To enter other keys, you can record, or use the Keys dialog. Examples: key "Indya" ;;type text
45 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • '"Indya" ;;type text (key can be replaced to ' ) key F12 ;;press F12 key LLLL ;;press Left Arrow four times key Cv ;;press Ctrl+V key A{ep} ;;press Alt+E+P outp Paste text. This command also can be entered through the Text dialog. Examples: outp "New Zealand" ;;paste text "New Zealand" ;;paste text (outp can be omitted) outp s ;;paste variable s out Display something (numbers, strings, variables, etc) in the QM output. Useful when debugging macros or learning (experimenting with) various commands and functions. This command also can be entered through the Text dialog. Examples: out "I am here" ;;display text out i ;;display variable i mes Display something in a message box. This command also can be entered through the Message Box dialog. Examples: mes "Important information" if(mes("Continue?" "" "YN")='N') ;;if the user clicks No ret ;;exit run Run file, open document, web page, etc. This command also can be entered through the Run File dialog or other dialogs from the same menu. Or, you can drag and drop a file from the desktop or Windows Explorer. Examples: run "C:\WINDOWS\system32\notepad.exe" run "http://www.quickmacros.com" act Activate window. This command also can be entered through the Window dialog. Example: act "Notepad" win Find window and return window handle that can be used in other commands. This command also can be entered through the Find Window Or Control dialog. Example: int hwnd=win("Internet Explorer" "IEFrame")
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • act hwnd wait Insert simple delay, or wait for some event (e.g., window). This command also can be entered through the Wait dialog. Examples: wait 5 ;;wait 5 seconds 0.5 ;;wait 0.5 second (wait can be omitted) wait 30 "Internet Explorer" ;;wait max 30 s for Internet Explorer mac Launch a macro. This command usually is used in menus and toolbars. To enter it quickly, drag and drop the macro from the list. Example: mac "Macro9" Note: mac is not used to call functions. The launched macro is independent from the current macro, and the current macro does not wait for it. err Continue macro if an error occurs. Example: wait 2 "Window" ;;wait for Window max 2 s, and then throw error err ;;on error continue ret Exit (end current macro or function). Example: if(i=0) ret ;;if variable i is 0, exit goto Go to another line. Example: if(i>=10) goto g1 ;;if variable i is >= 10, go to the g1 label out "This line is executed only if i is < 10" g1 out "This line is executed always" if, else Execute or skip commands if certain condition is true. The commands must be preceded by a tab, unless they are in the same line. Example: if i<5 ;;if variable i is less than 5, execute the following two commands out "variable i is < 5" i+1 else ;;else execute the following one command out "variable i is >= 5" rep, break Repeatedly execute commands. The commands must be preceded by a tab, unless they are in the same line. Examples:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • rep 10 ;;press Right Arrow 10 times key R Repeat while variable i is <= 5: int i ;;declare variable i rep i+1 if i>5 out "i is > 5" break ;;exit the loop out i int, str Declare variables. Use int for numeric integer variables. Use str for string variables. Examples: str s="Notepad" ;;declare str variable s, and store "Notepad" int x y ;;declare int variables x and y int+ g_var ;;declare global int variable g_var
IMPLEMENTATION USING QUICKMACROS • I Q M Take the Plan, and start executing the Test Cases. It would have been redundant to mention everywhere that 1st and foremost requirement of Test Automation is what you as a tester is supposed to Automate. Exact Test Steps (Test Plans) and associated results. This is irrespective of all the Automation Tools and all the scripting lanaguages / technologies used in the book for demonstration Test Case/Test Steps: 1. Invoke the Test Tool (QuickMacros in our case). 2. Click the Record button. 3. The Quickmacros will get minimized. 4. Invoke a calculator application. 5. Type 12345679*8 and click the return key. 6. Observe and Record the Test Results.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
7. Exit the Calc Application
Test Results: Displays 87654321
Expected behavior: same as above. 1. The minimized window has an insert field/checkbox. 2. Click the checkbox. 3. The Quickmacros home screen will get invoked and there will be code/script written by the Automation Tool.
Validate the Automated Test: 1. Run the script then and there on exactly the same environment and observe the results. 2. It really works. Validate The Automated Test to Another machine with exactly the same hardware as well as the the same OS flavor.
Test Results: It will fail.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure Number range FigureQ M
Analysis would reveal that the application has not failed, but its the in-efficiency of the tool in our case.
Lets do Another Automation Project using the same tool.
2. Test Case/Test Steps: 3.
Inorder to do the automation of the test Plan, perform the following steps: 1. Invoke the Test Tool.
50 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range Illustration.: Navigating to QuickMacros
2. Click the Record button. 4. The Window will get minimized. 5. Click the Record button 6. For easy identification, it remains as an icon on the windows taskbar. Please refer to screen capture.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure: Quick Macros Dialog Box
7. Invoke Microsoft Word, type something, save the document and then close it. 8. On the minimized Test Tool icon, click Insert. 9. It will write the code for the manual test we did.
What happens: 1. Record the Test Results 2. Save the Test Results
Like the earlier this also generates the code. Validating the Code on the Same Setup: Run the Test Automation Script. Observe/ Record the Test Results Result: It won't work in the vary 1st attempt itself.
Try Analyzing: Go via code, check out the error msg displayed
Try to Evaluate the findings:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
It did work for calc in the 1st go, but didn't work for Microsoft Word in the vary first attempt. Why? The Answer is Complexity handling by the automation tool as the said tool is not that matured as other 2 that we have used, and that is the reason the 2nd Test of Microsoft Office Application will work in vary first attempt. So the price is justified and the industry has been paying it. What does it take you to do it in case you do not have the test automation tool available with you. I had tried the same.
Conclusions and Interesting Findings: I tried to do it using Active Perl and the result were really interesting: 1. I could automate in lesser amount of time. 2. It worked across on all windows platforms. 3. Saving a cost of one of the most expensive tools in the world. Active Perl is free. 4. 4 Preventing licensing costs and also saving a huge amount of resources that the automation tool demands.
One of the myth that i have seen is people generally treat automation tools and jargon fancy, is this they have to spend money or is it ignorance. Whatever, but does not deserve to be spared.
I have listed a couple of functionalities of the current test tool, and could not find anything that cannot be done in lets say Microsoft's inherent scripting language.
What is a macro How Quick Macros work Quick Macros window Creating new macro, menu or toolbar Adding macro commands Recording Adding menu items and toolbar buttons
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure : Error in Test Automation - Office A
Launching macros, menus and toolbars How to know that a macro is running How to end a running macro How to assign a toolbar to a window How to create an "auto-hide" toolbar Managing toolbars How to create text-replacement ("autotext") macros Can macros run in any program? Can two macros run simultaneously? Can macros run in the backround?
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Can I run a macro without running QM? Can the macro run when an error occurs? What is the System folder and other default items in the list? What is the list that pops up when I type a period (.)? Disabling commands, adding comments Using strings; inserting newlines and quotes Repeating Using variables Programming in QM (if, goto, functions, etc) What is a macro A macro is a list of commands that are executed when the macro runs. The commands can perform the same actions as you do manually each day: type text, click menus, run files, etc. To launch a macro, you can assign it a trigger (e.g., hot key), or place it in a toolbar, or use some other way. That is, instead of manually doing the same sequence of actions again and again, you can place it in a macro, and launch it with just a single click or keystroke. Macros are easy to create. In many cases, you can simply record your actions (click the Record button on the toolbar, ...).
55 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure: Inherint Scripting/Macros/Objects...support as a default My Findings: as far as Macros go, the Microsoft provides the support anyway. Although a little more effort is required. Please refer to the screen capture
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
How Quick Macros Works: Insights •
As a layman/tester, you just see the UI. But what happens in the background is something that always remains hidden for a Tester. I have tried to throw some light on that. Lets start from the vary first home screen/window.
In the Quick Macros window, you can create macros, as well as items of other types (functions, toolbars, etc). You create a macro for each task you want to automate. You can assign it a trigger (e.g., hot key), or place it in a toolbar or menu.
Quick Macros runs continuously (although normally it is hidden most of the time). It watches for triggers that are assigned to macros. When a trigger occurs (e.g., hot key pressed), is launched the macro to which the trigger is assigned. When the macro runs, are executed commands it contains. Quick Macros also manages your toolbars and menus. Quick Macros window To show the Quick Macros window, click icon in the system tray or in the Start menu window parts: Left side - list of QM items - macros, functions, menus, toolbars, TS menus. Right side - text (list of commands) of the currently selected item. Bottom-right - QM output or tips. In the output are displayed error descriptions and other messages. Bottom - status bar. Top - menu, toolbar, floating toolbar. Creating new macro, menu or toolbar You create a macro for each task you want to automate. To create new macro, click the New Macro button on the toolbar, and type a name in the small field that appears at the top of the list. To create an item of other type (function, toolbar, etc), use the popup menu that appears when you click the small arrow beside the New Macro button. Adding macro commands Macros are stored as simple text. Usually, each command is stored in a separate line. To add macro commands, you can use dialogs from the floating toolbar. However, QM does
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • not provide dialogs for all commands, and therefore you have to type other commands directly. You can find all commands listed in the Reference topic. You can also type a period (.) to view and insert available commands, functions, etc. The simplest way to add macro commands - record. Recording You can record keystrokes, mouse, and some other commands. You can record complete macro, or only some parts. To start recording, click the Record button on the toolbar. When recording is finished, click Insert or other button in the QM Recording dialog. Recorded macro often is not perfect. You have to review and possibly edit it. For example, recorded macro may run too fast, and you have to insert delays or change the wait time in wait commands. Recorded window names often contain document name, which causes error when running the macro when there is open other document. In this case, remove document name. When recording, it is recommended to use the keyboard instead of the mouse (where possible), especially to select menu items (use Alt and underlined letters). It is because menus and other objects next time may be in another place, even when recorded mouse coordinates are relative to window (default). Adding menu items and toolbar buttons Menu text is simply a list of macro commands. Each command is preceded by some text (label), space and colon. The same with toolbars. Each line creates menu item or toolbar button, which will execute the command. Lets say for Example: Macro18 :mac "Macro18" Notepad :run "notepad.exe" Email :key "[email protected]" * text.ico Login :key "abcd"; key T; key "1234"; key Y * text.ico To quickly add a macro, drag and drop it from the list to the menu/toolbar text. To add a file, use the Run File dialog from the floating toolbar, or drag the file from the desktop or Windows Explorer. To add other commands, use other dialogs from the floating toolbar. To add a separator, type - or |. To add icons, use the Icons dialog. Launching macros, menus and toolbars
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
You can use various ways to launch a macro: You can use the Run button on the QM toolbar. You can assign it a hot-key, mouse, window, or other trigger. To assign a trigger click the Properties button on the toolbar. You can place it in a toolbar. Open the toolbar and drag the macro onto the toolbar text. Similarly, you can place it in a popup menu. A toolbars can be attached to a window. You can schedule it to run at a certain time or on some other event. Click the Schedule button in the Properties dialog. You can create a shortcut on the desktop. Click the Shortcut button in the Properties dialog. Other programs can launch QM macros using command line. You can use the mac command to launch it from another macro. T.S. menu commands are executed when you type a text. Menus, toolbars, T.S. menus and functions can be launched using the same ways as macros. How to know that a macro is running When a macro is running, the QM tray icon is red . When items of other types (functions, menus, etc) are running, the tray icon does not change. You can see running functions in the Threads dialog (menu Tools -> Threads). How to end a running macro When a macro is running, the QM tray icon is red. To end running macro manually, press Pause key (you can change this in Options). If input is blocked (BlockInput is used in the macro), at first press Ctrl+Alt+Delete.
If it is a function, you cannot end it with Pause. Right click the QM tray icon, click End Function, select the function in the Threads dialog, and click End Thread. Or, if you use the AddTrayIcon function to add a separate icon, you can Ctrl+click it. Or, you can place special code in the macro or function. Lets say for Example: rep ifk(F12) ret ;;exit if F12 is pressed ... How to assign a toolbar to a window Assign it a window trigger. To assign a window trigger, select Window in the Properties
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • dialog, drag the "Drag" picture and drop onto the window. You may have to edit or remove window name, e.g., remove the name of the currently open document.
How to create an "auto-hide" toolbar Click menu File -> New -> Templates -> Toolbar Top or Bottom, or Toolbar Left or Right. To launch it, you for example can assign a mouse trigger. Or, launch it at startup. Managing toolbars You can drag toolbars with the right mouse button. To drag or delete buttons, Shift+Drag. You can drag macros, files and Internet links and drop onto a toolbar. To change toolbar style, or quickly open to edit, use the right-click menu. If you have lost a toolbar (it is running but invisible), use the Toolbars dialog (menu Tools -> Toolbars). How to create text-replacement ("autotext") macros In QM, instead of creating separate macro for each replacement, you create one or several T.S. menus, and place the replacements there. Can macros run in any program? Macros can run in any Windows program. You can make a macro to run only in a certain program(s). Some triggers may not work in DOS and console programs, especially in full screen mode and on Windows 98/Me/Vista. Try low level hooks (in Options). Also, macros may not run properly in some games. Can two macros run simultaneously? Several macros cannot run simultaneously. If a macro (macro2) is launched while a macro is already running, macro2 will not run. Similarly, several instances of the same macro cannot run simultaneously. Several functions can run simultaneously. If a function is launched while a macro or function is already running, they will run simultaneously. Can macros run in the backround? Macros that use keyboard, mouse and some other commands, cannot run in the background. However, it is often possible (but not so easy, and will not always succeed) to replace keyboard and mouse commands to other commands that usually can work in the background. To insert such commands, use dialogs from the floating toolbar's "Windows, Controls" menu.
60 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Can I run a macro without running QM? Yes. You can create executable programs from macros and functions. Can macro run when an error occurs? When a macro is launched, at first it is compiled. This includes error checking. If the macro contains errors, it is not executed. Error description is displayed in the QM output, and error place is highlighted. When macro runs, some commands may fail (e.g., due to a missing file or window). Then macro ends. To continue on run-time error, you can use err. Lets say for Example: run "abc.exe" ;;macro would end if file "abc.exe" does not exist err ;;allows macro to continue when an error occurs in the preceding command What is the System folder and other default items in the list? Many QM features are written in QM language. It includes the floating toolbar, all dialogs from it, the Dialog Editor, and many useful functions that you can use in macros. All this code is placed in the System folder. That is why you should add System to each file you use (by default, it is added automatically). The System folder actually is loaded from separate file System.qml. Macros in the Default Macros folder aren't necessary but can be useful to you. The Samples folder is absolutely not necessary, and is provided only for learning. What is the list that pops up when I type a period (.)? The list contains functions and other identifiers that you can use in macro. Double click an item in the list to insert it in macro. At the top of the list, are listed categories - collections of related functions. When you double click a category in the list, pops up another list containing functions from that category. Disabling commands, adding comments Lines that begin with a space are disabled. This also can be used to add comments. To quickly disable/enable single line, right click the selection bar (thin gray bar at the left of the macro text). To disable/enable several lines, select them and right click the selection bar. To add comments at the end of line, use two semicolons. Lets say for Example: comments key Cv ;;press Ctrl+V Using strings; inserting newlines and quotes
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Strings (text) must be enclosed in double quotes. To insert a double quote in a string, use '' (two apostrophes). To insert a newline, use []. All this is true when you enter strings directly. When you enter strings through dialogs (except the Keys dialog), all this is done automatically. Lets say for Example: out "This is[]multiline string[]with ''double quotes''" Repeating Assume you want to repeatedly execute two commands: lef 100 200 key Y Insert rep before. Then select the commands and press Tab key. This tab-indents the selected lines: rep lef 100 200 key Y Repeat 10 times: rep 10 lef 100 200 key Y Using variables You can use variables almost everywhere in code. To store numeric integer values, use variables of type int. To store floating-point values - double. To store text - str. Lets say for Example: without variables lef 100 200 "Notepad" use variables str s="Notepad" ;;declare str variable s, and store "Notepad" int x y ;;declare int variables x and y x=100 y=x*2 lef x y s To share a variable between macros, declare it with +:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • int+ global_var You can use variables in dialogs (floating toolbar). To use a variable in a text field, enclose it in parentheses. Parentheses are not used in fields that accept numeric values. Top 20 commands Creating and understanding macros is easier if you know the following 20 commands. Some of them can be entered using dialogs, or recorded, but often it is quicker to write them directly. lef, rig Click mouse left or right button. The easiest way to enter these commands is recording. Or, use the Mouse dialog. Examples: lef 100 200 ;;left click at 100x200 pixels lef 90 40 "Notepad" ;;left click at 90x40 pixels in Notepad window key Press keyboard keys. Type text. To type text, enclose the text into quotes. To enter other keys, you can record, or use the Keys dialog. Examples: key "Indya" ;;type text '"Indya" ;;type text (key can be replaced to ' ) key F12 ;;press F12 key LLLL ;;press Left Arrow four times key Cv ;;press Ctrl+V key A{ep} ;;press Alt+E+P outp Paste text. This command also can be entered through the Text dialog. Examples: outp "New Zealand" ;;paste text "New Zealand" ;;paste text (outp can be omitted) outp s ;;paste variable s out Display something (numbers, strings, variables, etc) in the QM output. Useful when debugging macros or learning (experimenting with) various commands and functions. This command also can be entered through the Text dialog. Examples: out "I am here" ;;display text out i ;;display variable i mes Display something in a message box. This command also can be entered through the Message Box dialog. Examples: mes "Important information" if(mes("Continue?" "" "YN")='N') ;;if the user clicks No
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • ret ;;exit run Run file, open document, web page, etc. This command also can be entered through the Run File dialog or other dialogs from the same menu. Or, you can drag and drop a file from the desktop or Windows Explorer. Examples: run "C:\WINDOWS\system32\notepad.exe" run "http://www.quickmacros.com" act Activate window. This command also can be entered through the Window dialog. Lets say for Example: act "Notepad" win Find window and return window handle that can be used in other commands. This command also can be entered through the Find Window Or Control dialog. Lets say for Example: int hwnd=win("Internet Explorer" "IEFrame") act hwnd wait Insert simple delay, or wait for some event (e.g., window). This command also can be entered through the Wait dialog. Examples: wait 5 ;;wait 5 seconds 0.5 ;;wait 0.5 second (wait can be omitted) wait 30 "Internet Explorer" ;;wait max 30 s for Internet Explorer mac Launch a macro. This command usually is used in menus and toolbars. To enter it quickly, drag and drop the macro from the list. Lets say for Example: mac "Macro9" Note: mac is not used to call functions. The launched macro is independent from the current macro, and the current macro does not wait for it. err Continue macro if an error occurs. Lets say for Example: wait 2 "Window" ;;wait for Window max 2 s, and then throw error err ;;on error continue ret Exit (end current macro or function). Lets say for Example: if(i=0) ret ;;if variable i is 0, exit goto Go to another line. Lets say for Example: if(i>=10) goto g1 ;;if variable i is >= 10, go to the g1 label out "This line is executed only if i is < 10"
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
g1 out "This line is executed always" if, else Execute or skip commands if certain condition is true. The commands must be preceded by a tab, unless they are in the same line. Lets say for Example:
if i<5 ;;if variable i is less than 5, execute the following two commands out "variable i is < 5" i+1 else ;;else execute the following one command out "variable i is >= 5" rep, break Repeatedly execute commands. The commands must be preceded by a tab, unless they are in the same line. Examples: rep 10 ;;press Right Arrow 10 times key R Repeat while variable i is <= 5: int i ;;declare variable i rep i+1 if i>5 out "i is > 5" break ;;exit the loop out i
int, str Declare variables. Use int for numeric integer variables. Use str for string variables. Examples: str s="Notepad" ;;declare str variable s, and store "Notepad" int x y ;;declare int variables x and y int+ g_var ;;declare global int variable g_var A lot of critics say negative things about all the test automation tools available, but there are a lot of handy stuff out there. Please refer to the Screen capture as displayed in Figure 19. There are a lot of good and handy things about QuickMacros which I personally found impressive with the cost that is associated with the Product.
65 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Illustration Number range Illustration.: Sticky Tools and Supportive/Intuitive Interface of The Tool
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
USING QUICK TEST PROFESSIONAL (QTP) FOR TEST AUTOMATION
QTP happens to be another Test Automation Tool by Mercury Corp, and has an impressive market penetration. Testers rate high about it and very keen to explore as and when they have an opportunity.
Core Functionalities:
Supports VB, active x, and Web Applications Supports Record Playback modes Based on Object (OO) related concepts Generates vbs scripts for the actions preformed Alternatives to Standard Recording
Analog Recording
Relative to Screen
VB Script Statement : Desktop.RunAnalog "Track9"
Relative to Window
VB Script Statement : Window("Microsoft Internet").RunAnalog "Track8"
The track file called by the RunAnalog method contains all your analog data and is stored with the action.
Tip: To stop an analog step in the middle of a test run, click Ctrl + Esc, then
click Stop in the test toolbar.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Use low-level recording for recording tests in an environment or on an object not recognized by QuickTest or if the exact coordinates of the object are important for your test
The Web Event Recording Configuration dialog box offers 3 standard event-configuration levels.
Basic
Medium
High
By default, Quick Test uses the Basic recording -configuration level.
If Quick Test does not record all the events you need, you may require a higher event- configuration level.
To set Web Event Recording Configuration
Click Tools -> Web Event Recording Configuration
Virtual objects enable you to record and run tests on objects that are not normally recognized by Quick Test.
You can teach Quick Test to recognize any area of your application as an object by defining it as a virtual object.
Using the Virtual Object Wizard, you can map a virtual object to a standard object class, specify the boundaries and the parent of the virtual object, and assign it a logical name.
The virtual object collections displayed in the Virtual Object Manager are stored on your computer and not with the tests or components that contain virtual object steps.
To copy your virtual object collection definitions to another computer, copy the contents of your
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Using QTP for Office Automation.
Test Plan: 1. Invoke QTP. The 1st default screen comes with the following options. Select all first 2 options for Office Word testing.
Figure Number range Figure
With the default options, click the record button on the home page. 1. Invoke Microsoft Word 2. Type a couple of lines 3. Save the document 4. Close the document 5. Click the stop button
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 6. The Stop button will become disabled and the Run will get enabled. 7. Click the Run. 8. Observe the results 9. Record the results.
Findings:
Will work in the 1st Test Run.
Flaws and Workarounds:
if the same document has been saved at the location/the test is already run. The test will fail, but the application under test is not.
There are tonnes of other scenarios after the point 1 which will also fail.
Object Model of QTP:
Understanding Test Object, Run-Time Object, Properties & Methods
The test object property set for each test object is created and maintained by QuickTest. The run-time object property set for each run-time object is created and maintained by the object creator. (Microsoft for Internet Explorer objects, Netscape for Netscape objects, the product developer for ActiveX objects, and so forth.)
70 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Drawing Number range Drawing Figure Number range Figureobject model
Similarly, test object methods are methods that QuickTest recognizes and records when they are performed on an object while you are recording, and that QuickTest performs when your test or component runs. Run-time object methods are the methods of the object in your application as defined by the object creator. You can access and perform run-time object methods using the Object property.
For information about activating run-time methods using the Object property, see “Retrieving and Setting Test Object Property Values” on page 872.
Each test object method you perform while recording is recorded as a ➤ separate step in your test or component. When you run your test or component, QuickTest performs the recorded test object method on the run-time object.
Test object properties are the properties whose values are captured from the ➤ objects in your Web site or application when you record your test or component. QuickTest uses the values of these properties to identify runtime objects in your application during a run session.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Drawing Number range Drawingfunctionalities at a glance
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Drawing Number range DrawingObjectSpy
Start from the following steps (one after the other till the time the test passes)
Enable all the checks and capture again. Note that there will be a performance over head on the system go one by one, from simple to analog.
Further analog to low level. (where even low level fails)
Object Repository and captured details of the objects.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Other Insights into the Tool:
Drawing Number range DrawingHelp System Bug
75 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Chapter 3: Touching the OS Internals, insights to make a Test Automation Project a Success
Doing a fair amount of home work is a key to any success, and the same happens to be true in our case also. But its a whole world out there. I have tried to tailor what is required strictly from a Testers perspective.
1. What is Microsoft Windows
2. Architectural Insights into Microsoft Windows
3. Windows Kernel
4. Windows API’s
5. Advent of OOPS to COM/DCOM
6. OLE
7. WSH to handling GUI
8. Scripting Architectures
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
ARCHITECTURE OF MICROSOFT WINDOWS
Before We go to the Insights of Microsoft Windows Operating System, here is the Figure of the same (highest level abstract for a higher level underlying grey matter which matters the most. Scripting Language
Application Programming Interface (API)
Drivers/Windows Kernel Drivers/Windows Kernel
Hardware x86 In Our Case
Figure: Highest Level Architectural View of Microsoft Windows-- Block Diagram Illustration
Lets talk about Internals and associated Representation in block diagram: Please refer to the Figure. It talks about the hardware Architecture (X86) in our case. The basic memory management and associated processes/components.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Architectural View of Microsoft Windows Operating System Please refer to the block text and diagram view below. I think you should be able to map this to the above mentioned Architectures (Off course at an abstract level).
Scripts for Automation--> Various Scripting Languages like perl\python\vbs and so on
API Drivers\Windows Kernel
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Hardware
Windows Architecture: Scripting Perspectives for Interfacing
So what does it reveal? The Hardware happens to be at the lowest level.
And the topmost is the scripting languages
Technically the interface between the operating system and the user programs is defined by the set of "extended instructions" that the operating system provides. These extended instructions have been traditionally known as system calls, although they can be implemented in several ways.
The Windows architecture like any other operating system also consisits of instruction sets, memory organization, I/O, and bus structure.
The calls available in the interface vary from operating system to operating system (although the underlying concepts tend to be similar) irrespective of the Operating System.
➢ Bridgeing The Logical Divide: Test Automation Checks & mapping/interpretation to the Conventional Microsoft Windows insights.
The Figure shows a mapping of Checks in Test Automation Tools are a notation/jargon for Objects in Operating Systems (Little variations here and there; just for implementation, and nothing else)
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Architectural Mapping between Objects in Windows and Checks in Test Automation Tools
Too Confused? Don't worry guys. There is one of my all time favorite Quote from one of the smartest guys in Windows.
Learning Windows API's gives you vital insights into working of Microsoft Windows that are regardless of what you end up using to actually do the coding. Windows is a complex system, putting a programming layer on top of API which does not eliminate the complexity. It merely hides it, sooner or later that complexity is going to jump out and bite in the leg. Programming Windows 5th edition, by Charles Petzoid.
Microsoft Windows Kernel - A Brief Primer •
The reason I have given an architectural figure/description of Architecture is as to how the
80 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Operating System (Windows in our subject) handles/works internally. And how can we (as software Testers benefit from it. It also takes care of internal directory structures/files associations ...... and so on.
Other Technical Insights required from a Tester's perspective:
Boot Sequencing: Boot Loader Concepts of Microsoft Windows
On the Microsoft Operating Systems, I have not come across many testers who know the core of boot loaders of Windows. I have tried to tailor down just for the testing perspective. Sounds Geek, don't worry guys. There is always a 1st time.
I have pasted below a typical Boot Loader File from my Windows XP machine. Looks explanatory. It becomes all the more important when you need to have multiple OS'es flavors on the same vendor. Or even when you want to have a dual-boot with Linux flavor, wherein Linux itself has 2 popular boot loaders: Lilo, and Grub.
[boot loader]
timeout=0
default=multi(0)disk(0)rdisk(0)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Microsoft Windows XP Professional" /
noexecute=option /fastdetect
Explanation: fixmbr \Device\HardDisk0The device (drive) on which you intend to have the Boot Record.
The name can be obtained from the output of the map command. An example of a device name is:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • \Device\HardDisk0. Boot Record
It is not recommended to play with Master Boot Records unless in-editable due to multiple reasons. But understanding definitely is required by the testers time to time.
What are ini's? The “.ini” files come with .ini as an extension. Contains the hardware specific information.
What is Boot Record/ File Systems/associated configurations etc.
The Changing Trends of Partitioning in Windows: • The Changing Trends of Partitioning in Windows: Instead of a BIOS, computers with the Intel Titanium family of processors use the Extensible Firmware Interface (EFI) between the computer's firmware, hardware, and the operating system. EFI includes a new disk partitioning style, the GUID partition table (GPT). Compared to the disk partitioning style, which supports volumes up to 2 terabytes in size and up to 4 primary partitions per disk (or three primary partitions, one extended partition, and unlimited logical drives), GPT supports volumes up to 18 gigabytes in size and up to 128 partitions per disk. Unlike partitioned disks, data critical to platform operation is located in partitions instead of unpartitioned or hidden sectors. In addition, GPT partitioned disks have redundant primary and backup partition tables for improved partition data structure integrity. Disk Management describes disks with the GUID partition table (GPT) partitioning style as GPT disks and disks with the master boot record partitioning style as disks. You can perform the same tasks on GPT disks as you can on disks with the following exceptions: GPT disks are supported only by Windows XP 64-Bit Edition. You cannot move GPT • disks to computers running the 32-bit versions of Windows XP. From Disk Management on computers running a 32-bit version of Windows XP, GPT disks appear as basic disks with a single partition covering the whole disk, but the data on the partition cannot be accessed. The operating system loader and boot partition must reside on a GPT disk. Other hard •
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
disks can be either or GPT. You can have both and GPT disks in a single dynamic disk group. You can also have a • mix of basic GPT and disks, which are not part of disk groups. You can convert an disk to a GPT disk and vice versa only if the disk is empty. • You cannot use the GPT partitioning style on removable media, detachable disks that • use universal serial bus (USB) or IEEE 1394 (also called FireWire) interfaces, or on cluster disks that are connected to shared SCSI or Fiber Channel buses used by the Cluster service. You can create partitions on basic GPT disks using the EFI firmware utility diskpart.efi, the diskpart.exe command-line utility, or Disk Management in Windows XP 64-Bit Edition
What is a Process: The Process corresponds to the resources being used by CPU. Can be said to be an instance of a running program. The processes are created to run a program/application. Then, the app generates Process as defined by Address space
What is a Thread? An execution ext within a process Unit of scheduling (threads run, processes don’t run) All threads in a process share the same pre-process address spacing. Also, the services provided so that threads can synchronize access to shared resources (critical sections, mutexes, events, semaphores.. and so on). Primarily, all threading in Operating Systems is referred to as peers to all others. And this happens to be irrespective of other processes: let it be parent, address spacing and so on. System-wide, Address Space, Thread. I tried it and it was a breeze. In order to do this: Go to Start-Run. Type: tasklist /svc tasklist /svctasklist /svctasklist /svc
# Abhinav Vaid
# In All Things be Men
# Building Automated Test Systems- Windows
C:\Documents and Settings\naudee>cd\
C:\>svchost
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • C:\>tasklist /svc
Image Name PID Services
======
System Idle Process 0 N/A
System 4 N/A
smss.exe 432 N/A
csrss.exe 488 N/A
winlogon.exe 512 N/A
services.exe 556 Eventlog, PlugPlay
lsass.exe 568 PolicyAgent, ProtectedStorage, SamSs
svchost.exe 716 DcomLaunch, TermService
svchost.exe 816 RpcSs
svchost.exe 888 6to4, AudioSrv, Browser, CryptSvc, Dhcp, dmserver, ERSvc, EventSystem, FastUserSwitchingCompatibility, helpsvc, lanmanserver, lanmanworkstation, Netman, Nla, NwSapAgent, RasMan, Schedule, seclogon, SENS, SharedAccess, ShellHWDetection, srservice, TapiSrv, TrkWks, W32Time, winmgmt, wscsvc, wuauserv, WZCSVC svchost.exe 932 Dnscache
svchost.exe 1016 LmHosts, RemoteRegistry, SSDPSRV, WebClient
spoolsv.exe 1108 Spooler
svchost.exe 1204 BthServ
btwdins.exe 1224 btwdins
Apache.exe 1240 dev4_423
Mcdetect.exe 1292 McDetect.exe
McShield.exe 1312 McShield
McTskshd.exe 1364 McTskshd.exe
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
PAStiSvc.exe 1556 STI Simulator
svchost.exe 1568 stisvc
Apache.exe 1632 N/A
alg.exe 232 ALG
explorer.exe 880 N/A
mcagent.exe 2280 N/A
Residence.exe 2344 N/A
GoogleWebAccWarden.exe 2356 N/A
GoogleWebAccClient.exe 2416 N/A
mcvsshld.exe 2140 N/A
McVSEscn.exe 2612 N/A
IEXPLORE.EXE 2036 N/A
BBImpSec.exe 3356 N/A
AcroRd32.exe 3496 N/A
WINWORD.EXE 2060 N/A
cmd.exe 1984 N/A
tasklist.exe 4044 N/A
wmiprvse.exe 2264 N/A
C:\> Developers Need proofs: If thread stack doesn’t solve it, get a memory dump of the process address space & attach to the bug tracking system along with the bug details, Period. No Re-produceability debates. But guys, its easier said than done.
Drwtsn32.exe Drwtsn32.exe -p
85 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure Number range FigureConfiguring for Customized Crash Dumps configure as per Test Requirements.
Windows Crashes, the same old debates. There was developer's conference organized by Microsoft guys and the statistical data was generated from the inputs that were derived. Precisely, this is what it is: ~70% caused by 3rd party driver code ~15% caused by unknown (memory is too corrupted to tell) ~10% caused by hardware issues ~5% caused by Microsoft code Microsoft Windows Kernel writes memory but not the processes. In most of the cases
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
(definitely not always) most crashes related debugging doesn’t involve looking at process memory anyway Useful for large memory systems. Remember, whenever there is any exception raised, the options thrown are abort/debug, on selecting the later, the applications gets attached to the debugger. If you want to go deeper to understand: The latest Microsoft OS'es XP, Vista the minidump is always created, even if system set to full or kernel dump Can extract a minidump from a kernel or full dump using the debugger “.dump /m” command. To analyze, requires access to the images on the system that crashed At least must have have access to the Ntoskrnl.exe Microsoft Symbol Server now has images for Windows XP and later Set image path to same as symbol path (covered later) Writing a Crash Dump Crash dumps are written to the paging file Too risky to try and create a new file (no guarantee you will get a dump anyway) How is even this protected? When the system boots it checks HKEY_LOCAL_MACHINE\System\ CurrentrolSet\rol\Crashrol.The boot disk paging file’s on-disk mapping is obtained Relevant components are checksummed: Boot disk miniport driver Crash I/O functions Page file map On crash, if checksum doesn’t match, dump is not written At The Reboot
Confusing Jargon: guys, please do not confuse the image name mentioned to the System Images that we have been using for retrieval from crashes. I am attaching a screen cap to clarify.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure Number range FigureImage-TaskMgr-
1. Win32 API is the interface provided by Microsoft guys along with the Operating Systems. 2. API is used/referred to build applications on top of the OS, whether it is an application program/or a script to Test/Automate. 3. Intention is to give support for accessing and building components. There is a very old quote from one of the most widely accepted OS guru, and it happens to be one of my favorite one. It says Windows is a n extremely complicated and bulky system, with a programming layer on top of it (known as API) which merely hides it, and one day which will definitely come out and bite right on your leg. Now comes the identifications and usages associated. I have give a couple of examples, try them along with the help files of the OS.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Do a $gettemppath = new win32:: API'kernel32', 'kernel32.dll',)
Lets dig further: do a $gettemppath = new win32:: API'kernel32', 'gettemppath',NP',N')
3rd parameter listing consists of i/p parameter list; no of arguments required and the function and their types can also be passed as a string, can be identified as a parameter/list references; lets say for example “abcd” [a, b,c,d] note (a,b,c,d)@list and the 4th parameter returns the value returned by the function. $gettemppath = new win32:: API'kernel32', 'gettemppath','NP',N);
For example;
In win 9x, systems, we can use Quickview to know the functions that it exposes. There is a limitation however,
The 16 bit dlls have limitations however win32 based dlls can be used more than once.
Making a business Case for Test Automation This talks about Technologies/Engines available within Microsoft Windows that support automation, the ways to use them. How do the Test Tools use them, and how can we use them independently of using the Test Automation Tools.
Microsoft did manage to change the entire landscape of technologies associated with
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Computing. And it did manage to significantly penetrate the market. It had a very clear focus in place, usability and wizards driven technologies associated.
How ever there was 1 thing lagging, it was support for building applications over the OS. With everything hidden inside(unlike Linux), and OS not stable enough to work things out. But yes things have been changing positively. There was a time when you talked about windows kernel, people used to panic. So how does the change help us. The Component Object Model (COM) is a scripting language-independent and object-oriented programming model which defines the standards for implementation. It is definitely not a programming language. It is a binary standard that enables software to be implemented as units called components. The component can be edited in any programming/scripting language or scripting language associated with the same. After the binary unit has been built, it can communicate with other units that were produced in any other scripting language, on the same machine or on remote machines provided that the units were written as COM components. COM forms the core of all Microsoft technologies, and provides the skeleton for all it's future strategies. OLE is placed higher than COM and ActiveX sits on top of OLE. Most of the engineers associated with the Internet based development, ActiveX is a series of OLE classes. If you consider a Microsoft product like Visual Basic, you will find the concept of components. These components may have properties and events and other attributes. These components can be inserted into any container like VB, PB and so on etc. and later can be integrated into. For VB, that sounds fine, but the irony of the situation is that if we wanted to do the same thing from C++, then we will have a problem. It was extremely difficult to use ActiveX from C++, even though the entire ActiveX, OLE, COM families are based on C++. This was till the time Microsoft introduced Visual C++ 5.0 But its not as complicated as it might sound to you at this point of time, all the Microsoft Office Applications have been supporting them. And only thing is you might not have used it yourself. But its there. Please refer to the screen capture
90 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range FigureOLE Support Via Microsoft Word As many see it, the Component Object Model is elegant and powerful. The basic yet ingenious programming model provides the foundation for application-development. The power resides in that everything built on the same foundation can communicate with each other without any restrictions such as programming languages or platforms. When developing an application devoted to COM, components should encapsulate every functionality that the application requires. A task performed by the application would be executed by a component, and these components would be reused in other and future applications. Lets say for example, a Perl component can contain the process of sending an email or performing a series of regular expressions on text. The component can then be used to provide functionality for an Active Server Pages application or a Windows desktop application.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • In summary, although the scripting language in which a component is generated may vary from component to component, a binary standard overcomes the limitation of programming in different languages. It does not care about anything else but the binary form of the targeted processor's code that the once source code was turned into and the fact that is implemented into COM. In this sense, the component is not a regular script or executable, but a black box that performs one task and returns the projected result. Multiple languages can be used for one application because COM provides a binary standard. A binary standard also results in platform-independence provided that multiple platforms support COM. Units of code that are used in COM are known as components, and on Windows a component is normally file with the extension .dll; however, there are offshoots such as Windows Script Components, which have the extension .wsc. The purpose of the component is to perform the one task that it has been programmed for, and through COM, it can communicate with other components, languages, and platforms.
WHAT IS COM: AND HOW IS IT ASSOCIATED WITH TEST AUTOMATION
COM is just a set of standards laid down to compily/adhere to. It can also be termed as a a specifications laid down for Impmentation. And being both a specification and an implementation, the COM defines a standard for how the components are created and how they communicate - namely as objects. As a result, the specification solves the implementation issues of the following key issues:
1. The calling of a component 2. The locating of a component 3. The identification of a component 4. The creation of a component
Although the parts above are involved in the specification of how to implement components, the automation engineer normally does not have to worry much about them. A software development tool that holds high class will create a skeleton for a component, and leave it up to the automation engineer only to plug in the code in the right places. Thanks to this, the code itself can be developed, debugged, and experimented with as usual. And then when the
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • code is finished, it is placed in the skeleton, the right buttons are clicked, and the component pops out.
Little about COM Components Associated When wondering what makes up a component, recall that COM is an object-oriented programming model that requires every component to be implemented as an object, so from this we know that we will be talking about classes, methods, properties, and such entities that belong in an object-oriented programming model. However, let’s start at the bottom level and move our way up from interfaces, to classes, and to objects. COM Interfacing, What’s that: When the component is implemented, it exposes its features through one or more interfaces. The interface is a specification of a collection of methods and properties that are related to each other, so when a component is called, it must be called through one of its interfaces. In general, this means that an interface relates to an operation such as verifying a credit-card number or sending an email. Because the component may expose several interfaces and the method-calls pass by way of the correct interface, the interface that is called must be identifiable. It is a fact that names easily clash, and especially with a high pace of component development, so another solution is provided. An interface uses a globally unique identifier (GUID) which is a 128-bit integer. And through the GUID, the interface is identified to the COM.
Interfaces can very much be changed refer to prog id,s n registry, you will find no of entries for same @ sme will have version no associated. But all theclsid's will be same
COM Class: Next, the COM class implements, or maybe better 'inherits', one or more COM interfaces. What is a Class ID (CLSID) globally uniquely identifies the COM class, and while the interfaces are at the base level of the COM architecture, the COM class is on top of the interfaces - one step close to the application. In other words a Class ID is a globally unique identifier, which is used to identify control(something that is required to become comfortable to locate a specific control on your machine. The main purpose of the COM class is to generate, or initiate, the
COM object that represents the functionality of the unit of code.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Each window has got a control id associated. They further have the child windows. Such child windows have windows control, hitch uniquely identifies it irrespective of the machine thus taking care of genericness across any machine irrespective of fighting out for (checks); Remember I talked about, Grab the core/pulse and the rest will follow. COM Object: As previously mentioned, this part of COM simply is the instance of a COM class, or the entity from which all the features of the COM class are accessed. COM Component: In the midst of everything, where does the component fit into the picture? It can be easily summarized as that the component is used to include a COM class by identifying it by the CLSID, create an instance of the COM class as the COM object, and then let the application make use if the COM object on the machine.
CHAPTER 4: MAKING A BUSINESS CASE TO BUILD TEST AUTOMATION: USING 10 DIFFERENT
TECHNOLOGIES/LANGUAGES TO DO THE SAME TEST AUTOMATION DONE USING COMMERCIAL
TEST TOOLS
Relationship Insights: COM & Scripting Languages Automation, formerly OLE for Test Automation, makes it easier for the masses to access COM object servers such as components. In terms of a scripting language, the only way for a COM component to make its features available is through Automation. Automation is one level higher up than COM, in other words one step closer to the application, and it is the technology that lets software packages such as Microsoft Office expose its functionality in an object-model for scripting languages that support Automation. In the same manner as large applications like MS Office are exposed, smaller COM components are made available to a scripting language through Automation, as well. Without it, the scripting language would necessarily have to know all the interfaces of an object that it wants to use before calling it - too much information to build into a scripting language. Instead of bloating a scripting language with junk, COM defines the standard for accessing COM object servers: Automation. In turn, Automation makes sure that a standard interface that allows, in our case, script-access to the component is always available for object-access. This standard interface used in Automation is called a dispatch interface, dispinterface, or
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
automation interface.
The Significance of IDispatch Interface: A dispatch interface is somewhat different than the standard COM interface. Methods are associated with dispatch IDs (dispids), and methods have been provided for reading and writing the values of properties - the access to the data members in the underlying data structure. The standard dispatch interface for automation is called IDispatch. Through IDispatch, a component can expose as much as it wants to expose. Then in order for Perl to gain access to components, the interpreter needs to support an object-oriented programming model, have the ability to call methods through IDispatch, know how to return errors, and be able to destroy objects. A method that belongs to an interface of a COM component is retrieved by the dispatch interface and in Perl this can occur by what is known as late binding. A dispatch method called GetIDsOfNames is called with the name and the parameters of the method of the COM interface that is called. The dispatch ID of the method is returned. Next, the Invoke-method of IDispatch is called with all the processed information returned from GetIDsOfNames, and from that call the result is returned in an output parameter. This method is called late binding, and there are two things to notice about it: All data types that are sent as parameters through Invoke to the method of an interface must be passed as Variants or converted into Variants. Importance of Binding: When not using late binding, the application already is aware of all the dispatch IDs and does not call GetIDsOfNames at all. This is called early binding since the information needed for using the components have been built into the application and are used at run-time. Perl uses late binding, and the process of late binding is simple. First, a moniker is required, which is a name that uniquely identifies the COM object. The moniker is then used to locate the object, which next is either in running state or put into running state. When in running state, the server application can access the interfaces of the COM object, and when that is completed, an interface pointer is returned to it. The dispatch interface that makes COM objects accessible to a scripting language is implemented as a COM interface that uses IDispatch. And the IDispatch functions of this interface only calls the methods that are laid out in the COM interface, and these two interfaces are called dual interfaces because an
95 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • application that knows only early-binding can access it while another scripting language that knows only late-binding can access it, as well. Although the implementation of all of the above is hidden to Perl, there are some things to know. In most cases, Automation will accept and convert scalar data into the correct variant- type. However, it might be necessary to specify the type of data; so to solve this issue, you would convert Perl data types to the variants used
Scripting and Software Test Automation: Theories and Concepts-Technical Jargon Associated: What is Scripting: The concept of Scripting had started to Automate small and monotonous tasks, using/writing small scripts/programs to make them work. Let it be doing automating IT/admin tasks to building macros for MS Office Applications. There was always a question raised, Why Scripting and why not Programming. The answer is simplicity and genericness that most of the scripting engines provide rather than full-fledged programming languages. Also, scripts are easy to write and maintain. But no, scripts are definitely easy to write, but writing intelligent scripts is the real challenge. Especially when you are automating the Software Tests where the real challenge is to automate the whole test suite. And the tester should ideally be able to run lets say at the EOD, and the next morning, he has the results. This is the real challenge, for the scriptwriter to make it generic enough and handle errors (if any), that can make the test execution to stop in case of any error. Have U heard of “Onerror resume next”. We will cover this later in the book. What is Black Box Programming: this is the most convenient way of using procedures to achieve modularity.
What is a Function: is a procedure that returns a value.
What is Modularity: The process of organizing the code to modules, with an intention to make the code organized.
What is a What is a Procedure: Procedure is more of a general term that could mean a:
What is a QA and /or QC (Software Testing) and the thin like in between: There
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • cannot be any other term to start with especially with respect to our subject. Hundreds and thousands of books/jobs/references/links/tutorials/lectures etc etc that I have come across and each one talks about QA and not about QC/software testing. I think the industry is yet to reach such a maturity stage where these terms get defined. QA is a non technical job (a more of a process oriented targeted for certification, in most of the cases, although proves out to be an umbrella activity putting Software Testing under it, which is a pure technical job. The word QC still hasn't got associated to it as it is more specific to manufacturing and other engineering industries. Strictly speaking, QC is the Software Testing, where the Software System is Tested as per the Requirement Specifications laid out.
What is a Debugger: Used mostly to debug software programs intended to make them bug free. What is a IDE: Integrated Development environment, usually a GUI based interface. Run-time: This is mostly applicable for scripting languages, where the code runs line by line and gets executed simultaneously. What is a Binaries/byte-code: its mostly associated to the programming languages and IDE’s(dev environments), consisting of compilers, debuggers, assemblers, linkers and/or a combination of them to generate byte code/binaries/exe’s/installers which run at a later stage on deployment. The compile time and runtime are different in ideal conditions, but can be integrated also for effective optimizations. The same is true for Microsoft Windows. What is a White Box Testing: Also known as glass box testing is the Unit Level Testing, to be more specific it’s the code testing. And all the time, it reminds me of a continent about everyone wants to talk about as much as they possibly can. And when you are told, or try to go there everyone runs away like a thief saving for his life. Reason is simple, Whitebox testing is nothing but a glass box testing, where in each and every function/unit/module/subroutine..... is tested as per the coding guidelines specified. The only thing that a tester can do here is compliance checks with respect to the coding guidelines being followed or not. Having audits, incorporating comments and so on. Also this is true mostly for larger size organizations. For very small startups or thin engineering firms, multitasking makes more sense where the developers mostly do the development as well as testing. A detailed and mapping with respect to various demanding engineering needs has been explained in the end of the book.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • What is a Grey Box Testing: Falls between Black and White Box Testing What is a Black Box Testing: User Interface and System Testing What is a Macros/Macro: Series of instructions to perform the repetitive tasks What is a Subroutines:Are nothing but functions What is a Compiler/What is Interpreter:
A Compiler is used to convert to byte-code to run at a later stage
Interpreter executes during runtime So how does it impact us: Basically any scripting language can be implemented either with a compiler or with an interpreter. In a lot of implementations, its been observed that both of them are used.
The various Compiler guru's say/claim that a program translated by a compiler is often much faster than an interpreter executing the same program: even a 10:1 ratio is not uncommon. The mixed solution's efficiency is typically somewhere in between. The downsides of the "compiler" solution are the inherent complexity of a good implementation, and longer test run cycles. An Interpreter, on the other hand executes during run-time, line by line. It takes longer to run a program under an interpreter than to run the compiled code but it can take less time to interpret it than the total time required to compile and run it. This becomes all the more important when prototyping and testing code when an -interpret-debug cycle can often be much shorter than an -compile-run-debug cycle. The various Compiler gurus claim that Interpreting code is slower than running the compiled code because the interpreter is supposed to analyze each statement in the program each time it is executed and then perform the desired action whereas the compiled code just performs the action.
What is ASCII: An ASCII can be defined as a 7bit code, which means that it uses the bit patterns representable with seven binary digits (a range of 0 to 127 decimal) to represent character information. At the time ASCII was introduced, many computers dealt with 8bit groups
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
What is Graphical User Interface (GUI): which is also termed as GUI, sometimes pronounced "gooey") is a the directly interfacing with the Computer. Scripting Languages (commonly called scripting programming languages or script languages) are computer-programming languages initially designed for "scripting" the operations of a computer. A couple of such scripting languages happen to be complicated and generally used to write very lengthy programs, which are often still called scripts even though they go well beyond automating simple computer tasks. A script scripting language can be found at almost every level of a computer system. Besides being found at the level of the OS itself they appear in computer games, web applications, word processing documents, administrative tasks etc. Scripting is sometimes also used for connecting diverse pre-existing components to accomplish a new related task. Many scripting languages emerged as tools for executing one-off tasks. A batch file is typical example of the same, which is still being used. Scripts are normally stored only in their plain text form. Some scripting languages are designed for a specific domain, but often it is possible to write more general programs in that scripting language.
In very large Test Systems, a scripting language and a lower level programming scripting language can be used together. Intention is to get the best of each.
On any fresh installation of Windows Operating System, do a find all (Start--> Search) for any script, let it be batch, java script or vbs. Observe how the MS guys have implemented the stuff. I am sure you are going to enjoy this. Pasted below is a sample of js taken on Windows XP. Try it over other versions of windows with variety of languages.
Such high level commands simplify the process of writing code.
Programming features such as memory management and optimization is automatically taken care of. On the other hand, in lower level or non-scripting languages, managing memory and variables and creating data structures often take a lot of efforts of automation engineer and
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • also the code size also keeps increasing.
Although, for a developer it is a good idea but for the automation/test engineer/scripter normally doesn’t have much choice to optimize for speed and/or to conserve memory, and the main objective always remains to automate the test cycle and ensuring to bring genericness and effective test system.
For the reasons noted above, it is usually faster to program in a scripting language, and scripts/programs are relatively smaller lets say, equivalent C program files. The only downside could be a fractional (or even little more) time consuming, which should not make much of a big difference as the intent is to run the un-attended test cycles, when a tester can leave the tests running at the end of the day and when he is back the next day, he/she has the results in front of him.
Scripting languages are mostly interpreted (not always), can be relatively slower to execute and can have larger memory overheads. In many cases, however, e.g. with small scripts of some tens of lines, the write-time advantage far outweighs the run-time disadvantage.
I have been doing this for years now, and the latest fun we've had is leaving the un- attended tests at the end of the day, and as soon as the tests complete, we get an sms trigger along with the auto-generated test report. We implemented almost a month back. And it was a lot of fun. Of course when we are back the next day, the test results are there as configured.
The thin line between scripting languages and regular programming languages sill looks to be dithered.
What is Shell Scripting: A major class of scripting languages has grown out of the automation of job control starting and controlling the behavior of system programs. Many of these languages' interpreters double as the MS-DOS COMMAND.COM. For Lets say for Example: batch files still remains to be one of the favorites of our OS Vendors.
100 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
What are Macros: When the industry was maturing and moving also from command line based systems to GUI based systems. Thus came the need for for controlling the Controls/Widgets/Menus etc. The technological jargon named it to Macros. interact with the same graphic windows, menus, buttons and such that a person does. They are typically used to automate repetitive actions or configure a standard state. In theory they could be used to control any application running on a GUI-based computer, but in practice the support for such languages depend on the application and operating system. Vendors have even started supplying with the Tools, for example, Microsoft Office provides inbuilt support for Macros. Which can either be directly recorded or tailored manually using the code. Not only Microsoft, but other vendors are also giving support for Macros. Just for the Reference, please see the interface for using Macros in OpenOffice.
What are Application Specific languages A lot of scripting languages are tailored as per the needs of the software application and/or end user. Since I am currently associated to mobile gaming industry, let me take the example of gaming itself. In order to build games (whether mobile or desktop), the engineer’s normally use a custom scripting scripting language to express the programmed actions of characters and the gaming target environment. Now comes the trick, whether it is a mobile device(ARM architecture based) or a desktop computer the development machine would remain to be desktop computer. Only difference would be the usage of cross-compiler targeted for the ARM based architecture. There was a time when MIPS was the all time favorite environment for game development. In that case also, the desktop computer used to be used for game development generating binaries for MIPS based architecture again using cross compiler.
What is a Extensible Embeddable languages: A small number of languages have been designed for the purpose of replacing application-specific scripting languages, by being embeddable in application programs. The application automation engineer (working in C or another systems scripting language) includes "hooks" where the scripting scripting language
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • can control the application. These languages serve the same purpose as application-specific extension languages. For Lets say for Example: Java Script
What is a Tool command scripting language (TCL):Tcl was created as an extension scripting language but has come to be used more frequently as a general purpose scripting language in roles similar to Python, Perl, and Ruby. Also used along with Perl, specially for Test Automation
What is a Native and/or Cross Compiler Most compilers are classified as either native or cross-compilers. This is primarily due to the deployment environment. processor A compiler may produce binary output intended to run on the same type of computer and operating system as the compiler itself runs on. This is sometimes called a native-code compiler. Alternatively, it might produce binary output designed to run on a different platform. This is known as a cross compiler. Compilers and Interpreters: The thin line drawn in between It still looks to be a subjective and debatable topic, but again cannot be ignored. Some gurus divide higher-level programming languages into compiled languages and interpreted languages. Compilers as well as interpreters can be defined as implementations of languages targeted for varied audiences. The categorization usually reflects the most popular or widespread implementations of a scripting language – Lets say for example, Perl is thought of as an interpreted scripting language, and C a compiled one.
What goes inside a Compiler A typical compiler consists of the following components: . Passes 4 . Front End 5 . Backend 6 . Optimizers/Error Checkert consider myself TemplatesTemplates technically competent 7 enough to speak on this subject, but there are some points There was a time when compilers were divided into many passes to save space. A pass in this
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • context is a run of the compiler through the source code of the program to be compiled, resulting in the building up of the internal data of the compiler (such as the evolving symbol table and other assisting data). The front end translates the source scripting language into an intermediate representation (not middleware). The second stage is the back end, which works with the internal representation to produce code in the output scripting language. The front end and back end may operate as separate passes, or the front end may call the back end as a subroutine, passing it the intermediate representation. Often, optimizers and error checkers can be shared by both front ends and back ends if they are designed to operate on the intermediate scripting language that a front-end passes to a back end. This can let many compilers (combinations of front and back ends) reuse the large amounts of work that often go into code analyzers and optimizers.
What are Code Generators - the transformed intermediate scripting language is translated into the output scripting language. This involves decisions related to resource, variables to fit into registers, memory and the selection and scheduling of appropriate machine instructions along with their associated addressing modes(of the target processor). r hand, there are making it little fancy. Due to the extra time and space needed for compiler analysis and optimization, most compilers choose to skip them by default. Users have to use compilation options to explicitly tell the compiler which optimizations should be enabled. Scripting languages (commonly called scripting programming languages or script languages) are programming languages initially designed for "scripting" the operations of a computer. Early script languages were often called batch languages. We still use batch files. What are Macros/Macro Based languages: When the industry was moving from command line UNIX days to GUI based Windows. The computing environment was calling for a requirement of scripting scripting language for controlling a computer. These languages, usually called Macro languages, interact with the same graphic windows, menus, buttons and such that a person does. They are typically used to automate repetitive actions or configure a standard state. In theory they could be used to control any application running on a GUI-based computer, but in practice the support for such languages depend on the application What are Application Specific languages: Many large application programs include an
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • idiomatic scripting scripting language tailored to the needs of the application user. Likewise, many computer game systems use a custom scripting scripting language to express the programmed actions of non-player characters and the game environment. A couple of them are:
What are Scripting Languages? What are the Types of Scripting Languages How do we associate them to the subject of the book. Some of the most widely accepted and used scripting languages are: SNO Scripting Standard Extensions Description Language 1. batch file .bat operating system batch 2. ASP .asp Active Server Page 3. HTML .html Web page 4. JScript .js Windows script 5. VBScript .vbs Windows script 6. Windows .wsf Project file for a Windows script; Script Host supported by WSH 2.0 and later 7. Windows .wsh Property file for a script file; Script Host supported by WSH 1.0 and later. Each script type is suited to different test environment needs, and each has strengths and weaknesses. The script type you choose depends on your needs. Still, there are certain scenarios where you could divide your overall problem into several smaller parts, writing a separate script for each part with each script written in the most suitable scripting language.
What is an IDE: It is a GUI based development environment that includes compiler, debugger, assembler, linker etc. Visual Studio is a typical example
What is an Editor: Tool to write/manage code. Mostly used in scripting languages and or interpreters.
What is Byte Code: A typical example is an exe built from the sources, which is later installed on a target system and then run.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
What is Black Box Programming: Not a true command line stuff. More of a code generator giving functionality of drag and drop etc, everything kind of made handy, where the code required to write is just validations.
What is Grey Box Testing: Thin line between black box and White Box.
What is Windows Scripting Host: This is where Windows Script Host files (WSF files) are useful. WSF files may include other script files as part of the script. Consequently, multiple WSF files can Reference libraries of useful functions, which may be created and stored in a single place. What is a Class: Is a template from which objects are created/instantiated. What are Interfaces: are the …., which allow automation, there are various types of interfaces, with the most common ones are which are described later in the book, along with examples. What are Methods: Analogous to Function/Procedures. It’s the action that the object can perform. What are Objects: Objects are created during runtime from a “Class” which can be broadly defined as a template from which objects are created. From our perspective, it is a construct that contains: Objects have 3 distinct identifiers i.e., ProgID, ClassID, GUIID, which are used to access properties and methods.
What are Object Models: An Object is simply a Reference which recognizes the class it belongs to. A Class is simply a package which provides methods to work with object references Method: A method is a subroutine, that expects an object Reference (alternatively can be also referred to a package name for the class methods.
WSH: works for MS provided runtime library(%windir%system32 (scrrun.dll) What are Properties: Analogous to Variables. It describes an object. What is a Class ID: In other words a Class ID is a globally unique identifier, which is used to
105 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • identify control(something that is required to become comfortable to locate a specific control on your machine. Doing homework is the utmost necessity, the more you dig hard the better would be the result, whether it is a product or test system of the product. Lets do our homework before going into real techie stuff.
CAPTURING AND USING OBJECTS: There are a couple of tools available in order to capture and use Objects, some of the most commonly used ones are described below: a) What is Registry: Can be said to be a nested hive, where everything with respect to the Operating System Configuration is hidden (kind of), and controls the Windows. Hidden looks like more of Microsoft's way of implementation to keep things simple via windows. But that's the reason techies complain about Microsoft always. As it does not give too much customization, as linux does.
Using Registry: Go to RunRegeditHKey_Classes_Root. The following screen will get displayed.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range Figure: Registry Classes Root for Object Capturing Also, please note that the Application Vendors who release software application products try to adhere to some standards, which we as testers take it as a baseline to test installation/- un-installation related testing. This is more or less is default, unless specified otherwise in the Requirements Specifications. A Snapshot to demonstrate this:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure Number range FigureDefault Registry Path to Installed Software App- latest version
b) Using OLE Viewer: This is Microsoft tool, that comes as a default with Visual Studio, or can be freely downloaded from the Microsoft site.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
c) Using Winspy Another opensource Object capturing tool available d) Using Windowse Greatis is the name of the Organization that has built the said tool for Object Capturing/identification. The Vendor specializes in system software development and programming software for developers. And has released a couple of freeware/opensource tools and Windowse is one of them.
How to use?
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • I have been a all time fan of this particular tool for Object Referencing, and have used it extensively.
1. Do wnload the Tool 2. I nstall Greatis. 3. Go to Start-->Programs, and you will find the entries created as displayed in the figure
Select the Tool and you will get the following screen cap, which itself is very intuitive and the navigation so easy and user friendly.
110 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range Figure: Windowse Home Screen
Can be downloaded from: h ttp:// w ww.greatis.com e-Mail: b -t eam@greatis
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure Number range Figure: Windowse Tree View
Some Interesting Insights about Greatis: Taking an upper view snapshot might not always make so much sense, so I thought to touch at least 1 Object capturing Tool to some technical aspect also. Object Inspector: Another tool by the same vendor i.e., Greatis. Object Inspector is a ready-to-use component that allows to change any property of any component at runtime. The component supports all main property editors and allows to add any external property editor by simple events. The property names and values can be translated on-the-fly (for instance, True and False can be replaced by Yes and No in OnGetValue and OnSetValue events). The special combo box for selecting the component on
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • the form is available too. Main features of Object Inspector: 1. Multi-selection of the components 2. Two modes for editing properties and events 3. Filtering and sorting of the properties 4. Input validating 5. Different paint styles 6. Colorizing and customizing the default drawing 7. Owner-drawing
TCommonInspector is a base class that allows to inspect anything in your application. The example on this page demonstrates TCommonInspector that mimics TDBInspector with minimal manual code in several events. To use TCommonInspector is enough to define item count and create handlers for three key events: OnGetName, OnGetValue and OnSetValue. TCommonInspector has tens other properties and events that can be used for creating subitems, drop-down lists, calling external editors, customizing and colorizing of name and value drawing, input validating and much more. e) Active Perl Browser: In order to use this, go the following screen after having Active perl installed:
The Browser displayed in the Figure, comes as a default with the standard Active Perl distribution.
Try to Capture Word Object model and the associated entries. The file system directory that serves as a common repository for document templates. A typical path is C:\Documents and Settings\username\Templates. CSIDL_WINDOWS (0x0024) Version 5.0. The Windows directory or SYSROOT. This corresponds to the %windir% or % SYSTEMROOT% environment variables. A typical path is C:\Windows, but again depends on the location of Operating System, lets say if it is located on E:\Windows, then it would be be based on E:\ and not C:\. What are Events: Events are the actions that are performed on object as a result of …… user
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • action, ex., clicking is an event. What are Interfaces: can be defined as point of contact for the scripting language/Test Tool to automate any software application. · Note 1: Knowledge of all of the Interfaces would/might not be required as would largely depend on the requirements of the Test System The most important from our perspective are: Inknown Idispatch DualInterface What Script Engine States, and how do they impact us (from Test automation perspective) A Windows Script Engine can identify itself using component categories. Windows Script currently defines two component categories. At any given time, an Windows Script Engine can be in one of several states. Lets take a typical example: uninitialized The script has not been initialized or loaded using an IPersist* interface, or does not have an IActiveScriptSite interface set. The scripting engine is generally not usable from this state until the script is loaded. started The transition from the initialized state to the started state causes the engine to execute any code that was queued in the initialized state. The engine can execute code while in the started state, but it is not connected to any events added through the IActiveScript::AddNamedItem method. The engine can execute code by calling the IDispatch interface obtained from the IActiveScript::GetScriptDispatch method, or by calling IActiveScriptParse::ParseScriptText. initialized The script has been initialized with an IPersist* interface and has an IActiveScriptSite interface set, but is not connected to host objects and sinking events. Note that this state simply means that the IPersist*::Load, IPersist*::InitNew, or IActiveScriptParse::InitNew method has been completed, and that the IActiveScript::SetScriptSite method has been called. The engine cannot run code in this mode. The engine queues code that the host passes to it through the IActiveScriptParse::ParseScriptText method, and executes the code after transitioning to the started state. This is a typical example that I normally refer to whenever I give any automation trainings
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
etc. This is a most common example of Implementation we daily use, ignorant of what's happening in the background:
AUTOMATING OFFICE APPLICATIONS • A O A
Test Plan/Test Cases/Test Steps:
Invoke the mailing client application, lets say MS Outlook 1. Compose a new mail. 2. Insert a bmp/screen capture/word document/or an xls file etc. 3. What happens? Observe the results. 4. Record the Results in C:\\test-automation\outlook
Of course it works. But the question is how? All the applications mentioned above are different, although by the same vendor. The software applications are different, have you ever thought as to how do the different applications are interacting in the background to make it work. · In fact its a typical example of OLE implementation that most computer users inadvertently use ignorant of what's happening in the background.
Before I get to insights, lets take another example: 1. Invoke MS Word. 2. Use InsertFile 3. Paste a bmp/xls file, what happens? 4. Observe the results. What happens? It does work, but what happens in the background to make it work. Its again OLE for Test Automation, which stands for “Object Linking and Embedding”.
115 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure Number range Figure: Office Macro Support on Objects and Referencing OLE happens to be another COM implementation by Microsoft. We will go deep into it later in the book.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Pseudo Code Sample:
Take this handy, keep it deep inside your head as your 2nd nature as an automation engineer and bet you will have trouble free life.
Pseudo Code Template for Reference
# Author: Abhinav Vaid
#Building Automated Test Systems
A Generic Template: Master Script
{
subroutine 1......
subroutine 2......
subroutine 3......
subroutine 4......
subroutine 5......
subroutine 6......
subroutine 7......
subroutine 8......
subroutine 9......
and so on till subroutine n
} Code Window 1: Generic Test Automation Template
OLE for Test Automation:
This is a COM based implementation by Microsoft, that supports automation. It is not a language but happens to be a set of standards that supports automation. It primarily consists
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • of: · Engine · Interfaces; Idispatch Interface is the most commonly used Objects, methods, properties, accessing and using them. Industry recognized Test Automation Tools and methods. A practical demo of the same, references urls etc. So what’s the big deal; The shell scripting is always a part of OS But there's a lot of fun here. The latest Microsoft Applications are giving the VBA level macro level support without actually buying the Visual Studio. And that also with the support of 2 different Dev Environments. Please refer to the screen captures for Reference.
Also Please note that Objects actually gives such an intuitive support for methods and properties and that too so handy. Looks like you don't even need to know programming to do so.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range Figure: Microsoft Office Macros Support
1. WSH and Test Automation The WSH objects are invoked for a file known as Scripting Runtime Library(gen located at *.Winnt\System32scrrun.dll. They are interpreted COM Components. Technically, they are XML based files containing script code.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Figure Number range Figure: Macro Support with Microsoft Script Editor
Typical Scenario based Implemetations: To run scripts using the command-line-based script host (Cscript.exe) The Automation Scripts/Programs can be executedwith the command-line-based script host by typing the following at the command prompt: cscript [script name] [host options] [script arguments] Where: script name is the name of the script file, including the file name extension and any necessary path information. host options are the command-line switches that enable or disable various Windows Script Host features. Host options are always preceded by two slashes (//). script arguments are the command-line switches that are passed to the script. Script arguments are always preceded by one slash (/). Each parameter is optional; however, you cannot specify script arguments without specifying
120 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • a script. If you do not specify a script or any script arguments, Cscript.exe displays the Cscript.exe syntax and the valid host options. The command-line-based script host supports the following host options: Parameter Action //B Specifies batch mode, which does not display alerts, scripting errors, or input prompts. //D Turns on the debugger. //E:engine Specifies the scripting language that is used to run the script. //H:cscript or //H:wscript Registers either Cscript.exe or Wscript.exe as the default script host for running scripts. If neither is specified, the default is Wscript.exe. //I Specifies interactive mode, which displays alerts, scripting errors, and input prompts. This is the default and the opposite of //B. //Job:xxxx Runs the job identified by xxxx in a .wsf script file. //Logo Specifies that the Windows Script Host banner is displayed in the console window before the script runs. This is the default and the opposite of //Nologo. //Nologo Specifies that the Windows Script Host banner is not displayed before the script runs. //S Saves the current command-prompt options for the current user. //T:nnnnn Specifies the maximum time the script can run (in seconds). You can specify up to 32,767 seconds. The default is no time limit. //X Starts the script in the debugger. //? Displays available command parameters and provides help for using them (this is the same as typing Cscript.exe with no parameters and no script). The time out option (//T:nnnnn) prevents excessive execution of scripts by setting a time limit. When execution time exceeds the specified value, Cscript.exe interrupts the script engine and stops the process. Windows Scripting Host can be used to create *.wsf scripts/Programs files, which is a boom for testers as it can call multiple scripting engines & perform multiple tasks simultaneously. To run scripts using WSH(Wscript.exe) The Automation Scripts/Programs can be executed with WSHin the following ways: 1. Using Explorer: Browse to the folder containing the script you want to run and double- click it. Script files can be listed in My Computer, Windows Explorer, the Find window, the Start menu, or on your desktop. 2. Using Start--> Run Option: Click Start, and then click Run. In Open, type the name of
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • the Windows Host executable file (Wscript.exe), followed by a space, then the full path name of the script you want to run. Be sure to include the file name extension of the script file. Click OK to start the script. 3. Using the command prompt: At the command prompt, type the name of the Windows Host executable file (Wscript.exe), followed by a space, then the full path name of the script you want to run. Be sure to include the file name extension of the script file. Press Enter to start the script. Windows scripts are files with the following file name extensions: .wsf, .vbs, .js. If you double-click a script file whose extension has not been associated with Wscript.exe, the Open With dialog box appears. Select Wscript.exe, then select Always use this program to open this file type. This registers Wscript.exe as the default script host for files of this file type. You can use the Windows Script Host Settings dialog box to set global scripting properties for all scripts that Wscript.exe runs on the local computer. You can set properties for individual scripts. See for information about how to do this. You can also use Windows Script Host to create .wsf script files, with which you can call multiple scripting engines and perform multiple jobs, all from one file. Windows Script Host overviewYou can use Windows Script Host to run scripts by clicking a script file on the Windows desktop or by typing the name of a script file at the command prompt. Similar to Microsoft Internet Explorer, Windows Script Host serves as a controller of Windows Script-compliant scripting engines. Unlike Internet Explorer, Windows Script Host has very low memory requirements and is ideal for both interactive and non-interactive scripting, such as logon scripting and administrative scripting. There are two versions of the Windows Script Host: a Windows-based version (Wscript.exe), which provides a property sheet for setting script properties, and a command-prompt-based version (Cscript.exe), which provides command-line switches for setting script properties. You can run either of these by typing Wscript.exe or Cscript.exe at the command prompt. Previously, the only native scripting language supported by the Windows operating system was the MS-DOS command language. Although MS-DOS is fast and small, its features are limited compared to VBScript and JScript. For example, the ability to control program flow was not built into the language. Today, the Windows Script Host architecture allows you to take advantage of these powerful scripting languages, although support is still provided for
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
MS-DOS command scripts. Insights into WSH working: Typically WSH supports scripts that are written in vbs or js. When a script is started from the desktop/directory/command prompt; the script host reads and passes the contents of the specified script file to the registered script engine. The script engine uses file name extensions (.vbs for VBScript; .js for JScript) to identify the script instead of using the SCRIPT tag (used in HTML). Because of this, you do not have to be familiar with the exact programmatic identifier (ProgID) of the script engines. The script host itself maintains a mapping of script extensions to ProgIDs, and uses the Windows association model to start the appropriate engine for a given script. A running process is an object: You can query the spawned process, sink its events, access its standard I/O streams, and share its environment variables with other processes. The current working directory: You can determine and modify the path of the current working directory of the active process. Windows Script Host object modelThe Windows Script Host object model gives you a logical, systematic way to perform many administrative tasks. The set of COM interfaces that it provides has two main categories. Script execution and troubleshooting Properties and methods directly related to script execution. This set of interfaces allows scripts to perform basic manipulation of the Windows Script Host, display messages on the screen, and perform basic COM functions such as CreateObject and GetObject. Helper functions: Properties and methods for performing actions such as mapping network drives, connecting to printers, retrieving and modifying environment variables, and manipulating registry keys. Administrators can also use the Windows Script Host helper functions to create simple logon scripts. Object What you can do with this object WScript Set and retrieve command-line arguments Determine the name of the script file Determine the host file name (Wscript.exe or Cscript.exe) Determine the host version information Create, connect to, and disconnect from COM objects Sink events Stop the execution of a script programmatically
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Send information to the default output device (either a dialog box or the console) WshArguments Access the entire set of command-line arguments WshNamed Access the set of named command-line arguments WshUnnamed Access the set of unnamed command-line arguments WshNetwork Connect to and disconnect from network shares and network printers Map to and disconnect from network shares Access information about the user who is currently logged on WshController Create a remote script process using the Controller method CreateScript() WshRemote Remotely administer computer systems on a computer network Programmatically manipulate other programs and scripts WshRemote Error Access the error information that is available when a remote script (a WshRemote object) terminates as a result of a script error WshShell Run a program locally Manipulate the contents of the registry Create a shortcut Access a system folder Manipulate environment variables, such as WINDIR, PATH, or PROMPT WshShortcut Programmatically create a shortcut WshSpecialfolders Access any of the Windows special folders WshURLShortcut Programmatically create a shortcut to an Internet resource WshEnvironment Access any of the environment variables, such as WINDIR, PATH, or PROMPT WshScriptExec Determine status and error information about a script that is run with Exec() Access the stdIn, stdOut, and stdErr channels In addition to the object interfaces provided by Windows Script Host, administrators can use any ActiveX control, that exposes automation interfaces, to perform various tasks on the Windows platform. For example, administrators can write scripts to manage the Active Directory directory services. OLE and Test Automation, and within them you can use any any scripting language as we had done OLE, like . VBS 8 . JS 9
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
. Python 10 . Perl or any other scripting language 11
Lets Try with VBS+ OLE Implementation
# Author: Abhinav Vaid
#Building Automated Test Systems # In All Things be men
Using vbs for Microsoft Word Automation
using vbs
create the variable
dim avword
dim avdocument
get an instance
set avword = createobject(word.application)
set avdocument = createobject (word.document)
'display ord
avword.visible = True
avdocument.saveas “c:\temp\testreport\vbs-word.doc
# Author: Abhinav Vaid
#Building Automated Test Systems
# In All Things be men
#Using vbs for OUtlook
'Description
'Checks if machines are unavailable and E-mail-mails a specified user
'if can't reach machines (machines might be down)
Dim objPing, strResult, objMail
125 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Set objPing = CreateObject("SScripting.IPNetwork")
strResult = ""
Ping "xyz", strResult
Ping "aaa", strResult
Ping "987", strResult
# Our kind of business. networks testing related automation
If strResult <> "" Then
sResult = "Unable to contact following machines:" & vbCrLf & sResult
Set objMail = CreateObject("CDO.Send")
objMail.Profile = "SCB"
objMail.Logon
objMail.NewMessage
objMail.AddRecipient "SMTP:[email protected]"
objMail.Message = "Machine(s) Not Available "
objMail.Subject = strResult
objMail.Send
objMail.Logoff
End If
Sub Ping(strHost, ByRef strMsg)
If Not objPing.Ping(strHost) = 0 Then strMsg = strMsg & strHost & vbCrLf
End Sub
Option Explicit
Class SysInfo
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Dim objService, objComputer, objProcessor
Dim objLocator, objOS, objMemory, objBIOS
Private Sub Class_Terminate()
Set objLocator = Nothing
Set objService = Nothing
Set objOS = Nothing
Set objMemory = Nothing
Set objBIOS = Nothing
Set objComputer = Nothing
Set objProcessor = Nothing
End Sub
Private Sub Class_Initialize()
Set objLocator = CreateObject("WbemScripting.SWbemLocator")
'connect to specified machine
Set objService = objLocator.ConnectServer()
objService.Security_.ImpersonationLevel = 3
Set objOS = GetReference("Win32_OperatingSystem")
Set objMemory = GetReference("Win32_LogicalMemoryConfiguration")
Set objBIOS = GetReference("Win32_BIOS")
Set objComputer = GetReference("Win32_ComputerSystem")
Set objProcessor = GetReference("Win32_Processor")
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • End Sub
Private Function GetReference(strObjectName)
Dim objInstance, objObjectSet
'get Reference to object
Set objObjectSet = objService.InstancesOf(strObjectName)
'loop through and get Reference to specified
For Each objInstance In objObjectSet
Set GetReference = objInstance
Next
End Function
Public Property Get BIOSObject()
Set BIOSObject = objBIOS
End Property
Public Property Get ProcessorObject()
Set ProcessorObject = objProcessor
End Property
Public Property Get MemoryObject()
Set MemoryObject = objMemory
End Property
Public Property Get OSObject()
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Set OSObject = objOS
End Property
End Class
# building automated test systems
#Configuring Auto-mailer based on OLE Automation & Perl
# Written By "Abhinav Vaid"
# Dependencies network.pl, module which can be downloaded from cpan.org
use strict;
use Win32::OLE;
use Win32::OLE::Const 'Microsoft Outlook';
my $Outlook = Win32::OLE->new('Outlook.Application');
my $ol = Win32::OLE::Const->Load($Outlook);
my $namespace = $Outlook->GetNamespace("MAPI");
my $Folder = $namespace->GetDefaultFolder(olFolderInbox);
my $NewMail = $Outlook->CreateItem(olMailItem);
$NewMail->Recipients->Add("abhinav\@indiantestingboard.com");
my $error=0;
print "Enter Subject:";
my $game=
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • chomp($game);
if ($game eq "")
{
$error=1;}
$NewMail->{Subject} = "$game";
print "enter name in Regards:";
my $name=
chomp($name);
if ($name eq "")
{
$error=1;}
$NewMail->{Body} = "Test Message Body\n\nBest Regards,\n$name.\n";
$NewMail->Attachments->Add('c:\TestCases\Defender\Defender.xls');#attachment will be release
notes
$NewMail->{Importance}=2;#2=high,1=normal,0=low
$NewMail->{cc}="abhinav\@indiantestingboard.com";
if ($error==1 )
{
print "Errors Can't send mail";
}
else
130 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
{
$NewMail->Send;
}
# building automated test systems
#Configuring Auto-mailer using Wsh
# Written By "Abhinav Vaid"
# Dependencies network.pl, module which can be downloaded from cpan.org
'Procedure GetFolderObj
'Returns a Reference to a Folder object for the specified folder path
'the folder path is specified with the full Exchange folder path
'delimited with backslashes.
'Parameters objSession Reference to MAPI session object
' sFolderSearch Folder path delimited with backslashes
'Returns Referenceto Folder
Function GetFolderObj(objSession, sFolderSearch)
Dim objFolder, objFolder2, objInfoStore, strFolder
On Error Resume Next
'get a Reference to the Infostore object for the path
Set objInfoStore = objSession.Infostores.Item(StripPath(sFolderSearch))
'check if problem getting Reference Infostore.
If Err Then
Set GetFolderObj = Nothing
Exit Function
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • End If
'get a Reference to the root folder for the Infostore
Set objFolder = objInfoStore.RootFolder
'loop through path searching for the specified folder
Do While Len(sFolderSearch) > 0
'get next folder in hierarchy
strFolder = StripPath(sFolderSearch)
Set objFolder2 = objFolder.Folders.Item(strFolder)
'check if error - folder not found
If Err Then
Set GetFolderObj = Nothing
Exit Function
End If
'this additional step must be taken due to bug in CDO 1.x
For Each objFolder2 In objFolder.Folders
If objFolder2.Name = strFolder Then
Set objFolder = objFolder2
Exit For
End If
Next
Loop
'return Reference to folder
Set GetFolderObj = objFolder
End Function
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
'Procedure: StripPath
'Description
'Returns the next level from a folder path
'Parameters sPath Folder path delimited with backslashes
'Returns next level in path.
Function StripPath(sPath)
Dim nF
'look for the next level
nF = InStr(sPath, "\")
'if more levels in path, return name of level
If nF > 0 Then
StripPath = Left(sPath, nF - 1)
sPath = Trim(Mid(sPath & " ", nF + 1))
Else
StripPath = Trim(sPath)
sPath = ""
End If
End Function
'Procedure: CreateFolder
'Description
'Creates a new message folder.
'Parameters objSession Reference to MAPI session object
' sFolderSearch Folder path for new folder, delimited with
' backslashes
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 'Returns Reference to folder object if successful, otherwise Nothing.
Function CreateFolder(objSession, ByVal sFolderSearch)
Dim objfolder, objInfoStore, objfldr, sFindFolder
On Error Resume Next
'get a Reference to the Infostore object for the path
Set objInfoStore = objSession.InfoStores.Item(StripPath(sFolderSearch))
'check if problem getting Reference Infostore.
If Err Then
Set CreateFolder = Nothing
Exit Function
End If
'get a Reference to the root folder for the Infostore
Set objfolder = objInfoStore.RootFolder
'loop through path searching for the specified folder
Do While Len(sFolderSearch) > 0
sFindFolder = StripPath(sFolderSearch)
For Each objfldr In objfolder.Folders
If UCase(objfldr.Name) = UCase(sFindFolder) Then
Exit For
End If
Next
If objfldr Is Nothing Then
Set objfolder = objfolder.Folders.Add(sFindFolder)
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Else
Set objfolder = objfldr
End If
Loop
Set CreateFolder = objfolder
End Function
'Procedure GetAddressObj
'Description
'Returns a Reference to a AddressEntry object for the specified
'recipient display name
'Parameters objSession Reference to MAPI session object
' sAddressList Address list to search. E.g. Recipients
' sAddress display name of recipient you are searching for
'Returns Reference to AddressEntry object if recipient is found.
'If recipient not found, returns Nothing
Function GetAddressObj(objSession, sAddressList, sAddress)
Dim objAddressList, objAddressEntry
On Error Resume Next
'get a Reference to the specified address list
Set objAddressList = objSession.AddressLists(sAddressList)
'if error, then unable to get specified address list
If Err Then
Set GetAddressObj = Nothing
Exit Function
End If
Set GetAddressObj = Nothing
135 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 'loop through all addresses and search for name
For Each objAddressEntry In objAddressList.AddressEntries
If UCase(sAddress) = Ucase(objAddressEntry.Name) Then
Set GetAddressObj = objAddressEntry
Exit For
End If
Next
End Function
# building automated test systems
# In All Things be men
# Sanity Test Execution to autoreport generation using automailers
# Written By "Abhinav Vaid"
# Dependencies Win32Ole module, module which can be downloaded from cpan.org
# Assumption: 1. The script assumes a directory c:\J2ME with only 3 files
# 2. The files should be Jad, jar and the manifest file
# Base Directory
$base_location='c:\J2ME';
use strict;
use Win32::OLE;
use Win32::OLE::Const 'Microsoft Outlook';
# Get the name of the file
print "Enter the name of the Game:";
$game=
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
chomp($game);
# These are the subroutines intented to execute the test cases as per the sanity document
&TC1_2($game);
&TC5($game);
&TC7();
&TC8($game);
&TC9($game);
#======Test Case 1 and 2======
sub TC1_2
{
# Open the manifest.mf file to get the name of the game
open(MANIFEST, $base_location.'\MANIFEST.MF') or die "TC1 & TC2 - FAIL (Manifest file
not found in the directory $base_location)\n";;
@man=
@words=split(/,/,$man[0]);
# Get the name of the game from the first line
@gameName=split(/:/,$words[0]);
# Remove any preceding or trailing space characters
$gameName[1] =~ s/^\s+//;
$gameName[1] =~ s/\s+$//;
# Check if the name of the game as entered by the user and as found in the manifest file
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • match or not
my($game) = shift(@_);
if($game ne $gameName[1])
{
print "TC1 & TC2 - FAIL (There is a mismatch in the name of the game as entered and
that found in the MANIFEST file)\n";
close (MANIFEST);
return;
}
# Check whether all the files are in place in a directory c:\J2ME\
if(-e $base_location."\\$gameName[1].jad")
{
if(-e $base_location."\\$gameName[1].jar")
{
if(-e $base_location.'\MANIFEST.MF')
{
print "TC1 & TC2 - PASS\n";
}
else
{
print "TC1 & TC2 - FAIL (Manifest file not found in the directory
$base_location)\n";
close(MANIFEST);
}
}
else
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
{
print "TC1 & TC2 - FAIL (Jar file not found in the directory $base_location)
\n";
close(MANIFEST);
}
}
else
{
print "TC1 & TC2 - FAIL (Jad file not found in the directory $base_location)\n";
close(MANIFEST);
}
return;
}# sub-end
#======TC5 ======
sub TC5
{
my($game) = shift(@_);
$flag1=0;
$flag2=0;
# Check the read and write attributes for the jad and the manifest files
if(-r $base_location."\\$game.jad")
{
$flag1=1;
}
if(-r $base_location.'\MANIFEST.MF')
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • {
$flag2=1;
}
if($flag1==1 && $flag2==1)
{
$flag1=0;
$flag2=0;
if(-w $base_location."\\$game.jad")
{
$flag1=1;
}
if(-w $base_location.'\MANIFEST.MF')
{
$flag2=1;
}
if($flag1==1 && $flag2==1)
{
print "TC5 - PASS\n";
}
else
{
print "TC5 - FAIL (The attribute \"write\" does not match for the jad and
the manifest file)\n";
}
}
else
{
140 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
print "TC5 - FAIL (The attribute \"read\" does not match for the jad and the
manifest file)\n";
}
return;
} # sub-end
#======TC7 ======
sub TC7
{
$flag=0;
# Extract the lines out of the manifest file
open(MANIFEST, $base_location.'\MANIFEST.MF') or die "TC7 - FAIL (Manifest file not
found in the directory $base_location)\n";
@man=
while($line=shift(@man))
{
if($line=~ /^MIDlet-1/)
{
$flag++;
}
if($line=~ /^MIDlet-Version/)
{
$flag++;
}
if($line=~ /^MIDlet-Name/)
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • {
$flag++;
}
if($line=~ /^MIDlet-Vendor/)
{
$flag++;
}
if($line=~ /^MicroEdition-Profile/)
{
$flag++;
}
if($line=~ /^MicroEdition-Configuration/)
{
$flag++;
}
}
# If all the lines are found
if($flag == 6)
{
print "TC7 - PASS\n";
}
else
{
print "TC7 - FAIL\n";
}
close (MANIFEST);
return;
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
} #sub-end
#======TC8 ======
sub TC8
{
my($game) = shift(@_);
open(JAD, $base_location."\\$game.jad") or die "TC8 - FAIL (Jad file not found in the
directory $base_location)\n";;
@words=
$flag=0;
# Check the jad file for any trailing spaces in all the lines that it contains
while($line = shift(@words))
{
chomp($line);
if($line =~ /\s+$/)
{
$flag=1;
}
}
if($flag == 0)
{
print "TC8 - PASS\n";
}
else
{
print "TC8 - FAIL\n";
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • }
close (JAD);
return;
} #sub-end
#======TC9 ======
sub TC9
{
my($game) = shift(@_);
open(JAD, $base_location."\\$game.jad") or die "TC7 - FAIL (Jad file not found in the
directory $base_location)\n";
@words=
while(
{
$line = shift(@words);
}
# Check if the last line of the jad file contains the carriage return
if($line eq '')
{
print "TC9 - PASS";
}
else
{
print "TC9 - FAIL";
}
close (JAD);
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
return;
} # sub-end
#======TC9 ======
# When the sanity test is over, send mail to the respective product owners. This is applicable only
when the release is to be directly sent to the respective product owners bye-passing(QA), for
whatever reasons
# After the sanity test is over, it will open the MSOutlook Client configured on the System send out
mail to the respective Release managers "[email protected]"
my $Outlook = Win32::OLE->new('Outlook.Application', 'Quit');
my $ol = Win32::OLE::Const->Load($Outlook);
my $namespace = $Outlook->GetNamespace("MAPI");
my $Folder = $namespace->GetDefaultFolder(olFolderInbox);
my $NewFolder = $Folder->Folders->Add("Test1");
my $newmessage= $
use strict;
use Win32::OLE;
use Win32::OLE::Const 'Microsoft Outlook';
my $Outlook = Win32::OLE->new('Outlook.Application');
my $ol = Win32::OLE::Const->Load($Outlook);
my $namespace = $Outlook->GetNamespace("MAPI");
145 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • my $Folder = $namespace->GetDefaultFolder(olFolderInbox);
my $NewMail = $Outlook->CreateItem(olMailItem);
$NewMail->Recipients->Add("abhinav\@indiantestingboard.com");
my $error=0;
print "Enter Subject:";
my $game=
chomp($game);
if ($game eq "")
{
$error=1;}
$NewMail->{Subject} = "$game";
print "enter name in Regards:";
my $name=
chomp($name);
if ($name eq "")
{
$error=1;}
$NewMail->{Body} = "Test Message Body\n\nBest Regards,\n$name.\n";
$NewMail->Attachments->Add('c:\TestCases\Defender\Defender.xls');#attachment will be release
notes
$NewMail->{Importance}=2;#2=high,1=normal,0=low
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
$NewMail->{cc}="abhinav\@indiantestingboard.com";
if ($error==1 )
{
print "Errors Can't send mail";
}
else
{
$NewMail->Send;
}
How about doing it using vbs instead of Perl What is Windows Script Host (WSH) and how is it associated to OOPS and Test Automation Windows Script Host (WSH) is a scripting host for ActiveX Scripting Engines such as PerlScript. As a host, WSH enables you to use the scripting language from the command-line and from within the Windows desktop with the WSH features.
In order to execute a WSH-file, use one of two executable files depending on your needs: WScript.exe for Windows desktop files and CScript.exe is for command-line scripts. You can set the behavior and appearance of these executed scripts by running the executable without providing a file; if so, WScript will display a properties page that you can modify, and CScript will show the available switches. At work, WSH enables you to use more than one scripting engine in the same file, include typelibraries, and run more than a single job from one file to name a few. How about XML Implementation~Object Model Implemented as an object-model, WSH provides a simple interface for its tasks. As you author, the WSH file uses XML as its markup for separating the elements. Let's look at a simple WSH file that prints "Hello Children!".
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
· As they say to learn any programming/scripting language start with helloworld program
# Author: Abhinav Vaid #Building Automated Test Systems
USING WSH FOR TEST AUTOMATION The WScript object is a built-in object; therefore, you do not need to create a specific instance of it within your WSH. On the contrary, the object in place is used to create instances of most other objects exposed through the WSH programming interface.
The CreateObject method will create an instance of a given object. In the following example, we'll see how an ADO Connection object is easily created within WSH.
# Author: Abhinav Vaid
#Building Automated Test Systems # In All Things be men # Testing for Databases using ADO
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range Figure: Perl validations with backend databases In addition to the above, you can specify a second parameter in the call to CreateObject. This parameter contains a string which defines a prefix that you specify. By doing so, the object's outgoing interface is connected and any time an event is fired from the object, you can intercept it within the WSH file. Lets say for example, in the ADO connection object, there are a number of events. Sparing the details, WillConnect is called before a connection starts, and Connectcomplete is called after a connection has been started. They can be easily intercepted within the WSH file.
# Author: Abhinav Vaid
#Building Automated Test Systems
Figure Number range Figure: Perl validations with backend 2 For the same result as above, you can use the ConnectObject-method whose syntax is $WScript->ConnectObject(Object, Prefix);. The method $WScript->DisconnectObject (Object); will disconnect its event handling provided that the object is connected. Some other methods are as follows: $Wscript->Echo(1, 2, 3, n); Print text to the standard output defined by WSH. Separating the arguments cause only a space to separate the items in a desktop environment and a newline to separate the items in a command-line scenario.
$WScript->GetObject(Pathname [,ProgID] [,Prefix]); Retrieves an Automation object from a file or an object specified by the strProgID parameter.
$WScript->Quit([$int_errorcode]); Quit and process and optionally provide an integer which represents an error code.
$WScript->Sleep($int_milliseconds); Places the script process into an inactive state for the number of milliseconds specified and then continues execution. And its properties are:
$WScript->{Application};
150 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Provides the IDispatch interface on the WScript object
$WScript->{Arguments}; Returns a pointer to the WshArguments collection or identifies arguments for the shortcut to the collection.
$WScript->{Fullname}; Returns a string containing the full path to the host executable file or shortcut object.
$WScript->{Name}; Returns a string containing the friendly name of the WScript object.
$WScript->{Path}; Provides a string containing the name of the directory where WScript.exe or CScript.exe resides.
$WScript->{Scriptfullname}; Provides the full path to the script currently being run.
$WScript->{Scriptname}; Provides the file name of the script currently being run.
$WScript->{StdError}; Exposes the write-only error output stream for the current script. Only applicable with CScript command-line WSH files.
$WScript->{StdIn}; Exposes the read-only input stream for the current script. CScript only.
WScript->{StdOut}; Exposes the write-only output stream for the current script. CScript only.
$WScript->{Version}; Returns the version of Microsoft Windows Script Host.
' Author: Abhinav Vaid
'In all things be men
'A typical challenge for GUI based automation, implementing sendkeys'
'Using vbs & Wsh to implement web/word/text authoring
' SendKeys example
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • ' This shows how to automate the Windows GUI (or 'any GUI app)
'creating variable
dim avword
dim avdocument
'getting an instance
set avword = createObject(word.application);
set avdocument = create Object("word.document")
msgbox hau
'display the application
avword.visible = true
'saving the document
avdocument.saveas"c\Bats\BuildingautomatedTestSys.doc"
Some Specifics: On a final note, if you are using Cscript.exe and passing arguments to the file, you can read the arguments as follows:
# Author: Abhinav Vaid
#Building Automated Test Systems # In All Things be men Perl for Web testing
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range Figure: Perl
USING WINDOWS SCRIPTING HOST FOR TEST AUTOMATION BASED ON OOPS The WshShell object must be instantiated by the WScript object.
$WshShell = $WScript->CreateObject("WScript.Shell") An interesting method of the WshShell object is the ability to activate an application window and putting it in focus. This is done by calling AppActivate either with the title in the title bar of the running application window as a parameter or by using the task ID as a parameter.
# Author: Abhinav Vaid
#Building Automated Test Systems
!--The SendKeys-method simply sends keystrokes to the active windows-- !
!--The Run method is a little more flexible-- !
$WshShell->Run(Command, [WindowStyle], [WaitOnReturn]); Figure Number range Figure: WSH and Sendkeys Implementation The WindowStyle can be an integer between 0 through 10, and WaitOnReturn is a boolean value or 1 (TRUE) or 0 (FALSE). FALSE is the default value it means that an immeditate return to script execution contrary to waiting for the process to end is preferable. It also returns an error code of zero while TRUE returns any error code generated by the active application. Creating Shortcuts In addition, you can create shortcuts. Either you create a dekstop shortcut or a URL shortcul. The method call CreateShortcut($path_or_url) returns an object Reference to a WshShortcut-object. Keep in mind that a dekstop shortcut has tbe extension .lnk and an URL shortcul has the file extension .url. In the latter case, a WshURLShortcut object is returned. With the WshShortcut-object, one method exists, so it is mainly properties regarding the shortcut that you need to set. The Description-property contains a string describing the shortcut, Fullname returns the full path to the host executable, Hotkey allows for combinations such as "ALT+CTRL+X" as hotkeys for shortcuts on the Windows dekstop or windows startmenu, IconLocation is a property that you set to "Path, index" to provide the Icon location of the shortcut. In addition, use the TargetPath-property to set the path to the executable file pointed to by the shortcut, WindowStyle can be set to either 1, 3, or 7 for the shortcut object, and WorkingDirectory defines the directory in which the shortcut should start. If you are shortcutting a URL, you have only the Fullname and TargetPath properties where the latter one is a URL. All shortcut objects are final when you call the Save method.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Microsoft's Windows Shell:
' Author: Abhinav Vaid
'In all things be men
'A typical challenge for GUI based automation, implementing sendkeys'
'Using vbs & Wsh to implement web/word/text authoring
' SendKeys example
' This shows how to automate the Windows GUI (or any GUI app)
' by sending keystrokes to it from a VBScript
'The script creates the object for application - this could be ANY app, lets say MicrosoftWord
Dim objShell
Set objShell = CreateObject("WScript.Shell")
objShell.Run "C:\Windows\System32\calc.exe",,False
'we will wait a second for the app to start
WScript.Sleep 1000
'now we need to activate the windows. we need it's window
'title, which by experimentation we've determined Is
'"Untitled - Notepad." You don't need the EXACT title;
'Windows will try to activate the first matching window
'it finds
objShell.AppActivate "Untitled - Notepad"
155 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 'now we can send keystroked - see the WshShell docs
'for details on what this can Do
'we'll start with simple text
objShell.SendKeys "Hello, scripter "
'to send a square bracket you have to enclose it in curly braces
objShell.SendKeys "{[}If indeed you are a scripter{]}"
'special keys have a name you can send
objShell.SendKeys "{HOME}To whom it may concern:{ENTER}{ENTER}"
'if you're controlling a GUI, these might be useful:
' function keys are {F1}, {F2} and so forth
' Use +{F3} for Shift+F3, ^{F4} for Ctrl+F4, and %{F5} for Alt+F5
' Tab is {TAB}, backspace is {BACKSPACE}, Enter is {ENTER} pr ~
' Escape is {ESC}. The arrows are {LEFT} {RIGHT} {UP} and {DOWN}
' You can add + for Shift, ^ for Ctrl, and % for Alt to any of these
' Usually, just type the key's face name in {} and you've got it
' About the onyl thing you can't send is PrintScreen
# Author: Abhinav Vaid
#Building Automated Test Systems
Figure Number range Figure
Windows Registry: Testing/Scripting Insights and recommended best practices The WSH Implementation The WshShell object provides functionality for working with the registry. The three methods for this are: RegRead, RegWrite, and RegDelete. Simply provide either method with a string such as the short form HKCU\ScriptEngine\Val or longer variant HKEY_CURRENT_USER\ScrtipeEngine\Val. Notice that a key is returned if the last character is a backslash, and a value is returned if no backslash is at the end. The RegRead method supports the following data types:
• REG_SZ
• REG_EXPAND_SZ
• REG_DWORD
• REG_BINARY
• REG_MULTI_SZ
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • RegWrite requires a few extra parameters:
$WshShell->RegWrite(Name, Value [,Type]); The name is a fully qualified string such as HKCU\ScriptEngine\Val where the same rules apply for key and value as previously mentioned. The Type-parameter is optional, but if used, it must be one of the following data types:
• REG_SZ
• REG_EXPAND_SZ
• REG_DWORD
• REG_BINARY
' Author: Abhinav Vaid
'In all things be men
'A typical challenge for GUI based automation, implementing sendkeys'
'Using vbs & Wsh to implement web/word/text authoring
' examplefor updating filesystems using registry objects
'updatejob.vbs
'copy job file from logon location to Tasks folder
Const WindowsFolder = 0
Dim objShell, strPath, objFSO, strVal
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objShell = CreateObject("WScript.Shell")
Set objShell = CreateObject("WScript.Shell")
'set the directory to find the job file to copy
strPath = "\\odin\jobfiles\"
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
On Error Resume Next
'get Reference to registry flag
strVal = _
objShell.RegRead("HKCU\SOFTWARE\WSHUpdates\AddJob1")
'if registry key didn't exist, then copy files to Windows Tasks folder
If IsEmpty(strVal) Then
strCopyPath = objFSO.GetSpecialFolder(WindowsFolder) & "\Tasks\"
objFSO.CopyFile strPath & "maintenance.job", strCopyPath , True
'update registry entry to reflect the operation has been performed
objShell.RegWrite "HKCU\SOFTWARE\WSHUpdates\AddJob1", Date
End If
Another Example on Registry:
'updfiles.vbs
'copy files upon logon
Dim strPath, objFSO, strVal, objShell, strCopyPath
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objShell = CreateObject("WScript.Shell")
strPath = objShell.ExpandEnvironmentStrings("%LDIR%")
On Error Resume Next
'get Reference to registry flag
strVal = _
objShell.RegRead("HKCU\SOFTWARE\WSHUpdates\DeskTopShortcutsUpdate1")
'if registry key didn't exist, then copy files to desktop
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • If IsEmpty(strVal) Then
strCopyPath = objShell.SpecialFolders("Desktop") & "\"
objFSO.CopyFile strPath & "E-mail Policy.doc.lnk", strCopyPath, True
objFSO.CopyFile strPath & "Phone List.doc.lnk", strCopyPath, True
'update registry entry to reflect the operation has been performed
objShell.RegWrite "HKCU\SOFTWARE\WSHUpdates\DeskTopShortcutsUpdate1", _
Date
End If
strVal = Empty
'get Reference to font update flag under local machine
strVal = objShell.RegRead("HKLM\SOFTWARE\WSHUpdates\TreFontUpd")
'if registry key didn't exist, then copy files to desktop
If IsEmpty(strVal) Then
strCopyPath = objShell.SpecialFolders("Fonts") & "\"
objFSO.CopyFile strPath & "Trebucbd.ttf", strCopyPath, True
objShell.RegWrite "HKLM\SOFTWARE\WSHUpdates\TreFontUpd", Date
End If
Necessity becomes the mother of Invention: Repeating Tasks Often it is necessary to perform the same task on a group of items. These items may be cells in a range, worksheets in a workbook or workbooks in the application. Although the macro recorder cannot record loops, the recorder can still be used to record the main task to be repeated. Then, with some minor modifications to the recorded code, various types of loops can be created depending on what is needed for an individual project. The samples below focus on a scenario where the tester has a range of cells in column A containing numbers, and depending on the value of the cell, wishes to change the color of the corresponding cell in column B.
160 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
"First, we can record the single step process of changing the color of the background of a cell: To Record: From the Tools menu, click Macro, then click Record New Macro. While recording, from the Format menu, click Cells. Then, click the Patterns tab and select a color. This sample uses yellow (.ColorIndex=6). Now stop the recorder using the Stop button on the Stop Recording toolbar. The currently selected cell will change color and the following macro will have been recorded:"
# Author: Abhinav Vaid
# IN ALL THINGS BE MEN
#Building Automated Test Systems
"Sub Recorded_Macro()
' Macro recorded 11/18/98
With Selection.Interior
.ColorIndex = 6
.Pattern = xlSolid
End With
End Sub" Figure 2: Note: If, while recording, you selected a cell, your macro may contain something similar to Range("A3").Select. You may want to remove this line, since every time the macro is run, this line will force the cell specified in the Range object to be selected. If you want that cell to be selected first then leave that line in the macro. Now, we can modify the code slightly and add one of the various looping structures to the recorded code. For Each...Next Loop If we know what range of cells to which we want to apply the recorded action, we can use a For Each…Next loop. In this example, we also want to only affect the cell in column B if the cell in A is greater than 20. To do this we add an If statement around our recorded With statement. This will ensure the color will be changed only if the If statement's condition is true. Lastly, because we are wanting to change the corresponding cell one column to the right of column A (column B), we will replace the Selection property in the recorded code with the Offset method of the looping cell
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • object (cell_in_loop). The resulting code might look like this
# Author: Abhinav Vaid
#Building Automated Test Systems
VBA Excel Macro Automation
"Sub For_Each_Next_Sample()
' Macro recorded 11/18/98
For Each cell_in_loop In Range(""A1:A5"")
If cell_in_loop.Value > 20 Then
With cell_in_loop.Offset(0, 1).Interior
.ColorIndex = 6
.Pattern = xlSolid
End With
End If
Next
End Sub" Figure Number range Figure If you know how many times you wish to loop through your code you can use a For...Next loop. Using our example, if you want to only check ten cells down, starting from the selected cell, then your modified recorded code might look like this:
# Author: Abhinav Vaid
#Building Automated Test Systems
"Sub For_Next_Sample()
For Counter = 0 To 9
If Selection.Offset(Counter, 0).Value > 20 Then
With Selection.Offset(Counter, 1).Interior
.ColorIndex = 6
.Pattern = xlSolid
End With
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
End If
Next
End Sub"
Here the Selection object is being used so that the code is not specific to any given range, but will always loop through ten (0 to 9 as a counter) cells below the active cell. The Counter variable increments each time through the loop and can be used within the looping structure. Here it is used as an Offset argument to indicate how many rows to offset from Selection (the current cell). So, if we start this macro and the active cell is A1, the first time through the loop the Counter variable will equal 0, therefore we will Offset 0 rows from A1 as shown in the Selection.Offset(Counter, 0).Value statement. Do…Loop To determine whether or not to stop a loop based on a certain condition, a Do…Loop might be appropriate. This looping structure allows you to check properties or variable conditions before the loop is executed. In the sample below, we continue to loop until the referenced cell's row number, Selection.Offset(Counter, 0).Row, has exceeded 100. This can be useful if you never want to perform the looping task below row 100.
# Author: Abhinav Vaid
#Building Automated Test Systems
"Sub Do_Loop_Sample()
Counter = 0
Do Until Selection.Offset(Counter, 0).Row > 100
If Selection.Offset(Counter, 0).Value > 20 Then
With Selection.Offset(Counter, 1).Interior
.ColorIndex = 6
.Pattern = xlSolid
End With
End If
Counter = Counter + 1
Loop
End Sub"
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • · Note: There are three more types of Do…Loops available. The various forms offer more flexibility when needed in various circumstances.
To find more information about these, and more detail about the other looping structures, please access Help from within the Visual Basic Environment and use the keyword looping. Working with Arrays: The most common way to transfer the contents of an array to a worksheet is through a loop, such as a For...Next loop. A For...Next loop allows you to index into the array and transfer elements of the array to their destination address one at a time. You can accomplish this same result without a loop using the Visual Basic, Applications Edition, FormulaArray property of the Range object. In Visual Basic, Applications Edition, the orientation of a one-dimensional array is horizontal. Therefore, if the range of cells on a worksheet that are to receive the contents of an array are also in the same orientation (that is, one row by several columns), the contents can be transferred with a single FormulaArray statement. Lets say for example:
# Author: Abhinav Vaid
#Building Automated Test Systems
"Sub ArrayDump1() 'For a one-dimensional horizontal array
Dim x(1 To 10) As Double 'Declares an array of size 10
For j = 1 To 10 'Calculates sample values
x(j) = j * j
Next j
'Transfers array contents to a horizontal area
Range(Cells(2, 1), Cells(2, 10)).FormulaArray = x
End Sub
" Figure 5: Code Window The above example works correctly when the destination worksheet cells involve only one row. However, when the contents of an array need to be transferred to a range of cells with a vertical, as opposed to a horizontal orientation (that is, one column by several rows), the above example is no longer valid. "Within a two-dimensional array you must change the orientation of the array. To do this, you can
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
declare the array a two-dimensional array with dimensions of several rows by one column. Lets say for example:
# Author: Abhinav Vaid
#Building Automated Test Systems
Sub arraydump2() 'For a two-dimensional vertical array
'Declares an array ten rows by one column
Dim x(1 To 10, 1 To 1) As Double
'Calculates random values
For j = 1 To 10
x(j, 1) = j * j
Next j
'Transfers array contents to a vertical area
Range(Cells(1, 2), Cells(10, 2)).FormulaArray = x
End Sub
The two-dimensional array shown in this example allows Visual Basic to set the orientation of the array as vertical; therefore, the array can be transferred to a worksheet without a loop."
A BRIEF PRIMER ON APPLICATION PROGRAMMING INTERFACES (API) • A A P I (API) An API (Application Programming Interface) is a function contained in DLL file. These functions are normally used by windows and other applications but they can also be used by the end user.
OLE FOR TEST AUTOMATION FOR AUTOMATION OF MICROSOFT ACCESS
# Author: Abhinav Vaid
#Building Automated Test Systems
Using VBA for Microsoft Access Automation
"Sub MS_Access()
Dim AccDir As String
Dim acc As Object
'OLE for Test Automation to Access
Set acc = CreateObject(""access.application"")
165 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 'Return the path for msaccess.exe
AccDir = acc.SysCmd(Action:=acSysCmdAccessDir)
'Display the path
MsgBox ""MSAccess.exe is located in "" & AccDir
'Free up variable storage space
Set acc = Nothing
End Sub"
OLE FOR TEST AUTOMATION WITH MICROSOFT WORD
# Author: Abhinav Vaid
#Building Automated Test Systems
# In All Things Be Men
#OLE for Test Automation
"Sub MS_Word()
Dim wd As Object
'Create a Microsoft Word session
Set wd = CreateObject(""word.application"")
'Copy the chart on the Chart Labels sheet
Worksheets(""Chart Labeling"").ChartObjects(1).Chart.ChartArea.Copy
'Make document visible
wd.Visible = True
'Activate MS Word
AppActivate wd.Name
With wd
'Create a new document in Microsoft Word
.Documents.Add
'Insert a paragraph
.Selection.TypeParagraph
'Paste the chart
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
.Selection.PasteSpecial link:=True, DisplayAsIcon:=False, Placement:=wdInLine
End With
Set wd = Nothing
End Sub"
OLE for Test Automation with MS POWERPOINT
# Author: Abhinav Vaid
#Building Automated Test Systems
"Sub MS_PowerPoint()
Dim ppt As Object, pres As Object
'Create a Microsoft PowerPoint session
Set ppt = CreateObject(""powerpoint.application"")
'Copy the chart on the Chart Labels Demo sheet
Worksheets(""Chart Labeling"").ChartObjects(1).Copy
'Create a new document in Microsoft PowerPoint
Set pres = ppt.Presentations.Add
'Add a slide
pres.Slides.Add 1, ppLayoutBlank
'Make PowerPoint visible
ppt.Visible = True
'Activate PowerPoint
AppActivate ppt.Name
'Paste the chart
ppt.ActiveWindow.View.Paste
Set ppt = Nothing
End Sub"
Automating MICROSOFT OUTLOOK
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • "This example creates and adds information to a new Outlook task. Run Outlook and click Tasks on the Outlook bar to see the new task.
# Author: Abhinav Vaid
#Building Automated Test Systems
VBA for Microsoft Outlook Automation
"Sub MS_Outlook()
Dim ol As Object, myItem As Object
'Create a Microsoft Outlook session
Set ol = CreateObject(""outlook.application"")
'Create a task
Set myItem = ol.CreateItem(olTaskItem)
'Add information to the new task
With myItem
.Subject = ""New VBA task""
.Body = ""This task was created via Automation from Microsoft Excel""
.NoAging = True
.Close (olSave)
End With
'Remove object from memory
Set ol = Nothing
End Sub"
' Add a new worksheet to this workbook
Set NewBook = Workbooks.Add
' Loop through all of the fields, returning the field names to the worksheet.
For i = 0 To rst.Fields.Count - 1
NewBook.Sheets(1).Range("a1").Offset(0, i).Value = rst.Fields(i).Name
Next i
' Copy the recordset to the new worksheet.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
NewBook.Sheets(1).Range("a2").CopyFromRecordset rst
' Close the recordset.
Set rst = Nothing
' Close the Connection.
conn.Close
End Sub Figure Number range Figure What is WSH WSH is another technology by Microsoft that supports automation, the same automation has been done by using this technology and 2 popular scripting languages. Internals of WSH: WSH works with the support of runtime scripting lib, provided by Microsoft. In case you are interested, you can go to %windir%\system32\scrunn.dll This also could have been done using all the above mentioned scripting languages. · Perl/Active Perl: Using WSH and Perl for Test Automation · Vbs: Using VBS and WSH for Test Automation So what you going to witness is same thing being automated using 10 different technologies, sounds interesting, isn't it. Wish, I had read a similar book would have been Windows Scripting Host (WSH) is a set of 3 files: Wscript.exe Cscript.exe Wsh.ocx Automating/Scripting Playing with Internals:Developers up the wall!!! There was a mention of touching the extremes with respect to the bug logging practices and avoiding the reproduce ability related fire fighting between testing and development Teams. · Another typical recommendation with respect to the above topic is having a bug scrub periodically where in all the product owners are present, and take a call on prioritization, severities, and so on. Although the book is not intended for this but the current topic definitely is.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Although it calls for a fair amount of hardwork to be done from a Testers perspective as it requires a fair amount of understanding of Operating System Internals in addition to the Scripting skill sets. Who said that life was an easy game !!! Expands the requested environment variable from the running process:
$WshShell->ExpandEnvironmentStrings($string); In addition, log an event in the NT event log or WSH.log (Windows 9x) file using:
$WshShell->LogEvent(Type, Message [,Target]); Target is the name of the system on NT, thus only applicable on NT. The Type of event is either
0 (SUCCESS)
1 (ERROR)
2 (WARNING)
4 (INFORMATION),
8 (AUDIT_SUCCESS)
16 (AUDIT_FAILURE) This method returns a boolean value indicating success or failure. Another method is Popup, which sends a Windows messagebox up on the screen.
$retval = $WshShell->Popup(Text, [SecondsWait], [Title], [Type]); The method allows you to define the text to pop up, alternatively seconds to wait before closing window, the title of the window, and lastly the type of buttons available in the window. They can be:
0 (Ok)
1 (Ok and Cancel)
2 (Abort, Retry, and Ignore)
3 (Yes, No, and Cancel)
4 (Yes and No)
5 (Retry and Cancel)
Using Reference Element
170 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
You can import external type libraries by using the Reference element. By importing a type library, you will be able to naturally access the constants that belongs to it, too.
"Syntactically this is what it is"
Engineers Need Proofs Updates: Which Propdcess Is Consuming CPU? The Processes tab does indicate what is using the CPU (sort of) Note that CPU Time column is mostly 0 (explained later) To determine which process is consuming the most CPU time, sort processes by “CPU” usage column Processes And Threads What is a process? Represents an instance of a running program You create a process to run a program Starting an application creates a process Process defined by Address space Resources (e.g., open handles) Security profile (token) What is a thread? An execution ext within a process Unit of scheduling (threads run, processes don’t run) All threads in a process share the same preprocess address space Services provided so that threads can synchronize access to shared resources (critical sections, mutexes, events, semaphores) All threads in the system are scheduled as peers to all others, without regard to their “parent” process Per-process address space System-wide Address Space Thread Thread Thread Type: tasklist /svc tasklist /svctasklist /svctasklist /svc (C) 1985-2001 Microsoft Corp. C:\Documents and Settings\naudee>cd\ C:\>svchost
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • C:\>tasklist /svc Image Name PID Services ======System Idle Process 0 N/A System 4 N/A smss.exe 432 N/A csrss.exe 488 N/A winlogon.exe 512 N/A services.exe 556 Eventlog, PlugPlay lsass.exe 568 PolicyAgent, ProtectedStorage, SamSs svchost.exe 716 DcomLaunch, TermService svchost.exe 816 RpcSs svchost.exe 888 6to4, AudioSrv, Browser, CryptSvc, Dhcp, dmserver, ERSvc, EventSystem, FastUserSwitchingCompatibility, helpsvc, lanmanserver, lanmanworkstation, Netman, Nla, NwSapAgent, RasMan, Schedule, seclogon, SENS, SharedAccess, ShellHWDetection, srservice, TapiSrv, TrkWks, W32Time, winmgmt, wscsvc, wuauserv, WZCSVC svchost.exe 932 Dnscache svchost.exe 1016 LmHosts, RemoteRegistry, SSDPSRV, WebClient spoolsv.exe 1108 Spooler svchost.exe 1204 BthServ btwdins.exe 1224 btwdins Apache.exe 1240 dev4_423 Mcdetect.exe 1292 McDetect.exe McShield.exe 1312 McShield McTskshd.exe 1364 McTskshd.exe PAStiSvc.exe 1556 STI Simulator svchost.exe 1568 stisvc
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Apache.exe 1632 N/A alg.exe 232 ALG explorer.exe 880 N/A mcagent.exe 2280 N/A Residence.exe 2344 N/A GoogleWebAccWarden.exe 2356 N/A GoogleWebAccClient.exe 2416 N/A mcvsshld.exe 2140 N/A McVSEscn.exe 2612 N/A IEXPLORE.EXE 2036 N/A BBImpSec.exe 3356 N/A AcroRd32.exe 3496 N/A WINWORD.EXE 2060 N/A cmd.exe 1984 N/A tasklist.exe 4044 N/A wmiprvse.exe 2264 N/A C:\> Engineers Need proofs: If thread stack doesn’t solve it, get a memory dump of the process address space and send to developer Use Adplus in Debugging Tools Can snapshot a group of processes at the same time Can also run Drwtsn32.exe Drwtsn32.exe -p
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 130,000 total drivers seen on XP (88 revised / day) Many Devices Over 680,000 distinct Plug and Play (PnP) IDs 1,600 PnP IDs added every dayCrash Dumps Options Small Memory Dump (aka minidump) Default for Windows 2000/XP Professional/Home Only 64kb (128kb on 64-bit systems) ains minimal crash information Creates a unique file name in \Windows\Minidump after reboot Kernel Writes OS memory and not processes most crash debugging doesn’t involve looking at process memory anyway Useful for large memory systems Overwrites every time Full Default for Server Writes all of RAM Overwrites every time Minidumps On Windows XP and Windows Server 2003, minidump is always created, even if system set to full or kernel dump Can extract a minidump from a kernel or full dump using the debugger “.dump /m” command To analyze, requires access to the images on the system that crashed At least must have have access to the Ntoskrnl.exe Microsoft Symbol Server now has images for Windows XP and later Set image path to same as symbol path (covered later) Writing a Crash Dump Crash dumps are written to the paging file Too risky to try and create a new file (no guarantee you will get a dump anyway) How is even this protected? When the system boots it checks
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
HKEY_LOCAL_MACHINE\System\ CurrentrolSet\rol\Crashrol.The boot disk paging file’s on- disk mapping is obtained Relevant components are checksummed: Boot disk miniport driver Crash I/O functions Page file map On crash, if checksum doesn’t match, dump is not written At The Reboot Session Manager NtCreatePagingFile Paging File Memory.dmp WinLogon SaveDump User mode Kernel mode 1 2 3 4 At The Reboot Session Manager process (\Windows\system32\smss.exe) initializes paging file NtCreatePagingFile NtCreatePagingFile determines if the dump has a crash header Protects the dump from use Note: crash dump portion of paging file is in use during the copy, so virtual memory can run low while the copy is in progress WinLogon calls NtQuerySystemInformation to tell if there’s a dump to extract If there’s a dump, Winlogon executes SaveDump (\Windows\system32\savedump.exe) Writes an event to the System event log SaveDump writes ents to appropriate file On XP or later, checks to see if Windows Error Reporting should be invoked 1 2 3 4 Why Would You Not Get A Dump?
175 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Most common reasons: Crash occurred before paging file was open E.g. a crash during driver initialization Paging file on boot volume is too small Not enough free space for extracted dump Less common: The crash corrupted components involved in the dump process Miniport driver doesn’t implement dump I/O functions Windows 2000 and beyond storage drivers must implement dump I/O to get a Microsoft® signature How do you troubleshoot a system that is crashing with no resulting dump? Online Crash Analysis (OCA) By Default, after a reboot XP/Server 2003 prompts you to send information to http://watson.microsoft.com Can be configured with Computer Properties->Advanced->Error Reporting Can be customized with Group Policies Do/do not show UI Send dump to an internal error reporting server Windows Error Reporting Internals Savedump checks if kernel error reporting is enabled Checks two values under HKLM\Software\Microsoft\PCHealth\ErrorReporting: IncludeKernelFaults and DoReport If crash reporting is enabled, Savedump: Extracts a minidump from the dump file (if system set to full or kernel dumps) Writes the name of the minidump under HKLM\Software\Microsoft\PCHealth\ErrorReporting\KernelFaults Adds a command to execute Dumprep.exe to HKLM\Software\Microsoft\Windows\CurrentVersion\Run This will cause it to run at the first user log on Dumpprep then:Generates an XML description of system version, drivers present, loaded plug and play drivers and depending on the configuration Displays the message box (if enabled) to send the dump Submits to dump for automatic analysis Manual Crash Analysis Open the crash dump with Windbg When you open a crash dump with Windbg or Kd youget a basic crash analysis:Stop code and parameters A guess at offending driver The analysis is the result of the automated execution ofthe !analyze debugger command !
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • analyze uses heuristics to determine what the likely cause of the crash isDebugger tells you to type “!analyze –v” to get detailed debugging informationExplains the bugcheck code Interprets the bugcheck parameters Symbol Files In order to analyze a crash dump, Debugger needs access to (at least) the symbol file for Ntoskrnl.exe Symbol files ain global function and variable names Easiest to use Microsoft Symbol Server for symbol access Windbg: click on File->Symbol File Path Enter “srv*c:\symbols*http://msdl.microsoft.com/download/symbols” If a minidump, must also configure image path to point to location of images (File->Image File Path) Use same string as for symbol server (XP and beyond) OutlineCrash dumps and toolsAnalysis basicsIRQLsStacksAnalyzing an “easy” crash Un-Analyzable crashesCrash transformationBuffer overrunCode overwrite Manual analysisHung SystemsWhen there is no crash Key IRQLs PASSIVE_LEVEL:No interrupts are maskedUser mode code always executes at PASSIVE_LEVEL Kernel-mode code executes at PASSIVE_LEVEL most of thetimeDISPATCH_LEVEL:Highest software interrupt levelScheduler is off Page faults cannot be handled and are illegal operations StacksThe stack is the #1 analysis resourceIt requires that a driver get “caught in the act” Sometimes that’s not possible without the Driver Verifier’s helpEach thread has a user-mode and kernel-mode stackThe user-mode stack is usually 1 MB on x86The kernel-mode stack is typically 12 KB on x86 systemsStacks allow for nested function invocationParameters can be passed on the stack Stores return addressServes as storage for local variables Crash dumps and toolsAnalysis basicsIRQLsStacksAnalyzing an “easy” crashUn-Analyzable crashesCrash transformationBuffer overrun Code overwriteManual analysisHung SystemsWhen there is no crash dumpNotMyFault.exe Test program and driver todemonstrate common crashscenariosIt loads MyFault.sysMyFault.Sys has an IOCTLinterface that implementsdifferent bugsGet it
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • at:http://www.sysinternals.com/files/notmyfault.zip MyFault.sys IOCTL Interface User Mode Kernel Mode enerating a Straight-Forward Crash Run NotMyFault and select “High IRQL fault (kernel mode)” Allocates paged pool buffer Frees the buffer Raises IRQL ≥ DISPATCH_LEVEL Touches the buffer Paged buffers that are marked “not present” but are touched when IRQL ≥ DISPATCH_LEVEL result in theDRIVER_IRQL_NOT_LESS_OR_EQUAL bug check Page fault handler calls KeBugCheckEx from page faulthandlerThe IRQL is not less than or equal to the maximum IRQL atwhich the operation is legal (which is < DISPATCH_LEVELHigh IRQL Fault Analysis !analyze says “memory corruption”!analyze –v easily identifies MyFault.sys by looking at the KeBugCheckEx parametersDisplays the faulting IP in Myfault.sysCrash dumps and tools Analysis basics IRQLs Stacks Analyzing an “easy” crash Un-Analyzable crashes Crash transformation Buffer overrun Code overwrite Manual analysis Hung Systems When there is no crash dump Crash Transformation Many crashes can’t be analyzed The “victim” crashed the system, not the criminal The analyzer may point at Ntoskrnl.exe or Win32K.sys or other Windows components
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Or, you may get many different crash dumps all pointing at different causes You’re goal isn’t to analyze impossible crashes… Its to try to make an “unanalyzable” crash into one that can be analyzed 4-33 Crash Transformation Recipe The tool for crash transformation is the Driver Verifier (Verifier.exe – not in Start menu) Introduced in Windows 2000 Helps developers test their drivers and systems administrators identify faulty drivers Run Verifier.exe Choose “Create Custom Settings” Choose “Select Individual Settings from a List” Enable all options except Low Resource Simulation Using Driver Verifier Then, need to decide which drivers to verifier 1. First, try any “suspicious” drivers (recently updated, known to be problematic, etc) 2. If no crash, try enabling verification on all 3rd party drivers and/or all unsigned drivers 3. If still no crash, enable verification on all drivers May significantly slow down your system
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 4-35 Un-Analyzable Crash: Buffer Overrun Result when a driver goes past the end (overrun) or the beginning (underrun) of a buffer Usually detected when overwritten data is referenced Another driver or the kernel makes the reference There can be a long delay between corruption and detection Driver Buffer Another Driver’s Buffer Pool Structures Higher Addresses Causing a Buffer Overrun Run NotMyFault and select “Buffer Overrun” Allocates a nonpaged pool buffer Writes a string past the end Note that you might have to run several times since a crash will occur only if: The kernel references the corrupted pool structures A driver references the corrupted buffer The crash tells you what happened, but not why A Buffer Overrun Bluescreen In this example, where the crash was the result of the kernel tripping on corrupt pool tracking structures, the Bluescreen tells you what to do What is Special Pool? Special pool is a kernel buffer area where buffers are sandwiched with
180 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • invalid pages Conditions for a driver allocating from special pool: Driver Verifier is verifying driver Special pool is enabled Allocation is slightly less than one page (4 KB on x86) Invalid Invalid Buffer Signature Higher Addresses Page n Page n+1 Page n+2 4-39 Turning on Special Pool You get Special Pool when you enable the Verifier The Verifier Catching Buffer Overrun The Driver Verifier catches the overrun when it occurs The blue screen tells you who’s fault it is !analyze explains the crash and also tells you the buggy driver name The stack shows where the driver bug is 4-41 Un-Analyzable Crash: Code Overwrite Caused when a bug results in a wild pointer A wild pointer that points at invalid memory is easily detected A wild pointer that points at data is similar to buffer overrun Might not cause a problem for a long time
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Crash makes it look like its something else’s fault System code write protection catches code overwrite, but it’s not on if: It’s a Windows 2000 system with > 127 MB memory It’s a Windows XP or Server 2003 system with > 255 MB Causing a Code Overwrite Run NotMyFault and select “Code Overwrite” Overwrites first bytes of nt!ntreadfile Function is most common entry to I/O system so a random thread will cause the crash The crash hints that the fault occurred in NtReadFile The last user-mode address is ZwReadFile The ebx register in the exception frame points at NtReadFile NtReadFile’s start location looks scrambled (u ntreadfile) 4-43 System Code Write Protection If at least one driver is being verified then system code write protection is enabled You can also enable it manually: Set HKLM\System\CurrentrolSet\rol \Session Manager\Memory Management LargePageMinimum REG_DWORD 0xFFFFFFFF EnforceWriteProtection REG_DWORD 1 Reboot to take effect Rerun NotMyFault Crash occurs immediately and even the blue screen points at MyFault.sys: !analyze shows the address of the write and the target (NtReadFile) Outline Crash dumps and tools Analysis basics IRQLs
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Stacks Analyzing an “easy” crash Un-Analyzable crashes Crash transformation Buffer overrun Code overwrite Manual analysis Hung Systems When there is no crash dump Manual Analysis Sometimes !analyze –v isn’t enough Doesn’t tell you anything useful You want to know what was happening at the time of the crash Useful commands: List loaded drivers: lm kv Make sure drivers are all recognized and up to date Look at memory usage: !vm Make sure memory pools are not full If full, use !poolused (requires pool tagging to be on) Examine current thread: !thread May or may not be related to the crash List all processes: !process 0 0 Make sure you understand what was running on the system If a Verifier detected deadlock: !deadlock Booting in Debugging Mode If you have hangs or crashes with no resulting dump (or other “spontaneous reboots”) …you need to get into the kernel debugger at the time of the crash Boot in Debugging Mode Two options to boot in debugging mode: 1. Press F8 during the boot and choose “Debugging Mode” 2. Or, edit the target’s boot.ini file to configure:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • /debugport=comX /baudrate=XXX (note: default baud rate in Debugging Mode is 19200) Windows XP and 2003 support 1394 In either case, this loads the kernel debugger at boot time Does not affect performance But, system will not auto reboot after a crash, even if configured to do so! 4-47 Debugging System in Debugging Mode When system hangs or crashes, attach a kernel debugger and analyze In Windbg, choose File->Kernel Debug Configure baud rate and COM port Click OKIf a crash, Debugger should connect and display the bugcheck codeIf a hung system, must break in with “Debug->Break” Type !analyze –v and/or perform manual analysis commands described earlier To save complete memory dump for offline analysis, use “.dump”(or “.dump /f” to capture a full dump) Note: this will be slow over a serial cable Analyzing a “Sick” System Sometimes a system is still responsive, but you know that something is wrong with it You want to look at its kernel state, but… You don’t want to take it off line by crashing it or connecting a debugger to it You can get a “dump” of a live system with LiveKd (free download from Sysinternals.com) Use it to run Windbg or Kd Use .dump to snapshot live system 4-49 Hung Systems Scenerio: “hard hangs” (e.g. no keyboard or mouse response) Two techniques: both require prior setup and a
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
reboot 1. Manually crash the hung system and hope you get a dump to analyze offline 2. Boot the system in debugging mode and when it hangs, break in with the kernel debugger and analyze system
4-50 Crashing Hung Systems Set the keystroke crash option Set HKEY_LOCAL_MACHINE\System\ CurrentrolSet\Services\i8042prt\ Parameters\CrashOnCtrlScroll to 1 Reboot Rerun NotMyFault Enter right-ctrl+[scroll-lock, scroll-lock] to crash the system !analyze won’t help Use !thread to see what’s running Use ~ command to look at each CPU One CPU is executing MyFault code grabbing a spinlock Spinlock hangs are much easier to debug than dispatcher object (events, semaphores, mutexes) and resource hangs 1. Process & Thread Troubleshooting 2. Understanding & Troubleshooting Memory Problems 3. Troubleshooting with Filemon & Regmon 4. Crash Dump Analysis 5. Boot & Startup Troubleshooting 5-2 Introduction Kinds of problems we're addressing:
185 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Error messages during boot Crashes and hangs during boot Errors messages during the logon process Typical causes: 3rd party drivers and applications System file corruption due to hardware problems or blue screens (from 3rd party drivers) Malware, viruses… Common response: "Reinstall Windows" You can do better than that by understanding the boot and startup process and the tools available to track down and repair problems
The boot process MBR corruption Boot sector corruption Boot.ini misconfiguration System file corruption Crashes or hangs 5-4 Boot Process Terminology Boot begins during installation when Setup writes various things to disk System volume: Boot Record (MBR) Boot sector NTLDR – NT Boot Loader NTDETECT.COM BOOT.INI SCSI driver – Ntbootdd.sys Boot volume: System files – %SystemRoot%: Ntoskrnl.exe, Hal.dll, etc.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
The Boot Process 1. MBR ains small amount of code that scans partition table 4 entries First partition marked active is selected as the system volume Loads boot sector of system volume 2. Boot sector (NT-specific code) Reads root directory of volume and loads NTLDR C: 5-6 x86 and x64 Boot Process 3. NTLDR (screen is black) Moves system from 16-bit to 32-bit mode and enables paging Reads and uses Ntbootdd.sys to perform disk I/O if the boot volume is on a SCSI disk Uses BIOS to read from system volume’s disk This is a copy of the SCSI miniport driver used when the OS is booted Reads Boot.ini Boot.ini selections point to boot drive Specifies OS boot selections and optional switches (most for debugging/troubleshooting) that passed to kernel during boot If more than one selection, NTLDR displays boot menu (with timeout) If you select a 64-bit installation, NTLDR moves the CPU into 64-bit mode NTLDR Once boot selection made, user can type F8 to get to special boot menu Last Known Good, Safe modes, hardware profile, Debugging mode NTLDR executes Ntdetect.com to perform BIOS hardware detection (x86 and x64 only) Later saved into HKLM\Hardware\Description NTLDR loads the SYSTEM hive (HKLM\System), boot drivers, Ntoskrnl.exe, Hal.dll and transfers rol to main entry point of Ntoskrnl.exe Boot driver: critical to boot process (e.g. boot file system driver)
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 4. Ntoskrnl (splash screen appears) Initializes kernel subsystems in two phases: First phase is object definition (process, thread, driver, etc) Second builds on the base that the objects provide This is done in the ext of a kernel- mode system thread that becomes the idle threadI/O Manager starts boot-start drivers and then loads and starts system-start drivers inally, Ntoskrnl creates the Session Manager process (\Windows\System32\Smss.exe), the first user-mode process
Runs programs specified in BootExecute e.g. autochk, the native API version of chkdsk Processes “Delayed move/rename” commands Used to replace in-use system files by hotfixes, service packs, etc. Get Pendmoves from Sysinternals to see registered commands Initializes the paging files and rest of Registry (hives or files) Note: crashes before this point will not result in a crash dump Loads and initializes kernel- mode part of Win32 subsystem(Win32k.sys)Starts Csrss.exe (user-mode part of Win32 subsystem) Starts Winlogon.exe 6. Winlogon.exe: Starts LSASS (Local Security Authority) Loads GINA (Graphical Identification and Authentication) to wait for logon default is Msgina.dll Starts Services.exe (the service roller) 7. Services.exe starts Win32 services marked as “automatic” start Also includes any drivers marked Automatic start (Start value is 2) Service startup inues asynchronous to logons End of normal boot process The Recovery Console Description: Simple repair-oriented command-line environment
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Built on a minimal NT kernel Bootable from Win2K/XP/Server 2003 Setup CD Type “r” to repair and then select the installation Installable onto hard disk (winnt32.exe /cmdcons) 5-13 The Recovery Console Capabilities: File commands: rename, move, delete, copy Service/Driver commands: listsvc, enable, disable MBR/Boot sector commands: fixmbr, fixboot Limitations: Must “log into” the system with the Administrator password Limits on what you can access: Only access system directory, \System Volume Information directories, and root of non-removable media Can only copy files onto system, not off You can override these in the Local Security Policy editor (secpol.msc) on the installation when its running No networking, file editing, or registry editin Agenda The boot processMBR corruptionBoot sector corruption Boot.ini misconfigurationSystem file corruptionCrashes or hangs MBR Corruption Symptoms: Hang at a black screen after BIOS executes “Invalid Partition Table”, “Error loading operating system” or “Missing operating system” message on black screen Cause:MBR is corruptResolution:Boot into Recovery Console Execute the RC’s “fixmbr” commandIf the partition table is corrupt you have to rely on restoring a backup MBR or use 3rd-party disk repair tools AgendaThe boot processMBR corruptionBoot sector corruption
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Boot.ini misconfigurationSystem file corruption Crashes or hangs Boot Sector Corruption Symptoms: Black screen hang“A disk read error occurred”, “NTLDR is missing” or “NTLDR is compressed” error message on black screenCause:Boot sector corruption Troubleshooting:Boot into RCExecute “fixboot” command AgendaThe boot processMBR corruptionBoot sector corruption Boot.ini misconfigurationSystem file corruptionCrashes or hangs Boot.ini Problems Symptom: NTOSKRNL complains that boot device is inaccessible Cause: Boot.ini is missing or corrupt Boot.ini is out-of-date because a partition has been added
5-20 Boot.ini Problems Troubleshooting: Boot into RC Run Bootcfg /rebuild 5-21 Agenda The boot process MBR corruption Boot sector corruption System file corruption Boot.ini misconfiguration Crashes or hangs
190 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Registry SYSTEMHive Corruption ymptom: NTLDR reports that System hive is corrupt Causes:Disk is corruptSystem hive is corrupted or deleted Registry SYSTEM Hive Corruption Troubleshooting: Boot into RC Run Chkdsk If no errors, need to get a backup of the SYSTEM hive On XP with System Restore enable, look in \System Volume Information Go to \_restore{xxx}\RPnnn\snapshot Copy _REGISTRY_MACHINE_SOFTWARE to \Windows\System32\Config\System Otherwise, copy from \Windows\Repair This backup is created by Windows Setup Backing up “System State” with Windows Backup update the Repair directory System File Corruption Symptom:Error message indicating that NTLDR, NTOSKRNL.EXE, HAL.DLL or other system file is missing or corrupt Blue screen with corruption message System File Corruption Causes: Disk is corrupt File is missing or corrupt Troubleshooting: Boot into RC Run Chkdsk If no chkdsk errors obtain clean copy of file and replace file Check in \Windows\System32\DLLCache for backup If not there, get a replacement from an identical system i.e. from same hotfix or service pack
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • If can’t find replacement use Automated System Recovery (ASR) or some other disk image restore utility AgendaThe boot processMBR corruption Boot sector corruptionBoot.ini misconfiguration System file corruptionCrashes or hangs5-28 Post-Splash Screen Crash or Hang Symptoms: System blue screens on boot Hang before logon prompt appears NOTE: If system auto-reboots on crash you won’t see the blue screen! Causes: Buggy driver Registry corruption of non-System hive Troubleshooting: Last Known Good or Safe Mode Accessing Last Known Good Enable it by pressing F8 and selecting it in the Advanced Options boot menu 5-30 LKG Description Last Known Good (LKG) Uses backup of registry rol set last used to boot successfully A rol Set is core startup configuration HKLM\System\rol00n rol set only includes core OS and driver configuration rol set does not include Software, SAM, Security, or Users HKLM\System\Select\Current points at active rol Set LKG Description
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Boot rol makes a copy of the rol set that booted the system Copy is rolSet00n, where 00n is the next available number After a successful boot: 1. LastKnownGood is set to the copy 2.The previous LastKnownGood is deleted By default, “Successful boot” is determined when All the auto-start services have started successfully A successful interactive log in Can be overridden programmatically 5-32 LKG Capabilities Restores bootable configuration when: A new driver was installed since the last successful Boot A driver’s settings were modified since the last successful boot System settings were modified since the last successful boot LKG Limitations Doesn’t work if: An existing driver was updated A latent driver bug for some reason becomes active Files or registry hives are missing or corrupt 5-34 Leveraging the Failed rol Set When you use LKG the rol set you avoid is saved as the Failed rol set 1. Look at the Failed value in the Select key – this is the rol set that you aborted 2. Export the current rol set and failed rol set to .reg files 3. Massage the text so that there are no differences in the rol set name
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 4. Windiff or Fc to see what’s different Safe Mode Description Try Safe Mode if LKG doesn’t work Accessible from same boot menu as LKG Idea is to only include core set of drivers/services Modeled after Safe Mode in Windows 95 Avoids third-party and unnecessary drivers, which hopefully are what’s causing the boot problem 5-36 Safe Mode Description HKLM\System\CurrentrolSet\Safeboot guides safe mode by specifying names and groups of drivers Normal, Network, Command-Prompt No networking in Normal Networking includes networking services Command-Prompt is same as Normal except launches Command Prompt instead of Explorer as shell for when Explorer shell extensions cause logon problems Directory Services Restore Mode: not for boot troubleshooting (for repairing or restoring Active Directory database from backup) Safe Mode Internals Registry keys guide what’s in safe modes: HKLM\System\CurrentrolSet\SafeBoot\Minimal is for Normal and Command-Prompt HKLM\System\CurrentrolSet\SafeBoot\AlternateShell specifies shell for Command-Prompt boot HKLM\System\CurrentrolSet\SafeBoot\Network is for Network rivers and services must be listed by name or by group to be loaded Exception: all boot-start drivers load regardless
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
What are Windows Script Components? How are they associated to OOPS and/or COM/DCOM Windows Script Components provide you with an easy way to create powerful, reusable COM components in script. You create script components using any scripting scripting language that supports the Microsoft ActiveX® Scripting interfaces. Script languages that support these interfaces include JScript, Microsoft Visual Basic® Scripting Edition (VBScript), PERLScript, PScript, and Python. This new script component technology supports common types of COM components, such as Automation, and is extensible with add-ons such as DHTML behaviors. Lets be more specific: Are small and efficient. Are easy to create, maintain, and deploy. Provide the ability to create COM components. Provide access to a broad range of system services. Using script components, you can create COM components for a variety of tasks, such as performing middle-tier business logic, accessing and manipulating database data, adding transaction processing to applications, and adding interactive effects to a Web page using DHTML Behaviors
Windows Script Components (WSC): Fundamentally, the highest aspiration/expectation from WSC's was building COM Components, by simplifying the process.
Like Other implementations, WSC's can also be implemented using any scripting language. But yes there is a limitation: Support for Interfaces is mandatory. But on the positive side, it also supports Perl code directly embedded into Windows Script Components (WSC's). As mentioned earlier, COM is just a set of guidelines for compliances. And from Tester's automation perspective, I would not recommend to go in depth of COM as there is a whole world out there.
Creating Windows Script Component's (WSC's) How about creating WSC's yourself. Its not as geek as it sounds. If you have played with
195 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • ASP/PHP (that too at a basic level), should be able to build it within no time. WSC's are typically are Extensible Markup Language (XML) files, no previous experience with XML is necessary in order to build a successful component.
Building your Own Windows Scripting Components (WSC's) In order to create a WSC, you can use the Windows Script Component Wizard. The wizard will produce a valid skeleton for your WSC, and all you will have to do is enter your PerlScript. Let's create a simple WSC that can be instantiated within a script and holds the basic functionality of being passed a string and retuns the reversed string
In order to do this, perform the following steps:
1. Open the Windows Script Component Wizard 2. Enter "Easy" in the component name-field; progID automatically becomes Easy.WSC, which is fine. 3. Click "Next" 4. Set the "Language"-option for the component to "Other" and enter "PerlScript" 5. Click "Next" 6. Click "Next" in the "Properties" window 7. Enter "SayHello" as a method with no parameters 8. Click "Next" 9. Click "Next" in the "Events" windows 10. Click "Finish"
Implementing the PerlScript XML Following the simple steps above will create a nice skeleton to hold your code. With the exception for the classid, which is unique each time one is generated, it will look as illustrated.
# Author: Abhinav Vaid
#Building Automated Test Systems # In All Things be men Implementing the PerlScript XML
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
description="Easy"
progid="Easy.WSC"
version="1.00"
classid="{74bb1ba9-2e69-4ad6-b02c-c52f3cbe153b}"
As seen above, you have a Window Script Component that is an XML file. The first declaration of the WSC enables strict XML. In that mode, the elements and attributes are case-sensitive, and attribute values must be enclosed within single quotes or double quotes. You may omit the XML declaration on top of the document and it will not be compiled as strictly, but in these examples, we will stick with XML conformity and leave the declaration in each document. Note: XML elements is that they, like HTML, have tag pairs like "
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • whenever you keep more than one component in your WSC file. Is is the package element. It will as a single element enclose all components; however, as mentioned, it is not required when you have one component within the file. Next, the registration element contains the information about your component such as the progid, classid, description, and version number. Description is a string in which you can write a short abstract summary the funcitonality of your component. The progid is used by the program which creates an instance of your component, and the version number should be incremented if you release a new version of your component. The version number can also be appeneded to the progid as as Easy.WSC.1.00 when creating the instance of your component. After the registration element, the data and functionality that the component expose are defined. The public element will hold properties, methods, and events. We declare a method by the name "SayHello" and then skip on down to the script-elements. As you can tell, there is no source code, so we need to fill that out. In the empty space, enter the following:
sub SayHello {
my($param) = shift @_;
return reverse($param);
}
Eventually this is going to be the code we going to arrive at
description="Easy"
progid="Easy.WSC"
version="1.00"
classid="{74bb1ba9-2e69-4ad6-b02c-c52f3cbe153b}"
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
How do we do it Manually: The art of Registering Components Now, it's time to register the component on the system so that it can be used. If we assume that the file is saved as c:\easy.wsc, there are two ways to do this: You can type regsvr32 c:\easy.wsc in the command window. • Or you can locate it through Windows Explorer, right-click on the file and then choose "Register". Either one of the above methods for registering a component should notify you upon success or failure. After registering the component, you can use it from within Active Server Pages by authoring a small script.
<%@Language=PerlScript%>
<%
$obj = $Server->CreateObject('Easy.WSC');
5 $retval = $obj->SayHello("Hello Children");
$Response->Write($retval);
%>
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
WSC with Properties and Notification Next, let's look at how to get a few properties included in the component, too. The following example will display how to read and write properties. It also includes comments, and, in addition, a custom subroutine is run when the component is registered and unregistered. What will happen is that when the component has been registered, a message box pops up with the text "Windows Script Component says: First.WSC has been registered!" and a similar tailored message when the component is unregistered by either one of the ways previously shown. So, open up the Windows Script Component wizard, again, and this time enter the read/write property "YourName", and the method "SayHello." The property is a global variable, and it internally is read and written by subroutines that implement the functionality needed for performing the given operation. The property can either use "get" and "set" attributes that point to the mentioned subroutines or there can be separate "get" and "set" elements as in the example. Their values point to internal Perl subroutines that do their thing on the property. It is a simple example, and it is as follows.
# Author: Abhinav Vaid
#Building Automated Test Systems
description="First"
progid="First.WSC"
version="1.00"
classid="{d0ccb637-bd0c-4c90-a4bd-7473f499d35a}">
USING PERL FOR SOFTWARE TESTING Using Perl with OOPS The World of Objects can vary from one programming/scripting language to another. As referred to earlier also, COM is just a set of standards which has been defined. Its just a model defined which various techies/organizations/products/projects adhere to w.r.t. Implementation. COM is not a programming language. Just a model. And their implementations have some kind of variations here or there. And Perl is a typical example that can be quoted to explain this.
Advent of Perl Architectural Insights and background. The latest Perl Implementation is derived from the same old Perl which was already in place. And it made a lot of sense to enhance it and extend it rather than changing the vary architecture itself. And it would have made lot more sense to have it and enhance it so that it does not break the level of maturity that per had managed to reach.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Also note that perl is not built for making business or use as a commercial tool. It was engineered by people like us, who had a common ajenda to contribute to the industry either as a contribution or as a support.
So in order to support for COM and Object referencing a methods and support via various modules, packages, functions, references and so on.
The Perl object architecture relies heavily on packages, subroutines, and references, so if you're skipping around in this book, please go back to the beginning. Ready? Here we go.
Win32::Internet
use Win32::Internet;
$INET = new Win32::Internet();
$file = $INET->FetchURL(“http://www.indyacellular.in”);
binmode STDOUT;
print $file;
Open HTTP session
use Win32::Internet;
$INET= new Win32::Internet();
$INET->HTTP($HTTP, “hostname”, “ username”, “password”);
#------
# @ in email address must be escaped e.g.,
# abhinav\@indiantestingboard.com
#------
($statuscode, $headers, $file)=$HTTP->Request(“/”);
binmode STDOUT;
print $file;
Open FTP session
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • use Win32::Internet;
$INET= new Win32::Internet();
$INET->FTP($FTP, “hostname”, “username”, “password”);
$FTP->Cd (“path”);
$FTP->Binary ();
$FTP->Get(“myfile.zip”); # file saved to myfile.zip
$FTP->Close();
FTP Function Reference
$FTP->Ascii();
$FTP->Binary();
Sets the ASCII (Binary) transfer mode for
this FTP session.
$FTP->Cd("/pub"); Changes the current directory on the FTP
remote host.
$FTP->Delete("yourfile.zip"); Deletes a file on the FTP remote host.
$FTP->Get("myfile.zip"); Gets the remote FTP file and saves it.
$FTP->Put("newfile.zip"); Upload a file to the server.
@files = $FTP ->List("*.txt"); Returns a list containing the files in
current directory.
$FTP->Mkdir("NextBuild"); Creates a directory on the FTP remote
host.
$path = $FTP ->Pwd(); Returns the current directory on the FTP
server.
$FTP->Rmdir("olddir"); Removes a directory on the FTP remote
host.
$FTP->Rename("old.zip",
"new.zip"); Renames a file on the FTP remote host.
Win32 API Functions
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
#------
# Some info about window
#------
use Win32;
print "*** Login Name: ", Win32::LoginName(), "\n";
print "*** Domain Name: ", Win32::DomainName(), "\n";
print "*** File System: ", Win32::FsType(), "\n";
print "*** OS version: ", Win32::GetOSVersion(), "\n";
print "*** IsWindowNT: ", Win32::IsWinNT(), "\n";
print "*** TickCount: ", Win32::GetTickCount(), "\n";
Win32::SetCwd(NEWDIRECTORY)
Win32::GetCwd()
Sets/Gets current active drive and directory.
Win32::SetLastError(ERROR)
Win32::GetLastError()
Sets/Gets the value of the last error encountered to ERROR.
Win32::InitiateSystemShutdown(MACHINE, MESSAGE, TIMEOUT,
FORCECLOSE, REBOOT)
Shutsdown the specified MACHINE, notifying users with the supplied
MESSAGE, within the specified TIMEOUT interval. Forces closing of all
documents without prompting the user if FORCECLOSE is true, and reboots
the machine if REBOOT is true. This function works only on WinNT.
Win32::AbortSystemShutdown(MACHINE)
Aborts a system shutdown (started by the InitiateSystemShutdown function)
on the specified MACHINE.
Win32::IsWinNT()
Win32::IsWin95()
205 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Returns non zero if the Win32 subsystem is Windows NT (Windows 95).
Win32::GetTickCount()
Returns the number of milliseconds elapsed since the last system boot.
Win32::GetOSVersion()
Returns the array (STRING, MAJOR, MINOR, BUILD, ID), where the
elements are, respectively: An arbitrary descriptive string, the major version
number of the operating system, the minor version number, the build number,
and a digit indicating the actual operating system. For ID, the values are 0 for
Win32s, 1 for Windows 9X and 2 for Windows NT. In scalar context it
returns just the ID.
Win32::MsgBox(MESSAGE [, FLAGS [, TITLE]])
Create a dialogbox containing MESSAGE. FLAGS specifies the required
icon and buttons according to the following table:
0 = OK
1 = OK and Cancel
2 = Abort, Retry, and Ignore
3 = Yes, No and Cancel
4 = Yes and No
5 = Retry and Cancel
MB_ICONSTOP "X" in a red circle
MB_ICONQUESTION question mark in a bubble
MB_ICONEXCLAMATION exclamation mark in a yellow triangle
MB_ICONINFORMATION "i" in a bubble
TITLE specifies an optional window title. The default is ``Perl''.
The function returns the menu id of the selected push button:
0 Error
1 OK
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
2 Cancel
3 Abort
4 Retry
5 Ignore
6 Yes
7 No
Win32::DomainName()
Returns the name of the Microsoft Network domain name.
Win32::FsType()
Returns the name of the filesystem of the currently active drive (like 'FAT' or
'NTFS'). In list context it returns three values: (FSTYPE, FLAGS,
MAXCOMPLEN). FSTYPE is the filesystem type as before. FLAGS is a
combination of values of the following table:
0x00000001 supports case-sensitive filenames
0x00000002 preserves the case of filenames
0x00000004 supports Unicode in filenames
0x00000008 preserves and enforces ACLs
0x00000010 supports file-based compression
0x00000020 supports disk quotas
0x00000040 supports sparse files
0x00000080 support s reparse points
0x00000100 supports remote storage
0x00008000 is a compressed volume (e.g. DoubleSpace)
0x00010000 supports object identifiers
0x00020000 supports the Encrypted File System (EFS)
MAXCOMPLEN is the maximum length of a filename component (the part
between two backslashes) on this file system.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Win32::GetChipName()
Returns the processor type: 386, 486 or 586 for Intel processors, 21064 for the
Alpha chip.
Win32::Sleep(TIME)
Pauses for TIME milliseconds. The timeslices are made available to other
processes and threads.
Yes there are some. Each Programming/scripting language have some inherent differences in terms
of implementation of OOPS.
An object is simply a Reference that happens to know which class it belongs to.
Objects are blessed, references are not.
A class is simply a package that happens to provide methods to deal with object
references.
If a package fails to provide a method, the base classes as listed in @ISA are
searched.
A method is simply a subroutine that expects an object Reference (or a package
name, for static methods) as the first argument.
Methods can be applied with:
METHOD OBJREF PARAMETERS; alternatively
OBJREF->METHOD PARAMETERS
Advocating for Perl's Success & Some Proofs: Perl has been used by thousands of different corporations to tackle and solve different problems. For most people, it has reduced the development time for their desired application by days, weeks, or even months. Below is a sample of the bigger companies that have used Perl. I’ve tried to include testimonials and deeper examples of how Perl was the better solution, where the information has been available. Amazon.com, one of the Internet’s best known and most successful e-commerce sites, used Perl to develop an entire editorial production and control system. This integrates the authoring, maintenance (including version control and searching), and output of the editorial content of the entire Amazon.com
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • website. Netscape engineers wrote a content management and delivery system, with logging, analysis, and feedback on use, in three months using Perl. In order to get around many cross-platform development problems, SPEC (the Standard Performance Evaluation Corporation) used Perl as a wrapper around the C code that is used to test performance. With Perl’s ability to import and dynamically use external C code in combination with its object-oriented abilities, SPEC generated a test system that was easily portable from Unix to the Windows NT platform. On the Internet, ASP serves documents of its own filetype as opposed to familiar document types like HTML documents. This is so that the ASP document can be processed on the server side before returned to the client's browser. An ASP document has the file extension .asp, and it provides both existing features derived from ASP and the ability to execute script commands within the document. The file extension is by the webserver recognized in a special way which results in the underlying technology executing script commands that are within the file and treat other markup such as HTML in a standard fashion. The source code of your saved file's PerlScript script commands will by default never be available to the user when choosing to view the document source in the browser, so any script that you write can not be copied. The reason for this is because everything is processed on the server side and what is returned to the browser is a standard format such as HTML. For a closure, it deserves to be repeated that if you are unfamiliar with object-oriented programming in Perl, it is strongly recommended that you sometime browse the Perl documentation on object-oriented programming to get a basic understanding of the aspects of an object and its origin. Next up, let's look at how to embed PerlScript commands within an ASP document.
The Script Delimiters To successfully execute the script commands within an ASP file, you must specify what scripting language ASP should use to interpret the script commands. You can do this within the .asp document by either setting a default scripting language in which all script commands within the script-delimiters <% and %> will be executed by the specified scripting language. Either you can apply that setting within the Microsoft Management Console's IIS snap-in or you can use what ASP recognizes as a page command, which is indicated by the special directive @. The following example is for illustrational purposes only and it shows you how to set the default scripting language for the ASP file to PerlScript; as a result, all commands will
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • be treated by the PerlScript engine; however, please note that no actual output or result is being displayed by the script, so if you run the code shown in the example below, you won't see an output although the code is being executed and interpreted.
<%@ Language=PerlScript %>
<%
for($i=0; $i<=10; $i++) {
#
# Your Perl code here
#
}
%> Figure Number range Figure In addition to page-commands, you can set the scripting language for a defined scope by using the delimiters. This results in the scripting language only executing the code within the
The Scripting Host When Active Server Pages acts as the scripting host for PerlScript, it is initalized as a host first when an ASP file on the web-server is requested by the way of a URL (such as http://indyacellular.in/home.asp). The web-server locates the file, and the scripting host
210 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • locates the script commands that are within the script delimiters. The chunks of code found within the delimiters are passed on to PerlScript, which creates a Perl interpreter that treats the code. Afterwards, it returns to the scripting host, which waits for the next incoming call. Certain rules apply to when you write code to interact with scripting hosts such as ASP, and let's look at one of those necessary things to know by illustrating it as a parallel to writing a command-line Perl script. When you began programming Perl, among the first things you probably learned was how to print text to the screen, and it is not unusual that such a script looks as follows.
print "Hello Children"; As seen, the "print()"-function of Perl outputs the string "Hello Children" to what is known as the Standard Output. PerlScript, too, will send any output to what it knows as Standard Output, and when executed within the ASP scripting host, the PerlScript engine will use the features of the host's object-model in order to reach the standard output. Essentially, the host provides a set of features, and you pick the scripting language in which you wish to write your application to interact with the host. In ASP, you need to know the host because it decides what method and what object you need to deploy of the host's in order to output a string. A scripting language would become extremely bloated if it had to keep track of all those things. In contrast, the scripter needs to keep track of such features. :-) To send "Hello Children" to the client's browser by using ASP, you call the "Write()"-method of the Response object as in the next example. $Response->Write("Hello Children"); In essence, it is of most importance that you know the object-model of the ASP scripting host within which you use PerlScript so that you know what object to call and how to get the best performance out of your application.
Creating An ASP Page When looking back, the little we've seen about ASP so far is that it contains a set of objects that you can use for your programming. It also requires a scripting language that is used to access these objects, and for that purpose, PerlScript can be used to execute Perl commands that are mixd and embedded within the HTML of your Active Server Pages document. Lastly, the ASP document has its own file extension .asp which is recognized by the web server as a
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • document parsed for first script commands and then HTML tags. 1) Create a file named PerlScript.asp 2) Edit it and enter " Hello Children " 3) Save the file where you can access it by a URL such as http://localhost/. On IIS, the following path is often valid
If it instead of finding the file reports an error such as "File Not Found," you need to open Windows Explorer and reassure yourself of that the file is saved with the .asp file-extension and in the correct physical location for being accessible by the browser. If the browser finds the file and "Hello Children" is outputted, you can place a few script commands in the file and rest assured that ASP will execute everything as for which it was built. Edit the file so that is looks as follows.
# Author: Abhinav Vaid
#Building Automated Test Systems
<%@Language=PerlScript%>
<%
for($i=0; $i<=10; $i+=2) {
$Response->Write("");
$Response->Write("Hello Children!
");
}
%>
The script shown is used only to output HTML and text. Essentially, the Response object's "Write()"-method sends the HTML and variable value of $i to the browser. However, $i is a part of the HTML Font element, so the result is that the font size is increased by 2 for each time "Hello Children" is sent to the client's browser. You could also write the script this way:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Author: Abhinav Vaid
'In all things be men
'A typical challenge for GUI based automation, implementing sendkeys'
'Using vbs & Wsh to implement web/word/text authoring
' SendKeys example
' This shows how to automate the Windows GUI (or any GUI app)
' by sending keystrokes to it from a VBScript
'first we'll launch Notepad - this could be ANY app
'we'll specify that the script NOT wait for it
'to finish running
Dim objShell
Set objShell = CreateObject("WScript.Shell")
objShell.Run "C:\Windows\System32\Notepad.exe",,False
'we will wait a second for the app to start
WScript.Sleep 1000
'now we need to activate the windows. we need it's window
'title, which by experimentation we've determined Is
'"Untitled - Notepad." You don't need the EXACT title;
'Windows will try to activate the first matching window
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 'it finds
objShell.AppActivate "Untitled - Notepad"
'now we can send keystroked - see the WshShell docs
'for details on what this can Do
'we'll start with simple text
objShell.SendKeys "Hello, scripter "
'to send a square bracket you have to enclose it in curly braces
objShell.SendKeys "{[}If indeed you are a scripter{]}"
'special keys have a name you can send
objShell.SendKeys "{HOME}To whom it may concern:{ENTER}{ENTER}"
'if you're controlling a GUI, these might be useful: ' function keys are {F1}, {F2} and so forth ' Use +{F3} for Shift+F3, ^{F4} for Ctrl+F4, and %{F5} for Alt+F5 ' Tab is {TAB}, backspace is {BACKSPACE}, Enter is {ENTER} pr ~ ' Escape is {ESC}. The arrows are {LEFT} {RIGHT} {UP} and {DOWN} ' You can add + for Shift, ^ for Ctrl, and % for Alt to any of these ' Usually, just type the key's face name in {} and you've got it ' About the onyl thing you can't send is PrintScreen A closure to this example, there is a shorthand for writing $Response->Write($variable), and it is not available when already within script delimiters; however, to easily mix variables with HTML you can automatically call the mentioned method by <%=$variable%> where the equal-sign automatically translates into $Response->Write().
Lets say for example:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
# Author: Abhinav Vaid
#Building Automated Test Systems
<% @Language=PerlScript %>
<% for($i=0; $i<=10; $i+=2) {
%>
COLOR=#000000>
Hello Children!
<% } >%
Figure Number range Figure
The advent of Objects and the recommended best practices
In object-oriented programming, the tester is required to create an instance of the object the tester wishes to use. The ASP objects, on the other hand, are "intrinsic", which means that they are already instantiated and available to your scripting language engine. No pre-work at all is associated with these objects and you can access them at any time. Now, each object encapsulate a set of very specific services and provides shorthand access to common tasks for the web developer such as reading the data submitted from a web-form, reading and writing cookies, redirecting the client's browser, and also extended functionality such as the ability to identify each user as a unique client and associate private server-side variables with her. Before diving into the syntax, these are the objects as of ASP 3.0: Application The Application Object an object that is a place-holder for global variables stored on the server side. A global variable in this sense is a variable that is global to what is known as the IIS application. The IIS application defines the physical space of a webserver such as the range on the hard disk for the URL http://members.someperlsite.com/. A webserver can have
215 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • more than one IIS application, but there is only one ASP application object per IIS application. Confused? Hang on a minute. The Application object, like the Session object, has what is called a collection. The collection contains variables created by you. Whenever you want to create a variable, the variable must be put on the contents collection, and the following is one way of doing it.
# Set an application variable # $Application->Contents->SetProperty('Item', 'myName', 'Tobias');
The contents collection can store scalar variables, arrays, or objects that somehow are associated with all users; for example, a visitor counter or a connection-string used to connect to a database through the Microsoft universal ADO application-programming interface. The variable can be accessed easily, too.
# Access an application variable #
$Application->Contents->Item('myName');
However, when you're comfortable within the Microsoft Management Console and can set and define your own IIS applications, you will notice that there is only one ASP Application object per IIS Applcation and the variables placed in the Contents colleciton are only available within that IIS application and no other. If it's a bit confusing, it's guaranteed to have cleared out after playing around with this for a while. It's the most abstract part about ASP if you are not used to defining IIS applications. On the downside, Personal Web Server does not enable you to define more than one IIS application.
Debugging and Error Handling in ASP The ASPError object is the only new object that shipped with ASP 3.0. It contains detailed information about an error which occured during the processing of an ASP page. Unless told otherwise by the administrator, ASP 3.0 redirects the client to a page named 500-100.asp when an error occurs, and in that page, the ASPError object is deployed to show detailed
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • descriptions about the error. You can with PerlScript easily also extend the use the 500- 100.asp page to perform custom tasks such as emailing you a formatted page of the contents of the ASPError object on an arror or writing it to a special log. Some of its useful properties include: SourceCode for returning the source code of the part of a script that generated the error, Category for tracking down the categoru of the error to ASP, scripting language, or an object, File for the name of the file processed when error was generated, and Description to return a description of the error; however, please note that ASPError objects are only returned by the Server object's GetLastError method.
# Return the ASPError object
#
$ASPError = $Server->GetLastError();
# Output the error's description
#
$Response->Write( $ASPError->{Description} );
ObjectContext The ObjectContext object is used for a page that is run as a transaction. This object enables you to commit or abort a transaction, which is an important functionality for deployed business applications. Its only methods are SetAbort() and SetComplete(). They are used to commit or abort the transaction. Before this can be used, your page must include a @TRANSACTION page command in the page, whereas you are enabled only to define @ once per file; however, you can use multiple directives within the @ such as <% @directiveA=something directiveB=somethingelse%>. If you use PerlScript a lot, you are better off setting it as the default language in the Microsoft Management Console. Setting the default language in the MMC is unfortunately not an option for Personal Web Server. PWS is good for practicing a scripting language and ASP, but it is not good for tailormaking your webservice in terms of IIS applications and other tasks that are administrative. Lets say for example, it does not have the Microsoft Management Console. What PWS mainly does is to allow you to publish and develop Active Server Pages within reson while it hides the
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • administrative tasks. Request For good reason, the Request object is a widely used object. It contains a collection of information such as form-data or cookies gathered from the client. With this object you can gather the contents submitted from any type of web-form, read server variables or the contents of a digital client certificate. You can also read binary data such as an uploaded file by first determining the number of bytes sent with the TotalBytes-property and then read it with the BinaryRead($bytes_to_read) method.
# Get the number of total bytes in the request
10 #$bytes_to_read = $Request->{TotalBytes};
# Read and return a variant array of type VT_U1
$read_bytes=$Request->BinaryRead( $bytes_to_read );
In the Request object, collections that you can read includes ClientCertificate for digital certificated, Cookies for cookies sent with the HTTP request, Form for data posted by a web form using the POST method, QueryString for data passed with the QueryString or GET method from a web form, and ServerVariables for HTTP environment variables such as the server software or the users browser.
# Let's see what's in the ALL_HTTP variable # $Response->Write( $Request->ServerVariables('ALL_HTTP')->Item() ); Response The Response Object is responsible for sending data from the client to the server. Lets say for example, you can print strings such as HTML or scalar variables, and you can also print binary data such as images, set cookies, control cache, character sets, and content-types, plus send the client status headers, append data to the log, set PICS labels, and see if the client is still connected. A cookie can contain the attributed Domain, Expires, HasKeys, Path, and Secure, and you first declare a cookie before setting these attributes.
# Author: Abhinav Vaid
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
#Building Automated Test Systems # In All Things be men # Validating Client Script Interactiveness $Response->Cookies->SetProperty( 'Item', 'someCookie', 'Hi! This is a cookie!' ); # ======# Let's set some attributes of the cookie! # ======# This is the domain that can read the cookie; to specify it further, # use the Path-attribute to define a more complete URL #$Response->Cookies('someCookie')->{Domain} = 'somesite.com'; # The expiration date for the cookie $Response->Cookies('someCookie')->{Expires} = 'January 1, 1999'; # Not a secure cookie; 1 defines it as secure $Response->Cookies('someCookie')->{Secure} = 0;
Testing for Server validations The Server object provides certain server-side functions. Its only property is Timeout which defines for how many seconds a script should try executing before returning an error. This setting can also be set in the Microsoft Management Console. In terms of methods, you can create instances of COM objects in your page, encoding HTML and URL's, mapping the physical path to a file by providing its URL, execute other .asp files, and seamlessly transfer the client to a new page while maintaining the state of the current ASP objects, which means that any form data that was gathered can be transferred between pages.
# Open the file in the virtual path /scripts/forum.asp/
#
15 open( FILE, $Server->MapPath('/scripts/forum.asp') );
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
You can use CreateObject('ProgID') to return a valid COM object to the Active Server Page. Lets say for example, to instantiate an ADO object, you'd do the following:
# Create an ADO Connection object # $ADOConnObj = $Server->CreateObject('ADODB.Connection');
When using strings, HTMLEncode($string) encodes charaters such as greater than and less than into its valid character represenation within an HTML page. This prevents, for example, users to post HTML to a chatroom or messageboard. The URLEncode($string)-method encodes a string for valid represenation as a URL. New in IIS 5, you can Execute('/scripts/page.asp') ASP scripts in other files, and to transfer the user without losing the state of the Request object's existing variables such as the contents of the Form-collection you can use the Transfer-method.
# Transfer the user to a different page
#
$Server->Transfer('/step2.asp');
Worth noting about the transfer method is that it takes place on the server side. There is never a request sent to the users browser telling the browser to redirect, thus a roundtrip is spared. Session
The Session object enables you to associate variables with each unique client and keep the variables stored until the tester leaves the domain. It has the same abilities as the Application object, but with the important difference that these variables will only be available to the single user and the script engine, making it ideal for applications such as shopping carts. The syntax is the same as the Application object in terms of adding entities to the Contents collection. In addition, you can set the Timeout-property to the number of minutes that you
220 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • wish the Session to last before it is ended or you can deliberately end the Session by calling the Abandon-method and then redirect to a new page. The redirection is very important! Other properties that you can use is CodePage which is used for Symbol Mapping, LCID for the locale identifier, and SessionID which uniquely identifies the client. For both the Session and Application object, the Contents collections contain a Remove() and RemoveAll() method. Remove() takes either an index or a keyname for the variable in the Collection to remove, and RemoveAll() removes all variables stored in the collection. In example, consider that you have stores a property called "myName" in the Session s Contents collection, you would remove it as follows
USING JAVA SCRIPT FOR TEST AUTOMATION • U J S T A Had I been the audience, I would have asked “Why Js for Test Automation”, and the question does make sense to me. The answer is clear and simple, Js is mostly targetted for Browser related development (whether it is for client/server or from any xyz vendor microsoft, netscape etc. And at times, Js could be better and simple than any other scripting language. Object Implementation in Java Scripting: Object Oriented Implementation in JS is relatively simple and breeze. It identifies everything as objects. 1. Objects have properties 2. Objects perform actions 3. Objects have methods, interfaces associated which we use to do Automation The most important from our subject I find is Window objects. Lets take an example for testing Interactiveness:
var name = window.prompt("What is your name ?",""); Implementing/Testing Url's/ The location object identifies the current document and consists of a complete URL. URL stands for Information can be sent to an another web page using a search string. DOCUMENT OBJECTS The document object has all the HTML elements attached to it. Anchors, forms, history and link objects. document objects and the objects . You have already used the write method of the document object when you have written a message to the web browserYou can also make anchors and links using JavaScript. You use the String object and the anchor() method of the string object to make an anchor in Java Script. A string object is
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • automatically made for you when you use a string value . Strings values are text enclosed in double quotes like "I am a string". The string object holds the string text. string_name.anchor (anchorName); var top = "you are at the top"; document.write(top.anchor("top")); You use the String object and the link() method of the string object to make a link in Java Script. string_name.link(href); var jump = "click here to go to the top"; document.write(jump.link("#"+"top")+"
");
Type the above code into an HTML file type Hello Children. Invoke the Browser. Runtime dependency: The Browser should have js enabled Variables in JavaScript can be used right when you need them. The variable can represent a string or a number depending on how it is used. We can demonstrate the use of a variable by using a JavaScript window object and the prompt method of the window object to get an input string from the keyboard. You will learn about all of the JavaScript objects shortly. Its good to knowWe use the prompt method on a window object to get a person’s name from the keyboard.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
var name = window.prompt (“What is your name ?”, ""); The window prompt method allows you to display a message and an optional initial value to appear in the place where you enter your name. We have set this initial value to "" which means display no value.
Using Javascript for interactiveness testing/Validations
document.write (“HELLO” + name);
The “+” joining operator is used to join messages or messages and variables together.
Once we get the person’s name we can write it to the screen with a friendly message using the write
method of the document object.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Dependencies on Tools etc: Its very much possible do the entire scripting using notepad/wordpad. But also there are some 3rd party tools also available like editplus etc. Some freeware as well as shareware. MS also provides a tool that can be downloaded from http://msdn.microsoft.com/scripting/scriptlets/wz10en.exe Other optional components that can make the work easier are: Script component wizard: can be downloaded from http://msdn.microsoft.com/scripting/scriptlets/sce10.exe A copy of Script Component documentation can be obtained from: http://msdn.microsoft.com/scripting/scriptlets/serverdocs.html Runtime Dependencies, if any: Scrobj.dll is the interpreter used Other Dependencies to run .exe files under scripts”
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Can be executed as a command line in DOS. Lets say for example, you need to run a command line along with vbs. In order to do this you should be able to access the Runtime method, within WshShell Object, and the syntax would be “c:\\Program Files\resource kit\inuse.exe” f:\\server\winnt\system32\test.dll The syntax would be: wscript.run(command,[windowstyle],[waitonretrun]). Wait on return in this case would be either either true or false(Boolean). Set ObjWS = Wscript.CreateObject(Wscript.shell”) ObjWS.Run (INUSE.EXE f:\test.dll \server\winnt\System32\test.dll”, 0, “true”) What is Programming: And how is it associated with Script Engine & Associated States The Script Engines can be used various scenarios based on requirements. Test Automation happens to be one of them. It becomes all the more important to keep its architecture as simple as possible. Lets say for example, a server-based host may need to preserve a multi threaded design while using Windows Script in an efficient manner. At the same time, a host that does not use threading, such as a typical application, should not be burdened with threading management. Windows Script achieves this balance by restricting the ways a free- threaded scripting engine can call back to the host, freeing hosts from this burden. The way its been built, synching happens to be the responsibility of the scripting engine. For scripting engines that are not internally reentrant, or for scripting language models that are not multithreaded, synchronization could be as simple as serializing access to the scripting engine with a mutex. Of course certain methods, such as IActiveScript::InterruptScriptThread, should not be serialized in this way so that a stuck script can be terminated from another thread. In India, there is a very popular story, that's there from generations about a rabit, who on seeing a Cat closes his eye believing that the tester will not see him and thus his life will be spared. Ignorance cannot be a bliss, esp if it is killing you. The same is true with the Computing world also. Either it has to high(1) or low(0), and if you try to make a tri-state, you are destined to perish. So the real fact remains that IActiveScript is typically free-threaded generally implies that the IActiveScriptSite interface and the host's object model should be free-threaded as well. This would make implementation of the host quite difficult, particularly in the common case where the host is a single-threaded Microsoft Windows-based application with single-threaded or
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • apartment-model ActiveX Controls in its object model. For this reason, the following constraints are placed on the scripting engine's use of IActiveScriptSite: The script site is always called in the context of a host thread. That is, the scripting engine never calls the script site in the context of a thread that the scripting engine created, but only from within a scripting engine method that was called from the host through IActiveScript and its derivatives, through the exposed scripting engine's dispatch object, through a Windows message, or from an event source in the host's object model. The script site is never called from within the context of a simple thread state control method (for example, the IActiveScript::InterruptScriptThread method) or from the IActiveScript::Clone method. Objects and Interfaces: What's different/What's Common: Interfaces: IUnknown and IDispatch are the most commonly used, which is described later in the chapter. The main Object identifiers are: Class ID, Prog ID, and GUI ID, primarily used to access properties and methods. VBS For Test Automation: The process more or less remains the same as we have seen for other scripting languages, and its only the syntax that varies:
# Author: Abhinav Vaid
#Building Automated Test Systems
‘ building automated test systems
‘ In All Things be men
'Authored by: Abhinav Vaid
‘Creating the variable
Dim avword ‘dim is the syntax used in VBS unlike Perl where we used…
Dim avdocument
‘Creating instane
setavword = Createobject(Word.Application);
setavdocument = Createobject(Word.document);
230 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
‘Display Word
avword.visible = True
‘save the document
avdocument.saveas “c:\abhinav\vbsdoc.doc”
'# $scripting language = "VBScript"
'# $interface = "1.0"
' This script uses the GetObject() call to first attempt to connet to
' a running instancd of MS Word. If that fails a call to CreateObject()
' attempts to start Word, make it visible and maximizes it.
Sub Main
Dim obj
' Request to handle errors ourselves so we can handle possible failure
' of GetObject() to connect to Word...
'
On Error Resume Next
' Gets an instancd of Word if it is already running
'
Set obj = GetObject(,"Word.Application")
If TypeName(obj) <> "Application" Then
' It wasn't already running so start it running.
Set obj = CreateObject("Word.Application")
End If
obj.Visible = True
obj.WindowState = 1 ' maximized
End Sub Code Window 2: VBS
2 Perl/ Active Perl For Test Automation
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • A couple of years back (in the year 2000 when I was a part of Motorola India), when I was told that we can use perl for testing, my reaction was "wow!!!". Reason, freeware available tool, platform independent, and lots of other things to explore which I used to think about. Active Perl can be downloaded free from www.activestate.com Dependencies/Constraints: Frankly speaking I don’t see any. It is supported by people like us, and as per requirements, a couple of modules are placed @ www.cpan.org, which we can download (for free) and use within our code. www.perlmonks.com is another site, which I have found extremely useful. Perl is considered good when it comes scripting, let it be CGI, or system administration, etc etc. Then where does testing chip in. Read on… Perl is best when it comes to text manipulation, searching directories, files etc etc. Windows exposes some objects, which can be used for scripting. In order to access the application's objects, windows can use COM scripting. Microsoft OLE is a tool freeware available to know the exposed window objects. How to do it, read on… Specifically, a scripting scripting language uses COM automation to access the remote application's objects. These are organized as a hierarchy of tree-like structures. Lets say for example, the top-level object is called the "Application" object. It has a number of dependent (child) objects, including the "Documents" collection, the "CommandBars" collection, and the "Windows" collection. The "Documents" (note the plural) collection in turn is a container for one or more "Document" (note the singular) objects, and so on. This map of an application's objects is called its object model. Most of the objects within an object model can be remotely accessed and controlled using, as the glue, any batch scripting language that supports COM. Typically, an object is a virtual representation of some real-world object. A particular Document object, for instance, represents an individual Word document. But an object by itself does nothing. Instead, what is important for controlling an object programmatically are the features that the object exposes. You might, for example, want to check the spelling in a Word document programmatically. Or you might want to display a document without allowing the user to modify it in any way. In an object model, these features are implemented as properties, methods, and events. Each object usually supports a number of properties, which represent attributes of the object. Lets say for example, a Word Window object has a "Caption" property, which can be used to either retrieve or to define the caption that would appear in that window's title bar. Each
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Word Document object also has a "ReadOnly" property; you'd set its value to "True" to prevent users from modifying the document. Similarly, objects support methods, which represent functionality that the object makes publicly available. Lets say for example, in a windows application say MSWord, the Document object supports a "SaveAs" method that allows you to save a document to dsk. And to check the spelling of a document programmatically, you'd call the Document object's "CheckSpelling" method. Finally, many objects support events, which are fired in response to some change in the object's environment. When an object's event is fired, any code that your program assigns to that object's event is automatically executed. For instancd, the Document object supports a "Close" event; if you assign code to a particular Document object's "Close" event, that code will execute by design when the document closes.
Using OLE for Test Automation with Perl:
# Author: Abhinav Vaid
#Building Automated Test Systems
use strict;
use Win32::OLE;
use Win32::OLE::Const 'Microsoft Word';
my $Word = Win32::OLE->new('Word.Application', 'Quit');
# $Word->{'Visible'} = 1;
$Word->Documents->Open("C:\\DOCUMENTS\\test.doc")
|| die("Unable to open document ", Win32::OLE->LastError());
$Word->ActiveDocument->PrintOut({
Background => 0,
Append => 0,
Range => wdPrintAllDocument,
Item => wdPrintDocumentContent,
Creating and Terminating an Instance of Microsoft Excel
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Let's start with the simplest possible script, one that creates an instance of Microsoft Excel and then adds a new workbook to that instance:
Set objExcel = CreateObject("Excel.Application") objExcel.Workbooks.Add
That was easy enough, wasn't it? Save your new script as a .vbs file and run it; when you do, your desktop should look something like this (depending, of course, on the wallpaper you're using):
Where Do We Go From Here?
OWA has been designed to be as modular as possible, and to provide as rich a scriptable interface as possible when used with Internet Explorer 5.5
Using the IE Objects for Automation
Since the Internet Explorer supports Automation, you can access its functionality from Visual Basic. You can make the objects in the Internet Explorer available to Visual Basic 4.0 through the References dialog box. The References dialog box lists all the objects available to Visual Basic. You access the dialog box by selecting Tools/References from the menu barshows the References dialog box with a Reference set to the Internet Explorer object library. The proper Reference is described in the dialog box as "Microsoft Internet Controls."
Once a Reference is set in Visual Basic, you can exploit the objects in code. Lets say for example, to get an instance of the Internet Explorer browser that can be accessed from Visual Basic, you could use the following code:
Dim MyBrowser As SHDocVw.InternetExplorer
Set MyBrowser = New SHDocVw.InternetExplorer
The Internet Explorer Scripting Object Model
Just as you can access the Internet Explorer object model from Visual Basic, you can access the scripting object model from Microsoft Visual Basic, Scripting Edition. The scripting object model, which provides access to the browser, is a collection of objects that can be accessed
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
by any scripting engine. VBScript accesses the scripting object model when it is executing code in an HTML page. Developers can also exploit the scripting object model in VBScript code. The scripting object model supports a variety of properties and methods for use directly in VBScript. shows the complete scripting model, which is discussed in detail in the sections that follow.
The Window Object
The Window object is the object that represents the Internet Explorer. The properties, methods, and events defined in Tables 3-3, 3-4, and 3-5 can be called directly in VBScript without qualification—meaning that the Window object does not need to be specified. The following code shows an example using the Confirm method:
Table 3-3.Window Object Properties
Creating the Interface
There is a book on the said topic, which is pretty exhaustive and one of the best I have read. Perl Debugger pocket Reference by Richard Foley
UNTOLD TRUTHS OF WINBATCH/CLI/DOS BASED PROGRAMMING • U T W /CLI/DOS BASED P On a fresh installed Operating System, try to find *.bat. There will be a huge list of batch files the vendor still uses for many short cut jobs and it makes a lot of sense. Reason is simplicity, practicality and the highest is the speed. The GUI's are the ones that suck a lot of processing power,which if done via CLI based tools makes it a much easier and faster quick aruond solution. Ask a geek and he will smile. For Software Testing also, it can be used to do a lot of tasks in much simpler manner. And I have been doing it for years now. Lets get hands on:
235 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • At the highest level, winbatch can be cateogorized into 2 parts:
1. Internal Scripts: straignt executes line by line as the normal scripting languages work
2 External Scripts: incokes the specific program and returns the result. Lets say invoking microsoft word.application using winbatch. But there are so many tricks and
Internal handling of batch files.
they are treated as internal dos commands in the windows environment.
Fun comes with a price associated, please take some precautions:
The name of any script created should not be same as a dos command. The way DOS engine has been implemented within windows is such that it treats all internal commands as . exe/com/winbatch.
All external commands are treated as ext commans and no internal checks are done by the OS. .com and .exe are the typical examples of external commands.
Other Important factors to implement/evaluate winbatch for test automation
Timing plays a very big role in developing test systems. And most of us are not even aware that DOS supports it to some extent. Yes it really does.
PAUSE:
IF: I have seen if playing a very critical role as far as automating sysadmin tasks are concerned.
Comparing and assigning: winbatch is based on teh same old concepts of assignment and comparator operators (=, and == respectively).
Error Handling and interpretations: DOS has a command "CHOICE" which plays a major role in handling errors and intrrpratons: . Lets say if any program ends by Ctrl + C, the resultant value is 3, and if it is null, the errr level evaluates to 0
The return value from the keyboard is taken as error level b ......
ex: CHOICE[string][c:keys][/S][/N][/t:ke,secs]
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
str...# strign 2 be displayed when CHOICE command is run
/C: keys defines the possible keys to be pressed. if options are mentioned then the default Y/N keys are used. ex: CHOICE/C:A1T) ; says that A.1,T and O are the possible keys during execution if undefined key is clicked would resut in a beep and the script would keep executing.
/s- makes case sensitive
/n shows possible keys in brackets when the program is executed
/t:key;secs defines the key which would trigger in case the certain amount of time has passed.
Another Very Important Aspect associated with Winbatch Implementation
Scheduling of offloading tasks/events/jobs: AtSchedules commands and programs to run on a computer at a specified time and date. You can use at only when the Schedule service is running. Used without parameters, at lists scheduled commands.
Syntax
at [\\ComputerName] [{[ID] [/delete]|/delete [/yes]}]
at [[\\ComputerName] hours:minutes [/interactive] [{/every:date[,...]|/next:date[,...]}] command]
Parameters
\\ComputerName
Specifies a remote computer. If you omit this parameter, at schedules the commands and programs on the local computer.
ID
Specifies the identification number assigned to a scheduled command.
/delete
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Cancels a scheduled command. If you omit ID, all of the scheduled commands on the computer are canceled.
/yes
Answers yes to all queries from the system when you delete scheduled events.
hours:minutes
Specifies the time when you want to run the command. Time is expressed as hours:minutes in 24-hour notation (that is, 00:00 [midnight] through 23:59).
/interactive
Allows command to interact with the desktop of the user who is logged on at the time command runs.
/every:
Runs command on every specified day or days of the week or month (for example, every Thursday, or the third day of every month).
date
Specifies the date when you want to run the command. You can specify one or more days of the week (that is, type M,T,W,Th,F,S,Su) or one or more days of the month (that is, type 1 through 31). Separate multiple date entries with commas. If you omit date, at uses the current day of the month.
/next:
Runs command on the next occurrence of the day (for example, next Thursday).
command
Specifies the Windows command, program (that is, .exe or .com file), or batch program (that is, .bat or .cmd file) that you want to run. When the command requires a path as an argument, use the absolute path (that is, the entire path beginning with the drive letter). If the command is on a remote computer, specify Universal Naming Convention (UNC) notation for the server and share name, rather than a remote drive letter.
/?
Displays help at the command prompt.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Remarks
Schtasks is a more powerful superset command-line scheduling tool and includes all the functionality found in the at command-line utility. Use schtasks instead of at for all command-line scheduling tasks. For more information about schtasks, see Related Topics.
Using at
To use at, you must be a member of the local Administrators group.
Loading Cmd.exe
At does not automatically load Cmd.exe, the command interpreter, before running commands. If you are not running an executable (.exe) file, you must explicitly load Cmd.exe at the beginning of the command as follows:
cmd /c dir > c:\test.out
Viewing scheduled commands
When you use at without command-line options, scheduled tasks appear in a table formatted similar to the following:
Status ID Day Time Command Line
OK 1 Each F 4:30 PM net send group leads status due
OK 2 Each M 12:00 AM chkstor > check.file
OK 3 Each F 11:59 PM backup2.bat
Including identification number (ID)
When you include identification number (ID) with at at a command prompt, information for a single entry appears in a format similar to the following:
Task ID: 1
Status: OK
Schedule: Each F
Time of Day: 4:30 PM
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Command: net send group leads status due
After you schedule a command with at, especially a command that has command-line options, check that the command syntax is correct by typing at without command-line options. If the information in the Command Line column is incorrect, delete the command and retype it. If it is still incorrect, retype the command with fewer command-line options.
Viewing results
Commands scheduled with at run as background processes. Output is not displayed on the computer screen. To redirect output to a file, use the redirection symbol (>). If you redirect output to a file, you need to use the escape symbol (^) before the redirection symbol, whether you are using at at the command line or in a batch file. For example, to redirect output to Output.text, type:
at 14:45 c:\test.bat ^>c:\output.txt
The current directory for the executing command is the systemroot folder.
Changing system time
If you change the system time at a computer after you schedule a command to run with at, synchronize the at scheduler with the revised system time by typing at without command-line options.
Storing commands
Scheduled commands are stored in the registry. As a result, you do not lose scheduled tasks if you restart the Schedule service.
Connecting to network drives
Do not use a redirected drive for scheduled jobs that access the network. The Schedule service might not be able to access the redirected drive, or the redirected drive might not be present if a different user is logged on at the time the scheduled task runs. Instead, use UNC paths for scheduled jobs. For example:
at 1:00pm my_backup \\server\share
Do not use the following syntax, where x: is a connection made by the user:
at 1:00pm my_backup x:
240 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
If you schedule an at command that uses a drive letter to connect to a shared directory, include an at command to disconnect the drive when you are finished using the drive. If the drive is not disconnected, the assigned drive letter is not available at the command prompt.
Examples
To display a list of commands scheduled on the Marketing server, type:
at \\marketing
To learn more about a command with the identification number 3 on the Corp server, type:
at \\corp 3
To schedule a net share command to run on the Corp server at 8:00 A.M. and redirect the listing to the Maintenance server, in the Reports shared directory, and the Corp.txt file, type:
at \\corp 08:00 cmd /c "net share reports=d:\marketing\reports >> \\maintenance\reports\corp.txt"
To back up the hard drive of the Marketing server to a tape drive at midnight every five days, create a batch program called Archive.cmd, which contains the backup commands, and then schedule the batch program to run, type:
at \\marketing 00:00 /every:5,10,15,20,25,30 archive
To cancel all commands scheduled on the current server, clear the at schedule information as follows:
at /delete
To run a command that is not an executable (that is, .exe) file, precede the command with cmd /c to load Cmd.exe as follows:
cmd /c dir > c:\test.out
Formatting legend
Format Meaning
Italic Information that the user must supply
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Bold Elements that the user must type exactly as shown
Ellipsis (...) Parameter that can be repeated several times in a command line
Between brackets ([]) Optional items
Between braces ({}); choices separated by pipe (|). Example: {even|odd} Set of choices from which the user must choose only one
Courier font Code or program output
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.
C:\Documents and Settings\Administrator>cd\
C:\>help
For more information on a specific command, type HELP command-name
ASSOC Displays or modifies file extension associations.
AT Schedules commands and programs to run on a computer.
ATTRIB Displays or changes file attributes.
BREAK Sets or clears extended CTRL+C checking.
CACLS Displays or modifies access control lists (ACLs) of files.
CALL Calls one batch program from another.
CD Displays the name of or changes the current directory.
CHCP Displays or sets the active code page number.
CHDIR Displays the name of or changes the current directory.
CHKDSK Checks a disk and displays a status report.
CHKNTFS Displays or modifies the checking of disk at boot time.
CLS Clears the screen.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
' building automated test systems
' Written By "Abhinav Vaid"
' Automating Notepad.exe using Winbatch
'notepad.vbs
Const WshNormalFocus = 1
Dim objShell
Set objShell = WScript.CreateObject("WScript.shell")
objShell.Run "Notepad.exe", WshNormalFocus ,True
CHAPTER 5: TECHNOLOGICAL ADVANCEMENTS/DIVERSIFICATION AND AUTOMATION
CHALLENGES: SUGGESTED WORKAROUNDS AND BEST PRACTICES Desktop Application Software Test Automation
Office App Sys Admin Security Testing Web Testing Mailer Testing Mobile Testing Protocol Testing Device Drivers Testing Testing Embedded Systems Data Driven Testing Boundary Testing Regression testing Performance Testing Stress testing Autobuild to Installer Testing
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Notifications related Testing Program Management Testing
CHAPTER 6: TOUCHING THE EXTREMES IN TEST AUTOMATION Making a measureable Significant Impact Bringing Intelligence into our Daily Lives with Automation Artificial Intelligence
CHAPTER 7: STEPPING UP FOR VERTICAL GROWTH IN TEST AUTOMATION/ASSOCIATED
TECHNOLOGIES.
6.4.5 Effective Reporting Diagnostics Tools in Windows: Remember we talked about Driving Developers up the wall: Do your homework properly and rest will follow. which captures exceptions\Errors\Logs that can possibly catch exceptions...., will undisputedly be accepted by the dev team. Trust me on this guys, this works wonders. For capturing the logs, and exceptions/triggers etc, there is a default feature that comes with Windows, just do a %temp%, and will display the directory as displayed in figure.
To configure settings to collect crash dumps, go to Control Panel and click on the System. Select the Advanced panel, and click on the Startup and Recovery Settings button.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range Figure%temp% Figure You will see the dialog box shown on this slide. Select the 64KB dump size. The dump directory shows where to get dump files you will send us. %SystemRoot% generally is c:\windows for XP, c:\winnt for 2K.
245 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Configuring to Capture Kernel Dumps Sometimes, a mini-dump will not provide enough data.
Follow the procedure on the previous slide, and select a Kernel or a Complete dump instead.
As shown, the somewhat larger dump does not configure to create a new file for each dump, but will overwrite previous dump files.
Typical location is c:\windows\memory.dmp
If a system goes into a hung state, Microsoft has provided a mechanism to break out, and invoke the crash dump mechanism.
To enable this feature:
Use regedit to find HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services\i8042prt\Parameters.
Add DWORD CrashOnCtrlScroll with the value 1.
Once enabled, you can crash your system (and create a dump) by hitting CTRL-ScrollLock twice. http://oca.microsoft.com/EN/FAQ.asp
Microsoft Windows itself has a couple of tools and support associated which can be used. I have tried to capture a couple of handy references from our/Testers perspective. Otherwise there really is a whole world out there to be digged out.
But again, who said that life was easy???
Using Dr Watson for Effective Reporting Dr. detects information about system and program failures and records the information in a log file. In the event of a program error, Dr watson would start automatically.
Invoke Dr Watson
To open Dr. , click Start, and then click Run. In the Open box, type drwtsn32.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range Figure: Using Dr Watson for Effective Reporting Dr. cannot prevent errors from occurring, but the information recorded in the log file can be used by technical support personnel to diagnose the problem.
Task managerMicrosoft's Windows Task Manager can be used for Tracking the memory being used by specific programs. A snapshot is given below.
For the options that can be tracked using task Manager, I have provided just 1 screen cap for the purpose of brievity. Its more or less self explainatory.
Windows provides information about computer performance and displays details about programs and processes running on your computer. If you are connected to a network, you
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • can also view network status and see at a glance how your network is functioning. Depending on your work environment and whether you share your computer with other users, you can
Figure Number range Illustration.: Task Manager Support by Microsoft Windows view additional information about those users. Using Window, you can also end programs or processes, start programs, and view a dynamic display of your computer's performance.
1. Memory
2. Inherint Memory
3. Dynamic and Static Memory
4. Memory Testing
5. Using CPU for Capturing the data:
6. Caputuring Process Ids and Associations for Effective Reporting. pid's
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Event Viewer and tricks to effective Usages: Microsoft Windows has also come a long way starting from old 3.1 3.11 In order to access this in the latest OS'es, just right click on my computer, and click the manage button. Microsoft Guys called it Microsoft Management Console (MMC to be presise).
In one snapshot it almost gives the entire computer management including System Tools, Storages to Services and Applications, and that also can be tailed as per own requirements.
The topmost one is:
1. Application Log Viewer:
2. Security log
3. Application log
4. System Log:
Performance Logs and Alerts:
Performance Logs and Alerts takes Windows to a very higher level maturity in terms of effective tracking and debugging support to the extent of adding Process Ids, Objects, and Counters associated.
And this also to the extent of pre-configuring the timing to log, along with locations and other support.
I have just provided a screen capture for a handy Reference. I have been using this exhaustively a lot and it really happens to be amazing wizard.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure Number range FigureLog Generation
Reporting on Reproduceability issues. Lets say for example; Microsoft Word and classic problems associated which still exists after 12 years of my experience. Might have been a couple of years more.
Problem: Microsoft Word hangs when the document size increases to …. It hangs
forever with no option but to kill the process, this is a question that my current manager, Dr Ravi Shankar had asked me recently.
He did not seem to be convinced by my answer, the techies need live proofs. So here it is.
If thread stack doesn’t solve it, get a memory dump of the process address space and attach it to bugzilla. This will save you from the miserable task of re-produceing everytime.
250 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
To generate dump, perform the following steps: 1. Drwtsn32.exe -p
Testing for System Processes CPU usages
The “System” process is another “multi-service process” Windows NT 4.0: PID 2, Windows 2000: PID 8, Windows XP: PID 4 Contains kernel-mode system threads Functions in OS and some drivers that need to run as real threads E.g., need to run concurrently with other system activity, wait timers, perform background “housekeeping” work System threads can be in other processes Look for threads with 100% kernel mode time
Tip: CSRSS contains threads running pieces of Win32k.sys
System Threading and intented Interpretations If System threads are consuming CPU time, need to find out what code is running, since it could be any one of a variety of components Pieces of OS (Ntoskrnl.exe) File server worker threads (Srv.sys) Other drivers: avedump checks if kernel error reporting is enabled Checks two values under HKLM\Software\Microsoft\PCHealth\ErrorReporting: IncludeKernelFaults and DoRepor What Are Error Messages in Windows and Interpretations : there are 3 types of errors w.r.t. Ms windows i.e.,
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 1. Exception Error
2. Illegal Operation Errors
3. Kernel Errors
Playing and Testing With Windows Registry
A database repository for information about a computer's configuration. The registry contains information that Windows continually references during operation, such as:
1. Profiles for each user.
2. The programs installed on the computer and the types of documents each can create.
3. Property settings for folders and program icons.
4. What hardware exists on the system.
5. Which ports are being used.
The registry is organized hierarchically as a tree and is made up of keys and their subkeys, hives, and value entries. registry boot
The default boot option used by most Windows DNS servers. When registry boot is used, DNS is started and initialized using DNS parameters and their values as they are stored in the Windows registry. You can use a Berkeley Internet Name Domain (BIND) boot file as an alternative to this method of boot configuration for DNS. registry size limit (RSL)
A universal maximum for registry space that prevents an application from filling the paged pool with registry data. The total amount of space that can be consumed by registry data (hives) is restricted by the registry size limit. relative distinguished name
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
The part of an object's distinguished name that is an attribute of the object itself. For most objects this is the Common Name attribute. For security principals, the default common name is the security principal name, also referred to as the SAM account name.
Using System Information for Generating Information: found under system tools can be used to gather system information right from hardware to software. Starts from Hardware Resources to Software environment, device drivers to OLE Registration and Windows Error reporting.
Please refer to the screen capture below. I have marked/highlighted the areas to make it intuitive.
Figure Number range FigureUsing System Information Tool to generate Information
Some Third Party Automation Tools: Comments and Technical details: Load and Performance Test Tools
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Loadea - Stress testing tool runs on WinXP; free evaluation version for two virtual users. Capture module provides a development environment, utilizes C# scripting and XML based data. Control module defines, schedules, and deploys tests, defines number of virtual users, etc. Analysis module analyzes results and provides reporting capabilities.
Embarcadero Extreme Test - Performance test tool from Embarcadero Technologies. Capabilities include capture/playback/scripting and test probes/agents that work with a wide variety of applications. Load can be generated against HTTP and JDBC, allowing testing of Web sites, Web-based applications, and relational database systems.
HostedToolbox - Hosted load testing service from hostedLABS, LLC. Browser based test script recording, no downloads or system requirements Works with any client or server. Executed from hostedLAB's distributed infrastructure with servers in multiple locations.
Test Complete Enterprise - Automated test tool from AutomatedQA Corp. includes web load testing capabilities.
WebPartner Test and Performance Center - Test tool from WebPartner for stress tests, load performance testing, transaction diagnostics and website monitoring of HTTP/HTTPS web transactions and XML/SOAP/WSDL web services.
QuotiumPro - Web load testing tool from Quotium Technologies SA. Capabilities include: cookies managed natively, making the script modelling phase shorter; HTML and XML parser, allowing display and retrieval of any element from a HTML page or an XML flux in test scripts; option of developing custom monitors using supplied APIs; more.
LoadDriver - Load test tool from Inforsolutions emphasizes ease of use; directly drives multiple inces of MSIE, rather than simulating browsers. Supports browser-side scripts/objects, HTTP 1.0/1.1, HTTPS, cookies, cache, Windows authentication. Tests can be scriptlessly parameterized with data from text files or custom ODBC data source, for individual userID, password, page to start, think times, data to enter, links to click, cache, initial cache state, etc.
Test Perspective Load Test - Do-it-yourself load testing service from Keynote Systems for Web applications. Utilizes Keynote's load-generating infrastructure on the Internet; conduct realistic outside-the-firewall load and stress tests to validate performance of entire Web
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
application infrastructure.
SiteTester1 - Load test tool from Pilot Software Ltd. Allows definition of requests, jobs, procedures and tests, HTTP1.0/1.1 compatible requests, POST/GET methods, cookies, running in multi-threaded or single-threaded mode, generates various reports in HTML format, keeps and reads XML formatted files for test definitions and test logs. Requires JDK1.2 or higher.
httperf - Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high- performance, extensible tool. Available free as source code.
NetworkTester - Tool (formerly called 'NetPressure') from Agilent Technologies uses real user traffic, including DNS, HTTP, FTP, NNTP, streaming media, POP3, SMTP, NFS, CIFS, IM, etc. - through access authentication systems such as PPPOE, DHCP, 802.1X, IPsec, as necessary. Unlimited scalability; GUI-driven management station; no scripting; open API. Errors isolated and identified in real-time; traffic monitored at every step in a protocol exchange (such as time of DNS lookup, time to logon to server, etc.). All transactions logged, and detailed reporting available.
WAPT - Web load and stress testing tool from SoftLogica LLC. Handles dynamic content and HTTPS/SSL; easy to use; support for redirects and all types of proxies; clear reports and graphs.
OpenLoad - Affordable and completely web-based load testing tool from OpenDemand; knowledge of scripting languages not required - web-based recorder can capture and translate any user action from any website or web application. Generate up to 1000 simultaneous users with minimum processor.
ANTS - Advanced .NET Testing System from Red Gate Software. A load and stress testing tool focused on .NET web applications, including XML Web Services. ANTS generates multiple concurrent users via recordable Visual Basic .NET scripts and records the user experiences, at the same time performance counter information from Windows system is integrated into the results.
Apache JMeter - Java desktop application from the Apache Software Foundation designed to
255 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types; can make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load.
TestMaker - Free open source utility maintained by PushToTest.com and Frank Cohen, for performance, scalability, and functional testing of Web application. A framework and utility to build and run intelligent test agents that implement user behaviors and drive the system as users would. Features an XML-based scripting scripting language and library of test objects to create test agents. Includes capability to check and monitor email systems using SMTP, POP3, IMAP protocols. Java-based tool - runs on any platform.
Webhammer - Low-cost utility by Stephen Genusa designed to test Web applications and servers. Configurable 1-16 connections per system CPU.
Web Polygraph - Freely available benchmarking tool for caching proxies, origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high- performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration scripting language, and portable open-source implementation. C++ source available; binaries avail for Windows.
OpenSTA - 'Open System Testing Architecture' is a free, open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.
PureLoad - Java-based multi-platform performance testing and analysis tool from Minq Software. Includes 'Comparer' and 'Recorder' capabilities, dynamic input data, scenario editor/debugger, load generation for single or distributed sources.
ApacheBench - Perl API for Apache benchmarking and regression testing. Intended as foundation for a complete benchmarking and regression testing suite for transaction-based mod_perl sites. For stress-testing server while verifying correct HTTP responses. Based on the
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Apache 1.3.12 ab code. Available via CPAN as .tar.gz file.
Torture - Bare-bones Perl script by Lincoln Stein for testing web server speed and responsiveness and test stability and reliability of a particular Web server. Can send large amounts of random data to a server to measure speed and response time of servers, CGI scripts, etc.
WebSpray - Low-cost load testing tool from CAI Networks; includes link testing capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. . eValid - Web test tool from Software Research, Inc that uses a 'Test Enabled Web Browser' test engine that provides browser based 100% client side quality checking, dynamic testing, content validation, page performance tuning, and webserver loading and capacity analysis.
WebPerformance Trainer - Load test tool emphasizing ease-of-use, from WebPerformance Inc. Supports all browsers and web servers; simulates up to 750 users per playback machine at various connection speeds; records and allows viewing of exact bytes flowing between browser and server; no scripting required. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. , Linux, Solaris, most UNIX variants.
WebSizr/WebCorder - Load testing and capture/playback tools from Technovations. WebSizr load testing tool supports authentication, SSL, cookies, redirects. Recorded scripts can be modified manually. .
FORECAST - Load testing tool from Facilita Software for web, client-server, network, and database systems. Capabilities include proprietary, Java, or C++ scripting; windows browser or network recording/playback. Network traces can also be taken from over 15 third party tracing tools. Virtual user data can be parameterized. Works with a wide variety of platforms. e-Load - Load test tool from Empirix Software; for use in conjunction with test scripts from their e-Tester functional test tool. Allows on-the-fly changes and has real-time reporting capabilities. Includes script editor with advanced debugging and maintenance capabilities. Works with a wide variety of platforms.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • QALoad - Compuware's QALoad for load/stress testing of web, database, and char-based systems. Integration with other Compuware tools provides an in-depth view by monitoring its operating system, database and network components, as well as the application itself. Works with a variety of databases, middleware, ERP.
Microsoft WCAT load test tool - Web load test tool from Microsoft for load testing of MS IIS servers; other MS stress tools also listed.
Portent Web Load test tool - Loadtesting.com's low-priced web load testing tool. Has minimal processor requirements. Page validation via matching string in page. Written in Java; multi- platform.
SilkPerformer - Enterprise-class load-testing tool from Segue. Can simulate thousands of users working with multiple protocols and computing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity. SilkPerformer Lite version also available forup to 100 simulated users.
Radview's WebLoad - Load testing tool from Radview Software, also available as part of their TestView web testing suite. Capabilities include over 75 Performance Metrics; can view global or detailed account of transaction successes/failures on individual Virtual Client level, assisting in capturing intermittent errors; allows comparing of running test vs. past test metrics. Test scripting via visual tool or Javascript. Wizard for automating non-GUI-based services testing; DoS security testing.
Loadrunner - Mercury's load/stress testing tool for web and other applications; supports a wide variety of deployment environments, platforms, and databases. Large suite of network/app/server monitors to enable performance measurement of each tier/server/component and tracing of bottlenecks. Integrates with other Mercury testing and monitoring producs.
Koalog Code Coverage - Code coverage analyzer for Java applications from Koalog SARL. Includes: in-process or remote coverage computation, capability of working directly on Java method binaries (no recompilation), predefined (XML, HTML, LaTex, CSV, TEXT) or custom report generation, and session merging to allow compilation of overall results for distinct executions. Integrates with Ant and JUnit.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
TestSmith - Functional/Regression test tool from Quality Forge. Includes an Intelligent, HTML/DOM-Aware and Object Mode Recording Engine, and a Data-Driven, Adaptable and Multi-Threaded Playback Engine. Handles Applets, Flash, Active-X controls, animated bitmaps, etc. Controls are recorded as individual objects independent of screen positions or resolution; playback window/size can be different than in capture. Special validation points, such as bitmap or text matching, can be inserted during a recording, but all recorded items are validated and logged 'on the fly'. Fuzzy matching capabilities. Editable scripts can be recorded in SmithSript scripting language or in Java, C++ or C++/MFC. 90-day evaluation copy available.
QA Wizard - Automated functional web test tool from Seapine Software. Advanced object binding reduces script changes when Web-based apps change. Next-generation scripting scripting language eliminates problems created by syntax or other scripting language errors. Includes capability for automated scripting, allowing creation of more scripts in less time. Supports unlimited set of ODBC-compatible data sources as well as MS Excel, tab/comma delimited file formats, and more. Free Demo and est Script available. platforms.
HttpUnit - Open source Java program for accessing web sites without a browser, from SourceForge.net/Open Source Development Network, designed and implemented by Russell Gold. Ideally suited for automated unit testing of web sites when combined with a Java unit test framework such as JUnit. Emulates the relevant portions of browser behavior, including form submission, basic http authentication, cookies and automatic page redirection, and allows Java test code to examine returned pages as text, an XML DOM, or containers of forms, tables, and links. Includes ServletUnit to test servlets without a servlet container.
MaxQ - Free open-source web functional testing tool from Tigris.org, written in Java. Works as a proxy server; includes an HTTP proxy recorder to automate test script generation, and a mechanism for playing tests back from the GUI and command line. Jython is used as the scripting scripting language, and JUnit is used as the testing library.
Compuware TestPartner - Automated software testing tool from Compuware designed specifically to validate Windows, Java, and web-based applications. The 'TestPartner Visual Navigator' can create visual-based tests, or MS VBA can be used for customized scripting.
WebKing - Web site functional, load, and static analysis test suite from ParaSoft. Maps and
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • tests all possible paths through a dynamic site; can enforce over 200 HTML, CSS, JavaScript, 508 compliance, WML and XHTML coding dards or customized dards. Allows creation of rules for automatic monitoring of dynamic page content. Can run load tests based on the tool's analysis of web server log files. , Linux, Solaris.
Rational Functional Tester - IBM's (formerly Rational's) automated tool for testing of Java, . NET, and web-based applications. Enables data-driven testing, choice of scripting languages and editors. and Linux.
Winrunner - Functional/regression test tool from Mercury; includes support for testing Web, Java, ERP, etc.
Compuware's QARun - QARun for functional/regression testing of web, Java, and other applications. Handles ActiveX, HTML, DHTML, XML, Java beans, and more.
SilkTest - Functional test tool from Segue for Web, Java or traditional client/server-based applications. Features include: test creation and customization, test planning and management, direct database access and validation, recovery system for unattended testing, and IDE for developing, editing, compiling, running, and debugging scripts, test plans, etc.
Security Test Tools – Targetted for the Web
Watchfire's AppScan - Tool suite from Watchfire automates web application security testing, produces defect analyses, and offers recommendations for fixing detected security flaws. Assessment module can be used by auditors and compliance officers to conduct comprehensive audits, and to validate compliance with security requirements.
Acunetix Web Vulnerability Scanner - Web site security testing tool from Acunetix first identifies web servers from a particular IP or IP range. It then crawls entire site, gathering information about every file it finds, and displaying website structure. After this discovery stage, it performs an automatic audit for common security issues. Applications utilizing CGI, PHP, ASP, ASP.NET can all be tested for vulnerabilities such as cross site scripting, SQL injection, CRLF injection, code execution, directory traversal and more. Requires Windows and MSIE.s
Codenomicon HTTP Test Tool - Tool for discovering and eradicating security-related flaws in
260 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
HTTP implementations via robustness testing. Tool systematically generates very large numbers of protocol messages containing exceptional elements simulating malicious attacks, in order to induce component crashes, hanging, and denial-of-service situations which may affect component/application security. (HTTP implementations may be utilized in web servers, browsers, network appliances, proxies, protocol analyzers, PDAs and cell phones.)
SecurityMetrics Appliance - Integrated software and processor device includes Intrusion Detection and Prevention Systems and Vulnerability Assessment. Operates as a Layer 2 Bridge - no network configuration needed. Automatically downloads latest IDS attack signatures, vulnerability assessment scripts and program enhancements nightly.
Lightning Console - Security management tool from Tenable Network Security for multiple security and network administrators across multiple organizations. Scheduled vulnerability scanning, real-time IDS event analysis, asset management, remediation management of vulnerabilities, network topology discovery and organizational executive reporting for hundreds of administrators via easy-to-use web interface.
SARA - 'Security Auditor's Research Assit' Unix-based security analysis tool from Advanced Research Corp. Supports the FBI/SANS Top 20 Consensus; remote self scan and API facilities; plug-in facility for third party apps; SANS/ISTS certified, updated bi-monthly; CVE dards support; based on the SATAN model. Freeware. Also available is 'Tiger Analytical Research Assit' (TARA), an upgrade to the TAMU 'tiger' program - a set of scripts that scan a Unix system for security problems.
Qualys Free Security Scans - Several free security scan services from Qualys, Inc. including SANS/FBI Top 20 Vulnerabilities Scan, network security scan, and browser checkup tool.
Qualys Guard - Online service that does remote network security assessments; provides proactive 'Managed Vulnerability Assessment', inside and outside the firewall,
Perimeter Check - SecurityMetrics 'Perimeter Check' service analyzes external network devices like servers, websites, firewalls, routers, and more for security vulnerabilities which may lead to interrupted service, data theft or system destruction. Includes instructions to help immediately remedy security problems. Can automatically schedule vulnerability assessment of designated IP addresses during low traffic times.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Nessus Security Scanner - Free, open-source remote network security auditing tool, by Renaud Deraison, based on 'never trust the version number' and 'never trust that a given service is listening on the good port'. Nessus is made up of two parts : a server and a client; the server (nessusd) manages the 'attacks', whereas the client is a frontend designed to collect the results. Includes more than 1000 tests in 23 vulnerability categories, and Nessus Attack Scripting scripting language. Works with a variety of operating systems.
NetRecon - Network vulnerability assessment scanner from Symantec. Scans systems and services on the network and safely simulates common intrusion or attack scenarios. Utilizes a root-cause and path-analysis engine to illustrate exact sequence of steps taken to uncover vulnerabilities, enabling identification of exactly where to correct vulnerabilities to enforce corporate security policies. Learns about network as it scans, adapting penetration strategy based on previous results. Customizable reporting. Can scan a variety of operating systems; product runs on Windows.
Secure-Me - Automated security test scanning service from Broadbandreports.com for individual machines. Port scans, denial-of-service checks, 45 common web server vulnerability checks, web server requests-per-second benchmark, and a wide variety of other tests. Limited free or full licensed versions available.
SAINT - Security Administrator's Integrated Network Tool - Security testing tool from SAINT Corporation. An updated and enhanced version of the SATAN network security testing tool. Updated regularly; CVE compatible. Includes DoS testing, reports specify severity levels of problems. Single machine or full network scans. Also available is 'WebSAINT' self-guided scanning service, and SAINTbox scanner appliance. Runs on many UNIX flavors.
Tools – CM product\ Release\Versioning
SilkCentral Performance Monitor - Tool from Segue Software, Inc., actively monitors and evaluates the health of a site's live applications utilizing synthetic business transactions. Passive monitoring records real user sessions and reliably detects and logs all error situations encountered by real-users. Recorded transactions can then be analyzed when production errors occur and fed back into SilkPerformer for real-world performance testing scenarios.
Cuevision Network Monitor - Monitoring tool from Cuevision for monitoring website, server,
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • services, applications, and network; capabilities include notifications via email, net send, and popup, restart apps and services, etc. .
GFI Network Server Monitor - Server management tool from GFI Software Ltd. checks network and servers for failures and fixes them automatically. Alerts via email, pagers, SMS; automatically reboot servers, restart services, run scripts, etc. Freeware version of GFI Network Server Monitor is also available; includes modules to check HTTP and ICMP/ping for checking availability of HTTP and HTTPS sites.
IP Check - Server monitoring tool from Paessler GmbH. Alerts webmasters if a webserver is not working correctly because the web service is down, CGI scripts return errors or the SQL server is not reachable. It can contly monitor any number of servers and ports.
IBM Workplace Web Content Management - IBM's web content management product for Internet, intranet, extranet and portal sites; runs on both Lotus Domino and IBM WebSphere.
WebCheck - Windows application from ITUtils that runs in background and periodically checks a site for availability and correctness; searches for keywords; provides notification by displaying a message or sending an e-mail. Requires MSIE.
WS_FTP Pro - FTP/web publishing tool from Ipswitch; manage, upload, and update websites; automatically resume interrupted transfers; support more than 50 host file systems; drag- and-drop files; for Windows.
SiteScope - Mercury's product for agentless site monitoring and maintenance. Runs on servers and monitors server performance, links, connections, logs, etc.; scheduled and on- demand reporting; provides notifications of problems. Includes published API for creating custom monitors. Monitors mimic users' end-to-end actions. or Unix.
6.4.6 Log Analysis Tools HTTPD Log Analyzers list - Includes categories for Access Analyzers, Agent Analyzers, Referrer Analyzers, Error Analyzers, Other Log Analyzers. Most extensive log analysis tool listing on the net. Includes listing of other log analyzer lists. The access analyzers list includes more than 100 listed with short descriptions of each, Eclipse TPTP Testing Tools Project - TPTP (Test & Performance Tools Platform) is a subproject of Eclipse, an open platform for tool integration. TPTP provides frameworks for building testing tools by extending the TPTP
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Platform. The framework contains testing editors, deployment and execution of tests, execution environments and associated execution history analysis and reporting. The project also includes exemplary tools for JUnit based component testing tool, Web application performance testing tool, and a manual testing tool. The project supports the OMG UML2 Test Profile.
Test Architect - Keyword-driven test automation tool from LogiGear helps increase test coverage. Built-in playback support for web-based application and other platforms.
Networking and Server Test Utilities - Small collection of web server and other test utilities.
Morae - Usability test tool for web sites and software, from TechSmith Corp. for automated recording, analyzing and sharing of usability data. Consists of 3 components. A Recorder records and synchronizes video and data, creating a digital record of system activity and user interaction. A Remote Viewer enables geographically dispersed observers to watch usability tests from any location; it displays test user's computer screen along with a picture-in-picture window displaying the test participant's face and audio; Remote Viewer observers can set markers and add text notes. The Manager component includes integrated editing functionality for assembly of important video clips to share with stakeholders.
AutoTestFlash - Freeware tool by Tiago Simoes for recording and playing back UI Tests in flash applications.
Repro - Manual testing 'helper' tool that records desktop video, system operations in 7 different categories, system resource usage, and system configuration information. Allows user to save and review relevant information for bug reports, and compress the result into a very small file to replay, upload to a bug tracking system, and share with others. Instruments in memory the target application at runtime so no changes are required to application under test. .
URL2image.com - Service from HREF Tools to check web page appearance in different Browser/OS combinations. For anyone interested in css, web dards and elastic design; can specify the screen width(s), font magnification(s) and page position(s) for the proofs. Enter URL and receive back report with screenshots taken in real time on real processor.
FREEping - Free ping software utility from Tools4ever which will ping all your Windows-based
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
servers (or any other IP address) in freely-definable intervals. Will send a popup when one of the servers stops responding.
IP Traffic Test and Measure - Network traffic simulation and test tool from Omnicor Corp. can generate TCP/UDP connections using different IP addresses; data creation or capture and replay; manage and monitor throughput, loss, and delay.
Sandra - 'System ANalyser, Diagnostic and Reporting Assit' utility from SiSoftware. Provides large variety of information about a Windows system's processor and software. Includes CPU, mainboard, drives, ports, processes, modules, services, device drivers, ODBC sources, memory details, environment settings, system file listings, and much more. Provides performance enhancing tips, tune-up wizard, file system and memory bandwidth benchmarking, more. Reporting via save/print/fax/email in text, html, XML, etc. Free, Professional, and other versions available in multiple languages.
Dummynet - Flexible tool developed by Luigi Rizzo, originally designed for testing networking protocols, can be used in testing to simulate queue and bandwidth limitations, delays, packet losses, and multipath effects. Can be used on user's workstations, or on FreeBSD machines acting as routers or bridges.
MRTG - Multi Router Traffic Grapher - free tool utilizing SNMP to monitoring traffic loads on network links; generates reports as web pages with GIF graphics on inbound and outbound traffic.
SOFTWARE QA AND TESTING-RELATED ORGANIZATIONS AND CERTIFICATIONS • S QA T - O C As I said earlier, this can be used by engineer's, who aspire to get into building career in the subject. And with respect to our subject, it wouldn't have been worth it not talking about various Organizations and Certifications associated.
SEI - Software Engineering Institute web site; info about SEI technical programs, publications, bibliographies, some online documents, SEI courses and training, links to related sites.
IEEE Standards - IEEE web site; has Software Engineering dards titles and prices; the topical areas for publications of interest would include listings in the categories of Software
265 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Design/Development and Software Quality and Management.
American Society for Quality - American Society for Quality (formerly the American Society for Quality Control) web site; geared to quality issues in general, not just Software QA. ASQ is the largest quality organization in the world, with more than 100,000 members. Provides a wide variety of general quality-related certifications, as well as the CSQE (Certified Software Quality Engineer).
Society for Software Quality - Has chapters in San Diego, Delaware, and Washington DC area; each with monthly meetings.
QAI - Quality Assurance Institute
EOQ-SG - European Organization for Quality - Software Group, an independent not-for-profit organization founded in 1983. It is comprised of more than 30 national quality organizations and other institutions, enterprises and specialists.
Certification Information for Software QA and Test Engineers:
CSQE - ASQ (American Society for Quality) CSQE (Certified Software Quality Engineer) program - information on requirements, outline of required 'Body of Knowledge', listing of study references and more.
CSQA/CSTE - QAI (Quality Assurance Institute)'s program for CSQA (Certified Software Quality Analyst), CSTE (Certified Software Test Engineer), and Certified Software Project Manager (CSPM) certifications.
ISEB Software Testing Certifications - The British Computer Society maintains a program of 2 levels of certifications - ISEB Foundation Certificate, Practitioner Certificate.
ISTQB Certified Tester - The International Software Testing Qualifications Board is a part of the European Organization for Quality - Software Group, based in Germany. The certifications are based on experience, a training course and test. Two levels are available: Foundation and Advanced.
Another common type of approach for automation of functional testing is 'data-driven' or 'keyword-driven' automated testing, in which the test drivers are separated from the data
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • and/or actions utilized in testing (an 'action' would be something like 'enter a value in a text box'). Test drivers can be in the form of automated test tools or custom-written testing software. The data and actions can be more easily maintained - such as via a spreadsheet - since they are separate from the test drivers. The test drivers 'read' the data/action information to perform specified tests. This approach can enable more efficient control, development, documentation, and maintenance of automated tests/test cases.
CHAPTER 10: CONCLUSIONS AND SUMMARY Statistical Analysis: Benefits and Pitfalls of Test Automation Tools: It’s just a very small introduction so that you get going and explore deeper as you move on. There are some generics in commercially available automation tools which I have explained below in comparison to any automation script used/intented to be used for accomplishing the same automation task.
Capturing: During a Test Run, the Test Automation tool: Captures the test actions/inputs. Works on the OOPS concepts, and recognizes everything as checks, text checks, co-ordinate checks, bmp checks and so on. GUI element, which is selected (such as button, radio-box, toolbar, etc.) and capture all object characteristics (name, color, label, value), and not just the X/Y-coordinates of the mouse-clicks etc.
Test Script: The captured test steps are stored by the Software Test Automation Tools in a test script equivalent to C or Basic. With all the functions of a programming scripting language in the test script (case differentiation, loops, subroutines), even complex test processes can be implemented.
Checkpoints: In order to determine if the program being tested (SuT: Software under Test) is functioning correctly or if there have been any errors, the tester (during the test capture or in the script editing) can insert additional checkpoints in the test script. By this means the layout-relevant characteristics of windows-objects (color, position, size, etc.) can be verified along with functional characteristics of the SuT (a mask value, contents of a message box,
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • etc.).
Replaying Tests: Once captured, tests can be replayed and thus in principal are repeatable at any time. The aforementioned object-oriented test capture permits GUI elements to be re- recognized when the test is repeated, even if the GUI has meanwhile been modified by a change in the software. If the test object behaves differently during a repeat test or if checkpoints are violated, then the test fails. The Software Test Automation Tools records this as an error in the test object.
Tool vendors promote Software Test Automation Tools as a very fast and easy method of automating GUI testing. But in reality there are lots of traps or pitfalls which can impair the effectiveness of Software Test Automation Tools based testing. To understand where and why, we have to take a closer look at GUI's and GUI testing.
Pitfalls to check while Automation:
A GUI which is not designed fosr the Test Automation in perspective can be a daunting task. There are Object Implementations which do not give. Go for Scripting Interfaces, than GUI's. Backup registry & system image
Last known Good Configuration
Climbing up the Technical ladder network shortcut’s url collections specialforder
OS Architecture:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN • with respect to our subject line:
Now what happens in Embedded:
Although it is out of scope, still I wanted to give a higher level one shot description to make the audiences understand in a snapshot.
Here in I have taken Symbian, the most common Operating System found in most of the mobile devices we have today.
1. One of the most important factors is the test code/development code does not become bulky enough so that it results in making it self un-magenable. It always need to be there at the back of the minds and communicated to the stake holders that the objective of automation is testing, and not build a bulky system which itself becomes harder to manage.
2. Take backup of the registry in case you intend to put the application to threshold conditions/expect crashes.
3. Always take image of the system also, in case you intend to put the system to threshold conditions/expect crashes.
6.4.7 Some Clarifications Why I picked up MS Office as an example:
Reason being this is the 1st and foremost requirement for building any test system. Inside- out knowledge of the application that you intend to automate.
A COUPLE OF UN-ANSWERED AND DEBATE-ABLE QUESTIONS IN SOFTWARE TESTING • A - - S T 1. Testers Skill-Sets: Everyone wants to be a developer and not a tester, why? Because developers think that its a low skilled job. But testers knowledge skill-sets are very different and broad from development. It requires strong Operating Systems skillsets (including internals), databases, usability testing, scripting, networking, SDLC to processes compliances. Quality (CMM, ISO etc) and a lot more.
Problems with the Test Industry with respect to our subject: No tailored course and no specific benchmarking guidelines. All abstracts everywhere. But things have been changing,
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • with certifications (both tools and engineering). Things hav... but not enough to convince everyone.
It calls for: Fair insights into Operating System Concepts: Scripting/Programming: Once I was interviewing a guy for embedded tools testing and I asked him about the languages he's comfortable with for writing test plans. His answer was English. Is that wrong, i really do not know. I am referring to one of the world's biggest semiconductor company.
On the other hand, on a training session the traineer was referring to test scripts. And later it turned out to be simple english test steps. And this happened in one of the worlds best Security Org. Bit of everything Everything about 1~2 things.
Do I make sense???
2. Importance of FEVIFTTWH: Almost everyone I talk to refers to usability and usability standards. But who defines it, and what is the standardization. Who sets them, and how do they impact.
3. Has anyone seen an application with HWTTF....., ie, the menu starting from Help, Window, Tools and so on from the Left Hand Side. Similar happens to be with all the Usability/GUI's and more or less goes back to what Microsoft had implemented.
4. How would you define a bug that is least severity and still remains to be priority. Has anyone come across any product by Microsoft that displays Win instead of Windows. Bet you will not be able to find it as for Microsoft guys this would be of last severity but the top-most priority.
5. Measuring Quality: So how do you measure Quality. The simple answer is across what is set as a standard that the product claims to be capable of.
270 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
6. QA/QC: There cannot be any other term to start with especially with respect to our subject. Hundreds and thousands of books/jobs/references/links/tutorials/lectures etc etc that I have come across and each one talks about QA and not about QC/software testing. I think the industry is yet to reach such a maturity stage where these terms get defined. QA is a non technical job (a more of a process oriented targeted for certification, in most of the cases, although proves out to be an umbrella activity putting Software Testing under it, which is a pure technical job.
7. White Box Testing: And all the time, it reminds me of a continent about everyone wants to talk about as much as they possibly can. And when you are told, or try to go there everyone runs away like a thief saving for his life. Reason is simple, Whitebox testing is nothing but a glass box testing, where in each and every function/unit/module/subroutine..... is tested as per the coding guidelines specified. The only thing that a tester can do here is compliance checks with respect to the coding guidelines being followed or not. Having audits, incorporating comments and so on. Also this is true mostly for larger size organizations. For very small startups or thin engineering firms, multitasking makes more sense where the developers mostly do the development as well as testing.
8. Black Box Programming: Discussed earlier
9. Unit Testing—Is it white box or black-box: Lets do a File-->New in Microsoft Word. Is it a white box or a black-box. Before integrating various modules, it would be white- box and when it gets integrated as a product, then it would be a black box, makes sense. In case it doesn’t, please do not curse me.
10. White Box Expectations from Testers: Since the time I started by career, hardly has any interview/conference/training/event happened wherein the said subject has not been refered to.
Some Myths about Testing and Test Automation 1. Automation is different from Developement. Test development is no different from any kind of developement. The only difference is the expectations from the development, i.e., the end users. In our case would be software testers/engineers, who are supposed to have the domain expertise of the end user app.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • 2. Development is more challenging job that Automation.
6.4.8 Live Examples & Proofs: Can Web development be more techie job than lets say embedded testing.
Can VB development be more technically demanding than lets say Codec Testing(any form).
Knowledge base Test Certifications, climbing up the Technical Ladder What have I Automated till today, the safest bets with Zero percent Cost Implication:
This one comes directly from my heart.
1. Smoke Tests
2. Installer Tests
3. Sanity Tests
4. Office monotonous Tasks: MSWord/Excel to Mailing
5. Auto mailers
6. Bug Tracking Related Automation
6.4.9 Lessons Learnt from the Book The theory behind any innovation remains to be the same, objective in terms of the task to be accomplished is all the more important than the tool. Learning MS Word, Framemaker, pagemaker will not make you a technical writer. But if you are a good technical writer, you will learn any tool very fast.
Knowing the inside out functionality of whatever you need to achieve/automate is all the more important than the tool needed to do so. Know. Knowing the former will lower the struggle to a large extent. The best way to do it test test test, and only then automate.
After you have automated, test test test.
6.4.10Where to Go from Here: 1. Open any Application, lets say any component of Microsoft Office or alternatively any
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
software you are comfortable with and start.
2. Automating bit by bit using any scripting language you know/aspire to know.
3. Trying with 2nd scripting language to do the same stuff.
4. Then put small pieces (subroutines), together as a 1 single master script and all subroutines under it as we had discussed during start of the book.
A Test to Brush up and keep moving in Software Test Automation:
For Lets say for Example, lets say:
{
subroutine 1: Invoke MS Outlook
subroutine 2: Enter some Text
subroutine 3: Enter the destination address
subroutine 4: send the mail
} Code Window 26
Start putting in validations once you are able to do this. I am sure you are going to enjoy it. Do share your experience with me.
All The Best !!!
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
RESOURCES AND REFERENCES • R R Software Test Automation: effective use of test execution tools
Mark Fewster and Dorothy Graham
Addison Wesley, 1999. ISBN 0-201-33140-3.
6.4.11Touching the Extremes/Cherish towards Intelligence Perfection
This happens to be my all time favorite subject. Key to Intelligent Scripting and Automation
Auto Responders: Try to bring in interactiveness in mailing systems, so that even if you are physically not there, the automation infrastructure takes care.
Bill Gates example: during the early IT years, I had read that in his house, the electronic controls are automated right from lighting to the phones. Whereever he goes, the automation takes care of enabling and also disabling where he has left.
Home Automation Building Bluetooth proximity Systems: Today I was trying to do something with Bluetooth and eventualy ended up in putting a camera in the car and then monitoring it over the phone. Don't ask me the price guys, the cellphone I already had and web cams are available at throw away prices. Later I checked with the site of the cellphone vendor and the similar product was available at Inr 20,000/- and what I spent was just Rs 200 (Indian Currency) Robotics in our daily Life
Fuzzy Logics
Mobile Multiplayer gaming is a classic example
Wearable Computers
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Today I know that it is very much possible. Try it guys, its a lot of fun. Get stuck up, call me and I will give you tips.
GEARING UP FOR THE FUTURE: WHAT LIES AHEAD, AND WHAT ARE GOING TO BE THE • G F : W , W
TESTING CHALLENGES
1. The highly demanding customers have overpaced over development efforts & the world seems to go towards virtual with lesser physical Interactiveness. 2. road ahead ~ vista 3. convergence in technologies and implementations 4. security challenges 5. boot loaders, ini's, registry, dats, profiles file systems, ctrl panel. problems of bulkiness ms 2 pr speeds ~ Intel. When Microsoft was always working on to stabilize, the Intel towards speeding up the things. Each complementing the other. The times seem to be changing now. That's another reason Microsoft has really not been able to penetrate so far, and neither is Intel. 6. X86 2 MIPS, & ARM: Symbian to Smartphone and Win CE & challenges. Testing competencies & skill sets r always different from others. OS insights, scripting, methodologies. Usability insights to awareness. problems when industry itself is evolving @ greater pace before stabilizing. ex re?ent roll out of 11n products on the basis of 2.0 draft itself. 7. Vendors have already started pushing out end to end solutions targeted for various line of customers. For example, Microsoft itself releasing a MobileDevice, accompanied with OS (of course), apps, and additional support for various apps lets say WiFi. Not only this, the financial softwares to anti-viruses/anti-spammers based on the target audiences. Along with this the challenges for testing are going to grow manyfold.
Security Challenges:
2. Secuirity happens to be one of the biggest threats I have seen till date. What makes it worst is the lack of technical expertise of the law enforcers.
275 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Some Proofs: 1. The biggest aution site which was taken over by again the world's biggest player was being used to sell some offensive video by students. And the cops caught hold of the CEO and put him behind bars.
2. Yesterday, when I was travelling in the market, found some cops in Cyber Cafe. They were taken away and were given some non-sense reason. The guys were released but there entire infrastructure was taken over by the cops.
3. On the other hand, there are so many security flaws in almost everything where technology is involved right from banking to telecom.
Other General Observations and Technological Trends: Typical Desktop Office/SOHO/ Personal Machines:
Microsoft Operating Systems:
1. Microsoft Windows Vista and various available flavors I have been using it since a couple of months now. I had started on win3.1 and now it is vista. A great legendry penetration my Microsoft, which is remarkable. On the negative side, all the versions one after the other have been are also growing bulkier day by day. Each New Version heavier than the previous one, with zero exceptions Unfortunately we still do not see the stability even close to any of these versions. And now even the prior functionalities are cracking up with addition of new features and versions. I personally found hundreds of bugs during this period, and as of today, I find Compaq P1 with Windows98 Second Edition much faster than the dualcore highest version of SonyVaio as it comes with Vista as a default. I have pasted a screen cap below, and was amazed to see this. Opened earlier Microsoft Boxes, and tried, but they work.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range FigureBasic Failures - bugs in latest Windows Flavors
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS •
Figure Number range FigureSame old classic copy paste problems with additional bugs
This one is very interesting, you start using Vista a bit, you will keep getting the message box as displayed in Figure 30.
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Figure Number range FigureExplorerNotWorkingMessageBox
Lets Talk about Hardware/Processors and Peripherals:
And similar happens to be with the biggest Chipsets vendors also, they have focussing primarily towards the speed to meet the high demands. That's another reason why none of them have still not been able to make a significant penetration into embedded space.
There was a very old quote that I had read once which said “Windows is a complicated System with a programming layer on top of it (API), which merely hides. Sooner or later it is destined to come out and bite right on your leg.
Personally I definitely see a lot of change in the trend which has been over the past decade.
7 Conclusions and Summary:
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • During my Engineering School, there was a pal who was always more eager to do anything except studying. Once a professor asked him as to why he does not study, his answer was, “more studies more confusion, no studies no confusion”. The guy was very right to some extent.
Its been a couple of years since this idea of going ahead on this was in my mind. IT is a demanding career, and I was doing this along with regular job. Today I really feel that there are 100’s of things in the book that could have been done better. Maybe in the next version, I will address some of those issues and to improve on the feedback that I receive.
Also feel free to contact me for any querries/bugs/scope of improvement/suggestions. I will definitely address them and get back to you on that.
A couple of automation tools that I have used/tested:
data-dimensions.com
xprogramming.com
the-scientist.com
vb controls: mvps.org/ccrp
test auto snake oil satisfy.com/articles/testautomation
11mauto.htm
kaner.com
testing.com
io.com
Other Technical References:
Vb controls imp: mvps.org/ccrp
Test Auto Snake Oil: satisfy.com/articles/test automation
Lessons in test automation: SQTE Group
280 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Building Maintainable GUI Tests www.data-dimensions.com/testers/network/11mauto.htm kaner.com xprogramming.com/practices/ www.testers.com www.io.com qualitytree.com codecs.microsoft.com/isapi/ocget.dll www.the-scientist.com
Object Identifiers:
Progid, ClassID, Guid
Win scripting components: exposing the properties, methods and events
Various objects associated with Wsh
WSH:script object shell
Some helpful and my Favorite url's : http://www.ActiveState.com - The Perl for Win32 Home Page. http://www.perl.com - The Perl home page. http://language.perl.com - The Perl language home page. http://www.yahoo.com/Computers_and_Internet/Programming_Languages/Perl/ - http://www.automationworld.com/ Web Resources on Software Testing
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Software Testing
Automation www.perl.com www.activeperl.com www.cpan.org www.perlmonks.com http://msdn.microsoft.com/library/default.asp?url=/library/en-us/script56/html/scripting.asp http://www.geekarticles.com/Test-Automation http://www.winbatch.com/
Perl/Active Perl: with OLE and others (perlguitest etc) www.cpan.org
Rational
PurifyPlus for Windows
PurifyPlus for UNIX
PurifyPlus for Linux
PurifyPlus RealTime
Rational XDE Tester Rational XDE Tester1
Rational TestManager Rational TestManager
Rational Robot Rational Robot
Rational TestManager Rational Test 50 Virtual Test Pack2
Rational® PurifyPlus
Rational® Purify®
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Rational® PureCoverage®
Rational® Quantify
Qarun
Silk
Winrunner
Office Automation: Software
Microsoft's Word.application
Open Office
Uability Testing
Help Systems Testing www.sqatester.com www.stickyminds.com www.Hdtune.com www.testing.com www.io.com www.the-scientist.com www.qualitytree.com http://qualitytree.com
http://grantbibson.co.uk/misc/readyboost www.automationworld.com hklm\sw\ms\defrag\bootopt... Key & change enble 2 n
...... \currentversion\emdmgmt & searck for sb key of usb , db clk @ device-status dword
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • value to 2. www.indiantestingboard.com http://www.automationworld.com/
www.gutenberg.com
www.sourceforge.net
www.indyacellular.in
www.automationjunkies.com
7.4.2 Other References/ Technical Articles by Author (The published ones) 1. Using Perl for Software Testing
2. The Advent of Artificial Intelligence
3. Wireless Networks--> The Road Ahead
Books that I would recommended: had there been any book on this, then I would'not have come up with this. But yes there are a couple of books out there which impressed me but are targeted for specific audiences & technologies. I have read every bit of them and found them very useful.
1. Using Visual Basic for Testers
2. NET for Testers
3. Windows API
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Alphabetical Index
100p., 125 Generic Test Automation Template...... 98 : Installers Tracking and information...... 176 : Log Generation Support by Microsoft Windows...... 181 : Task Manager Support by Microsoft Windows...... 179 ' examplefor updating filesystems using registry objects...... 124 ' SendKeys example...... 122 'A typical challenge for GUI based automation, implementing sendkeys'...... 122, 124 'Using vbs & Wsh to implement web/word/text authoring...... 119 # In All Things be men...... 101 # Testing for Databases using ADO...... 116 # Validating Client Script Interactiveness...... 157 #OLE for Test Automation...... 132 $INET = new Win32::Internet();...... 143 Tips...... 9 Precautions and Suggested workarounds...... 9 Maturity in Test Automation...... 9, 13 A Generic Template: Master Script...... 98 Architectural Mapping between Objects in Windows and Checks in Test Automation Tools.... 72 Architectural Mapping between Objects in Windows and Checks in Test Automation Tools ... 72 Architectural View of Microsoft Windows Operating System...... 71 Code Window 1...... 98 Conventions...... 8 Figure 10: Perl validations with backend 2...... 118 Figure 18: Using Dr Watson for Effective Reporting...... 178 Javascript for interactiveness testing...... 162 Lets Try with VBS+ OLE Implementation...... 101 My $Outlook = Win32::OLE->new('Outlook...... 104, 113p. Ole Automation Implementation with Perl...... 104 Perl for Web testing ...... 120
285 Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Sanity Test Execution to autoreport generation using automailers...... 106 Using VBA for Microsoft Access Automation...... 131 Using vbs for Microsoft Word Automation...... 101 VBA Excel Macro Automation...... 128 VBA for Microsoft Outlook Automation...... 133 Winrunner for Microsoft Word Automation...... 22
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Code Windows
Chapter1: The Evolution of Software Testing to Test Automation...... 14 Preface and Introduction...... 14 Setting up the Stage for Test Automation...... 14 6.4.2Test Automation in Embedded Systems...... 17 6.4.3Test Automation Linux...... 18 Organizing the Grey Matter...... 18 Some Higher Level abstracts on Test Automation Tools...... 19 Getting into Insights of Test Automation...... 20 Chapter 2 : Using 3rd Party Tools for Test Automation...... 20 1.Using Winrunner for Test Automation...... 20 6.4.4How do The Automation Script Engines Work? ...... 27 Using LoadRunner ...... 32 Using QuickMacros for Test Automation...... 38 Implementation using QuickMacros...... 48 How Quick Macros Works: Insights...... 57 Using Quick Test Professional (QTP) for Test Automation...... 67 Chapter 3: Touching the OS Internals, insights to make a Test Automation Project a Success.. 75 Architecture Of Microsoft Windows...... 76 Microsoft Windows Kernel - A Brief Primer...... 79 The Changing Trends of Partitioning in Windows:...... 81 What is COM: And how is it associated with Test Automation ...... 91 Chapter 4: Making a Business Case to Build Test Automation: Using 10 different technologies/languages to do the same Test Automation done using Commercial Test Tools. 93 What are Object Models: ...... 104 Capturing and Using Objects:...... 105
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers PAGE NUMBERS • Automating Office Applications ...... 114 1.WSH and Test Automation...... 118 Using WSH for Test Automation...... 147 Using Windows Scripting Host for Test Automation based on OOPS...... 152 A brief primer on Application Programming Interfaces (API)...... 164 OLE for Test Automation for Automation of Microsoft Access...... 164 OLE for Test Automation with Microsoft Word...... 165 Using Perl for Software Testing ...... 201 Using Perl with OOPS...... 201 Using Java Script for Test Automation...... 220 Testing with/For Databases ...... 222 2 Perl/ Active Perl For Test Automation...... 230 Untold Truths of Winbatch/CLI/DOS BASED Programming...... 234 Chapter 5: Technological Advancements/diversification and Automation Challenges: Suggested workarounds and Best Practices...... 242 Chapter 6: Touching the Extremes in Test Automation...... 242 Chapter 7: Stepping up for Vertical Growth in Test Automation/associated technologies..242 6.4.5Effective Reporting Diagnostics tools in windows:...... 242 6.4.6Configuring to Capture Kernel Dumps...... 244 6.4.7Using Dr Watson for Effective Reporting...... 245 6.4.8Event Viewer and tricks to effective Usages: ...... 247 6.4.9Some Third Party Automation Tools: Comments and Technical details: ...... 252 6.4.10Log Analysis Tools ...... 262 Software QA and Testing-related Organizations and Certifications ...... 264 Chapter 10: Conclusions and Summary...... 266 6.4.11Some Clarifications...... 268 A couple of un-answered and debate-able questions in Software Testing...... 268 Some Myths about Testing and Test Automation...... 270 6.4.12Live Examples & Proofs: ...... 271 Knowledge base Test Certifications, climbing up the Technical Ladder...... 271 6.4.13Lessons Learnt from the Book...... 271 6.4.14Where to Go from Here:...... 271
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers AUTHOR: ABHINAV VAID IN ALL THINGS BE MEN •
Resources and References...... 273 6.4.15Touching the Extremes/Cherish towards Intelligence Perfection...... 273 Gearing up for the Future: What lies ahead, and What are going to be the Testing Challenges...... 274 Security Challenges: ...... 274 7Conclusions and Summary: ...... 278 7.4.2Other References/ Technical Articles by Author (The published ones)...... 283 5
End of the Book
Building Automated Test Systems –Microsoft Windows Version 2.0 Page numbers i Xml with OOPS Implementation