DOCSLIB.ORG

Robustness Testing of the Microsoft Win32 API

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Robustness Testing of the Microsoft Win32 API Robustness Testing of the Microsoft Win32 API Charles P. Shelton Philip Koopman Kobey DeVale ECE Department & ICES ECE Department ECE Department Carnegie Mellon University Carnegie Mellon University Carnegie Mellon University Pittsburgh, PA, USA Pittsburgh, PA, USA Pittsburgh, PA, USA [email protected] [email protected] [email protected] Abstract Unfortunately, Windows operating systems have ac- quired a general reputation of being less dependable than Although Microsoft Windows is being deployed in Unix-based operating systems. In particular, the infamous mission-critical applications, little quantitative data has "Blue Screen Of Death" that is displayed as a result of Win- been published about its robustness. We present the results dows system crashes is perceived by many as being far of executing over two million Ballista-generated exception handling tests across 237 functions and system calls more prevalent than the equivalent kernel panics of Unix involving six Windows variants, as well as similar tests operating systems. Additionally, it is a common (although conducted on the Linux operating system. Windows 95, meagerly documented) experience that Windows systems Windows 98, and Windows CE were found to be vulnerable need to be rebooted more often than Unix systems. How- to complete system crashes caused by very simple C ever, there is little if any quantitative data published on the programs for several different functions. No system dependability of Windows, and no objective way to predict crashes were observed on Windows NT, Windows 2000, whether the impending move to Windows 2000 will actu- and Linux. Linux was significantly more graceful at ally improve dependability over either Windows 98 or Win- handling exceptions from system calls in a dows NT. program-recoverable manner than Windows NT and Windows 2000, but those Windows variants were more Beyond the dependability of Windows itself, the com- robust than Linux (with glibc) at handling C library parative dependability of Windows and Unix-based sys- exceptions. While the choice of operating systems cannot tems such as Linux has become a recurring theme of be made solely on the basis of one set of tests, it is hoped discussion in the media and Internet forums. While the that such results will form a starting point for comparing most that can usually be quantified is mean time between dependability across heterogeneous platforms. reboots (anecdotally, Unix systems are generally said to op- erate longer between reboots than Windows NT), issues 1. Introduction such as system administration, machine usage, the behavior of application programs, and even the stability of underly- ing hardware typically make such comparisons problem- Different versions of the Microsoft Windows operating atic. It would be useful to have a comparison of reliability system (OS) are becoming popular for mission- and between Windows and Unix systems based on direct, re- safety-critical applications. The Windows 95/98 OS family producible measurements on a reasonably level playing is the dominant OS used in personal computer systems, and field. Windows NT 4.0 has become increasingly popular in busi- The success of many critical systems requires depend- ness applications. The United States Navy has adopted able operation, and a significant component of system de- Windows NT as the official OS to be incorporated into pendability can be a robust operating system. (Robustness onboard computer systems [15]. Windows CE and Win- is formally defined as the degree to which a software com- dows NT Embedded are new alternatives for embedded op- ponent functions correctly in the presence of exceptional erating systems. Thus, there is considerable market and inputs or stressful environmental conditions [6].) Of partic- economic pressure to adopt Windows systems for critical ular concern is the behavior of the system when confronted applications. with exceptional operating conditions and consequent ex- This is a reprint plus appendices of a paper appearing in the International Conference on Dependable Systems and Net- works, June 25-28, 2000. © 2000 IEEE. Personal use of this material is permitted. However,permission to reprint/repub- lish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE. 1 ceptional data values. Because these instances are by defi- separate pool defined for each data type being tested. nition not within the scope of designed operation, it is These pools of values contain exceptional as well as crucial that the system as a whole, and the OS in particular, non-exceptional cases to avoid successful exception han- react gracefully to prevent compromising critical operating dling on one parameter from masking the potential effects requirements. (Some systems, such as clustered web serv- of unsuccessful exception handling on some other parame- ers, can be architected to withstand single-node failures; ter value. Each test case (the execution of a single MuT however there are many critical systems in the embedded with a single test value selected for each required parameter computing world and mission-critical systems on desktops in the call) is executed as a separate task to minimize the oc- which cannot afford such redundancy, and which require currence of cross-test interference. A single Ballista test highly dependable individual nodes.) case involves selecting a set of test values, executing con- This paper presents a quantitative comparison of the structors associated with those test values to initialize es- vulnerability of six different versions of the Windows sential system state, executing a call to the MuT with the Win32 Application Programming Interface (API) to ro- selected test values in its parameter list, measuring whether bustness failures caused by exceptional function or system the MuT behaves in a robust manner in that situation, and call parameter values. These results are compared to the re- cleaning up any lingering system state in preparation for the sults of similarly testing Linux for exception handling ro- next test (including freeing memory and deleting tempo- bustness. rary files). Exception handling tests are performed using the Ballista testing looks only for non-robust responses Ballista robustness testing harness [1] for both Windows from software, and does not test for correct functionality. and Linux. In order to perform a reasonable This, combined with a data type-based testing strategy, Windows-to-Linux comparison, 237 calls were selected for rather than a functional testing strategy, results in a highly testing from the Win32 API, and matched with 183 calls of scalable testing approach in which the effort spent on test comparable functionality from the Linux API. Of these development tends to grow sub-linearly with the number of calls, 94 were C library functions that were tested with MuTs to be tested. An additional property of Ballista test- identical test cases in both APIs, with the balance of calls ing results is that in practice they have proven to be highly being system calls. Beyond C library functions, the calls repeatable. Virtually all test results reproduce the same ro- selected for testing were common services used by many bustness problems every time a brief single-test program application programs such as memory management, file representing a single test case is executed. and directory system management, input/output (I/O), and Ballista uses the CRASH scale [9] to measure robust or process execution/control. The results are reported in non-robust responses from MuTs. CRASH is an acronym groups rather than as individual functions to provide a for the different robustness failures that can occur. In reasonable basis for comparison in those areas where the Catastrophic failures, the most severe robustness failure APIs differ in the number and type of calls provided. type, the application causes a complete system crash that requires an OS reboot for recovery. In Restart failures, the 2. Background application enters a state where it "hangs" and will not con- tinue normal operation, requiring an application restart for The Ballista software testing methodology has been de- recovery. Abort failures are an abnormal termination of an scribed in detail elsewhere [3], [9] and is publicly available application task as the result of a signal or thrown exception as an Internet-based testing service [1] involving a central that is not specific enough to constitute a recoverable error testing server and a portable testing client that was ported to condition unless the task elects (or is forced by default) to Windows NT and Windows CE for this research. Thus, terminate and restart. Silent failures occur when a function only a brief summary of Ballista testing operation will be or call is performed with invalid parameter values, but the given. system reports that it was completed successfully instead of The Ballista testing methodology is a combination of returning an error indication. Finally, Hindering failures software testing and fault injection approaches. Spe- report an incorrect error indication such as the wrong error cifically selected exceptional values (selected via typical reporting code. Ballista can automatically detect Cata- software testing strategies) are used to inject faults into a strophic, Restart, and Abort failures; Silent failures and system via
Load more

Recommended publications
  • A Microkernel API for Fine-Grained Decomposition
    A Microkernel API for Fine-Grained Decomposition Sebastian Reichelt Jan Stoess Frank Bellosa System Architecture Group, University of Karlsruhe, Germany freichelt,stoess,[email protected] ABSTRACT from the microkernel APIs in existence. The need, for in- Microkernel-based operating systems typically require spe- stance, to explicitly pass messages between servers, or the cial attention to issues that otherwise arise only in dis- need to set up threads and address spaces in every server for tributed systems. The resulting extra code degrades per- parallelism or protection require OS developers to adopt the formance and increases development effort, severely limiting mindset of a distributed-system programmer rather than to decomposition granularity. take advantage of their knowledge on traditional OS design. We present a new microkernel design that enables OS devel- Distributed-system paradigms, though well-understood and opers to decompose systems into very fine-grained servers. suited for physically (and, thus, coarsely) partitioned sys- We avoid the typical obstacles by defining servers as light- tems, present obstacles to the fine-grained decomposition weight, passive objects. We replace complex IPC mecha- required to exploit the benefits of microkernels: First, a nisms by a simple function-call approach, and our passive, lot of development effort must be spent into matching the module-like server model obviates the need to create threads OS structure to the architecture of the selected microkernel, in every server. Server code is compiled into small self- which also hinders porting existing code from monolithic sys- contained files, which can be loaded into the same address tems. Second, the more servers exist | a desired property space (for speed) or different address spaces (for safety).
    [Show full text]
  • Microsoft Update Blue Screen Error Fixing BSOD
    kyoceradocumentsolutions.us Microsoft Update Blue Screen Error Fixing BSOD March 2021 Microsoft Update Blue Screen Error What is causing the Blue Screen error when I print using the KX Driver? Kyocera as well as many other companies began getting reports of Blue Screen of Death (BSOD) incidents when printing. Microsoft released 2 updates this week, KB5000802 and KB5000808. The updates caused the crash. What has Microsoft done since this was reported? Microsoft has removed both updates from the automatic pushed update list. If your PC has not yet downloaded the update it will not install them. If you manually update your system from within Windows Update the 2 problem updates are still there and can be manually selected and installed. Do not do that until a complete fix is released if your PC is used for printing. Microsoft has left them up for manual install for users that have systems not used for printing. How do we fix a PC that has already been updated? The best solution is to uninstall the 2 Updates. Follow the directions below and remember to reboot when done. 1. Open a Command Prompt with Administrator rights. Click on the Start button and type cmd. The following will be displayed. 2 Microsoft Update Blue Screen Error 2. Click Run as Administrator on the right to open the Command Prompt Window. Accept the prompt to open with elevated rights and make changes to the computer. 3. In the Command Prompt enter the following command and press enter. – wusa /uninstall /kb:5000802 4. The Stand alone install will launch and remove the Update.
    [Show full text]
  • Pexpect Documentation Release 4.8
    Pexpect Documentation Release 4.8 Noah Spurrier and contributors Apr 17, 2021 Contents 1 Installation 3 1.1 Requirements...............................................3 2 API Overview 5 2.1 Special EOF and TIMEOUT patterns..................................6 2.2 Find the end of line – CR/LF conventions................................6 2.3 Beware of + and * at the end of patterns.................................7 2.4 Debugging................................................8 2.5 Exceptions................................................8 2.6 Pexpect on Windows...........................................9 3 API documentation 11 3.1 Core pexpect components........................................ 11 3.2 fdpexpect - use pexpect with a file descriptor.............................. 23 3.3 popen_spawn - use pexpect with a piped subprocess.......................... 23 3.4 replwrap - Control read-eval-print-loops................................. 24 3.5 pxssh - control an SSH session...................................... 25 4 Examples 33 5 FAQ 35 6 Common problems 39 6.1 Threads.................................................. 39 6.2 Timing issue with send() and sendline()................................. 39 6.3 Truncated output just before child exits................................. 40 6.4 Controlling SSH on Solaris....................................... 40 6.5 child does not receive full input, emits BEL............................... 40 7 History 41 7.1 Releases................................................. 41 7.2 Moves and forks............................................
    [Show full text]
  • Openvms Record Management Services Reference Manual
    OpenVMS Record Management Services Reference Manual Order Number: AA-PV6RD-TK April 2001 This reference manual contains general information intended for use in any OpenVMS programming language, as well as specific information on writing programs that use OpenVMS Record Management Services (OpenVMS RMS). Revision/Update Information: This manual supersedes the OpenVMS Record Management Services Reference Manual, OpenVMS Alpha Version 7.2 and OpenVMS VAX Version 7.2 Software Version: OpenVMS Alpha Version 7.3 OpenVMS VAX Version 7.3 Compaq Computer Corporation Houston, Texas © 2001 Compaq Computer Corporation Compaq, AlphaServer, VAX, VMS, the Compaq logo Registered in U.S. Patent and Trademark Office. Alpha, PATHWORKS, DECnet, DEC, and OpenVMS are trademarks of Compaq Information Technologies Group, L.P. in the United States and other countries. UNIX and X/Open are trademarks of The Open Group in the United States and other countries. All other product names mentioned herein may be the trademarks of their respective companies. Confidential computer software. Valid license from Compaq required for possession, use, or copying. Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government under vendor’s standard commercial license. Compaq shall not be liable for technical or editorial errors or omissions contained herein. The information in this document is provided "as is" without warranty of any kind and is subject to change without notice. The warranties for Compaq products are set forth in the express limited warranty statements accompanying such products. Nothing herein should be construed as constituting an additional warranty.
    [Show full text]
  • How to Fix a Frozen Windows PC
    How to Fix a Frozen Windows PC Mashka/Shutterstock.com Windows PCs freeze for a variety of reasons. One instance might be a fluke, but repeated freezes suggest a problem you’ll want to fix. Here’s how to unfreeze and recover a stuck PC—and stop it from freezing again. How to Speed Up a Slow PC Volume 0% How to Unfreeze a Frozen Windows PC There are several ways you can recover your frozen PC, depending on what caused the problem. Sometimes, all you have to do is wait a few seconds—the PC might get hung up while doing some work and unfreeze itself a few seconds later. If a full-screen application, like a game, freezes and prevents you from leaving it, press Alt+F4. This closes the application if the game is just experiencing graphical problems, but it won’t work if the application has frozen completely. To see if the computer is still responding, press Ctrl+Alt+Delete. From this screen, you can open the Task Manager (and close any running applications), or log out of or restart your PC. If this screen doesn’t appear, you might not be able to recover your PC without rebooting it. If you can open the Task Manager, you might be able to recover from the freeze. (You can also press Ctrl+Shift+Esc to open the Task Manager.) Select the “Processes” tab—if you don’t see it, click “More Details.” Locate any processes using a lot of CPU—you can click the “CPU” column header to sort by CPU usage and see the most demanding processes at the top of the list.
    [Show full text]
  • Cygwin User's Guide
    Cygwin User’s Guide Cygwin User’s Guide ii Copyright © Cygwin authors Permission is granted to make and distribute verbatim copies of this documentation provided the copyright notice and this per- mission notice are preserved on all copies. Permission is granted to copy and distribute modified versions of this documentation under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one. Permission is granted to copy and distribute translations of this documentation into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Free Software Foundation. Cygwin User’s Guide iii Contents 1 Cygwin Overview 1 1.1 What is it? . .1 1.2 Quick Start Guide for those more experienced with Windows . .1 1.3 Quick Start Guide for those more experienced with UNIX . .1 1.4 Are the Cygwin tools free software? . .2 1.5 A brief history of the Cygwin project . .2 1.6 Highlights of Cygwin Functionality . .3 1.6.1 Introduction . .3 1.6.2 Permissions and Security . .3 1.6.3 File Access . .3 1.6.4 Text Mode vs. Binary Mode . .4 1.6.5 ANSI C Library . .4 1.6.6 Process Creation . .5 1.6.6.1 Problems with process creation . .5 1.6.7 Signals . .6 1.6.8 Sockets . .6 1.6.9 Select . .7 1.7 What’s new and what changed in Cygwin . .7 1.7.1 What’s new and what changed in 3.2 .
    [Show full text]
  • Introduction to Unix
    Introduction to Unix Rob Funk <[email protected]> University Technology Services Workstation Support http://wks.uts.ohio-state.edu/ University Technology Services Course Objectives • basic background in Unix structure • knowledge of getting started • directory navigation and control • file maintenance and display commands • shells • Unix features • text processing University Technology Services Course Objectives Useful commands • working with files • system resources • printing • vi editor University Technology Services In the Introduction to UNIX document 3 • shell programming • Unix command summary tables • short Unix bibliography (also see web site) We will not, however, be covering these topics in the lecture. Numbers on slides indicate page number in book. University Technology Services History of Unix 7–8 1960s multics project (MIT, GE, AT&T) 1970s AT&T Bell Labs 1970s/80s UC Berkeley 1980s DOS imitated many Unix ideas Commercial Unix fragmentation GNU Project 1990s Linux now Unix is widespread and available from many sources, both free and commercial University Technology Services Unix Systems 7–8 SunOS/Solaris Sun Microsystems Digital Unix (Tru64) Digital/Compaq HP-UX Hewlett Packard Irix SGI UNICOS Cray NetBSD, FreeBSD UC Berkeley / the Net Linux Linus Torvalds / the Net University Technology Services Unix Philosophy • Multiuser / Multitasking • Toolbox approach • Flexibility / Freedom • Conciseness • Everything is a file • File system has places, processes have life • Designed by programmers for programmers University Technology Services
    [Show full text]
  • Acronis True Image for Western Digital
    Acronis True Image for Western Digital USER GUIDE Table of contents 1 Introduction ....................................................................................................................5 1.1 What is Acronis True Image for Western Digital? ..................................................................... 5 1.2 Backups created in Acronis True Image ..................................................................................... 5 1.3 System requirements and supported media ............................................................................. 6 1.3.1 Minimum system requirements .................................................................................................................... 6 1.3.2 Supported operating systems ........................................................................................................................ 7 1.3.3 Backing up all data on your PC ....................................................................................................................... 7 1.3.4 Supported file systems .................................................................................................................................... 8 1.3.5 Supported storage media ............................................................................................................................... 8 1.4 Installing Acronis True Image for Western Digital ..................................................................... 9 1.5 Activating Acronis True Image for Western Digital .................................................................10
    [Show full text]
  • Pure User Mode Deterministic Replay on Windows
    Pure User Mode Deterministic Replay on Windows Atle Nærum Eriksen Thesis submitted for the degree of Master in Informatics: programming and networks 60 credits Department of Informatics Faculty of mathematics and natural sciences UNIVERSITY OF OSLO Autumn 2017 Pure User Mode Deterministic Replay on Windows Atle Nærum Eriksen August 1, 2017 Abstract The ability to record and replay program executions has many interesting applications such as debugging in the backwards direction, discovering and fixing the source of non-deterministic bugs and data races and retracing the steps of a system intrusion. Unfortunately, the power of deterministic replay tools is underutilized by the general public as the tools are either too difficult to deploy or unable to fulfill the performance and log size requirements of the user. As it happens, the majority of the research has been aimed at implementing such tools for Linux, and other platforms, including Windows, have mostly been neglected. In this thesis we look at whether it is possible to implement a determin- istic replay tool for the Windows platform that is easily deployable (user mode only without operating system support { this entails no OS modifi- cations or drivers), can record all system calls and their side-effects (even if unknown), works on large programs (1 GB+ RAM), and has a high recording performance (≈2x slowdown). We found that the challenges deterministic replay tools are facing in user mode are exacerbated on Windows due to a lack of documentation and a more restrictive API. Despite this we came up with a design proposal that solves all the problems necessary to implement a deterministic replay tool that satisfies all our requirements for the Windows platform.
    [Show full text]
  • Errors and Error Checking in Windows
    Errors and error checking in Windows Higham and Rushden U3A Let’s first of all look at probably the most serious but least common: Total Windows failure Earlier versions of Windows used to crash much more frequently than they do today, usually accompanied by the very unfriendly Blue Screen of Death (BSOD) shown on the left below, and/or one of the other error messages shown. Windows 10 has moved on from there. Apart from anything else the screen is a much friendlier blue and it has a nice emoticon as well! Joking aside, the blue screen in Windows 10 is normally followed by the PC restarting and quite regularly the machine will boot up normally as Windows has adjusted some settings in the background. It probably doesn’t do any harm to make a note of the error details at the bottom of the screen in case the restart fails to solve the problem, but any solutions you find online are likely to be complicated and technical, and may well contradict other suggested fixes. Be warned! Also be aware that the restart process may take a while. Other errors and warnings are often generated by User Account Control, which is Windows’ way of trying to keep you safe, so turn it off at your peril. When a background program tries to do something, Windows will often ask you if you want it to do that. This program was downloaded, so unless you are absolutely certain that you downloaded it deliberately, and have just clicked on it to start it or make changes, it is probably safer to say “no”.
    [Show full text]
  • Critical Vulnerabilities Bulletin
    January 18th, 2021 Critical Vulnerabilities Bulletin Confidential | Cyberint Copyright © All Rights Reserved 2020 CRITICAL VULNERABILITIES BULLETIN | INDUSTRY SECURITY BULLETIN JANUARY 14, 2021 EXECUTIVE SUMMARY Due to an unusual increase in Critical vulnerabilities found on Windows operating systems, Cyberint's Research Team has decided to release this bulletin to alert our customers of the potential impact. This bulletin will be expanded at the end of the month, as part of the Monthly Vulnerability bulletin we plan on releasing. The reason for this bulletin lies at 2 newly discovered vulnerabilities: 1. <No CVE assigned yet> - Corrupt NTFS OS partition without user interaction. 2. <No CVE assigned yet> - Trigger a BSOD by trying to access a system path. 3. CVE-2020-1398 - Potential bypass of Bitlocker protected systems, including Lock Screen bypass. We suspect that threat actors and updated malware may attempt to exploit these vulnerabilities to further impact potential victims. Confidential | Cyberint Copyright © All Rights Reserved 2020 2 CRITICAL VULNERABILITIES BULLETIN | INDUSTRY SECURITY BULLETIN JANUARY 14, 2021 NTFS CORRUPTION VULNERABILITY Published by a known vulnerability researcher, jonasLyk, it is possible to cause disk partition corruption by executing a cd command as a low privilege user, to a unique path, which causes the Windows Operating System to alert the user to a corrupted partition and force a restart. Whilst technical details are scarce, and the exact reason for the issue being unknown at this time, affected Windows builds using the NTFS file system include those from 1803 all the way to 20H2. According to SANS, the $bitmap is an attribute reserved for files marked as deleted, while $i30 is marked as an NTFS index attribute.
    [Show full text]
  • Openextensions POSIX Conformance Document
    z/VM Version 7 Release 1 OpenExtensions POSIX Conformance Document IBM GC24-6298-00 Note: Before you use this information and the product it supports, read the information in “Notices” on page 73. This edition applies to version 7, release 1, modification 0 of IBM z/VM (product number 5741-A09) and to all subsequent releases and modifications until otherwise indicated in new editions. Last updated: 2018-09-12 © Copyright International Business Machines Corporation 1993, 2018. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents List of Tables........................................................................................................ ix About This Document............................................................................................xi Intended Audience......................................................................................................................................xi Conventions Used in This Document.......................................................................................................... xi Where to Find More Information.................................................................................................................xi Links to Other Documents and Websites.............................................................................................. xi How to Send Your Comments to IBM....................................................................xiii Summary of Changes for z/VM
    [Show full text]