\

Undocumented DOS Second Edition A Programmer's Guide to Reserved MS-DOS® Functions and Data Structures

Andrew Schulman, Ralf Brown, David Maxey, Raymond J. Michels, and Jim Kyle

A . •• Addison-Wesley Publishing Company

Reading, Massachusetts Menlo Park, California New York Don Mills, Ontario Wokingham, England Amsterdam Bonn Sydney Singapore Tokyo Madrid San Juan Paris Seoul Milan Mexico City Taipei CONTENTS

CHAPTER 1 — Undocumented DOS: The Madness Continues 1 "Cruel Coding" and Tying Arrangements 3 Windows and DR DOS 5 Systems Rivalry and Smoking Guns 6 The Windows AARD Detection Code 7 A Gauntlet of Tests 9 A Gratuitous Gatekeeper 1 3 Does Beta Gode Really Matter? 15 So What? 16 Microsoft's Response 16 Documentation vs. Tying 1 7 Microsoft Windows Uses Undocumented DOS 18 WIN.COM Walks the SFT 20 BlockDevand INT2Fh Function 13h 21 DOSMGR: Windows' Connection to Undocumented DOS 21 CON CON CON CON CON 24 The Undocumented DOSMGR Callout API.:•...: 24 Implementing DOSMGR Functions 30 Patching DOS 32 DOS Knows About Windows 34 DOSMGR and the SDA 35 DOSMGR and the InDOS Flag 35 SYSTEM.INI Settings and Undocumented DOS 36 KRNL386 Grows the SFT 37 - KRNL386 and the PSP 37 Undocumented DOS and the Utilities Wars 38 Undocumented SmartDrive .-. 39 Undocumented DoubleSpace ?.. 40 UndocumentedEMM386.EXE 42 Microsoft Anti-Virus 43 No Problem? '. 45 DOS Documented 48 Why Leave Functionality Undocumented? 50 Documentation and Monopoly 52 Fear of Undocumented DOS 54 Ain't Misbehavin' 55 CHAPTER 2 — Programming for Documented and Undocumented DOS: A Comparison 59 Using Documented DOS Functions 60 DOS Calls from Assembly Language 61

III iv UNDOCUMENTED DOS, Second Edition

CHAPTER 2 (continued) DOS Calls from C 62 int86() 62 Inline Assembler 63 Register Pseudo-Variables 64 DOS Library Functions 65 DOS Calls from Turbo Pascal 65 DOS Calls from BASIC 66 Using Undocumented DOS 67 Disassembling DOS 68 Using the Interrupt List 70 No Magic Numbers 70 Undocumented DOS Calls from Assembly Language 72 DOS Versionitis 74 Accessing SysVars 75 Undocumented DOS Calls from C 76 What, No Structures? 78 Undocumented DOS Calls from Turbo Pascal 82 Undocumented DOS Calls from BASIC 84 When Not To Use Undocumented Features 85 Verifying Undocumented DOS 86 Making Modifications 87 An Important Special Case: Novell NetWare 93 Hooking DOS: Application Wrappers 95 On to Protected Mode 99 CHAPTER 3 — Undocumented DOS Meets Windows 101 Calling Undocumented DOS from Windows 103 It Doesn't Really Work! 108 The Dreaded GP Fault 110 A DPMI Shell 11 3 Trying Out Undocumented DOS from DPMISH Programs 11 7 The Windows DOS Extenders 119 Inside the DOSMGR DOS Extender 122 How DOSMGR Handles Undocumented DOS Calls / 126 Do Your Own XLAT .'. 128 DPMI Programming 128 Hiding DPMI .- 131 Fixing SFTWALK ; 138 Inside the DPMI Server in VMM 141 Back to Windows Programming 143 Windows and the SFT 143 Walking the Device Chain 144 Truename 148 Windows and the PSP 151 Peeking at DOS Boxes from a Windows Program 155 A Brief Introduction to VxD Programming 173 Timing DOS Calls 176 Contents v

CHAPTER 4 — Other DOSs: From DR DOS and NetWare to MVDMs in OS/2 and Windows NT 179 From CP/M to DR DOS to Novell DOS 181 The DR DOS Version Number 184 Undocumented Novell DOS 185 Watching DR DOS 189 Disassembling DR DOS 189 How Close Is DR DOS to MS-DOS? 190 SysVars, the Current Directory Structure, and the Redirector 191 The System File Tables and SHARE 192 Memory Control Blocks 192 TSRs and the Swappable Data Area 194 Additional DR DOS and Novell DOS Functionality 194 Novell NetWare 195 NETXandlNT21h 196 NetWare 4.0 and the Network Redirector 197 How NETX Changes INT 21 h 198 Undocumented NetWare 204 OS/2 2.x: "A Better DOS Than DOS"? 205 MVDMs and VDDs 207 So What Version of DOS Is This DOS Emulation Pretending To Be?. 208 Loading a Genuine DOS 210 OS/2 2.x and Undocumented DOS 211 New OS/2 Services for Old DOS Programs 213 DOS Emulation Under Windows NT 216 The Client/Server Model 217 NTVDM, NTIO, and NTDOS 218 Magic Pills and Bops 219 WhatisNTVDM.EXE? 220 DOS 5.50 222 Additional NTDOS Functionality 224 Undocumented NT 225 CHAPTER 5 — INTRSPY: A Program for Exploring DOS 229 Why a Script-Driven, Event-Driven Debugger? .- 229 A Guided Tour. L 230 Device Drivers 234 Watching XMS „ 237 Dynamic Hooks !". 238 INTRSPY User's Guide 241 UsinglNTRSPY.EXE 241 UsingCMDSPY.EXE 241 Script Language 241 Syntax 242 INCLUDE Syntax 242 STRUCTURE Syntax 242 INTERCEPT Syntax 243 GENERATE Syntax 245 RUN Syntax 245 REPORT, STOP and RESTART Syntax 246 Vi UNDOCUMENTED DOS, Second Edition

CHAPTER 5 (continued) DEBUG Syntax 246 Predefined Constants 247 Error Messages 247 GMDSPY Compilation Messages 247 CMDSPY and INTRSPY In Operation 248 INTRSPY Utility Scripts 248 UNDOC 248 LSTOFLST 251 Log Your Machine's Activity 253 Monitoring Disk I/O 254 MEM 258 Writing a Generic Interrupt Handler 258 The Problem with Intel's INT 260 Changed Implementation in INTRSPY 2.0 261 Implementation 261 Pitfalls I Fell In 262 The Future of INTRSPY 263 CHAPTER 6 — Disassembling DOS 265 What is MS-DOS? 266 Disassembling IO.SYS and MSDOS.SYS s. 267 Interrupt Vectors and Chaining 271 Tracing a DOS INT 21 h Call 280 Unassembling the Get/Set PSP Functions 287 Unassembling INT 21 h AH=33h 288 Examining the Low-Memory Stub for DOS=HIGH 289 Examining the INT 21 h Dispatch Function 291 Examining the INT 21 h Dispatch Table 297 Get SysVars and the Caller's Registers 299 A Very Brief Glance at File I/O 301 Tracing a DOS INT 2Fh Call 301 How Does DEBUG Trace Through an INT? 301 INTCHAIN 302 Examining The INT 2Fh Chain /. 306 The MSDOS.SYS and IO.SYS INT 2Fh Handlers 308 Examining the MSDOS.SYS Handler for INT 2Fh AH=12h 309 Locating the INT 2Fh AH=12h Dispatch Table 310 Really Disassembling DOS :. 313 Using NICEDBG 315 Examining a Few DOS Functions 324 Examining the DOS Lseek Function 326 Other Parts of DOS 332 Am I Going to Jail for This? 333 Use the Source, Luke! 336 Microsoft's DOS OEM Adaptation Kit (OAK) 340 CHAPTER 7 — MS-DOS Resource Management: Memory, Processes, Devices 343 Memory Management 343 Memory Control Blocks 344 The HMA and UMBs 346 Contents vii

CHAPTER 7 (continued) Making Use of UMBs., 347 The High Memory Area 349 How To Find the Start of the MCB Chain 350 How To Trace the MCB Chain 351 MCB Consistency Checks 354 A More Detailed UDMEM Program 356 Allocation Precautions 364 RAM Allocation Strategies 365 First-fit Strategy 366 Best-fit Strategy 366 Last-fit Strategy 366 Process Management 367 Program Files and Processes 367 The COM File Format 367 The EXE File Format 367 The PSP: How It Identifies a Process 368 History, Purpose, and Use 369 (Usually) Unique Process Identifier 369 Undocumented Areas of the PSP '. 370 DOS Termination Address 371 Other PSP Fields 372 Spawning Child Processes 373 Locating Parent Processes 373 Locating Ancestors 373 Use of this Capability 373 Device Management 374 Why Device Drivers Exist...., 374 Hardware-Dependent Details 375 Logically Required Functions 375 Congruence of Files and Devices 375 Tracing the Driver Chain 376 Organization of the Device Driver Chain 377 How Drivers Are Initialized ..377 Locating the Start of the Chain {. 378 Tracing It Through 378 Loading Device Drivers from the DOS Command Line 381 How DEVLOD Works f. 382 DEVLOD.C '.. 385 MOVUP.ASM 394 TESTNAME.ASM 395 CO.ASM 396 Make File 396 How Well Does DEVLOD Work? 397 CHAPTER 8 — The DOS File System and Network Redirector 401 A Quick Overview of the System 402 The DOS File System 407 Surfaces, Tracks, and Sectors 407 Partition and Boot Records 408 viii UNDOCUMENTED DOS, Second Edition r |

CHAPTER 8 (continued) The Boot Record and BIOS Parameter Block (BPB) 410 Logical Sector Numbers and the Cluster Concept 412 The File Allocation Table (FAT) 41 3 DOS Directory Structure 421 The Drive Parameter Block (DPB) 433 Buffers and Disk Caches 436 The Current Directory Structure (CDS) 443 Contents of the CDS 446 Walking the CDS Array 448 Detecting RAM Disks 449 DoubleSpace Drives 450 Stacker Drives 458 Novell NetWare Drives : 461 Manufacturing and Removing Drive Letters '. 462 System File Tables (SFTs) and ]ob File Table (]FT) 465 How Many FILES? 469 Filename From Handle 472 What Files Are Now Open? 475 Releasing Orphaned File Handles 482 More File Handles 485 System FCBs 489 The SHARE Hooks 490 The MS-DOS Network Redirector 494 Using the Network Redirector Interface 496 Front-End Hooks and Device Drivers vs. Back-End Redirectors 497 What DOS Provides : 498 What a Redirector Must Supply 503 Tracing an Open, Revisited 503 The Phantom 508 Phantom Implementation 509 Initializing the CDS 509 The Redirector INT 2Fh Handler 511 How Do We Know the Call Is for Us? ,: 51 3 Handling a Read 515 The Phantom XMS File System 51 7 Handling an Open 519 Handling Chdir 522 Handling Mkdir 523 Differences Between DOS Versions 525 The Network Redirector Specification 526 Using DOS Internal Functions 534 The Future of the DOS File System 539 CHAPTER 9 — Memory Resident Software: Pop-ups and Multitasking 541 TSR: It Sounds Like a Bug, But It's a Feature 542 Where Does Undocumented DOS Come In? 544 MS-DOS TSRs 547 The Generic TSR 548 Contents ; jx

CHAPTER 9 (continued) TSR Programming in Microsoft and Borland C/C++ 550 Keeping a C Program Resident 553 Not Going Resident.... 555 Jiggling the Stack 556 DOS Functions for TSRs 558 MS-DOS Flags 558 Get/Set PSP 560 • Extended Error Information 561 Extended Break Information 563 Interrupt 28h 564 Inside the Generic TSR 565 TSR Command Line Arguments 583 Writing TSRs with the DOS Swappable Data Area 584 TSRs and Task Managers 592 Removing a TSR 601 Sample TSR Programs 604 TSRFILE 604 TSRMEM 605 TSR2E 607 Multitasking TSR 609 Task Switching 610 MULTI Installation 610 Timer Interrupt 610 Idle Interrupt ..611 Keyboard Interrupt 611 Printing .'. 611 MULTI.C 611 CHAPTER 10 — Command Interpreters 619 lnsideCOMMAND.COM 620 Requirements of a Command Interpreter 622 Obtaining Operator Input 622 The DOS Prompt 623 Keystrokes .-. 623 Batch Files ' 623 Batch Enhancers and Compilers 624 Interpreting Operator Requests 626 Parsing for Inclusion in the PSP 626 SWITCHAR 627 Command Line Redirection and Pipes 627 Distinguishing Internal and External Commands 630 Finding and Executing Internal Commands 631 Dispatching Appropriate Processes 631 Locating and Loading External Commands 631 Dealing with BAT Files 632 Dealing with COM and EXE Files 632 The Exitcode Idea .633 Installable Commands 634 TSHELL, a Simple Command Interpreter 638 UNDOCUMENTED DOS, Second Edition

CHAPTER 10 (continued) How COMMAND.COM Works 641 Why Shells Are Their Own Parents 642 How and Why COMMAND.COM Reloads Itself 642 The Division Points 643 Resident, Initial, and Transient Portions 643 Where These Portions Are Loaded 644 Why Does F3 Sometimes Quit Working? 645 DOSKEY and How It Works 647 Using the Environment 648 How C0MMAND.COM Uses the Environment 648 Locating the Environment 649 Other Ways of Locating the Environment 651 Finding the "Active" Environment 656 Searching the Environment 656 INT 2Eh, the Back Door 659 The Function 659 Its Use 660 Alternatives to COMMAND.COM 663 4D0S.COM 663 A Total Replacement, Plus More 664 No Undocumented Features 664 Sample Program: An Environment Editor 665 Conclusion 674 APPENDIX — Undocumented DOS Functions and Data Structures 675 Acknowledgment 675 Sample Entry 675 GLOSSARY 831 ANNOTATED BIBLIOGRAPHY 835 INDEX 847