4620-1 ch05.f.qc 10/28/99 12:00 PM Page 157
Chapter 5 Troubleshooting TCP/IP
In This Chapter Troubleshooting TCP/IP in Windows 2000 Server TCP/IP troubleshooting steps Defining which is the best TCP/IP troubleshooting tool to solve your problem Mastering basic TCP/IP utilities
roubleshooting is, it seems, an exercise in matrix mathematics. That is, Twe use a variety of approaches and influences to successfully solve our problems, almost in a mental columns-and-rows format. These approaches may include structured methodologies, inductive and deductive reasoning, common sense, experience, and luck. And this is what troubleshooting is made of. Troubleshooting TCP/IP problems is really no different from troubleshooting other Windows 2000 Server problems mentioned in this book, such as instal- lation failures described in Chapter 2. Needless to say, Windows 2000 Server offers several TCP/IP-related tools and utilities to assist us, but more on the specifics in a moment.
TCP/IP Troubleshooting Basics The goal in TCP/IP troubleshooting is very simple: fix the problem. Too often, it is easy to become overly concerned about why something happened instead of just fixing the problem. And by fixing the problem, I mean cost effectively. 1. Be cost effective. Don’t forget that several hours’ worth of MCSE-level consulting could more than pay for the additional bandwidth that may easily solve your TCP/IP WAN-related problem. Don’t overlook such an easy fix when struggling to make a WAN connection between sites utilizing Windows 2000 Server. Too little bandwidth is typically the result of being penny wise and pound foolish. Oh, and it also causes nasty timeout conditions that can wreak havoc on your TCP/IP-based network. And if you want to make a complex database unhappy, just give it too little bandwidth on a TCP/IP-based WAN. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 158
158 Part II: TCP/IP
2. Experience is the best teacher. One of the more challenging corporate training assignments I frequently face is when I’m asked to deliver a custom TCP/IP and Windows 2000 Server troubleshooting session. The challenge is this: I’m not sure I can teach troubleshooting. It’s really just something you do and you’re ultimately skilled at it or not. TCP/IP troubleshooting ability is heavily based on experience. The good news is that the more time on the computer you put in (“stick time”), the better you will do. 3. Use inductive reasoning. Microsoft officially recommends pursuing a TCP/IP troubleshooting strategy of working from the bottom up, such as starting at the Physical Layer of the Open Standards Interconnections (OSI) model and proceeding to look at more broader influences such as the applications you are running. This enables you to isolate a problem. Such an approach is also known as induction or inductive reasoning, which Webster’s New World Dictionary defines as “a bringing forward of separate facts or instances, esp. so as to prove a general statement.” This mindset is largely the basis for this chapter, as individual tools and utilities that in reality would be used independently to solve a larger TCP/IP-related problem will be discussed. That is, you would start with a specific tool to solve a discrete problem and, as troubleshooting both goes and grows, resolve more global TCP/IP issues. In contrast, deductive reasoning is really better suited for the Windows 2000 Server developers in feature-set brainstorming sessions where the whole idea is to come up with great new features and then work down to the implementation specifics. Webster’s defines deduction as “Logic — the act or process of deducing; reasoning from a known principle to an unknown, from a general to the specific.” 4. Use the in-house help. Many wonderful tools are included in Windows 2000 Server for use in your TCP/IP troubleshooting efforts. These include native commands and utilities such as IPConfig and ping that will be reviewed in this chapter. And given that Windows 2000 Server is often bundled with the full version of BackOffice, don’t forget the full-featured version of Network Monitor included in Microsoft Systems Management Server (SMS). You will recall that tools such as Network Monitor were discussed at length in Part VI of this book, “Optimizing and Troubleshooting Windows 2000 Server.” 5. Don’t forget third-party tools. Not surprisingly, a wide range of third-party TCP/IP troubleshooting tools is available to assist you. One favorite, which will be discussed in Chapter 9, is PingPlotter, a low-cost shareware application from Richard Ness at Nessoft (www.nesssoft.com). PingPlotter is included on the CD-ROM that accompanies this book. This application tests ping connectivity and measures ping performance across WAN hops. 6. Always reboot. Last, but certainly not least, you must always reboot when modifying anything related to the TCP/IP protocol stack in Windows 2000 Server. Even though the “stack” has improved dramatically, I still don’t trust it completely. In my eyes, there is nothing like a complete reboot where you shut the computer down for 15 seconds after you’ve modified 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 159
Chapter 5: Troubleshooting TCP/IP 159
any TCP/IP protocol settings. And it’s an easy lesson to overlook! Here’s why. Let’s assume you switch your IP address from a dynamic DHCP- assigned address to a static IP address. So far, so good. But if at this moment, you run the IPConfig command that reports basic TCP/IP configuration information (discussed later in this chapter), you will note that the TCP/IP configuration information reports the new, updated IP address as if it were properly bound to the network adapter. Don’t you believe it for a minute! Always reboot. In fact, if you want my $59.95’s worth, I’d highly recommend you follow Step Zero — that is to completely cold-reboot your Windows 2000 Server prior to concluding you have any problems with TCP/IP. Don’t ask me why, but I’ve seen many Windows 2000 Server TCP/IP-related gremlins disappear this way. And that’s something you won’t read about in the official MCSE study guides. Trust me. First Step: Ask the Basic Questions So where do you go from here? Remember that troubleshooting any problem is a function of asking enough questions. Here is a short list of questions you can start your TCP/IP troubleshooting journey with. It is by no means inclusive. What’s working? What’s not working? What is the relationship between the things that work and the things that don’t? Did the things that don’t work now ever work on this computer or network? If the answer is yes, what has changed since they last worked? You can ask more specific questions in your quest to resolve your TCP/IP problems. These questions are presented and answered at the end of the chapter.
Second Step: Define the Tools Having completed this first step, you’re ready to begin troubleshooting TCP/IP in Windows 2000 Server. Table 5-1 provides a list of TCP/IP diagnostic utilities and troubleshooting tools, many of which will be discussed further in this chapter. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 160
160 Part II: TCP/IP
Table 5-1 Windows 2000 Server TCP/IP Troubleshooting Tools and Utilities Utility/Tool Description ARP Address Resolution Protocol. Enables you to view local computer ARP table entries to detect invalid entries. Hostname Typing this at the command line returns the current host name of the local computer. IPConfig Current TCP/IP information is displayed. Command line switches enable you to release and/or renew your IP address. Nbtstat Connections using NetBIOS over TCP/IP and protocol statistics are displayed. The LMHOSTS cache is updated (purged and reloaded). Netstat Active TCP/IP connections are displayed in addition to TCP/IP statistics. Nslookup Internet domain name servers are queried and recorded; domain host aliases, domain host services, and operating system information is returned. Ping Packet Internet Gopher. Tests connections and verifies configurations. Route Displays, prints, or modifies a local routing table. Tracert Checks the route from the local to a remote system. FTP File Transfer Protocol. This tool is used for two-way file transfers between hosts. TFTP Trivial File Transfer Protocol. Provides another form of two- way file transfer between hosts. Typically used when one host demands TFTP. I’ve used this in conjunction with router configuration and troubleshooting scenarios. Telnet Basic terminal emulation program that establishes a session with another TCP/IP host running a Telnet host. RCP Remote Copy Protocol. Enables you to copy files between TCP/IP-based hosts. RSH Remote Shell. Enables you to be authenticated by and run UNIX commands on a remote UNIX host. Rexec Enables you to be authenticated by and run processes on a remote computer. Finger System information is retrieved from a remote computer running TCP/IP and supporting the Finger command. Microsoft Internet Explorer Browser used for locating information and retrieving resources from the Internet. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 161
Chapter 5: Troubleshooting TCP/IP 161
Two important TCP/IP-related “tools” that are missing in Windows 2000 Server are native NFS client support and the whois command. For NFS client support, as discussed in Part VI of this book, “Optimizing and Troubleshooting Windows 2000 Server,” check with NetManage or WRQ, two independent software vendors that provide NFS client solutions for Windows 2000 Server. Typing whois at the Windows 2000 Server command line results in the following error message:
‘whois’ is not recognized as an internal or external command, operable program or batch file. In a moment, in the Telnet discussion, I will share a secret for using the whois command with Windows 2000 Server. Before going any further, it is important to establish your troubleshooting paradigm. Consider the following as you work with TCP/IP and read the remainder of this chapter. The following are considered TCP/IP diagnostic commands: ARP, hostname, IPConfig, nbtstat, netstat, ping, route, and tracert. These are considered connectivity commands: Finger, FTP, RCP, rexec, RSH, Telnet, and TFTP. Did you know that FTP, rexec, and Telnet not only use but also rely on clear- text passwords in a Windows 2000 scenario? That’s a huge departure from Windows 2000 Server’s basic reliance on encrypted password-based security. Be sure to think about this little fact the next time you use these tools. Third Step: Use the Tools Now the details. Having read about the TCP/IP troubleshooting tools and utilities found in Windows 2000 Server, you’re now ready to learn the finer points. This section includes an array of TCP/IP tools including: IPConfig FTP Ping TFTP ARP Telnet Nbtstat RCP Route RSH Netstat Rexec Tracert Finger Hostname Microsoft Internet Explorer But before using the tools, I want to spend a moment discussing how to “learn” the variables, command line entries, and switches associated with each tool. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 162
162 Part II: TCP/IP
The next several pages detail each of the suggested TCP/IP tools and utilities you may use in your troubleshooting efforts. Don’t forget that you may “capture” any command line details in Windows 2000 Server by redirecting the screen output to a text file. This is accomplished by appending your command line statement with the pipe or mathematical “greater than” sign (>). Observe:
C:\> dir >foo.txt
This command would direct the directory contents listing to the file foo.txt. That is a file that could easily be read in Notepad or another text editor. Be sure to use a filename without spaces when you redirect screen output to a text file. If you took the preceding example and directed the output to the filename “foo one.txt,” it would be stored under Windows 2000 Server as “foo” with no additional attributes such as the “.txt” extension. That’s problematic when you’ve created output with similar names such as “foo.txt” and “foo one.txt.” Later, when you try to open your important output with Notepad or WordPad, the filenames look nearly identical. Be careful when naming your output files from the command line. Be sure to use contiguous filenames such as “fooone.txt” to distinguish your filenames. Otherwise, as shown in Figure 5-1, the filenames are difficult to separate.
Figure 5-1: Incorrect naming of output files
IPConfig The IPConfig command line utility provides a baseline view of where your system is with respect to TCP/IP. Host TCP/IP connection parameters are verified, and you may observe whether the TCP/IP configuration has properly initialized. It is a good first step to take because it enables us to check the TCP/IP configuration on the computer having the alleged problem. There are several variations of the IPConfig command. These are implemented as command line switches: 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 163
Chapter 5: Troubleshooting TCP/IP 163
Windows 2000 IP Configuration USAGE: ipconfig [/? | /all | /release [adapter] | /renew [adapter] | /flushdns | /registerdns] | /flushdns | /registerdns | /showclassid adapter | /setclassid adapter [classidtoset] ] adapter Full name or pattern with ‘*’ and ‘?’ to ‘match’, * matches any character, ? matches one character. Options /? Display this help message. /all Display full configuration information. /release Release the IP address for the specified adapter. /renew Renew the IP address for the specified adapter. /flushdns Purges the DNS Resolver cache. /registerdns Refreshes all DHCP leases and re-registers DNS names /displaydns Display the contents of the DNS Resolver Cache. /showclassid Displays all the dhcp class IDs allowed for adapter. /setclassid Modifies the dhcp class id. The default is to display only the IP address, subnet mask and default gateway for each adapter bound to TCP/IP. For Release and Renew, if no adapter name is specified, then the IP address leases for all adapters bound to TCP/IP will be released or renewed. For SetClassID, if no class id is specified, then the class id is removed.
Examples: > ipconfig ... Show information. > ipconfig /all ... Show detailed information > ipconfig /renew ... renew all adapters > ipconfig /renew EL* ... renew adapters named EL.... > ipconfig /release *ELINK?21* ... release all matching adapters, eg. ELINK-21, myELELINKi21adapter. The most robust view of IPConfig is with the /all switch. Information for each physically bound network adapter card, modem connections, and even virtual bindings are displayed:
Windows 2000 IP Configuration Host Name ...... : TCI1 Primary Domain Name . . . . : Main.local Node Type ...... : Broadcast IP Routing Enabled. . . . . : No WINS Proxy Enabled. . . . . : No DNS Suffix Search List. . . : Main.local Ethernet adapter Local Area Connection: Adapter Domain Name . . . . : DNS Servers ...... : 209.20.130.35 209.20.130.33 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 164
164 Part II: TCP/IP
Description ...... : Intel(R) PRO/100+ PCI Adapter Physical Address...... : 00-90-27-78-A9-CC DHCP Enabled...... : No IP Address...... : 209.20.232.11 Subnet Mask ...... : 255.255.255.0 Default Gateway ...... : 209.20.232.1 Primary WINS Server . . . . : 10.0.0.2 When interpreting this IPConfig output, you can decipher whether a duplicate IP address has been configured. If the subnet mask appears as 0.0.0.0 for a particular IP address, that indicates that the said address is a duplicate IP address. Likewise, if you dynamically assign IP addresses to your network via DHCP, you can determine if your network adapter was unable to obtain an IP address. This is observed when the IP address appears as 0.0.0.0. Take that last point to heart. Most TCP/IP problems I can recall having on a Windows 2000 network centered on duplicate or unobtainable IP addresses. Hopefully, you can avoid such a fate. Don’t forget to make use of “|more” when executing the IPConfig command at the Windows 2000 Server command line. Otherwise, the TCP/IP information will rapidly scroll past without stopping, causing you to mis-key TCP/IP configuration information. This command is especially critical when the /all switch is used with the IPConfig command and lots and lots of important TCP/IP configuration information is displayed. If your system reports appropriate TCP/IP configuration and connection information, such as that displayed previously, proceed to use the ping command. Remember that Windows 95/98 uses the winipcfg command in place of the IPConfig command. That distinction is important for both the MCSE certification exams and working with client and server operating systems in the field. Ping Assuming you’ve successfully executed and interpreted the preceding IPConfig command, you’re ready to employ the ping command. Ping is my friend. It’s a low-level command that anyone can execute, and thus it’s a command that I ask clients to try while I’m performing over-the-telephone diagnosis. The answer to whether you have ping connectivity is either yes or no. In layperson’s terms, ping is used to diagnose connection-related failures. By executing the ping command, you can determine whether a particular TCP/IP- based host is available and responding in a non-dysfunctional manner. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 165
Chapter 5: Troubleshooting TCP/IP 165
Technically, the ping command is transmitting Internet Control Message Protocol (ICMP) packets between two TCP/IP-based hosts. Remember that ICMP relates to session management and special communications between hosts. With ICMP, messages and errors regarding packet delivery are reported. Great stuff! The ping command has several command line switches that increase its functionality. These switches, listed here, may be observed by typing ping /? at the command line:
Usage: ping [-t] [-a] [-n count] [-l size] [-f] [-i TTL] [-v TOS] [-r count] [-s count] [[-j host-list] | [-k host-list]] [-w timeout] destination-list Options: -t Ping the specified host until stopped. To see statistics and continue - type Control- Break; To stop - type Control-C. -a Resolve addresses to hostnames. -n count Number of echo requests to send. -l size Send buffer size. -f Set Don’t Fragment flag in packet. -i TTL Time To Live. -v TOS Type Of Service. -r count Record route for count hops. -s count Timestamp for count hops. -j host-list Loose source route along host-list. -k host-list Strict source route along host-list. -w timeout Timeout in milliseconds to wait for each reply. Typing the basic ping command followed by a host IP address results in the following information, which indicates that basic, low-level TCP/IP connectivity has been established:
Pinging 209.20.232.10 with 32 bytes of data: Reply from 209.20.232.10: bytes=32 time=15ms TTL=128 Reply from 209.20.232.10: bytes=32 time=16ms TTL=128 Reply from 209.20.232.10: bytes=32 time=16ms TTL=128 Reply from 209.20.232.10: bytes=32 time=16ms TTL=128
Ping statistics for 209.20.232.10: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 15ms, Maximum = 16ms, Average = 15ms If the host is unreachable, the ping command fails, as shown by
Pinging 10.0.0.5 with 32 bytes of data: Request timed out. Request timed out. Request timed out. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 166
166 Part II: TCP/IP
Ping statistics for 10.0.0.5: Packets: Sent = 4, Received = 1, Lost = 3 (75% loss), Approximate round trip times in milli-seconds: Minimum = 0ms, Maximum = 0ms, Average = 0ms For the MCSE exam and other official purposes, follow the traditional six-step ping command food chain and its relationship to IPConfig (see Figure 5-2). I call this phenomenon of IPConfig and ping working together as being reunited!
Step One - Run Ipconfig on the local workstation. Observe TCP/IP configuration information. Step Two - Ping 127.0.0.1 (Loopback Address) Step Three - Ping 204.107.6.111
Windows® 2000 Server 204.107.6.200 *Ethernet
Step Four - Ping Router 204.107.7.112 204-107.6.112 Step Six - Ping 204.107.7.111 Step Five - Ping 204-107.7.112
Router Token Ring 204-107.7.112 204.107.7.111
Figure 5-2: Six steps to success using the IPConfig and ping commands
STEPS: To use the IPConfig and ping commands
Step 1. Run the IPConfig command on the local workstation and observe the TCP/IP configuration information. Step 2. Ping the internal loopback address to verify that TCP/IP is installed and configured correctly on the local host computer. This address is 127.0.0.1, a reserved address that can’t be used as a real IP address on a network. See Chapter 9 for more information. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 167
Chapter 5: Troubleshooting TCP/IP 167
Step 3. Ping the address of the local host computer to ensure that TCP/IP is working correctly. Here we are typically pinging the network adapter card(s). Step 4. Ping the IP address of the router or default gateway so that you know and verify that the router or default gateway is functioning correctly. This also ensures that you have a functional infrastructure in place to communicate with a local host on the local network or subnet. Step 5. Ping the distant router across a WAN link if appropriate. This is a step I’ve added to the traditional scenario that you may or may not see in the MCSE texts or exams. However, this is based on real-world experience. Often you can ping a remote router yet are unable to ping the desired remote host. That’s because something as simple as a return route may not be programmed (see the route command discussion later in this chapter). Step 6. Ping the IP address of the remote host. Success at this stage establishes that you can communicate through the remote router and that the remote host is functional.
Typically, I use the ping command in bankruptcy law fashion. What do I mean by that? I mean I work backward (remember that in U.S. bankruptcy law, one starts with Chapter 11 and moves backward to Chapter 7... get it?). So I first ping the remote host (Step 6) and work backward. This approach better typifies the real-world need to communicate with another workstation/ server/host somewhere. If that doesn’t work, I back up and ultimately try to find the source of failure. The ping command is a great command for testing your implementation of IP security in Windows 2000 Server. If you’ve correctly implemented IP security, even the ping command will fail between two hosts that are not allowed to speak with each other. Further discussion on IP security can be found in Chapter 13 of this book. For a really good time, use the ping command to test the Windows Sockets- based name resolution on your network. This is accomplished by pinging a host name. For example, on the Internet, I might ping the domain name of my ISP with the following command:
ping nwlink.com If I enjoy successful replies, then I know there is no problem with address resolution or the network connection. However, if the ping command using a host name isn’t successful but the ping command using the IP address is, then I know that I only have an address resolution problem, not a network connection problem. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 168
168 Part II: TCP/IP
A few general steps to consider when the ping command doesn’t work include the following: Reboot the computer after TCP/IP was installed or modified (see my comments earlier under “Always reboot” in the “TCP/IP Troubleshooting Basics” section). Check that the local host’s address is valid as displayed in the Internet Protocol (TCP/IP) Properties tab sheet under Local Area Connection Properties, found under Network and Dial-up Connections, if it is a static IP address (see Figure 5-3). If necessary, make sure Windows 2000 Server-based routing is operational and a link exists between routers. In other words, perhaps you’re having a telephone company-related communications problem.
Figure 5-3: Verifying the host static IP address
ARP The Address Resolution Protocol (ARP) cache is composed of both dynamic and static addresses. In reality, ARP is used several ways, but fundamentally, ARP maps an IP address to a hardware address. This role is defined in RFC 826. The hardware address is the Media Access Control (MAC), or physical address. This hardware address can best be obtained by viewing the “Physical Address” entry returned by the IPConfig command. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 169
Chapter 5: Troubleshooting TCP/IP 169
Other ways to obtain physical hardware addresses include using the install utility on the driver disk that comes with your network adapter card or trapping packets with Network Monitor. The former approach is good for workstations that might be running IPX/SPX or NetBEUI, and the latter is good for non-mainstream workstation implementations in a Windows 2000 Server environment such as Macintosh clients. See Chapters 19 and 21 for further discussions of these techniques. The bottom line on network communications is that hosts must ultimately know each other’s physical addresses to communicate on a network. Address resolution via ARP is the act of converting the host’s IP address to its physical address. In that context, ARP is responsible for gathering the hardware addresses on broadcast-based networks. When operating in a dynamic discovery mode, this is accomplished by ARP issuing a local broadcast of the remote IP address to discover the physical address of the remote host. Having obtained the physical address, it adds it to the ARP cache. In fact, for a given host, both the IP address and the physical address are stored as one entry in the ARP table, for example:
Interface: 10.0.0.2 on Interface 0x2 Internet Address Physical Address Type 10.0.0.3 00-aa-62-c6-08-00 static Interface: 131.107.6.171 on Interface 0x3 Internet Address Physical Address Type 131.107.6.88 00-60-97-ba-f1-25 static The ARP cache is always read for an IP-physical address mapping before any ARP-related request broadcasts are initiated. Dynamic ARP table entries are maintained for ten minutes. This ensures the freshest ARP resolution information at all times. Static ARP table entries are maintained until the machine is rebooted. Remember that caching enables ARP to operate effici- ently. If you did not have any ARP caching, your network would have far too many ARP-type broadcasts travelling to resolve IP addresses to physical addresses (as shown in Figure 5-4). Not a wise way to manage a network. ARP broadcasts appear as shown in Figure 5-4 when viewed by Network Monitor. Remember that Windows 2000 Server makes heavy use of address caching in its TCP/IP implementation. That’s a helpful hint to keep in mind when things aren’t going smoothly and you decided to reboot the server to refresh the address cache. So how does ARP resolve an IP address? Before I discuss the steps that ARP undertakes, remember that the IP addresses for any two hosts must be “resolved” before a communications session may be established. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 170
170 Part II: TCP/IP
Figure 5-4: ARP broadcasting
1. Any time one host tries to communicate with another, an ARP request is initiated. An example of such a communication would be the ping command. If the IP address to physical address entry exists in the local table, then the address is resolved. End of story. 2. However, if no entry exists in the ARP table that resolves the IP address to a physical address, ARP starts asking questions in the form of a broadcast packet (such as the packet traffic displayed in Figure 5-4). 3. Every host on the local network evaluates the broadcast and determines whether the IP address in the ARP packet is the same as its IP address. 4. When a match is found, the destination host sends an ARP reply packet back to the source host. Again, this is akin to the traffic shown in Figure 5-4. 5. Routers in scenarios that span multiple subnets participate in ARP- related events in the following manner (also outlined in Figure 5-5): The ARP broadcast is forwarded to the default gateway for evaluation. The ARP broadcast packet is forwarded yet again to a remote router if necessary. Once a session is established with the remote router, the source host sends the ICMP-based request (such as a ping) to the remote router. The remote router resolves the request by sending the ping command to the destination host. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 171
Chapter 5: Troubleshooting TCP/IP 171
1: Host wants to "ping" Host C. 4: Match is made. No entries for Host C exist Host C returns in the ARP table for Host A. ARP reply packet.
Host A Host B Host C 204.107.6.111 204.107.6.110 204.107.6.109 3: ARP broadcast packet is evaluated by Host B and forwarded
ARP *Ethernet Packet
5: ARP broadcast handled by 2: ARP broadcast routers when multiple subnets packet is sent. are involved.
Router Router 204-107.6.112 204-107.6.112
IBM Compatible
Figure 5-5: An ARP broadcasting scenario
Common ARP-related problems Two common problem areas are associated with ARP. First is the problem of duplicate addresses. ARP operates on the FIFO principle. That means ARP table entries are made on a first-come, first-served basis. Thus it is possible that if duplicate IP addresses accidentally exist on the network, the wrong IP-based host may reply and cause an incorrect IP address-physical address entry to be added to the ARP table. The ARP case study that follows incorporates elements of this ARP problem. Second, broadcast storms strike when subnet masks are invalid and countless ARP broadcasts looking for a host are sent in vain on the network. In essence, the numerous broadcast packets being sent are the storm. You and your users know the outcome as decreased network performance. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 172
172 Part II: TCP/IP
ARP case study This is the case of the naughty ISDN router. Malcontent as it was, this ISDN router at the headquarters of an athletic club chain was incorrectly causing duplicate IP address errors on the network irrespective of the IP addresses we assigned to the hosts. We’d receive “duplicate IP address” errors at the Windows 95 workstation, whether we assigned the IP address to the workstation dynamically or statically. What gives, we pondered? We were only dealing with two dozen or so workstations that, by all accounts, didn’t have duplicate IP addresses. Finally, a breakthrough arrived. Using the ARP command, we were able to trace the “bogus” IP address (and there were a bunch of ‘em) back to the ISDN router’s MAC address. In short, one MAC address was mapped to several IP addresses. Whenever a Windows 95 workstation attempted to acquire what should have been a valid IP address, it of course got the duplicate IP address error. ARP bailed us out that day and got us a new, correctly functioning router for the athletic club. Microsoft’s position on ARP Microsoft’s position on TCP/IP troubleshooting is this: After running IPConfig and ping, you should then test IP-to-MAC address resolution using ARP. The bottom line? If two hosts can’t ping each other, try running ARP commands to see if the host computers have the correct MAC addresses.
Nbtstat One of the apparent dilemmas in a Microsoft Windows 2000 environment is the resolution of NetBIOS names to IP addresses. This is handled several ways, including Dynamic DNS, WINS server queries, local cache resolution, broadcasts, and LMHOSTS and HOSTS lookup. If you want to drop under the hood, you have the nbtstat command. In addition to acting as a name resolution troubleshooting tool, it enables you to correct or remove preloaded name entries. Options for the nbtstat command are as follows:
NBTSTAT [ [-a RemoteName] [-A IP address] [-c] [-n] [-r] [-R] [-RR] [-s] [-S] [interval] ] -a (adapter status) Lists the remote machine’s name table given its name -A (Adapter status) Lists the remote machine’s name table given its IP address. -c (cache) Lists NBT’s cache of remote [machine]names and their IP addresses -n (names) Lists local NetBIOS names. That is, names are displayed that were registered locally on the system by the server and redirector services. -r (resolved) Lists names resolved by broadcast and via WINS -R (Reload) Purges and reloads the remote cache 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 173
Chapter 5: Troubleshooting TCP/IP 173
name table (LMHOSTS) -S (Sessions) Lists sessions table with the destination IP addresses -s (sessions) Lists sessions table converting destination IP addresses to computer NETBIOS names. -RR (ReleaseRefresh) Sends Name Release packets to WINs and then, starts Refresh RemoteName Remote host machine name. IP address Dotted decimal representation of the IP address. interval Redisplays selected statistics, pausing interval seconds between each display. Press Ctrl+C to stop redisplaying statistics.
In the following nbtstat example, I run the command with the -S switch to list the current NetBIOS sessions, complete with status and statistics:
Local Area Connection: Node IpAddress: [10.0.0.2] Scope Id: [] NetBIOS Connection Table Local Name State In/Out Remote Host Input Output
SECRETS2 <03> Listening ADMINISTRATOR <03> Listening A trick you may use to ensure that you’re using a fresh local name cache is the nbtstat -r command. This command updates the local name cache immediately from such sources as the LMHOSTS file. Route The route command is discussed extensively in Chapter 5 in the context of IP gateways. However, a quick review is in order. Using the route command, you may view or modify the route table. The route table lists all current IP routes seen by the host. This includes routes that Windows 2000 Server creates automatically and routes learned by running the router information protocol (RIP). Common options for the route command are shown in Table 5-2.
Table 5-2 Common Options for the Route Command Command Function Route print Displays all current IP routes known by the host. Route add Used to add persistent and nonpersistent routes to the table. It is necessary to use the -p command line option with route add to create a persistent route. Otherwise, the route is lost when the machine is rebooted. Route delete Deletes routes from the table. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 174
174 Part II: TCP/IP
Netstat This command displays the current TCP/IP connections and protocol statistics. Options for the netstat command include the following:
NETSTAT [-a] [-e] [-n] [-s] [-p proto] [-r] [interval] -a Displays all connections and listening ports. -e Displays Ethernet statistics. This may be combined with the -s option. -n Displays addresses and port numbers in numerical form. -p proto Shows connections for the protocol specified by proto; proto may be TCP or UDP. If used with the -s option to display per-protocol statistics, proto may be TCP, UDP, or IP. -r Displays the routing table. -s Displays per-protocol statistics. By default, statistics are shown for TCP, UDP and IP; the -p option may be used to specify a subset of the default. interval Redisplays selected statistics, pausing interval seconds between each display. Press CTRL+C to stop redisplaying statistics. If omitted, netstat will print the current configuration information once.
Here is sample output from the netstat command using both the -e (Ethernet statistics), -a (all connections and listening ports), -r (route table), and -s (per-protocol statistics) command line options: netstat -e:
Interface Statistics Received Sent Bytes 291244 107280 Unicast packets 0 0 Non-unicast packets 1509 758 Discards 0 0 Errors 0 0 Unknown protocols 1128 netstat -a:
Active Connections Proto Local Address Foreign Address State TCP SECRETS2:echo SECRETS2:0 LISTENING TCP SECRETS2:discard SECRETS2:0 LISTENING TCP SECRETS2:daytime SECRETS2:0 LISTENING TCP SECRETS2:qotd SECRETS2:0 LISTENING TCP SECRETS2:chargen SECRETS2:0 LISTENING 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 175
Chapter 5: Troubleshooting TCP/IP 175
TCP SECRETS2:ftp SECRETS2:0 LISTENING TCP SECRETS2:name SECRETS2:0 LISTENING TCP SECRETS2:domain SECRETS2:0 LISTENING TCP SECRETS2:80 SECRETS2:0 LISTENING TCP SECRETS2:135 SECRETS2:0 LISTENING TCP SECRETS2:443 SECRETS2:0 LISTENING TCP SECRETS2:445 SECRETS2:0 LISTENING TCP SECRETS2:printer SECRETS2:0 LISTENING TCP SECRETS2:548 SECRETS2:0 LISTENING TCP SECRETS2:1025 SECRETS2:0 LISTENING TCP SECRETS2:1026 SECRETS2:0 LISTENING TCP SECRETS2:1028 SECRETS2:0 LISTENING TCP SECRETS2:1031 SECRETS2:0 LISTENING TCP SECRETS2:1034 SECRETS2:0 LISTENING TCP SECRETS2:1035 SECRETS2:0 LISTENING TCP SECRETS2:3389 SECRETS2:0 LISTENING TCP SECRETS2:5162 SECRETS2:0 LISTENING TCP SECRETS2:nbsession SECRETS2:0 LISTENING TCP SECRETS2:1027 SECRETS2:0 LISTENING TCP SECRETS2:nbsession SECRETS2:0 LISTENING UDP SECRETS2:echo *:* UDP SECRETS2:discard *:* UDP SECRETS2:daytime *:* UDP SECRETS2:qotd *:* UDP SECRETS2:chargen *:* UDP SECRETS2:name *:* UDP SECRETS2:135 *:* UDP SECRETS2:snmp *:* UDP SECRETS2:445 *:* UDP SECRETS2:1030 *:* UDP SECRETS2:1032 *:* UDP SECRETS2:1033 *:* UDP SECRETS2:9987 *:* UDP SECRETS2:domain *:* UDP SECRETS2:bootp *:* UDP SECRETS2:68 *:* UDP SECRETS2:nbname *:* UDP SECRETS2:nbdatagram *:* UDP SECRETS2:1029 *:* UDP SECRETS2:domain *:* UDP SECRETS2:bootp *:* UDP SECRETS2:68 *:* UDP SECRETS2:nbname *:* UDP SECRETS2:nbdatagram *:* netstat -r
======Interface List 0x1 ...... MS TCP Loopback interface 0x2 ...00 60 08 3a 04 cb ...... 3Com 3C90x Ethernet Adapter 0x3 ...00 60 97 bf a1 23 ...... ELNK3 Ethernet Adapter 0x4 ...00 53 45 00 00 00 ...... WAN (PPP/SLIP) Interface 0x5 ...00 53 45 00 00 00 ...... WAN (PPP/SLIP) Interface 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 176
176 Part II: TCP/IP
0x6 ...00 53 45 00 00 00 ...... WAN (PPP/SLIP) Interface 0x7 ...00 53 45 00 00 00 ...... WAN (PPP/SLIP) Interface ======Active Routes: Network Destination Netmask Gateway Interface Metric 10.0.0.0 255.0.0.0 10.0.0.2 10.0.0.2 1 10.0.0.2 255.255.255.255 127.0.0.1 127.0.0.1 1 10.255.255.255 255.255.255.255 10.0.0.2 10.0.0.2 1 127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1 1 131.107.6.0 255.255.255.0 131.107.6.171 131.107.6.171 1 131.107.6.171 255.255.255.255 127.0.0.1 127.0.0.1 1 131.107.255.255 255.255.255.255 131.107.6.171 131.107.6.171 1 224.0.0.0 224.0.0.0 10.0.0.2 10.0.0.2 1 224.0.0.0 224.0.0.0 131.107.6.171 131.107.6.171 1 255.255.255.255 255.255.255.255 10.0.0.2 10.0.0.2 1 ======Route Table Active Connections Proto Local Address Foreign Address State netstat -s:
IP Statistics Packets Received = 1646 Received Header Errors = 0 Received Address Errors = 685 Datagrams Forwarded = 0 Unknown Protocols Received = 0 Received Packets Discarded = 0 Received Packets Delivered = 997 Output Requests = 748 Routing Discards = 0 Discarded Output Packets = 0 Output Packet No Route = 0 Reassembly Required = 0 Reassembly Successful = 0 Reassembly Failures = 0 Datagrams Successfully Fragmented = 0 Datagrams Failing Fragmentation = 0 Fragments Created = 0 ICMP Statistics Received Sent Messages 12 6 Errors 0 0 Destination Unreachable 0 0 Time Exceeded 0 0 Parameter Problems 0 0 Source Quenchs 0 0 Redirects 0 0 Echos 0 0 Echo Replies 0 0 Timestamps 0 0 Timestamp Replies 0 0 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 177
Chapter 5: Troubleshooting TCP/IP 177
Address Masks 0 0 Address Mask Replies 0 0 TCP Statistics Active Opens = 0 Passive Opens = 0 Failed Connection Attempts = 0 Reset Connections = 0 Current Connections = 0 Segments Received = 0 Segments Sent = 0 Segments Retransmitted = 0 UDP Statistics Datagrams Received = 963 No Ports = 34 Receive Errors = 0 Datagrams Sent = 729 Tracert A route tracing utility, tracert utilizes the IP TTL field and ICMP error messages to discover host-to-host routes through the network. Options for the tracert command include the following:
Usage: tracert [-d] [-h maximum_hops] [-j host-list] [-w timeout] target_name Options: -d Do not resolve addresses to hostnames. -h maximum_hops Maximum number of hops to search for target. -j host-list Loose source route along host-list. -w timeout Wait timeout milliseconds for each reply. Here is sample output from the tracert command:
Tracing route to SECRETS2 [131.107.6.171] over a maximum of 30 hops: 1 <10 ms <10 ms <10 ms SECRETS2 [131.107.6.171] Trace complete. Hostname This is a very simple but useful command. It returns the NetBIOS computer name for the machine on which the command was executed. The hostname command is a quick and dirty way to find out (or remember!) which machine you are working on. This command line utility eliminates the need to find similar information via the MMC. It’s a favorite command of mine in part because of its simplicity. The hostname command and its output appear as
C:\>hostname SECRETS2 where SECRETS2 is the actual host name. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 178
178 Part II: TCP/IP
You may not change the host name via the hostname command. The -s option is not supported in Windows 2000 Server. You must use the Network applet in Control Panel to change the host name, followed by a reboot.
FTP This command remains very popular with Windows 2000 Server users. This is an important point because other TCP/IP processes from the same era (such as Gopher) have declined. Using Port 21, FTP is the basic command for transferring information from one Internet host to another. There are several “variations” of this command, two of which I will show you — the command line method and the Internet Explorer method. First, there is the command line version contained within Windows 2000 Server, which provides very basic, character-based two-way file transfer capabilities. Here is sample output with the FTP command using the fully qualified domain name:
C:\>ftp secrets2 Connect to SECRETS2. 220 SECRETS2 Microsoft FTP Service (Version 5.0) User (SECRETS2:(none)): anonymous 331 Anonymous access allowed, send identity (e-mail name) as password. Password: 230 Anonymous user logged in. ftp> Note that this session could have been initiated by using the IP address for SECRETS2 (for example, 10.0.0.2). The return codes listed in the preceding FTP session, along with all of the other FTP return codes, may be found at http://andrew2.andrew._cmu. edu/rfc/rfc640.html. Table 5-3 shows the commands that may be used during an FTP session such as the one just displayed. Note these commands are listed by typing “?” at the FTP prompt. You will note that most of these commands openly reveal their UNIX heritage.
Table 5-3 Commands for Use in an FTP Session ! debug ls put status ? dir mdelete pwd trace append disconnect mdir quit type ascii get mget quote user bell glob mkdir recv verbose bye hash mls remotehelp cd help mput rename 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 179
Chapter 5: Troubleshooting TCP/IP 179
close lcd open rmdir delete literal prompt send
Table 5-4 provides explanations of the more common FTP commands.
Table 5-4 Common FTP Commands Command Description ! Spawns an MS-DOS shell, but FTP remains active. Typing exit returns the user to the FTP prompt. !command Executes an MS-DOS command inside the FTP session on the local computer. Bye Terminates or ends the FTP session. Delete With appropriate permissions, files are deleted on the remote computer. Dir Lists the remote directory’s files and subdirectories. Get Copies a file to the local computer from a remote computer. Help Displays FTP command descriptions. Put Copies files from the local computer to the remote computer. Mkdir With appropriate permissions, enables you to create a directory on a remote computer.
The key to using the command-line FTP command in Windows 2000 Server is that the host you are “FTP-ing” to will accept your request and initiate a session. FTP management is configured via the FTP service in Microsoft Internet Information Server (IIS), which is included with Windows 2000 Server. Note that changing the default TCP port value used by FTP is one way to create a more secure FTP server site. Both hosts must agree to use the same TCP port value to initiate a session. Using a non-default TCP port can thwart intruders. Also, consider how you might really use this tool. Basically, I’ve used FTP for low-level file transfers when I don’t have an NFS-based solution to communicate with true UNIX hosts and Windows 2000 Server. Specifically, I once used FTP to transfer files from Sun workstations to a Windows 2000 Server for storage and printing. The FTP service in Microsoft Internet Information Server (IIS) is shown in Figure 5-6. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 180
180 Part II: TCP/IP
Figure 5-6: Managing the FTP service via the Internet Information Server application
TFTP Operating on Port 69 by default, Trivial File Transfer Protocol is a variation of FTP that is used to transfer files to and from a remote computer. I’ve used TFTP to transfer files from a Windows 2000 Server machine to a Cisco router. Whatever works! Here are the TFTP commands available to you:
TFTP [-i] host [GET | PUT] source [destination] -i Specifies binary image transfer mode (also called octet). In binary image mode the file is moved literally, byte by byte. Use this mode when transferring binary files. host Specifies the local or remote host. GET Transfers the file destination on the remote host to the file source on the local host. PUT Transfers the file source on the local host to the file destination on the remote host. source Specifies the file to transfer. destination Specifies where to transfer the file. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 181
Chapter 5: Troubleshooting TCP/IP 181
Telnet Telnet is of course a basic terminal emulation feature that enables you to establish a terminal-mode session with another host. You might, for instance, use Telnet to establish a session with your Windows 2000 Server over the Internet. Another valid use is programming a router, either internally or externally, via the Internet. When executing the Telnet command, you may save an extra step by appending the Telnet command with the IP address or host name of the server you intend to log on to. Such a command would appear as:
C:> telnet nwlink.com
Note that nwlink.com is the host name. Telnet is used each and every day. Honest. When I’m at a remote site that is connected to the Internet, I like to Telnet back to my ISP to check my e-mail. Such a session involves issuing the Telnet command with the fully qualified domain name of my ISP and then launching pine, a character-based e-mail program that is hosted by my ISP (see Figures 5-7 and 5-8).
Figure 5-7: Using the Telnet utility to access an ISP from a remote location 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 182
182 Part II: TCP/IP
Figure 5-8: Remote access of e-mail via Telnet and other e-mail applications
Because Windows 2000 Server doesn’t natively support the whois command, you should use the following trick to add this powerful command to your arsenal. Telnet to a bonafide UNIX server on the Internet. Your ISP should be your first choice. Next, issue the whois command at the UNIX command prompt as described next. Why use the whois command? To spy on thy Internet neighbor, of course! Just kidding, but the whois command enables you to see who sent that junk mail by performing the whois command against the e-mail’s Internet domain name to the right of the “@” symbol. More important, ISP customer service reps and perhaps you can use the whois command (see Figure 5-9) to see if a specific Internet domain name has been taken already (a.k.a. “registered”). At a minimum, the whois command returns important Internet domain registration information (see Figure 5-10). Note the whois command applies only to second-level (such as idgbooks.com), not third-level (springers._nwnexus.com) domain names. Note that in Windows 2000 Server, the Telnet screen with its limited size requires two screens to display the full Internet domain name registration information. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 183
Chapter 5: Troubleshooting TCP/IP 183
Figure 5-9: The whois command at the UNIX command prompt
Figure 5-10: Valuable Internet domain name registration information 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 184
184 Part II: TCP/IP
RCP The Remote Copy Protocol (RCP) enables you to copy files between TCP/IP- based hosts. Settings for this command include
RCP [-a | -b] [-h] [-r] [host][.user:]source [host][.user:] path\destination -a Specifies ASCII transfer mode. This mode converts the EOL characters to a carriage return for UNIX and a carriage return/line feed for personal computers. This is the default transfer mode. -b Specifies binary image transfer mode. -h Transfers hidden files. -r Copies the contents of all subdirectories; destination must be a directory. host Specifies the local or remote host. If host is specified as an IP address OR if host name contains dots, you must specify the user. .user: Specifies a user name to use, rather than the current user name. source Specifies the files to copy. path\destination Specifies the path relative to the logon directory on the remote host. Use the escape characters (\ , “, or ‘) in remote paths to use wildcard characters on the remote host. RSH This command launches a remote shell on a UNIX host. Settings for this command include the following:
RSH host [-l username] [-n] command host Specifies the remote host on which to run command. -l username Specifies the user name to use on the remote host. If omitted, the logged on user name is used. -n Redirects the input of RSH to NULL. command Specifies the command to run. Rexec This command enables you to run commands on remote hosts running the rexec service. Rexec authenticates the user name on the remote host before executing the specified command. Settings for this command include the following:
REXEC host [-l username] [-n] command host Specifies the remote host on which to run command. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 185
Chapter 5: Troubleshooting TCP/IP 185
-l username Specifies the user name on the remote host. -n Redirects the input of REXEC to NULL. command Specifies the command to run.
Finger This command displays information about a user on a specified system running the Finger service. Output varies based on the remote system. Settings for this command include the following:
FINGER [-l] [user]@host [...] -l Displays information in long list format. user Specifies the user you want information about. Omit the user parameter to display information about all users on the specified host. @host Specifies the server on the remote system whose users you want information about. Microsoft Internet Explorer Even though I have touted the benefits of a robust Internet browser such as Internet Explorer (IE) several times in this book, it’s worth repeating here. The “official” Microsoft party line is that IE is very much a troubleshooting tool for the TCP/IP protocol in Windows 2000 Server. That’s because IE increasingly supports TCP/IP protocol suite utilities such as FTP. This is a capability beyond the original browser, which basically has Hypertext Transfer Protocol (HTTP) support. In reality, I use IE every day to go up on the Internet and download resources to optimize my Windows 2000 Server installations.
Other TCP/IP Troubleshooting Angles Having reviewed the primary utilities that ship as part of the TCP/IP protocol suite in Windows 2000 Server, let’s explore a few other time-tested TCP/IP troubleshooting tricks.
Troubleshooting TCP/IP database files This section is written for the MCSE candidate in mind. In the real world, you and I rely on the GUI-interface presentation of TCP/IP information in Windows 2000 Server. However, whether you’re an old-timer in the industry or you’re trying to pass the demanding TCP/IP exams on the MCSE tracks, the files shown in Table 5-5 contain critical TCP/IP information. Everyone else can benefit by observing the file descriptions and contents. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 186
186 Part II: TCP/IP
Table 5-5 Windows 2000 Server UNIX-style Database Files File Name Description HOSTS Provides host name-to-IP address resolution for applications that are Windows Sockets-compliant. LMHOSTS Provides NetBIOS name-to-IP address resolution for Windows-based networking. Networks Provides network name-to-network ID resolution for TCP/IP management. Protocol Provides protocol name-to-protocol ID resolution for Windows Sockets applications. Services Provides service name-to-port ID resolution for Windows Sockets applications.
Sample output from the HOSTS file contained at \%systemroot%\system32\drivers\etc is as follows:
# Copyright (c) 1993-1999 Microsoft Corp. # # This is a sample HOSTS file used by Microsoft TCP/IP for Windows. # # This file contains the mappings of IP addresses to host names. Each # entry should be kept on an individual line. The IP address should # be placed in the first column followed by the corresponding host name. # The IP address and the host name should be separated by at least one # space. # # Additionally, comments (such as these) may be inserted on individual # lines or following the machine name denoted by a ‘#’ symbol. # # For example: # # 102.54.94.97 rhino.acme.com # source server # 38.25.63.10 x.acme.com # x client host
127.0.0.1 localhost Sample output from the LMHOSTS (lmhosts.sam) file contained at \%systemroot% \system32\drivers\etc is as follows:
# Copyright (c) 1993-1999 Microsoft Corp. # # This is a sample LMHOSTS file used by the Microsoft TCP/IP for Windows. # # This file contains the mappings of IP addresses to computernames # (NetBIOS) names. Each entry should be kept on an individual line. # The IP address should be placed in the first column followed by the # corresponding computername. The address and the computername # should be separated by at least one space or tab. The “#” character 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 187
Chapter 5: Troubleshooting TCP/IP 187
# is generally used to denote the start of a comment (see the exceptions # below). # # This file is compatible with Microsoft LAN Manager 2.x TCP/IP lmhosts # files and offers the following extensions: # # #PRE # #DOM:
188 Part II: TCP/IP
# # The following example illustrates all of these extensions: # # 102.54.94.97 rhino #PRE #DOM:networking #net group’s DC # 102.54.94.102 “appname \0x14” #special app server # 102.54.94.123 popular #PRE #source server # 102.54.94.117 localsrv #PRE #needed for the include # # #BEGIN_ALTERNATE # #INCLUDE \\localsrv\public\lmhosts # #INCLUDE \\rhino\public\lmhosts # #END_ALTERNATE # # In the above example, the “appname” server contains a special # character in its name, the “popular” and “localsrv” # server names are # preloaded, and the “rhino” server name is specified so # it can be used # to later #INCLUDE a centrally maintained lmhosts file # if the “localsrv” # system is unavailable. # # Note that the whole file is parsed including comments # on each lookup, # so keeping the number of comments to a minimum will # improve performance. # Therefore it is not advisable to simply add lmhosts file # entries onto the # end of this file. I’ve highlighted in bold the two most valuable lines from this sample file. First, the line 102.54.94.97 rhino #PRE #DOM:networking is an entry type that I’ve used to solve pesky resolution problems. Sometimes preloading the entry (the #PRE statement) and forcing a domain name for the host (#DOM) will solve nasty timeout conditions over slow WAN links. Been there, done that. The other interesting entry is the 102.54.94.102 “appname \0x14” line. What is occurring here, in English, is that the full 15 positions of the host name are being filled out or padded. This is necessary in some resolution scenarios. Most likely you will be working with Microsoft Technical Support when you get to the point at which this becomes necessary. As stated in the preceding sample file, the \0x14 represents nonprinting characters. In fact, one time that I can recall where I worked extensively with the LMHOSTS file was when I was troubleshooting the dickens out of a Microsoft Exchange performance problem across two domains. Upon reflection years later, I now see that this was an exercise in using TPC/IP tools to troubleshoot an integration problem between a Microsoft BackOffice component and the underlying Windows NT Server operating system. Such lofty insights, garnered from bumps, bruises, and general maturity with Windows NT Server, have made me a more effective network professional. I’m sure you will enjoy the same positive results from ascending both the Windows 2000 Server and TCP/IP learning curves. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 189
Chapter 5: Troubleshooting TCP/IP 189
If you want to convert the lmhosts.sam file from being a sample file to acting as your real lmhosts file, you will need to remove the .sam file extension. Sample output from the Networks file contained at \%systemroot% \system32\drivers\etc is as follows:
# Copyright (c) 1993-1999 Microsoft Corp. # # This file contains network name/network number mappings for # local networks. Network numbers are recognized in dotted # decimal form. # # Format: # #
loopback 127
Sample output from the Protocol file contained at \%systemroot% \system32\drivers\etc is as follows:
# Copyright (c) 1993-1999 Microsoft Corp. # # This file contains the Internet protocols as defined by RFC 1700 # (Assigned Numbers). # # Format: # #
ip 0 IP # Internet protocol icmp 1 ICMP # Internet control message protocol ggp 3 GGP # Gateway-gateway protocol tcp 6 TCP # Transmission control protocol egp 8 EGP # Exterior gateway protocol pup 12 PUP # PARC universal packet protocol udp 17 UDP # User datagram protocol hmp 20 HMP # Host monitoring protocol xns-idp 22 XNS-IDP # Xerox NS IDP rdp 27 RDP # “reliable datagram” protocol rvd 66 RVD # MIT remote virtual disk
Sample output from the Services file contained at \%systemroot% \system32\drivers\etc is as follows:
# Copyright (c) 1993-1999 Microsoft Corp. # # This file contains port numbers for well-known services # as defined by # RFC 1700 (Assigned Numbers). 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 190
190 Part II: TCP/IP
# # Format: # #
echo 7/tcp echo 7/udp discard 9/tcp sink null discard 9/udp sink null systat 11/tcp users #Active users systat 11/tcp users #Active users daytime 13/tcp daytime 13/udp qotd 17/tcp quote #Quote of the day qotd 17/udp quote #Quote of the day chargen 19/tcp ttytst source #Character generator chargen 19/udp ttytst source #Character generator ftp-data 20/tcp #FTP, data ftp 21/tcp #FTP. control telnet 23/tcp smtp 25/tcp mail #Simple Mail Transfer Protocol time 37/tcp timserver time 37/udp timserver rlp 39/udp resource #Resource Location Protocol nameserver 42/tcp name #Host Name Server nameserver 42/udp name #Host Name Server nicname 43/tcp whois domain 53/tcp #Domain Name Server domain 53/udp #Domain Name Server bootps 67/udp dhcps #Bootstrap Protocol Server bootpc 68/udp dhcpc #Bootstrap Protocol Client tftp 69/udp #Trivial File Transfer gopher 70/tcp finger 79/tcp http 80/tcp www www-http #World Wide Web kerberos-sec 88/tcp krb5 #Kerberos kerberos-sec 88/udp krb5 #Kerberos hostname 101/tcp hostnames #NIC Host Name Server iso-tsap 102/tcp #ISO-TSAP Class 0 rtelnet 107/tcp #Remote Telnet Service pop2 109/tcp postoffice #Post Office 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 191
Chapter 5: Troubleshooting TCP/IP 191
Protocol – Version 2 pop3 110/tcp #Post Office Protocol – Version 3 sunrpc 111/tcp rpcbind portmap #SUN Remote Procedure Call sunrpc 111/udp rpcbind portmap #SUN Remote Procedure Call auth 113/tcp ident tap #Identification Protocol uucp-path 117/tcp nntp 119/tcp usenet #Network News Transfer Protocol ntp 123/udp #Network Time Protocol epmap 135/tcp loc-srv #DCE endpoint resolution epmap 135/udp loc-srv #DCE endpoint resolution netbios-ns 137/tcp nbname #NETBIOS Name Service netbios-ns 137/udp nbname #NETBIOS Name Service netbios-dgm 138/udp nbdatagram #NETBIOS Datagram Service netbios-ssn 139/tcp nbsession #NETBIOS Session Service imap 143/tcp imap4 #Internet Message Access Protocol pcmail-srv 158/tcp #PCMail Server snmp 161/udp #SNMP snmptrap 162/udp snmp-trap #SNMP trap print-srv 170/tcp #Network PostScript bgp 179/tcp #Border Gateway Protocol irc 194/tcp #Internet Relay Chat Protocol ipx 213/udp #IPX over IP ldap 389/tcp #Lightweight Directory Access Protocol https 443/tcp MCom https 443/udp MCom microsoft-ds 445/tcp microsoft-ds 445/udp #? kpasswd 464/tcp # Kerberos (v5) #? kpasswd 464/udp # Kerberos (v5) isakmp 500/udp ike #Internet Key Exchange exec 512/tcp #Remote Process Execution biff 512/udp comsat login 513/tcp #Remote Login 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 192
192 Part II: TCP/IP
who 513/udp whod cmd 514/tcp shell syslog 514/udp printer 515/tcp spooler talk 517/udp ntalk 518/udp efs 520/tcp #Extended File Name Server router 520/udp route routed timed 525/udp timeserver tempo 526/tcp newdate courier 530/tcp rpc conference 531/tcp chat netnews 532/tcp readnews netwall 533/udp #For emergency broadcasts uucp 540/tcp uucpd klogin 543/tcp #Kerberos kshell 544/tcp krcmd #Kerberos remote shell new-rwho 550/udp new-who remotefs 556/tcp rfs rfs_server rmonitor 560/udp rmonitord monitor 561/udp ldaps 636/tcp sldap #LDAP over TLS/SSL doom 666/tcp #Doom Id Software doom 666/udp #Doom Id Software kerberos-adm 749/tcp #Kerberos administration kerberos-adm 749/udp #Kerberos administration kpop 1109/tcp #Kerberos POP phone 1167/udp #Conference calling ms-sql-s 1433/tcp #Microsoft-SQL- Server ms-sql-s 1433/udp #Microsoft-SQL- Server ms-sql-m 1434/tcp #Microsoft-SQL- Monitor ms-sql-m 1434/udp #Microsoft-SQL- Monitor wins 1512/tcp #Microsoft Windows Internet Name Service wins 1512/udp #Microsoft Windows Internet Name Service ingreslock 1524/tcp ingres l2tp 1701/udp #Layer Two Tunneling Protocol pptp 1723/tcp #Point-to-point tunnelling protocol 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 193
Chapter 5: Troubleshooting TCP/IP 193
radius 1812/udp #RADIUS authentication protocol radacct 1813/udp #RADIUS accounting protocol nfsd 2049/udp nfs #NFS server knetd 2053/tcp #Kerberos de-multiplexor ttcp 5001/tcp #TTCP ttcp 5001/udp #TTCP man 9535/tcp #Remote Man Server Reinstalling TCP/IP Clearly one trick that is always available is to uninstall and reinstall the TCP/IP protocol suite in Windows 2000 Server. I mentioned previously that the Microsoft TCP/IP protocol stack is a weaker stack, and unfortunately this holds true in the heat of battle on occasion (and it always seems to be at the enterprise level, not at my home basement lab!). So let’s assume that you’ve tried just about every TCP/IP troubleshooting approach mentioned in this chapter and nothing, absolutely nothing, is solving your problem. Time for drastic action. Simply stated, remove and reinstall the TCP/IP protocol stack. Correctly done, this approach will work wonders. But of course it isn’t that simple. When reinstalling the TCP/IP protocol suite, it’s entirely plausible that you will receive an error message that indicates “The Registry Subkey Already Exists.” The fix is obvious. To fully remove TCP/IP, you must remove the “embedded” Registry entries the protocol suite made during installation. Experienced Microsoft Exchange users will immediately recognize what’s going on here. To completely remove Microsoft Exchange from a Windows 2000 server, you must manually remove its related Registry entries. The same can be said for TCP/IP. The TCP/IP protocol suite and related services entries that should be removed from the Registry are as follows (note I’m assuming some services such as WINS have been installed, otherwise ignore the Registry references). Connectivity utilities Assuming you have removed the TCP/IP protocol “component,” then you must remove
HKEY_LOCAL_MACHINE\Software\Microsoft\NetBT HKEY_LOCAL_MACHINE\Software\Microsoft\Tcpip HKEY_LOCAL_MACHINE\Software\Microsoft\TcpipCU HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\DHCP HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\LMHosts HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\’NetDriver’\Parameters\Tcpip HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\NetBT 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 194
194 Part II: TCP/IP
SNMP service Assuming you have removed the SNMP service, you must remove
HKEY_LOCAL_MACHINE\Software\Microsoft\RFC1156Agent HKEY_LOCAL_MACHINE\Software\Microsoft\Snmp HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\Snmp TCP/IP network printing support These entries relate to the LPDSVC line printer components. You must remove
HKEY_LOCAL_MACHINE\Software\Microsoft\Lpdsvc HKEY_LOCAL_MACHINE\Software\Microsoft\TcpPrint HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\LpdsvcSimple TCP\IP Services The next two entries related to the simple TCP/IP service. You must remove
HKEY_LOCAL_MACHINE\Software\Microsoft\SimpTcp HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\SimTcp DHCP Server service Assuming you have removed the HCP Server service, you must remove
HKEY_LOCAL_MACHINE\Software\Microsoft\DhcpMibAgent HKEY_LOCAL_MACHINE\Software\Microsoft\DhcpServer HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\DhcpServer WINS Server service Assuming you have removed the WINS Server service, you must remove
HKEY_LOCAL_MACHINE\Software\Microsoft\Wins HKEY_LOCAL_MACHINE\Software\Microsoft\WinsMibAgent HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\Wins DNS Server service Assuming you have removed the DNS Server service, you must remove
HKEY_LOCAL_MACHINE\Software\Microsoft\Dns HKEY_LOCAL_MACHINE\Software\Microsoft\WinsMibAgent HKEY_LOCAL_MACHINE\SYSTEM\CCS\Services\Wins Few Windows 2000 Server professionals know about the required TCP/IP- related Registry housekeeping just detailed (removing Registry entries and so on). By following the preceding suggestions, you’ve set yourself apart from the 2000 pack! TCP/IP Q & A As promised, here are more specific TCP/IP troubleshooting-related questions and answers. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 195
Chapter 5: Troubleshooting TCP/IP 195
Is TCP/IP correctly installed on my Windows 2000 Server? This question isn’t as easy to answer as you might think, given the possibilities of corrupt TCP/IP protocol stack components that won’t easily reveal themselves during the moment of need. However, you can always stick to a few basics. First, ping the loopback address of 127.0.0.1. Assuming this worked and a reply occurred, you’re possibly home free. If it failed, however, observe the event logs and see what types of TCP/IP- related information were recorded in the system log. Second, ping another address (a host machine, another workstation, or such) and see if you receive a reply. If so, yet another TCP/IP-related hurdle has been passed. I receive an Error 53 when connecting to a server. What is it? Error 53 is returned when host name resolution fails. That’s the bottom line. Possible resolution paths include confirming that the host name is spelled correctly (such as in the UNC format) when you attempt resolution (this assumes of course that the other computer is running TCP/IP). The preceding advice is valid for a remote host located on the same or a different subnet. However, if you are crossing to another subnet, the resolution scenario becomes more complex. For mixed Windows 2000 Server networks, you might also check that the WINS database contains the same type of name-to-IP address mappings. Heck, if you’re really old fashioned, it wouldn’t hurt to see if the LMHOSTS file contains the appropriate name-to-IP address mapping entries. I’m relying on the LMHOSTS file for resolution. I’ve added a new name mapping, but I’m experiencing long connect times or timeout conditions. Why? Supposing you have a large LMHOSTS file (of course this would only occur at the enterprise level), your new entries may be too far down the list for speedy name resolution. Therefore, it is better for you to preload the entry with the #PRE command. This and other LMHOST file goodies were discussed earlier in this chapter. You may also place your mapping higher in the LMHOSTS file. I am having a difficult time connecting to a specific server. What gives? Run the nbtstat -n to determine without doubt what names, including the server you are seeking, are registered on the network. See the nbtstat section earlier in the chapter for more information. I cannot connect to a foreign host when using host names, but I can connect with IP addresses. Why? Simply stated, you’re having problems with DNS-related resolution. Check that the DNS Server setup for the TCP/IP protocol is correct. Make sure that the DNS addresses are correct and in the proper order. If you are using the HOSTS file, make sure that the remote computer name is spelled correctly with proper capitalization. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 196
196 Part II: TCP/IP
I’m communicating with a remote host, and the TCP/IP connection appears hung. I’ve confirmed this by observing TCP/IP-related errors in my error log. Why? Using the netstat -a command, you can observe the port activity for TCP and UDP. A good TCP connection typically has 0 bytes in the send/receive queues. Blocked data will reveal itself as a connection problem. If this is not the case, then you’re experiencing application delays or network problems. Don’t overlook the possibilities of application delay. There’s nothing like a client/server database suffering from contention problems that are disguised and appear as TCP/IP connection problems. If you’re an experienced network professional, you probably have similar epic stories. When using Telnet, the banner on the title bar isn’t correct, but I’m sure I’ve specified the correct IP address. Why? It’s important to verify that the DNS name and HOSTS table are current. Also, verify that no two computers have the same IP addresses on the same network. Imposter problems such as this are among the most difficult to track down and resolve. Using the arp -g command, you will see mapping from the ARP cache. This will display the Ethernet (MAC) address for the particular remote computer, possibly enabling you to delete the import entry with arp -d if you know the erroneous MAC address. After undertaking these steps, try pinging the remote host address, an action that forces an ARP. Finally, check the ARP table again with arp -g. I’ve received a message “Your default gateway does not belong to one of the configured interfaces.” How can I solve this? Basically, the default gateway doesn’t appear to be on the same logical network as the computer’s network adapter. This can be determined and resolved by comparing the network ID portion of the default gateway’s IP address with the network ID(s) of any of the computer’s network adapters. I can’t ping across a router when using TCP/IP as a RAS client. Why? The RAS Phonebook is the culprit here. If you have selected “Use default gateway on remote network” under the TCP/IP settings in the RAS Phonebook, you will have this problem. The resolution is this: Using the route add command, add the route of the subnet you’re attempting to connect to.
Additional TCP/IP Troubleshooting Resources Just a short note to wrap up TCP/IP troubleshooting. Remember that the event logs such as System and Application will assist your troubleshooting efforts, as will Microsoft TechNet, the online Microsoft Knowledge Base found at www.microsoft.com; Microsoft Technical Support; and Performance Monitor. If you use Performance Monitor, make sure that you’ve installed the SNMP service on your Windows 2000 Server so that you have the full suite of TCP/IP-related object:counters. 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 197
Chapter 5: Troubleshooting TCP/IP 197
Summary This chapter explored all the important topics of troubleshooting TCP/IP: Learning how to troubleshoot TCP/IP Understanding TCP/IP troubleshooting steps Understanding TCP/IP utilities and tools used for troubleshooting Selecting the best tool to solve your TCP/IP-related problem 4620-1 ch05.f.qc 10/28/99 12:00 PM Page 198