INTERNET TECHNOLOGY AND WEB PROGRAMMING
1 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 CONTENTS LESSON I: Introduction to Networking · Networking concepts and Technology (LANs and WANs) ...... · Serial Networking (SLIP, PPP) ...... · Internet Protocol (IP) and Domain Name System (DNS) ...... · What is the Internet ......
LESSON II: Internet Access Hardware and Media · HARDWARE: Modems, Terminal Adapters, Routers ...... · MEDIA: PTSN, ISDN, Kilostream ......
LESSON III: Internet Services · Electronic Mail; Newsgroups ...... · File Transfer Protocol (FTP) and Hypertext Transfer Protocol (HTTP) . . . . . · Internet databases: WAIS, Archie, gopher, WWW search databases . . . . .
LESSON IV: Using E-Mail and other Clients · Electronics Mail ...... · Other Internet Clients ...... · FTP ...... · Newsgroups ...... · Telnet ......
LESSON V: Media & Active Content · Object & Active Content ......
· Types of Browser Plug-ins ...... 2 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · Additional Media File Formats ...... · Images File Formats ......
LESSON VI: Internetworking Servers · Server Implementation ...... · Content Servers ...... · Performance Servers ...... · Database Servers ...... · Mirrored Servers ...... · Popular Server Products ......
LESSON VII: Web Servers and Databases · Databases ...... · Introduction to Database Gateways for Web Servers ...... · Common Gateway Interface (CGI) ...... · Server Application Programming Interfaces (SAPIs) ...... · JavaScript ...... · ASP ...... · PHP ...... · HTML ...... · Java & Java Service ...... · JSP ...... · ColdFusion ...... · Database Connectivity 3 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · ODBC ...... · JDBC ......
LESSON VIII: Internet Security · What is Security? ......
· The cracker Process ...... · Types of Attacks ...... · Defending Your Networks ...... · Firewalls ...... · Defending Your Computer ...... · Defending Your Transmitted Data ......
Lesson I: (Introduction to Networking)
1. Network concepts and Technology (LANs and WANs)
4 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 LANs, WANs, and Other Area Networks
Computer networks come in many different shapes and sizes. Over the years, the networking industry has coined terms like "LAN" and "WAN" attempting to define sensible categories for the major types of network designs. The precise meaning of this terminology remains lost on the average person, however.
Area Networks
For historical reasons, the industry refers to nearly every type of network as an "area network." The most commonly-discussed categories of computer networks include the following -
· Local Area Network (LAN)
· Wide Area Network (WAN)
· Metropolitan Area Network (MAN)
· Storage Area Network (SAN)
· System Area Network (SAN)
· Server Area Network (SAN)
· Small Area Network (SAN)
· Personal Area Network (PAN)
· Desk Area Network (DAN)
· Controller Area Network (CAN)
· Cluster Area Network (CAN)
LANs and WANs were the original flavors of network design. The concept of "area" made good sense at this time, because a key distinction between a LAN and a WAN involves the physical distance that the network spans. A third category, the MAN, also fit into this scheme as it too is centered on a distance-based concept.
5 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · LAN Basics
A LAN connects network devices over a relatively short distance. A networked office building, school, or home usually contains a single LAN, though sometimes one building will contain a few small LANs, and occasionally a LAN will span a group of nearby buildings. In IP networking, one can conceive of a LAN as a single IP subnet (though this is not necessarily true in practice).
Besides operating in a limited space, LANs include several other distinctive features. LANs are typically owned, controlled, and managed by a single person or organization. They also use certain specific connectivity technologies, primarily Ethernet and Token Ring.
· WAN Basics
As the term implies, a wide-area network spans a large physical distance. A WAN like the Internet spans most of the world! A WAN is a geographically-dispersed collection of LANs. A network device called a router connects LANs to a WAN. In IP networking, the router maintains both a LAN address and a WAN address.
WANs differ from LANs in several important ways. Like the Internet, most WANs are not owned by any one organization but rather exist under collective or distributed ownership and management. WANs use technology like ATM, Frame Relay and X.25 for connectivity.
LANs and WANs at Home
Home net workers with cable modem or DSL service already have encountered LANs and WANs in practice, though they may not have noticed. A cable/DSL router like those in the Links family join the home LAN to the WAN link maintained by one's ISP. The ISP provides a WAN IP address used by the router, and all of the computers on the home network use private LAN addresses. On a home network, like many LANs, all computers can communicate directly with each other, but they must go through a central gateway location to reach devices outside of their local area.
What About MAN, SAN, PAN, DAN, and CAN?
Future articles will describe the many other types of area networks in more detail. After LANs and WANs, one will most commonly encounter the following three network designs: A Metropolitan Area Network (MAN) connects an area larger than a LAN but smaller than a WAN, such as a city, with dedicated or high-performance hardware. A Storage Area Network (SAN) connects servers to data storage devices through a technology like Fibre Channel. A System Area Network (SAN) connects high-performance computers with high-speed connections in a cluster configuration.
6 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Conclusion
To the uninitiated, LANs, WANs, and the other area network acronyms appear to be just more alphabet soup in a technology industry already drowning in terminology. The names of these networks are not nearly as important as the technologies used to construct them, however. A person can use the categorizations as a learning tool to better understand concepts like subnets, gateways, and routers.
A Simple Computer Network for File Sharing
Illustration: 1
This diagram illustrates the simplest possible kind of computer network. In a simple network, two computers (or other networkable devices) make a direct connection with each and communicate over a wire or cable. Simple networks like this have existed for decades. A common use for these networks is file sharing.
A Local Area Network (LAN)
7 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Illustration: 2
This diagram illustrates a typical local area network (LAN) environment. Local area networks often feature a group of computers located in a home, school, or part of an office building. Like a simple network, computers on a LAN share files and printers. Computers on one LAN can also share connections with other LANs and with the internet.
A Hypothetical Wide Area Network
Illustration: 3
This diagram illustrates a hypothetical wide area network (WAN) configuration that joins LANs in three metropolitan locations. Wide area networks cover a large geographic area like a city, a country or multiple countries. WANs normally connect multiple LANs and other smaller-scale area networks. WANs are built by large telecommunication companies and other corporations using highly-specialized equipment not found in consumer stores. The Internet is an example of a WAN that joins local and metropolitan area networks across most of the world.
8 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Wiring in Computer Networks
Illustration: 4
This diagram illustrates s e v e r a l c o m m o n forms of wiring in computer networks. In many homes, twisted-pair Ethernet cables are often used to connect computers. Phone or cable TV lines in turn connect the home LAN to the Internet Service Provider (ISP). ISPs, larger schools and businesses often stack their computer equipment in racks (as shown), and they use a mix of different kinds of cable to join this equipment to LANs and to the Internet. Much of the Internet uses high-speed fiber optic cable to send traffic long distances underground, but twisted pair and coaxial cable can also be used for leased lines and in more remote areas.
Network Topologies Bus, ring, star, and all the rest
In networking, the term topology refers to the layout of connected devices on a network. This article introduces the standard topologies of computer networking.
Topology in Network Design
One can think of a topology as a network's "shape" . This shape does not necessarily correspond to the actual physical layout of the devices on the network. For example, the computers on a home LAN may be arranged in a circle, but it would be highly unlikely to find an actual ring topology there.
Network topologies are categorized into the following basic types:
· bus
· ring
· star 9 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · tree
· mesh
More complex networks can be built as hybrids of two or more of the above basic topologies.
Illustration: 5
Bus Topology diagram
Bus Topology
Bus networks (not to be confused with the system bus of a computer) use a common backbone to connect all devices. A single cable, the backbone functions as a shared communication medium, that devices attach or tap into with an interface connector. A device wanting to communicate with another device on the network sends a broadcast
10 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 message onto the wire that all other devices see, but only the intended recipient actually accepts and processes the message.
Ethernet bus topologies are relatively easy to install and don't require much cabling compared to the alternatives. 10Base-2 ("ThinNet") and 10Base-5 ("ThickNet") both were popular Ethernet cabling options years ago. However, bus networks work best with a limited number of devices. If more than a few dozen computers are added to a bus, performance problems will likely result. In addition, if the backbone cable fails, the entire network effectively becomes unusable.
Ring Topology Diagram
Illustration: 6
Ring Topology
In a ring network, every device has exactly two neighbors for communication purposes. All messages travel through a ring in the same direction (effectively either "clockwise" or "counterclockwise"). A failure in any cable or device breaks the loop and can take down the entire network.
To implement a ring network, one typically uses FDDI, SONET, or Token Ring technology. Rings are found in some office buildings or school campuses.
Star Topology Diagram
11 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Illustration: 7
Star Topology
Many home networks use the star topology. A star network features a central connection point called a "hub" that may be an actual hub or a switch. Devices typically connect to the hub with Unshielded Twisted Pair (UTP) Ethernet.
Compared to the bus topology, a star network generally requires more cable, but a failure in any star network cable will only take down one computer's network access and not the entire LAN. (If the hub fails, however, the entire network also fails.)
Tree Topology Diagram
Illustrati on: 8
12 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Tree Topology
Tree topologies integrate multiple star topologies together onto a bus. In its simplest form, only hub devices connect directly to the tree bus, and each hub functions as the "root" of a tree of devices. This bus/star hybrid approach supports future expandability of the network much better than a bus (limited in the number of devices due to the broadcast traffic it generates) or a star (limited by the number of hub ports) alone.
Mesh Topology Diagram
Illustration: 9
Mesh Topology
Mesh topologies involve the concept of routes. Unlike each of the previous topologies, messages sent on a mesh network can take any of several possible paths from source to destination. (Recall that in a ring, although 13 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 two cable paths exist, messages can only travel in one direction.) Some WANs, like the Internet, employ mesh routing.
Conclusion
Topologies remain an important part of network design theory. You can probably build a home or small business network without understanding the difference between a bus design and a star design, but understanding the concepts behind these gives you a deeper understanding of important elements like hubs, broadcasts, ports, and routes
Serial Networking (SLIP)
What is SLIP?
The Shared Land Information Platform is a shared information delivery service which provides fast and easy access to the state’s spatial information. SLIP is the platform connecting WA Government spatial information. Driven by the Department of Land Information in Western Australia (DLI), SLIP is a ground-breaking project revolutionizing the way government spatial (land and property) information is used and shared, providing numerous benefits to government, business, industry and the community.
SLIP delivers online real-time access to spatial information in a seamless cross-Government manner, thereby overcoming the agency run-around currently experienced by industry and the public. Spatial information underpins and links a range of government activities, including planning, land use and development, environmental sustainability and emergency management. SLIP makes it easier to locate and use this information.
The implementation plan for SLIP was developed and endorsed by the WA Government in November 2004. SLIP involves the engagement of over 200 people across more than 20 State Government agencies, as well as local government, community groups and industry bodies. SLIP works with WALIS (WA Land Information System) to facilitate widespread engagement and collaboration.
SLIP is the platform connecting WA Government Spatial Information, with the endorsed program-of-work also pursuing four cross-agency whole-of-government business opportunities - emergency management, natural resource management, land development and register of interests.
The SLIP platform is implemented using an enabling framework built on current DLI infrastructure. The platform has been funded to connect fifteen (15) State Government agencies, provide access to a minimum of 60 data services (with access to up to 200 data services available when all business opportunities are fully implemented), together with services for security, management, metering, cataloguing and viewing of data.
SLIP Benefits
14 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 SLIP facilitates improvement of business processes and systems which use spatial information across networked government. Through the use of SLIP, Government agencies, local government, community groups, industry and private citizens will have a single point of access to spatial information. This improved information access will facilitate greater sharing of information amongst diverse groups; more consistent use of information; resulting in better management decisions; and the ability to more effectively "value-add" new information using a range of information sources.
Systems using spatial information will be able to be implemented with a reduced infrastructure overhead and at lower cost than otherwise previously achievable. Data integrity will improve through access to the single and most recent "authorities" source of spatial information, in an online real-time basis. The right information can be made available to the right people at the right time.
1. Serial Networking (PPP)
The Point-to-Point Protocol (PPP) originally emerged as an encapsulation protocol for transporting IP traffic over point-to-point links. PPP also established a standard for the assignment and management of IP addresses asynchronous (start/stop) and bit-oriented synchronous encapsulation, network protocol multiplexing, link configuration, link quality testing, error detection, and option negotiation for such capabilities as network layer address negotiation and data-compression negotiation. PPP supports these functions by providing an extensible Link Control Protocol (LCP) and a family of Network Control Protocols (NCPs) to negotiate optional configuration parameters and facilities. In addition to IP, PPP supports other protocols, including Novell's Internet work Packet Exchange (IPX) and DECnet.
PPP Components
PPP provides a method for transmitting diagrams over serial point-to-point links. PPP contains three main components:
• A method for encapsulating datagram’s over serial links. PPP uses the High-Level Data Link Control (HDLC) protocol as a basis for encapsulating diagrams over point-to-point links. "Synchronous Data Link Control and Derivatives," forzaxyzaxy more information on HDLC.)
• An extensible LCP to establish, configure, and test the data link connection.
• A family of NCPs for establishing and configuring different network layer protocols. PPP is designed to allow the simultaneous use of multiple network layer protocols.
General Operation
To establish communications over a point-to-point link, the originating PPP first sends LCP frames to configure and (optionally) test the data link. After the link has been established and optional facilities have been negotiated as needed by the LCP, the originating PPP sends NCP frames to choose and configure one or more network layer protocols. When each of the chosen network layer protocols has been configured, packets from each network layer 15 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 protocol can be sent over the link. The link will remain configured for communications until explicit LCP or NCP frames close the link, or until some external event occurs (for example, an inactivity timer expires or a user intervenes).
Physical Layer Requirements
PPP is capable of operating across any DTE/DCE interface. Examples include EIA/TIA-232-C (formerly RS-232-C), EIA/TIA-422 (formerly RS-422), EIA/TIA-423 (formerly RS-423), and International Telecommunication Union Telecommunication Standardization Sector (ITU-T) (formerly CCITT) V.35. The only absolute requirement imposed by PPP is the provision of a duplex circuit, either dedicated or switched, that can operate in either an asynchronous or synchronous bit-serial mode, transparent to PPP link layer frames. PPP does not impose any restrictions regarding transmission rate other than those imposed by the particular DTE/DCE interface in use.
PPP Link Layer
PPP uses the principles, terminology, and frame structure of the International Organization for Standardization (ISO) HDLC procedures (ISO 3309-1979), as modified by ISO 3309:1984/PDAD1 "Addendum 1: Start/Stop Transmission." ISO 3309-1979 specifies the HDLC frame structure for use in synchronous environments. ISO 3309:1984/PDAD1 specifies proposed modifications to ISO 3309-1979 to allow its use in asynchronous environments. The PPP control procedures use the definitions and control field encodings standardized in ISO 4335-1979 and ISO 4335-1979/Addendum 1-1979.
Six Fields Make Up the PPP Frame
Illustration: 10 T h e f ol lo w in g d escriptions summarize the PPP frame fields. • Flag—A single byte that indicates the beginning or end of a frame. The flag field consists of the binary sequence 01111110.
• Address—A single byte that contains the binary sequence 11111111, the standard broadcast address. PPP does not assign individual station addresses.
• Control—A single byte that contains the binary sequence 00000011, which calls for transmission of user data in an subsequences frame. A connectionless link service similar to that of Logical Link Control (LLC) Type 1 is provided. (For more information about LLC types and frame types, refer to Chapter 16.)
• Protocol—Two bytes that identify the protocol encapsulated in the information field of the frame. The most up-to-date values of the protocol field are specified in the most recent Assigned Numbers Request For Comments (RFC).
16 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 • Data—Zero or more bytes that contain the datagram for the protocol specified in the protocol field. The end of the information field is found by locating the closing flag sequence and allowing 2 bytes for the FCS field. The default maximum length of the information field is 1,500 bytes. By prior agreement, consenting PPP implementations can use other values for the maximum information field length.
• Frame check sequence (FCS)—normally 16 bits (2 bytes). By prior agreement, consenting PPP implementations can use a 32-bit (4-byte) FCS for improved error detection.
The LCP can negotiate modifications to the standard PPP frame structure. Modified frames, however, always will be clearly distinguishable from standard frames.
PPP Link-Control Protocol
The PPP LCP provides a method of establishing, configuring, maintaining, and terminating the point-to-point connection. LCP goes through four distinct phases. First, link establishment and configuration negotiation occur. Before any network layer datagram’s (for example, IP) can be exchanged, LCP first must open the connection and negotiate configuration parameters. This phase is complete when a configuration-acknowledgment frame has been both sent and received.
This is followed by link quality determination. LCP allows an optional link quality determination phase following the link-establishment and configuration-negotiation phase. In this phase, the link is tested to determine whether the link quality is sufficient to bring up network layer protocols. This phase is optional. LCP can delay transmission of network layer protocol information until this phase is complete. At this point, network layer protocol configuration negotiation occurs. After LCP has finished the link quality determination phase, network layer protocols can be configured separately by the appropriate NCP and can be brought up and taken down at any time. If LCP closes the link, it informs the network layer protocols so that they can take appropriate action. Finally, link termination occurs. LCP can terminate the link at any time. This usually is done at the request of a user but can happen because of a physical event, such as the loss of carrier or the expiration of an idle-period timer.
Three classes of LCP frames exist. Link-establishment frames are used to establish and configure a link. Link-termination frames are used to terminate a link, and link-maintenance frames are used to manage and debug a link. These frames are used to accomplish the work of each of the LCP phases.
· Internet Protocols
The Internet protocols are the world's most popular open-system (nonproprietary) protocol suite because they can be used to communicate across any set of interconnected networks and are equally well suited for LAN and WAN communications. The Internet protocols consist of a suite of communication protocols, of which the two best known are the Transmission Control Protocol (TCP) and the Internet Protocol (IP). The Internet protocol suite not only includes lower-layer protocols (such as TCP and IP), but it also specifies common applications such as electronic mail, terminal emulation, and file transfer. This chapter provides a broad introduction to specifications that comprise the Internet protocols. Discussions include IP addressing and key upper-layer protocols used in the Internet. Specific routing protocols are addressed individually later in this document. 17 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Internet protocols were first developed in the mid-1970s, when the Defense Advanced Research Projects Agency (DARPA) became interested in establishing a packet-switched network that would facilitate communication between dissimilar computer systems at research institutions. With the goal of heterogeneous connectivity in mind, DARPA funded research by Stanford University and Bolt, Beranek, and Newman (BBN). The result of this development effort was the Internet protocol suite, completed in the late 1970s.
TCP/IP later was included with Berkeley Software Distribution (BSD) UNIX and has since become the foundation on which the Internet and the World Wide Web (WWW) are based. Documentation of the Internet protocols (including new or revised protocols) and policies are specified in technical reports called Request For Comments (RFCs), which are published and then reviewed and analyzed by the Internet community. Protocol refinements are published in the new RFCs. To illustrate the scope of the Internet protocols, maps many of the protocols of the Internet protocol suite and their corresponding OSI layers. This chapter addresses the basic elements and operations of these and other key Internet protocols.
Internet protocols span the complete range of OSI model layers.
Illustration: 11
18 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Internet Protocol (IP)
The Internet Protocol (IP) is a network-layer (Layer 3) protocol that contains addressing information and some control information that enables packets to be routed. IP is documented in RFC 791 and is the primary network-layer protocol in the Internet protocol suite. Along with the Transmission Control Protocol (TCP), IP represents the heart of the Internet protocols. IP has two primary responsibilities: providing connectionless, best-effort delivery of datagram’s through an internetwork; and providing fragmentation and reassembly of datagram’s to support data links with different maximum-transmission unit (MTU) sizes.
IP Packet Format
Illustration: 12
19 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Fourteen fields comprise an IP packet.
The following discussion describes the IP packet fields illustrated in :
• Version—indicates the version of IP currently used.
• IP Header Length (IHL)—Indicates the datagram header length in 32-bit words.
• Type-of-Service—specifies how an upper-layer protocol would like a current datagram to be handled, and assigns datagram’s various levels of importance.
• Total Length—specifies the length, in bytes, of the entire IP packet, including the data and header.
• Identification—contains an integer that identifies the current datagram. This field is used to help piece together datagram fragments.
• Flags—consist of a 3-bit field of which the two low-order (least-significant) bits control fragmentation. The low-order bit specifies whether the packet can be fragmented. The middle bit specifies whether the packet is the last fragment in a series of fragmented packets. The third or high-order bit is not used.
• Fragment Offset—indicates the position of the fragment's data relative to the beginning of the data in the original datagram, which allows the destination IP process to properly reconstruct the original datagram.
• Time-to-Live—maintains a counter that gradually decrements down to zero, at which point the datagram is discarded. This keeps packets from looping endlessly.
20 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 • Protocol—Indicates which upper-layer protocol receives incoming packets after IP processing is complete.
• Header Checksum—helps ensure IP header integrity.
• Source Address—specifies the sending node.
• Destination Address—specifies the receiving node.
• Options—Allows IP to support various options, such as security.
• Data—Contains upper-layer information.
IP Addressing
As with any other network-layer protocol, the IP addressing scheme is integral to the process of routing IP datagrams through an internetwork. Each IP address has specific components and follows a basic format. These IP addresses can be subdivided and used to create addresses for subnetworks, as discussed in more detail later in this chapter.
Each host on a TCP/IP network is assigned a unique 32-bit logical address that is divided into two main parts: the network number and the host number. The network number identifies a network and must be assigned by the Internet Network Information Center (InterNIC) if the network is to be part of the Internet. An Internet Service Provider (ISP) can obtain blocks of network addresses from the InterNIC and can itself assign address space as necessary. The host number identifies a host on a network and is assigned by the local network administrator.
IP Address Format
The 32-bit IP address is grouped eight bits at a time, separated by dots, and represented in decimal format (known as dotted decimal notation). Each bit in the octet has a binary weight (128, 64, 32, 16, 8, 4, 2, 1). The minimum value for an octet is 0, and the maximum value for an octet is 255.
Illustration: 13
An IP address consists of 32 bits, grouped into four octets.
21 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 IP Address Classes
IP addressing supports five different address classes: A, B, C, D, and E. Only classes A, B, and C are available for commercial use. The left-most (high-order) bits indicate the network class. Provides reference information about the five IP address classes.
Illustration: 14
Reference Information About the Five IP Address Classes
High- IP Order No. Bits Address Bit(s) Address Range Network/Host M a x . Class Format Purpose Hosts
A N.H.H.H Few large 0 1.0.0.0 to 7/24 16777214 1 organizations 126.0.0.0 2 (224 - 2)
B N.N.H.H Medium-size 1, 0 128.1.0.0 to 14/16 6 5 5 3 4 organizations 191.254.0.0 (216 - 2)
C N.N.N.H R e l a t i v e l y 1, 1, 0 192.0.1.0 to 21/8 254 (28 - s m a l l 223.255.254.0 2) organizations
D N/A M u l t i c a s t 1, 1, 224.0.0.0 to N/A (not for N/A groups (RFC 1, 0 239.255.255.255 c o m m e r c i a l 1112) use)
E N/A Experimental 1, 1, 240.0.0.0 to N/A N/A 1, 1 254.255.255.255
N= network number, H= host number.
One address is reserved for the broadcast address, and one address is reserved for the network.
Illustrates the format of the commercial IP address classes. (Note the high-order bits in each class.)
22 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 The class of address can be determined easily by examining the first octet of the address and mapping that value to a class range in the following table. In an IP address of 172.31.1.2, for example, the first octet is 172. Because 172 falls between 128 and 191, 172.31.1.2 is a Class B address. Summarizes the range of possible values for the first octet of each address class.
Illustration: 15
A range of possible values exists for the first octet of each address class.
IP Subnet Addressing 23 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 IP networks can be divided into smaller networks called subnetworks (or subnets). Subnetting provides the network administrator with several benefits, including extra flexibility, more efficient use of network addresses, and the capability to contain broadcast traffic (a broadcast will not cross a router). Subnets are under local administration. As such, the outside world sees an organization as a single network and has no detailed knowledge of the organization's internal structure.
A given network address can be broken up into many subnetworks. For example, 172.16.1.0, 172.16.2.0, 172.16.3.0, and 172.16.4.0 are all subnets within network 171.16.0.0. (All 0s in the host portion of an address specifies the entire network.)
IP Subnet Mask
A subnet address is created by "borrowing" bits from the host field and designating them as the subnet field. The number of borrowed bits varies and is specified by the subnet mask. Shows how bits are borrowed from the host address field to create the subnet address field.
Illustration: 16
Bits are borrowed from the host address field to create the subnet address field.
Subnet masks use the same format and representation technique as IP addresses. The subnet mask, however, has binary 1s in all bits specifying the network and subnetwork fields, and binary 0s in all bits specifying the host field.
Illustration: 17 A sample subnet mask consists of all binary 1s and 0s.
24 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Subnet mask bits should come from the high-order (left-most) bits of the host field, as illustrates. Details of Class B and C subnet mask types follow. Class A addresses are not discussed in this chapter because they generally are subnetted on an 8-bit boundary. Illustration: 18
Subnet mask bits come from the high-order bits of the host field.
Vari ous types of subnet masks exist for Class B and C subnets. The default subnet mask for a Class B address that has no subnetting is 255.255.0.0, while the subnet mask for a Class B address 171.16.0.0 that specifies eight bits of subnetting is 255.255.255.0. The reason for this is that eight bits of subnetting or 28 - 2 (1 for the network address and 1 for the broadcast address) = 254 subnets possible, with 28 - 2 = 254 hosts per subnet.
The subnet mask for a Class C address 192.168.2.0 that specifies five bits of subnetting is 255.255.255.248.With five bits available for subnetting, 25 - 2 = 30 subnets possible, with 23 - 2 = 6 hosts per subnet. The reference charts shown in table 30-2 and table 30-3 can be used when planning Class B and C networks to determine the required number of subnets and hosts, and the appropriate subnet mask.
Illustration: 19
25 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Class B Subnetting Reference Chart
Number of Number of Number of Bits Subnet Mask Subnets Hosts
2 255.255.192.0 2 16382
3 255.255.224.0 6 8190
4 255.255.240.0 14 4094
5 255.255.248.0 30 2046
6 255.255.252.0 62 1022
7 255.255.254.0 126 510
8 255.255.255.0 254 254
9 255.255.255.128 510 126
10 255.255.255.192 1022 62
11 255.255.255.224 2046 30
12 255.255.255.240 4094 14
13 255.255.255.248 8190 6
14 255.255.255.252 16382 2
Illustration: 20 Class C Subnetting Reference Chart
Number of Bits Subnet Mask Number of Subnets Number of Hosts
2 255.255.255.192 2 62
3 255.255.255.224 6 30
4 255.255.255.240 14 14
26 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 5 255.255.255.248 30 6
6 255.255.255.252 62 2
How Subnet Masks are Used to Determine the Network Number
The router performs a set process to determine the network (or more specifically, the subnetwork) address. First, the router extracts the IP destination address from the incoming packet and retrieves the internal subnet mask. It then performs a logical AND operation to obtain the network number. This causes the host portion of the IP destination address to be removed, while the destination network number remains. The router then looks up the destination network number and matches it with an outgoing interface. Finally, it forwards the frame to the destination IP address. Specifics regarding the logical AND operation are discussed in the following section.
Logical AND Operation
Three basic rules govern logically "ANDing" two binary numbers. First, 1 "ANDed" with 1 yields 1. Second, 1 "ANDed" with 0 yields 0. Finally, 0 "ANDed" with 0 yields 0. The truth table provided in illustration 21 illustrate the rules for logical AND operations.
Illustration: 21
Rules for Logical AND Operations
Input Input Output
1 1 1
1 0 0
0 1 0
0 0 0
Two simple guidelines exist for remembering logical AND operations: Logically "ANDing" a 1 with a 1 yields the original value, and logically "ANDing" a 0 with any number yields 0.
Illustrates that when a logical AND of the destination IP address and the subnet mask is performed, the subnetwork number remains, which the router uses to forward the packet.
Illustration: 22
Applying a logical AND the destination IP address and the subnet mask produces the subnetwork number.
27 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Address Resolution Protocol (ARP) Overview
For two machines on a given network to communicate, they must know the other machine's physical (or MAC) addresses. By broadcasting Address Resolution Protocols (ARPs), a host can dynamically discover the MAC-layer address corresponding to a particular IP network-layer address. After receiving a MAC-layer address, IP devices create an ARP cache to store the recently acquired IP-to-MAC address mapping, thus avoiding having to broadcast ARPS when they want to recontact a device. If the device does not respond within a specified time frame, the cache entry is flushed.
In addition to the Reverse Address Resolution Protocol (RARP) is used to map MAC-layer addresses to IP addresses. RARP, which is the logical inverse of ARP, might be used by diskless workstations that do not know their IP addresses when they boot. RARP relies on the presence of a RARP server with table entries of MAC-layer-to-IP address mappings.
Internet Routing
Internet routing devices traditionally have been called gateways. In today's terminology, however, the term gateway refers specifically to a device that performs application-layer protocol translation between devices. Interior gateways refer to devices that perform these protocol functions between machines or networks under the same administrative control or authority, such as a corporation's internal network. These are known as autonomous systems. Exterior gateways perform protocol functions between independent networks.
Routers within the Internet are organized hierarchically. Routers used for information exchange within autonomous systems are called interior routers, which use a variety of Interior Gateway Protocols (IGPs) to accomplish this purpose. The Routing Information Protocol (RIP) is an example of an IGP. Routers that move information between autonomous systems are called exterior routers. These routers use an exterior gateway protocol to exchange information between autonomous systems. The Border Gateway Protocol (BGP) is an example of an exterior gateway protocol.
IP Routing
IP routing protocols are dynamic. Dynamic routing calls for routes to be calculated automatically at regular intervals by software in routing devices. This contrasts with static routing, where routers are established by the network administrator and do not change until the network administrator changes them.
28 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 An IP routing table, which consists of destination address/next hop pairs, is used to enable dynamic routing. An entry in this table, for example, would be interpreted as follows: to get to network 172.31.0.0, send the packet out Ethernet interface 0 (E0). IP routing specifies that IP datagram’s travel through internetworks one hop at a time. The entire route is not known at the onset of the journey, however. Instead, at each stop, the next destination is calculated by matching the destination address within the datagram with an entry in the current node's routing table.
Each node's involvement in the routing process is limited to forwarding packets based on internal information. The nodes do not monitor whether the packets get to their final destination, nor does IP provide for error reporting back to the source when routing anomalies occur. This task is left to another Internet protocol, the Internet Control-Message Protocol (ICMP), which is discussed in the following section.
· The Domain Name System (DNS)
The initial solution for name resolution on the Internet was a file named Hosts.txt that was used on the now obsolete Advanced Research Projects Agency network (ARPANET), the predecessor of the modern day Internet. When the number of hosts on the ARPANET was small, the Hosts.txt file was easy to manage because it consisted of unstructured names and their corresponding IPv4 addresses. Computers on the ARPANET periodically downloaded Hosts.txt from a central location and used it for local name resolution. As the ARPANET grew into the Internet, the number of hosts began to increase dramatically and the centralized administration and manual distribution of a text file containing the names for computers on the Internet became unwieldy.
The replacement for the Hosts.txt file needed to be distributed, to allow for a hierarchical name space, and require minimal administrative overhead. The original design goal for DNS was to replace the existing cumbersome, centrally administered text file with a lightweight, distributed database that would allow for a hierarchical name space, delegation and distribution of administration, extensible data types, virtually unlimited database size, and reasonable performance.
DNS defines a namespace and a protocol for name resolution and database replication: 2. The DNS namespace is based on a hierarchical and logical tree structure. 3. The DNS protocol defines a set of messages sent over either User Datagram Protocol (UDP) port 53 or Transmission Control Protocol (TCP) port 53. Hosts that originate DNS queries send name resolution queries to servers over UDP first because it is faster. These hosts, known as DNS clients, resort to TCP only if the returned data is truncated. Hosts that store portions of the DNS database, known as DNS servers, use TCP when replicating database information.
Historically, the most popular implementation of the DNS protocol is Berkeley Internet Name Domain (BIND), which was originally developed at the University of California at Berkeley for the 4.3 Berkeley Software Distribution release of the UNIX operating system.
DNS Components
29 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Requests for Comments (RFCs) 974, 1034, and 1035 define the primary specifications for DNS. From RFC 1034, DNS comprises the following three components:
1. The domain namespace and resource records DNS defines a specification for a structured namespace as an inverted tree in which each node and leaf of the tree names a set of information. Resource records are records in the DNS database that can be used to configure the DNS database server (such as the Start of Authority [SOA] record) or to contain information of different types to process client queries (such as Address [A] records or Mail Exchanger [MX] records). Typical resource records contain resources by name and their IP addresses. Name queries to DNS database servers are attempts to extract information of a certain type from the namespace. The name query requests a name of interest and a specific type of record. For example, a name query would provide a host name and ask for the corresponding IPv4 or IPv6 address.
2. Name servers Name servers store resource records and information about the domain tree structure and attempt to resolve received client queries. DNS database servers, hereafter referred to as name servers or DNS servers, either contain the requested information in their resource records or have pointer records to other name servers that can help resolve the client query. If the name server contains the resource records for a given part of the namespace, the server is said to be authoritative for that part of the namespace. Authoritative information is organized into units called zones.
3. Resolvers Resolvers are programs that run on DNS clients and DNS servers and that create queries to extract information from name servers. A DNS client uses a resolver to create a DNS name query. A DNS server uses a resolver to contact other DNS servers to resolve a name on a DNS client's behalf. Resolvers are usually built into utility programs or are accessible through library functions, such as the Windows Sockets gethostbyname () or getaddrinfo () functions.
DNS Names
DNS names have a very specific structure, which identifies the location of the name in the DNS namespace. A fully qualified domain name (FQDN) is a DNS domain name that has been constructed from its location relative to the root of the namespace (known as the root domain). FQDNs have the following attributes: · FQDNs consist of the series of names from the name of the host or computer to the root domain. · A period character separates each name. · Each FQDN ends with the period character, which indicates the root domain. 30 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · Each name within the FQDN can be no more than 63 characters long. · The entire FQDN can be no more than 255 characters long. · FQDNs are not case-sensitive. · RFC 1034 requires the names that make up a FQDN to use only the characters a-z, A-Z, 0-9, and the dash or minus sign (-). RFC 2181 allows additional characters and is supported by the DNS Server service in Microsoft® Windows Server™ 2003 operating systems.
Domains and Subdomains
The DNS namespace is in the form of a logical inverted tree structure. Each branch point (or node) in the tree is given a name that is no more than 63 characters long. Each node of the tree is a portion of the namespace called a domain. A domain is a branch of the tree and can occur at any point in the tree structure. Domains can be further partitioned at node points within the domain into subdomains for the purposes of administration or load balancing. The domain name identifies the domain's position in the DNS hierarchy. The FQDN identifies the domain relative to the root. You create domain names and FQDNs by combining the names of the nodes from the designated domain node back to the root and separating each node with a period (.). The root of the tree has the special reserved name of "" (null), which you indicate by placing a final period at the end of the domain name (such as www.sales.example.com.). Domains and subdomains are grouped into zones to allow for distributed administration of the DNS namespace.
Illustration: 23
Shows the DNS namespace as it exists for the Internet.
Shows a few of the top-level domains and example hosts in the "microsoft.com." domain. A trailing period designates a domain name of a host relative to the root domain. To connect to that host, a user would specify the name "www.microsoft.com." If the user does not specify the final period, the DNS resolver automatically adds it to the specified name. Individual organizations manage second-level domains (subdomains of the top level domains) and their name servers. For example, Microsoft manages the "microsoft.com." domain.
31 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 DNS Servers and the Internet
Domains define different levels of authority in a hierarchical structure. The top of the hierarchy is called the root domain. The DNS namespace on the Internet
· Root domain
· Top-level domains
· Second-level domains
The root domain uses a null label, which you write as a single period (.). In the United States, the Internet Assigned Names Authority (IANA) manages several root domain name servers.
The next level in the hierarchy is divided into a series of nodes called the top-level domains. The top-level domains are assigned by organization type and by country/region. Some of the more common top-level domains are the following: · com – Commercial organizations in the United States (for example, microsoft.com for the Microsoft Corporation). · edu – Educational organizations in the United States. · gov – United States governmental organizations. · int – International organizations. · mil – United States military organizations. · net - Networking organizations. · org – Noncommercial organizations. · xx – Two-letter country code names that follow the International Standard 3166. For example, “.fr” is the country code for France. · arpa – Used to store information for DNS reverse queries.
Each top-level domain has name servers that IANA administers. Top-level domains can contain second-level domains and hosts. Second-level domains contain the domains and names for organizations and countries/regions. The names in second-level domains are administered by the organization or country/region either directly (by placing its own DNS server on the Internet) or by using an Internet service provider (ISP) who manages the names for an organization or country/region on its customer's behalf.
Zones
A zone is a contiguous portion of a domain of the DNS namespace whose database records exist and are managed in a particular DNS database file stored on one or multiple DNS servers. You can configure a single DNS server to manage one or multiple zones. Each zone is anchored at a specific domain node, referred to as the zone's root domain. Zone files do not necessarily contain the complete branch (that is, all subdomains) under the zone's root domain. For example, you can partition a domain into several subdomains, which are controlled by separate DNS servers. You might break up domains across multiple zone files if you want to distribute management of the domain across different groups or make data replication more efficient.
Illustration: 24 Shows the difference between domains and zones.
32 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 In the example, "microsoft.com" is a domain (the entire branch of the DNS namespace that starts with the microsoft.com. node), but the entire domain is not controlled by one zone file. Part of the domain is in a zone for "microsoft.com." and part of the domain is in a zone for the "dev.microsoft.com." domain. These zones correspond to different DNS database files that can reside on the same or different DNS servers.
Name Resolution
The two types of queries that a DNS resolver (either a DNS client or another DNS server) can make to a DNS server are the following:
· Recursive queries
In a recursive query, the queried name server is requested to respond with the requested data or with an error stating that data of the requested type or the specified domain name does not exist. The name server cannot just refer the DNS resolver to a different name server. A DNS client typically sends this type of query.
· Iterative queries
In an iterative query, the queried name server can return the best answer it currently has back to the DNS resolver. The best answer might be the resolved name or a referral to another name server that is closer to fulfilling the DNS client's original request. DNS servers typically send iterative queries to query other DNS servers.
DNS Name Resolution Example
To show how recursive and iterative queries are used for common DNS name resolutions, consider a computer running a Microsoft Windows® XP operating system or Windows Server 2003 connected to the Internet. A user types http://www.example.com in the Address 33 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 field of their Internet browser. When the user presses the ENTER key, the browser makes a Windows Sockets function call, either gethostbyname () or getaddrinfo(), to resolve the name http://www.example.com to an IP address. For the DNS portion of the Windows host name resolution process, the following occurs:
· The DNS resolver on the DNS client sends a recursive query to its configured DNS server, requesting the IP address corresponding to the name "www.example.com". The DNS server for that client is responsible for resolving the name and cannot refer the DNS client to another DNS server.
The DNS server that received the initial recursive query checks its zones and finds no zones corresponding to the requested domain name; the DNS server is not authoritative for the example.com domain. Because the DNS server has no information about the IP addresses of DNS servers that are authoritative for example.com. or com., it sends an iterative query for www.example.com. to a root name server.
· The DNS server that received the initial recursive query checks its zones and finds no zones corresponding to the requested domain name; the DNS server is not authoritative for the example.com domain. Because the DNS server has no information about the IP addresses of DNS servers that are authoritative for example.com. or com., it sends an iterative query for www.example.com. to a root name server.
3. The root name server is authoritative for the root domain and has information about name servers that are authoritative for top-level domain names. It is not authoritative for the example.com. Domain. Therefore, the root name server replies with the IP address of a name server for the com. top-level domain.
4. The DNS server of the DNS client sends an iterative query for www.example.com. to the name server that is authoritative for the com. top-level domain.
5. The com. name server is authoritative for the com. domain and has information about the IP addresses of name servers that are authoritative for second-level domain names of the com. domain. It is not authoritative for the example.com. domain. Therefore, the com. name server replies with the IP address of the name server that is authoritative for the example.com. domain.
6. The DNS server of the DNS client sends an iterative query for www.example.com. to the name server that is authoritative for the example.com. domain.
7. The example.com. name server replies with the IP address corresponding to the FQDN www.example.com.
8. The DNS server of the DNS client sends the IP address of www.example.com to the DNS client.
Illustration: 25
Example of recursive and iterative queries in DNS name resolution
34 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 All DNS queries are DNS Name Query Request messages. All DNS replies are DNS Name Query Response messages. In practice, DNS servers cache the results of queries on an ongoing basis. If a DNS server finds an entry matching the current request in its cache, it does not send an iterative DNS query. This example assumes that no cache entries were in any of the DNS servers to prevent the sending of the iterative name queries.
Forward lookups are queries in which a DNS client attempts to resolve an FQDN to its corresponding IP address. Zones that contain FQDN-to-IP address mappings are known as forward lookup zones.
Reverse Queries
In a reverse query, instead of supplying a name and asking for an IP address, the DNS client provides the IP address and requests the corresponding host name. Reverse queries are also known as reverse lookups, and zones that contain IP address-to-FQDN mappings are known as reverse lookup zones. Because you cannot derive the IP address from a domain name in the DNS namespace, only a thorough search of all domains could guarantee a correct answer. To prevent an exhaustive search of all domains for a reverse query, reverse name domains and pointer (PTR) resource records were created.
An example of an application that uses reverse queries is the Tracert tool, which by default uses reverse queries to display the names of the routers in a routing path. If you are going to use reverse queries, you must create reverse lookup zones and PTR records when you administer a DNS server so that reverse queries can be satisfied.
Reverse Queries for IPv4 Addresses
35 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 To support reverse lookups for IPv4 addresses, a special domain named in-addr.arpa. was created. Nodes in the in-addr.arpa domain are named after the numbers in the dotted decimal representation of IPv4 addresses. But because IPv4 addresses get more specific from left to right and domain names get more specific from right to left, the order of IPv4 address octets must be reversed when building the in-addr.arpa domain name corresponding to the IPv4 address. For example, for the generalized IPv4 address w.x.y.z, the corresponding reverse query name is z.y.x.w.in-addr.arpa. IANA delegates responsibility for administering the reverse query namespace below the in-addr.arpa domain to organizations as they are assigned IPv4 address prefixes.
Illustration: 26 Shows an example of the reverse lookup portion of the DNS namespace.
Within the in-addr.arpa domain, special pointer (PTR) resource records are added to associate the IPv4 addresses to their corresponding host names. To find a host name for the IPv4 address 157.54.200.2, a DNS client sends a DNS query for a PTR record for the name 2.200.54.157.in-addr.arpa. Reverse queries use the same name resolution process previously described for forward lookups (a combination of recursive and iterative queries). The DNS server finds the PTR record that contains the FQDN that corresponds to the IPv4 address 157.54.200.2 and sends that FQDN back to the DNS client.
Reverse Queries for IPv6 Addresses
IPv6 reverse lookups use the ip6.arpa. domain. To create the domains for reverse queries, each hexadecimal digit in the fully expressed 32-digit IPv6 address becomes a separate level in the reverse domain hierarchy in inverse order.
For example, the reverse lookup domain name for the address 3ffe:ffff::1:2aa:ff:fe3f:2a1c (fully expressed as 3ffe:ffff:0000:0001:02aa:00ff:fe3f:2a1c) is c.1.a.2.f.3.e.f.f.f.0.0.a.a.2.0.1.0.0.0.0.0.0.0.f.f.f.f.e.f.f.3.ip6.arpa.Just as in IPv4 addresses, PTR records in the reverse IPv6 domain map IPv6 addresses to FQDNs.
36 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Caching and TTL
For each resolved query (either recursive or iterative), the DNS resolver caches the returned information for a time that is specified in each resource record in the DNS response. This is known as positive caching. The amount of time in seconds to cache the record data is referred to as the Time To Live (TTL). The network administrator of the zone that contains the record decides on the default TTL for the data in the zone. Smaller TTL values help ensure that data about the domain is more consistent across the network if the zone data changes often. However, this practice also increases the load on name servers because positive cache entries time out more quickly.
After a DNS resolver caches data, it must start counting down from the received TTL so that it will know when to remove the data from its cache. For queries that can be satisfied by this cached data, the TTL that is returned is the current amount of time left before the data is flushed from the DNS cache. DNS client resolvers also have data caches and honor the TTL value so that they know when to remove the data.The DNS Client service in Windows XP and Windows Server 2003 and the DNS Server service in Windows Server 2003 support positive caching.
Negative Caching
As originally defined in RFC 1034, negative caching is the caching of failed name resolutions. A failed name resolution occurs when a DNS server returns a DNS Name Query Response message with an indication that the name was not found. Negative caching can reduce response times for names that DNS cannot resolve for both the DNS client and DNS servers during an iterative query process. Like positive caching, negative cache entries eventually time out and are removed from the cache based on the TTL in the received DNS Name Query Response message. The DNS Client service in Windows XP and Windows Server 2003 and the DNS Server service in Windows Server 2003 support negative caching.
Round Robin Load Balancing
DNS Name Query Response messages can contain multiple resource records. For example, for a simple forward lookup, the DNS Name Query Response message can contain multiple Address (A) records that contain the IPv4 addresses associated with the desired host. When multiple resource records for the same resource record type exist, the following issues arise:
· For the DNS server, how to order the resource records in the DNS Name Query Response message · For the DNS client, how to choose a specific resource record in the DNS Name Query Response message
To address these issues, RFC 1794 describes a mechanism named round robin or load sharing to share and distribute loads for network resources. The central assumption of RFC 1794 is that when multiple resource records for the same resource record type and the same name exist, multiple servers are offering the same type of service to multiple users. For example, the www.microsoft.com Web site is actually hosted by multiple Web servers with different IPv4 addresses. To attempt to distribute the load of servicing all the users who access www.microsoft.com, the DNS servers that are authoritative for microsoft.com modify the order of the resource records for the www.microsoft.com name in successive DNS Name
37 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Query Response messages. The DNS client uses the data in the first resource record in the response.
For example, if there were three A records for www.microsoft.com with the IPv4 addresses of 131.107.0.99, 131.107.0.100, and 131.107.0.101, the round robin scheme works as follows: 1. For the first request, the order of the resource records in the DNS Name Query Response message is 131.107.0.99-131.107.0.100-131.107.0.101. 2. For the second request, the order of the resource records in the DNS Name Query Response message is 131.107.0.100-131.107.0.101-131.107.0.99. 3. For the third request, the order of the resource records in the DNS Name Query Response message is 131.107.0.101-131.107.0.99-131.107.0.100.
The pattern repeats for subsequent queries. For an arbitrary number of resource records, the rotation process cycles through the list of resource records. A DNS server running Windows Server 2003 that is responding to a recursive query by default attempts to order the resource records according to the addresses that most closely match the IP address of the originating DNS client, and you can configure that server for round robin according to RFC 1794. To determine the addresses that are the closest match to the IPv4 address of the DNS client, the DNS Server service in Windows Server 2003 orders the addresses by using a high-order bit-level comparison of the DNS client's IPv4 address and the IPv4 addresses associated with the queried host name. This comparison technique is similar to the route determination process, in which IPv4 or IPv6 examines the IPv4 or IPv6 routing table to determine the route that most closely matches the destination address of a packet being sent or forwarded.
Name Server Roles
DNS servers store information about portions of the domain namespace. When name servers have one or more zones for which they are responsible, they are said to be authoritative servers for those zones. Using the example in Figure 8-2, the name server containing the dev.microsoft.com zone is an authoritative server for dev.microsoft.com.
Configuration of a DNS server includes adding name server (NS) resource records for all the other name servers that are in the same domain. Using the example on the previous page, if the two zones were on different name servers, each would be configured with an NS record about the other. These NS records provide pointers to the other authoritative servers for the domain.
DNS defines two types of name servers, each with different functions:
· Primary
A primary name server gets the data for its zones from locally stored and maintained files. To change a zone, such as adding subdomains or resource records, you change the zone file at the primary name server.
· Secondary
A secondary name server gets the data for its zones across the network from another name server (either a primary name server or another secondary name server). The process of 38 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 obtaining this zone information (that is, the database file) across the network is referred to as a zone transfer. Zone transfers occur over TCP port 53.
The following are reasons to have secondary name servers within an enterprise network:
· Redundancy: At least two DNS servers, a primary and at least one secondary, serving each zone are needed for fault tolerance.
· Remote locations: Secondary name servers (or other primary servers for subdomains) are needed in remote locations that have a large number of DNS clients. Clients should not have to communicate across slower wide area network (WAN) links for DNS queries.
· Load distribution: Secondary name servers reduce the load on the primary name server.
Because information for each zone is stored in separate files, the primary or secondary name server designation is defined at a zone level. In other words, a specific name server may be a primary name server for certain zones and a secondary name server for other zones.
When defining a zone on a secondary name server, you configure the zone with the name server from which the zone information is to be obtained. The source of the zone information for a secondary name server is referred to as a master name server. A master name server can be either a primary or secondary name server for the requested zone.
Illustration:27
Shows the relationship between primary, secondary, and master name servers.
39 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 When a secondary name server starts up, it contacts the master name server and initiates a zone transfer for each zone for which it is acting as a secondary name server. Zone transfers also can occur periodically (provided that data on the master name server has changed) as specified in the SOA record of the zone file. The "Resource Records and Zones" section of this chapter describes the SOA resource record.
Forwarders
When a DNS server receives a query, it attempts to locate the requested information within its own zone files. If this attempt fails because the server is not authoritative for the domain of the requested name and it does not have the record cached from a previous lookup, it must communicate with other name servers to resolve the request. On a globally connected network such as the Internet, DNS queries for names that do not use the second-level domain name of the organization might require interaction with DNS servers across WAN links outside of the organization. To prevent all the DNS servers in the organization from sending their queries over the Internet, you can configure forwarders. A forwarder sends queries across the Internet. Other DNS servers in the organization are configured to forward their queries to the forwarder.
Illustration: 28
Shows an example of intranet servers using a forwarder to resolve Internet names.
40 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 A name server can use a forwarder in non-exclusive or exclusive mode.
Forwarders in Non-exclusive Mode
In non-exclusive mode, when a name server receives a DNS query that it cannot resolve through its own zone files, it sends a recursive query to its forwarder. The forwarder attempts to resolve the query and returns the results to the requesting name server. If the forwarder is unable to resolve the query, the name server that received the original query attempts to resolve the query using iterative queries.
A name server using a forwarder in non-exclusive mode does the following when attempting to resolve a name: 1. Checks its local cache. 2. Checks its zone files. 3. Sends a recursive query to a forwarder. 4. Attempts to resolve the name through iterative queries to other DNS servers. Forwarders in Exclusive Mode
In exclusive mode, name servers rely on the name-resolving ability of the forwarders. When a name server in exclusive mode receives a DNS query that it cannot resolve through its own zone files, it sends a recursive query to its designated forwarder. The forwarder then carries out whatever communication is necessary to resolve the query and returns the results to the originating name server. If the forwarder is unable to resolve the request, the originating name server returns a query failure to the original DNS client. Name servers in exclusive
41 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 mode make no attempt to resolve the query on their own if the forwarder is unable to satisfy the request.
A name server using a forwarder in exclusive mode does the following when attempting to resolve a name:
1. Checks its local cache.
2. Checks its zone files.
3. Sends a recursive query to a forwarder.
Caching-Only Name Servers
Although all DNS servers cache queries that they have resolved, caching-only servers are DNS servers that only perform queries, cache the answers, and return the results. Caching-only servers are not authoritative for any domains and contain only the information that they have cached while attempting to resolve queries.
When caching-only servers are started, they do not perform any zone transfers because they have no zones and no entries exist in their caches. Initially, the caching-only server must forward queries until the cache has been built up to a point where it can service commonly used queries by just using its cache entries.
Resource Records and Zones
If your organization is connected to the Internet, in many cases you do not need to maintain a DNS infrastructure. For small networks, DNS name resolution is simpler and more efficient by having the DNS client query a DNS server that is maintained by an ISP. Most ISPs will maintain domain information for a fee. If your organization wants to have control over its domain or not incur the costs of using an ISP, you can set up your organization's own DNS servers.
In both cases, either going through an ISP or setting up separate DNS servers, the IANA must be informed of the domain name of the organization and the IP addresses of at least two DNS servers on the Internet that service the domain. An organization can also set up DNS servers within itself independent of the Internet.
At least two computers as DNS servers are recommended for reliability and redundancy—a primary and a secondary name server. The primary name server maintains the database of information, which is then replicated from the primary name server to the secondary name server. This replication allows name queries to be serviced even if one of the name servers is unavailable. Replication is scheduled based on how often names change in the domain. Replication should be frequent enough so that changes are reflected on both servers. However, excessive replication can have a negative impact on the performance of the network and name servers.
Resource Record Format
Resource records have the following format: owner TTL type class RDATA
· owner The domain name of the resource record.
42 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · TTL (Time to Live) The length of time in seconds that a DNS resolver should wait before it removes from its cache an entry that corresponds to the resource record.
· type The type of resource record.
· class The protocol family in use, which is typically IN for the Internet class.
· RDATA The resource data for the resource record type. For example, for an address (A) resource record, RDATA is the 32-bit IPv4 address that corresponds to the FQDN in the owner field.
Resource records are represented in binary form in DNS request and response messages. In text-based DNS database files, most resource records are represented as a single line of text. For readability, blank lines and comments are often inserted in the database files and are ignored by the DNS server. Comments always start with a semicolon (;) and end with a carriage return.
The following is an example A resource record stored in a DNS database file: srv1.dev.microsoft.com. 3600 A IN 157.60.221.205 Each resource record starts with the owner in the first column (srv1.dev.microsoft.com.). If the first column is blank, then it is assumed that the owner for this record is the owner of the previous record. The owner is followed by the TTL (3600 seconds = 1 hour), type (A = Address record), class (IN = Internet), and then the RDATA (Resource Data = 157.60.221.205). If the TTL value is not present, the DNS server sets the value to the TTL specified in the SOA (Start of Authority) record of the zone.
Resource Record Types
The DNS standards define many types of resource records. The most commonly used resource records are the following:
· SOA Identifies the start of a zone of authority. Every zone contains an SOA resource record at the beginning of the zone file, which stores information about the zone, configures replication behavior, and sets the default TTL for names in the zone.
· A Maps an FQDN to an IPv4 address.
· AAAA Maps an FQDN to an IPv6 address.
· NS Indicates the servers that are authoritative for a zone. NS records indicate primary and secondary servers for the zone specified in the SOA resource record, and they indicate the servers for any delegated zones. Every zone must contain at least one NS record at the zone root.
· PTR Maps an IP address to an FQDN for reverse lookups.
· CNAME Specifies an alias (synonymous name).
· MX Specifies a mail exchange server for a DNS domain name. A mail exchange server is a host that receives mail for the DNS domain name.
· SRV Specifies the IP addresses of servers for a specific service, protocol, and DNS domain.
43 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 RFCs 1035, 1034, 1183, and others define less frequently used resource records. The DNS Server service in Windows Server 2003 is fully compliant with RFCs 1034, 1035, and 1183.
The DNS Server service in Windows Server 2003 also supports the following resource record types that are Microsoft-specific:
· WINS Indicates the IPv4 address of a Windows Internet Name Service (WINS) server for WINS forward lookup. The DNS Server service in Windows Server 2003 can use a WINS server for looking up the host portion of a DNS name.
· WINS-R Indicates the use of WINS reverse lookup, in which a DNS server uses a NetBIOS Adapter Status message to find the host portion of the DNS name given its IPv4 address.
ATMA Maps DNS domain names to Asynchronous Transfer Mode (ATM) addresses.
For detailed information about the structure and contents of various types of DNS resource records, see the topic titled "Resource records reference" in Help and Support for Windows Server 2003.
Delegation and Glue Records
You add delegation and glue records to a zone file to indicate the delegation of a subdomain to a separate zone. For example, in Figure 8-2, the DNS server that is authoritative for the microsoft.com zone must be configured so that, when resolving names for the dev.microsoft.com, the DNS server can determine the following:
· That a separate zone for that domain exists.
A delegation is an NS record in the parent zone that lists the name server that is authoritative for the delegated zone.
· Where the zone for that domain resides.
A glue record is an A record for the name server that is authoritative for the delegated zone.
The name server for the microsoft.com. Domain has delegated authority for the dev.microsoft.com zone to the name server devdns.dev.microsoft.com at the IPv4 address of 157.60.41.59. In the zone file for the microsoft.com. Zone, the following records must be added: dev.microsoft.com. IN NS devdns.dev.microsoft.com. devdns.dev.microsoft.com. IN A 157.60.41.59 Without the delegation record for dev.microsoft.com, queries for all names ending in dev.microsoft.com would fail. Glue records are needed when the name of the name server that is authoritative for the delegated zone is in the domain of the name server attempting name resolution. In the example above, we need the A record for devdns.dev.microsoft.com. Because that FQDN is within the microsoft.com. Portion of the DNS namespace. Without this A record, the microsoft.com. DNS server would be unable to locate the name server for the dev.microsoft.com. Zone, and all name resolutions for names in the dev.microsoft.com domain would fail. A glue record is not needed when the name of the authoritative name server for the delegated zone is in a domain that is different than the domain of the zone file. In this case, the DNS server would use normal iterative queries to resolve the name to an IP
44 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 address. The DNS Server service in Windows Server 2003 automatically adds delegation and glue records when you delegate a subdomain.
The Root Hints File
The root hints file, also known as the cache file, contains the names and addresses of root name servers. For resolving domain names on the Internet, the default file provided with the DNS Server service in Windows Server 2003 has the records for the root servers of the Internet. For installations not connected to the Internet, the file should be replaced to contain the name servers authoritative for the root of the private network. This file is named Cached and is stored in the systemroot/System32/Dns folder.
Zone Transfers
Secondary name servers obtain zone files from a master name server using a zone transfer. The zone transfer replicates the set of records in the zone file from the master server to the secondary server. Zone transfers occur for all zones for which a DNS server is a secondary name server upon startup and on an ongoing basis to ensure that the most current information about the zone is reflected in the local zone file. The two types of zone transfers are full and incremental.
Full Zone Transfer
The original DNS RFCs defined zone transfers as a transfer of the entire zone file, regardless of how the file has changed since the last time it was transferred. In a full zone transfer, the following process occurs:
· The secondary server waits until the next refresh time (as specified in the SOA resource record) and then queries the master server for the SOA resource record for the zone.
2. The master server responds with the SOA resource record.
3. The secondary server checks the Serial Number field of the returned SOA resource record. If the serial number in the SOA resource record is higher than the serial number of the SOA resource record of the locally stored zone file, then there have been changes to the zone file on the master server and a zone transfer is needed. Whenever a resource record is changed on the master name server, the serial number in the SOA resource record is updated.
The secondary server sends an AXFR request (a request for a full zone transfer) to the master server.
4. The secondary server initiates a TCP connection with the master server and requests all of the records in the zone database. After the zone transfer, the Serial Number field in the SOA record of the local zone file matches the Serial Number field in the SOA record of the master server.
45 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Illustration: 29
Full zone transfer
If the secondary server does not receive a response to the SOA query, it retries SOA queries using a retry time interval specified in the SOA resource record in the local zone file. The secondary server continues to retry until the time elapsed since attempting to perform a zone transfer reaches an expiration time specified in the SOA resource record in the local zone file. After the expiration time, the secondary server closes the zone file and does not use it to answer subsequent queries. The secondary server keeps attempting to perform the zone transfer. When the zone transfer succeeds, the local zone file is opened and used for subsequent queries.
Incremental Zone Transfer
In a full zone transfer, the entire zone file is transferred. This can consume a substantial portion of processing resources and network bandwidth when the zone files are large and when zone records are frequently changed. To minimize the amount of information that is sent in a zone transfer for changes to zone records, RFC 1995 specifies a standard method of performing incremental zone transfers. In an incremental zone transfer, only the resource records that have changed (been added, deleted, or modified) are sent during the zone transfer.
In an incremental zone transfer, the secondary server performs the same query for the SOA record of the master server and comparison of the Serial Number field. If changes exist, the secondary server sends an IXFR request (a request for an incremental zone transfer) to the master server. The master server sends the records that have changed, and the secondary server builds a new zone file from the records that have not changed and the records in the incremental zone transfer.
Illustration: 30
An incremental zone transfer
46 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 For the master server to determine the records that have changed, it must maintain a history database of changes made to its zone files. The zone file changes are linked to a serial number so that the master server can determine which changes were made to the zone past the serial number indicated in the IXFR request from the secondary server. The DNS Server service in Windows Server 2003 supports incremental zone transfer.
DNS Notify
For both full and incremental zone transfers, the secondary server always initiates the zone transfer based on periodically querying the master server for its SOA record. The original DNS RFCs do not define a notification mechanism if the master server wanted to immediately propagate a large number of changes to its secondary servers.
To improve the consistency of data among secondary servers, RFC 1996 specifies DNS Notify, an extension of DNS that allows master servers to send notifications to secondary servers that a zone transfer might be needed. Upon receipt of a DNS notification, secondary servers request the SOA record of their master server and initiate a full or incremental zone transfer as needed.
Illustration: 31
Shows the DNS notify process.
To determine the secondary servers to which notifications should be sent, the master server maintains a notify list (a list of IP addresses) for each zone. The master server sends notifications to only the servers in the notify list when the zone is updated.The DNS Server service in Windows Server 2003 supports the configuration of a notify list (a list of IPv4 addresses) for each zone.
DNS Dynamic Update 47 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 DNS was originally defined as a name resolution scheme for relatively static names and addresses; DNS records contained information about servers, whose name and address configuration did not change often. Therefore, the manual administration of resource records in zone files was manageable. These original assumptions work well for an environment that is based on server and client computers that are statically configured, in which the client computers communicate only with the server computers and address configuration does not change. With the advent of peer-to-peer communications and applications and the Dynamic Host Configuration Protocol (DHCP), both of the assumptions of static DNS are challenged. In a Windows-based environment, client computers often communicate directly with each other and are automatically configured using DHCP. To communicate with each other, client computers must be able to resolve each other's names; therefore they must have corresponding DNS resource records. With DHCP, the address configuration of client computers could change every time they start. Manually administering DNS records for this environment is obviously impractical.
Therefore, RFC 2136 defines DNS dynamic update to provide an automated method to populate the DNS namespace with the current names and addresses for client and server computers by dynamically updating zone data on a zone's primary server. With DNS dynamic update, DNS records are automatically created, modified, and removed by either host computers or DHCP servers on their behalf. For example, a client computer that supports DNS dynamic update sends UPDATE messages to its DNS server to automatically add A, AAAA, and PTR records. The DNS server, which must also support DNS dynamic update, verifies that the sender is permitted to make the updates and then updates its local zone files.
The DNS Client service in Windows XP and Windows Server 2003 and the DNS Server service in Windows Server 2003 support DNS dynamic update.
· What is the Internet
The Internet was the result of some visionary thinking by people in the early 1960s who saw great potential value in allowing computers to share information on research and development in scientific and military fields. J.C.R. Licklider of MIT, first proposed a global network of computers in 1962, and moved over to the Defense Advanced Research Projects Agency (DARPA) in late 1962 to head the work to develop it. Leonard Kleinrock of MIT and later UCLA developed the theory of packet switching, which was to form the basis of Internet connections. Lawrence Roberts of MIT connected a Massachusetts computer with a California computer in 1965 over dial-up telephone lines. It showed the feasibility of wide area networking, but also showed that the telephone line's circuit switching was inadequate. Kleinrock's packet switching theory was confirmed. Roberts moved over to DARPA in 1966 and developed his plan for ARPANET. These visionaries and many more left unnamed here are the real founders of the Internet. When Senator Ted Kennedy heard in 1968 that the pioneering Massachusetts Company BBN had won the ARPA contract for an "interface message processor (IMP)," he sent a congratulatory telegram to BBN for their ecumenical spirit in winning the "interfaith message processor" contract.
The Internet, then known as ARPANET, was brought online in 1969 under a contract let by the renamed Advanced Research Projects Agency (ARPA) which initially connected four major computers at universities in the southwestern US (UCLA, Stanford Research Institute, UCSB, and the University of Utah). The contract was carried out by BBN of Cambridge, MA under Bob Kahn and went online in December 1969. By June 1970, MIT, Harvard, BBN, and Systems Development Corp (SDC) in Santa Monica, Cal. were added. By January 1971, 48 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Stanford, MIT's Lincoln Labs, Carnegie-Mellon, and Case-Western Reserve U were added. In months to come, NASA/Ames, Mitre, Burroughs, RAND, and the U of Illinois plugged in. After that, there were far too many to keep listing here.
Who was the first to use the Internet?
Charley Kline at UCLA sent the first packets on ARPANet as he tried to connect to Stanford Research Institute on Oct 29, 1969. The system crashed as he reached the G in LOGIN! The Internet was designed in part to provide a communications network that would work even if some of the sites were destroyed by nuclear attack. If the most direct route was not available, routers would direct traffic around the network via alternate routes. The early Internet was used by computer experts, engineers, scientists, and librarians. There was nothing friendly about it. There were no home or office personal computers in those days, and anyone who used it, whether a computer professional or an engineer or scientist or librarian, had to learn to use a very complex system.
Did Al Gore invent the Internet?
According to a CNN transcript of an interview with Wolf Blitzer, Al Gore said, "During my service in the United States Congress, I took the initiative in creating the Internet." Al Gore was not yet in Congress in 1969 when ARPANET started or in 1974 when the term Internet first came into use. Gore was elected to Congress in 1976. In fairness, Bob Kahn and Vint Cerf acknowledge in a paper titled Al Gore and the Internet that Gore has probably done more than any other elected official to support the growth and development of the Internet from the 1970's to the present. E-mail was adapted for ARPANET by Ray Tomlinson of BBN in 1972. He picked the @ symbol from the available symbols on his teletype to link the username and address. The telnet protocol, enabling logging on to a remote computer, was published as a Request for Comments (RFC) in 1972. RFC's are a means of sharing developmental work throughout community. The ftp protocol, enabling file transfers between Internet sites, was published as an RFC in 1973, and from then on RFC's were available electronically to anyone who had use of the ftp protocol. Libraries began automating and networking their catalogs in the late 1960s independent from ARPA. The visionary Frederick G. Kilgour of the Ohio College Library Center (now OCLC, Inc.) led networking of Ohio libraries during the '60s and '70s. In the mid 1970s more regional consortia from New England, the Southwest states, and the Middle Atlantic states, etc., joined with Ohio to form a national, later international, network. Automated catalogs, not very user-friendly at first, became available to the world, first through telnet or the awkward IBM variant TN3270 and only many years later, through the web. Ethernet, a protocol for many local networks, appeared in 1974, an outgrowth of Harvard student Bob Metcalfe's dissertation on "Packet Networks." The dissertation was initially rejected by the University for not being analytical enough. It later won acceptance when he added some more equations to it. The Internet matured in the 70's as a result of the TCP/IP architecture first proposed by Bob Kahn at BBN and further developed by Kahn and Vint Cerf at Stanford and others throughout the 70's. It was adopted by the Defense Department in 1980 replacing the earlier Network Control Protocol (NCP) and universally adopted by 1983. The Unix to Unix Copy Protocol (UUCP) was invented in 1978 at Bell Labs. Usenet was started in 1979 based on UUCP. Newsgroups, which are discussion groups focusing on a topic, followed, providing a means of exchanging information throughout the world. While Usenet is not considered as part of the Internet, since it does not share the use of TCP/IP, it linked UNIX systems around the world, and many Internet sites took advantage of the availability of newsgroups. It was a significant part of the community building that took place on the networks. Similarly, BITNET (Because It's Time Network) connected IBM mainframes around 49 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 the educational community and the world to provide mail services beginning in 1981. Listserv software was developed for this network and later others. Gateways were developed to connect BITNET with the Internet and allowed exchange of e-mail, particularly for e-mail discussion lists. These listservs and other forms of e-mail discussion lists formed another major element in the community building that was taking place.
In 1986, the National Science Foundation funded NSFNet as a cross country 56 Kbps backbone for the Internet. They maintained their sponsorship for nearly a decade, setting rules for its non-commercial government and research uses. As the commands for e-mail, FTP, and telnet were standardized, it became a lot easier for non-technical people to learn to use the nets. It was not easy by today's standards by any means, but it did open up use of the Internet to many more people in universities in particular. Other departments besides the libraries, computer, physics, and engineering departments found ways to make good use of the nets--to communicate with colleagues around the world and to share files and resources.
While the number of sites on the Internet was small, it was fairly easy to keep track of the resources of interest that were available. But as more and more universities and organizations--and their libraries-- connected, the Internet became harder and harder to track. There was more and more need for tools to index the resources that were available.
The first effort, other than library catalogs, to index the Internet was created in 1989, as Peter Deutsch and his crew at McGill University in Montreal, created an archiver for ftp sites, which they named Archie. This software would periodically reach out to all known openly available ftp sites, list their files, and build a searchable index of the software. The commands to search Archie were UNIX commands, and it took some knowledge of unix to use it to its full capability. McGill University, which hosted the first Archie, found out one day that half the Internet traffic going into Canada from the United States was accessing Archie. Administrators were concerned that the University was subsidizing such a volume of traffic, and closed down Archie to outside access. Fortunately, by that time, there were many more Archies available.At about the same time, Brewster Kahle, then at Thinking Machines, Corp. developed his Wide Area Information Server (WAIS), which would index the full text of files in a database and allow searches of the files. There were several versions with varying degrees of complexity and capability developed, but the simplest of these were made available to everyone on the nets. At its peak, Thinking Machines maintained pointers to over 600 databases around the world which had been indexed by WAIS. They included such things as the full set of Usenet Frequently Asked Questions files, the full documentation of working papers such as RFC's by those developing the Internet's standards, and much more. Like Archie, its interface was far from intuitive, and it took some effort to learn to use it well. Peter Scott of the University of Saskatchewan, recognizing the need to bring together information about all the telnet-accessible library catalogs on the web, as well as other telnet resources, brought out his Hytelnet catalog in 1990. It gave a single place to get information about library catalogs and other telnet resources and how to use them. He maintained it for years, and added HyWebCat in 1997 to provide information on web-based catalogs.
In 1991, the first really friendly interface to the Internet was developed at the University of Minnesota. The University wanted to develop a simple menu system to access files and information on campus through their local network. A debate followed between mainframe adherents and those who believed in smaller systems with client-server architecture. The mainframe adherents "won" the debate initially, but since the client-server advocates said they could put up a prototype very quickly, they were given the go-ahead to do a demonstration system. The demonstration system was called a gopher after the U of Minnesota mascot--the golden gopher. The gopher proved to be very prolific, and within a few years there were over 10,000 gophers around the world. It takes no knowledge of unix or 50 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 computer architecture to use. In a gopher system, you type or click on a number to select the menu selection you want.
Gopher's usability was enhanced much more when the University of Nevada at Reno developed the VERONICA searchable index of gopher menus. It was purported to be an acronym for Very Easy Rodent-Oriented Net wide Index to Computerized Archives. A spider crawled gopher menus around the world, collecting links and retrieving them for the index. It was so popular that it was very hard to connect to, even though a number of other VERONICA sites were developed to ease the load. Similar indexing software was developed for single sites, called JUGHEAD (Jonzy's Universal Gopher Hierarchy Excavation And Display). Peter Deutsch, who developed Archie, always insisted that Archie was short for Archiver, and had nothing to do with the comic strip. He was disgusted when VERONICA and JUGHEAD appeared.
In 1989 another significant event took place in making the nets easier to use. Tim Berners-Lee and others at the European Laboratory for Particle Physics, more popularly known as CERN, proposed a new protocol for information distribution. This protocol, which became the World Wide Web in 1991, was based on hypertext--a system of embedding links in text to link to other text, which you have been using every time you selected a text link while reading these pages. Although started before gopher, it was slower to develop. The development in 1993 of the graphical browser Mosaic by Marc Andreessen and his team at the National Center for Supercomputing Applications (NCSA) gave the protocol its big boost. Later, Andreessen moved to become the brains behind Netscape Corp., which produced the most successful graphical type of browser and server until Microsoft declared war and developed its Microsoft Internet Explorer.
MICHAEL DERTOUZOS 1936-2001
The early days of the web was a confused period as many developers tried to put their personal stamp on ways the web should develop. The web was threatened with becoming a mass of unrelated protocols that would require different software for different applications. The visionary Michael Dertouzos of MIT's Laboratory for Computer Sciences persuaded Tim Berners-Lee and others to form the World Wide Web Consortium in 1994 to promote and develop standards for the Web. Proprietary plug-ins still abounds for the web, but the Consortium has ensured that there are common standards present in every browser.
Since the Internet was initially funded by the government, it was originally limited to research, education, and government uses. Commercial uses were prohibited unless they directly served the goals of research and education. This policy continued until the early 90's, when independent commercial networks began to grow. It then became possible to route traffic across the country from one commercial site to another without passing through the government funded NSFNet Internet backbone. Delphi was the first national commercial online service to offer Internet access to its subscribers. It opened up an email connection in July 1992 and full Internet service in November 1992. All pretenses of limitations on commercial use disappeared in May 1995 when the National Science Foundation ended its sponsorship of the Internet backbone, and all traffic relied on commercial networks. AOL, Prodigy, and CompuServe came online. Since commercial usage was so widespread by this time and educational institutions had been paying their own way for some time, the loss of NSF funding had no appreciable effect on costs.
Today, NSF funding has moved beyond supporting the backbone and higher educational institutions to building the K-12 and local public library accesses on the one hand, and the research on the massive high volume connections on the other.
51 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Microsoft's full scale entry into the browser, server, and Internet Service Provider market completed the major shift over to a commercially based Internet. The release of Windows 98 in June 1998 with the Microsoft browser well integrated into the desktop shows Bill Gates' determination to capitalize on the enormous growth of the Internet. Microsoft's success over the past few years has brought court challenges to their dominance. We'll leave it up to you whether you think these battles should be played out in the courts or the marketplace. During this period of enormous growth, businesses entering the Internet arena scrambled to find economic models that work. Free services supported by advertising shifted some of the direct costs away from the consumer--temporarily. Services such as Delphi offered free web pages, chat rooms, and message boards for community building. Online sales have grown rapidly for such products as books and music CDs and computers, but the profit margins are slim when price comparisons are so easy, and public trust in online security is still shaky. Business models that have worked well are portal sites that try to provide everything for everybody, and live auctions. AOL's acquisition of Time-Warner was the largest merger in history when it took place and shows the enormous growth of Internet business! The stock market has had a rocky ride, swooping up and down as the new technology companies, the dot.com's encountered good news and bad. The decline in advertising income spelled doom for many dot.coms, and a major shakeout and search for better business models took place by the survivors.
A current trend with major implications for the future is the growth of high speed connections. 56K modems and the providers who supported them spread widely for a while, but this is the low end now. 56K is not fast enough to carry multimedia, such as sound and video except in low quality. But new technologies many times faster, such as cable modems and digital subscriber lines (DSL) are predominant now. Wireless has grown rapidly in the past few years, and travelers search for the wi-fi "hot spots" where they can connect while they are away from the home or office. Many airports, coffee bars, hotels and motels now routinely provide these services, some for a fee and some for free.
The next big growth area is the surge towards universal wireless access, where almost everywhere is a "hot spot". Municipal wi-fi or city-wide access, wiMAX offering broader ranges than wi-fi, Verizon's EV-DO, and other formats will joust for dominance in the USA in the months ahead. The battle is both economic and political.Another trend that is beginning to affect web designers is the growth of smaller devices to connect to the Internet. Small tablets, pocket PCs, smart phones, game machines, and even GPS devices are now capable of tapping into the web on the go, and many web pages are not designed to work on that scale.
Lesson II: Internet Hardware and Media
· HARDWARE: MODEMS
Short for Modulator Demodulator, a modem is a hardware device that enables a computer to transmit and receive information over telephone lines. The modem is responsible for 52 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 converting the digital data used by your computer into an analog signal used on phone lines and then converting it back once received on the other end. Because of the speed limitations of modems, many users are upgrading to a broadband connection such as a cable modem or DSL, which offer a faster download and upload speed when compared to the modem.
MODEM ABCs
Short for Modulator / Demodulator, which describes the method used to convert digital data used by computers into analog signals used by the phones and then back into digital data once received by the other computer.
Illustration: 32
The above pictures help represent a digital signal and an analog signal. All computer data is stored and transmitted within the computer in digital format 1s and 0s. In order for this data to be transmitted over analog phone lines, the data must be transmitted into an analog signal which is the noise you hear when connecting to another computer. Once the other computer receives this signal, it will then translate the signal back into its original digital format.
Typical modems are referred to as an asynchronous device, meaning that the device transmits data in an intermittent stream of small packets. Once received, the receiving system then takes the data in the packets and reassembles it into a form the computer can use.
Illustration: 33 S t o p D a t a S t a r t S t o p D a t a S t a r t 1 bit 8 bits 1 bit 1 bit 8 bits 1 bit P a c k e t P a c k e t 10 bits 10 bits
The above chart represents how an asynchronous transmission would be transmitted over a phone line. In asynchronous communication, 1 byte (8 bits) is transferred within 1 packet, which is equivalent to one character. However, for the computer to receive this information each packet must contain a Start and a Stop bit; therefore, the complete packet would be 10 bits. An example of what the above chart would transmit is the word HI which is equivalent to 2 bytes (16 bits).
There are two types of modems used in all computers.
53 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Internal: Modem which would be plugged into a slot located within or on the computer.
External: Modem which is located within a box and is hooked up externally to the computer, generally VIA the Serial Ports.
· Hardware: Terminal Adapters
A terminal adapter (TA) is a hardware interface between a computer and an Integrated Services Digital Network line. It's what replaces a modem when you are using an ISDN connection. Unlike "plain old telephone service," which carries signal in analog (voice) form between your computer and the telephone company's office, ISDN carries signals in digital form so there is no need to modulate and demodulate between analog and digital signals. The terminal adapter is what you have to install on a computer so that data can be fed directly into the ISDN line in digital form. Since ISDN service is not available from telephone companies in all areas, the terminal adapter is not usually built into a computer. You purchase and install it when you sign up for ISDN service.
· Hardware: Router
There are other important types of network devices besides the router, but understanding how a router works will go a long way toward your understanding the whole of internetworking. Before you can learn how to configure and manage routers, however, you need to know the basics of what makes one up. This chapter gives a general review of Cisco router hardware and software.
A dizzying array of hardware, software, telecommunications media, and technical expertise goes into internetworking. Switches, hubs, firewalls, packets, gateways, ports, access servers, interfaces, layers, protocols, serial lines, ISDN, frames, topologies—the list can seem endless. But there is a way to simplify things. A single, tangible entity makes sense of it all: the router. In the most basic terms, internetworking is about nothing more than linking machines and people through a maze of intermediary telecommunications lines and computing devices. This takes routing, which in essence involves just two fundamental missions: determine a path along which a link can be made and transmit packets across that path. It is within these two functions—which take place inside the router—that internetworking becomes easier to understand. This is because the router itself must cut all the complexity down to a level it can deal with. The router does this by working with everything, one IP packet at a time.
Looked at in this way, the router is the basic fabric of internetworks. Indeed, without the router, the Internet as we know it couldn’t even exist. This is because of the router’s unique and powerful capabilities: · Routers can simultaneously support different protocols (such as Ethernet, Token Ring, ISDN, and others), effectively making virtually all computers compatible at the internetwork level.
· They seamlessly connect local area networks (LANs) to wide area networks (WANs), which makes it feasible to build large-scale internetworks with minimum centralized planning—sort of like Lego™ sets.
· Routers filter out unwanted traffic by isolating areas in which messages can be “broadcast” to all users in a network.
54 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · They act as security gates by checking traffic against access permission lists
· Routers assure reliability by providing multiple paths through internetworks.
· They automatically learn about new paths and select the best ones, eliminating artificial constraints on expanding and improving internetworks.
In other words, routers make internetworks possible. They do so by providing a unified and secure environment in which large groups of people can connect. However, there are obstacles to bringing users together on internetworks, whether on a corporate intranet, a virtual private network, or the Internet itself. Figure 4-1 depicts how routing technology is the key to overcoming these obstacles.
Routers are like mini Towers of Babel. The router’s ability to support different protocols simultaneously is probably its most important feature because this capability lets otherwise incompatible computers talk with one another regardless of operating system, data format, or communications medium. The computer industry spent decades and billions of dollars struggling to attain compatibility between proprietary systems and met with limited success. Yet, in less than a decade, TCP/IP internetworking has built a common platform across which virtually all computer and network architectures can freely exchange information.
Illustration: 34
55 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 The router’s ability to filter out unwanted traffic is also important to internetworking. If users are bombarded with volumes of unwanted messages or if they feel their systems can be easily broken into, they will resist linking up to internetworks. Traffic filtering and access control provided by routers give users sufficient privacy and confidence to participate in internetworks. There are other important types of network devices besides the router, but understanding how a router works will go a long way toward your understanding the whole of internetworking. Before you can learn how to configure and manage routers, however, you need to know the basics of what makes one up. This chapter gives a general review of Cisco router hardware and software. In a nutshell, routers do exactly what their name says: They route data from a LAN to another router, then another router, and so on until data is received at its destination. Routers also act as traffic cops, allowing only authorized machines to transmit data into the local network so that private information can remain secure. In addition to supporting these dial-in and leased connections, routers also handle errors, keep network usage statistics, and handle security issues.
Routing for Efficiency
When you send an e-mail to your Aunt Sadie on the other side of the country, it’s routing technology that ensures she and she alone gets the message, and not every computer hooked up to the Internet. Routers direct the flow of traffic among, rather than within, networks. For instance, let’s consider how routers can be used within a LAN to keep information flowing.
Design-O-Rama, as shown in Illustration: 34, is a computer graphics company. The company’s LAN is divvied into two smaller LANs — one for the animators and one for the administration and support staff. The two subdivisions are connected with a router. Design-O-Rama employs eight people — four animators and four other staffers. When one animator sends a file to another, the large file will use a great deal of the network’s capacity. This results in performance problems for the others on the network.
56 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Illustration: 35
NOTE: Remember how Ethernet works. A single user can have such a dramatic impact on the network because each information packet sent by one computer is broadcast to all the other computers on the LAN. Then each computer examines the packet and decides if it was meant for them.
To keep the animators from constantly slowing down the network, the network was divided into two—one for the animators and one for everybody else. A router links the two networks and connects them both to the Internet. The router is the only device on the network that sees every message sent by any computer on either network. When an animator sends a file to a colleague, the router looks at the recipient’s address and keeps that piece of traffic isolated on that LAN. On the other hand, if the animator wants to query the human relations department about vacation time, the router knows to let that piece of traffic through to the HR department. Routers and the Internet In our previous example, we examined how a router could be used locally. Now, let’s broaden the scope of what routers do to include their functionality across the entire Internet. For the sake of comparison, let’s first talk about how a telephone call is routed across the country. Say it’s Aunt Sadie’s birthday and rather than send an e-mail, you want to call her. When you make a long-distance call, the telephone system establishes a stable circuit between your telephone and Aunt Sadie’s. The circuit may involve hopping through a number of steps, including fiber-optics, copper wires, and satellites. This end-to-end chain ensures that the quality of the line between you and Aunt Sadie will be constant. However, if the satellite goes offline or work crews cut the fiber-optic cable, your conversation with Aunt Sadie will be cut short. The Internet avoids this problem by making its “calls” in an entirely different way.
Whatever information is sent across the Internet (e-mail, Web page, and so on) is first broken into 1,500-byte packets. The packets are transmitted across a number of routers, each one sending the packet to the destination device. The packets will be transmitted via the best available route. This type of network is called a packet-switched network. Each 57 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 packet could take the same route, or none of the packets could take the same route. Once the packets show up at the destination computer, they are reassembled. This process goes so quickly that you wouldn’t even know that the file was chopped into 1,500-byte packets and then reassembled.
In illustration: 35, illustrates how a packet-switched network operates. The routers in the Internet are linked together in a web. The packets follow the path of least resistance to ensure they arrive at their destination in a reasonable amount of time. It seems logical that the packets would go through the least number of routers to get to its destination. However, sometimes that isn’t feasible, because there may be congestion clogging the ideal path. Routers send the traffic around the congested portions of the Internet for increased speed and efficiency.
Illustration: 36
This may seem like a very complicated system—as compared to the process followed when placing a telephone call—but the system works for two important reasons: The network can balance the load across different pieces of equipment on a millisecond-by-millisecond basis. If there is a problem with one piece of equipment in the network while a message is being transmitted, packets can be routed around the problem to ensure that the entire message is received.
The routers that make up the main back bone of the Internet can reconfigure the paths that packets take because they look at all the information surrounding the data packet, and they tell each other about line conditions, like problems sending and receiving data on various parts of the Internet.
All Shapes and Sizes
Not every router is responsible for the fate of packets whizzing across the Internet. Routers come in different sizes and do more or less, depending on how big and sophisticated they 58 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 are. For instance: If you have enabled Internet Connection Sharing between two Windows XP-based computers, the computer that is connected to the Internet is acting as a simple router. This router does very little—it just looks at data to see which computer it’s meant for.
■ Routers that are used to connect small offices to the Internet do more. They enforce rules about security for the office LAN, and they generally handle enough traffic that they tend to be stand-alone devices.
▲ The biggest routers (the ones used to handle data at the major traffic points on the Internet) handle a lot of information—millions of packets each second. These are stand-alone devices that look more like Maytag made them than a computer company.
Let’s consider the medium-sized router—it’s probably something humming away in a small room at your business. This router only has two networks to deal with—you’re LAN and the Internet. The office LAN connects to the router via an Ethernet connection. The router might also have two connections to your company’s ISP—a T3 and an ISDN connection. For the most part, your traffic comes and goes via the T3 line. However, the ISDN line is used in the event something goes awry with the T3 line. In fact, the router is configured to send data across the ISDN line, because the configuration table has been programmed to switch over in case of an emergency.
This router is also tasked with another function—it’s a layer of security against outside attacks. Although firewalls are routinely used to prevent attacks, the router is also configured to keep the bad people out.
However, the backbone of the Internet uses the third kind of router we listed. Cisco’s Gigabit Switch Router 12000 Series of routers is the kind of equipment used to run the Internet. These routers are designed and built like supercomputers. For instance, the 12000 Series uses 200 MHz MIPS R5000 processors, which are the same kind of processors used in the computers that make special effects for the movies. Cisco’s largest router—the 12816—can handle up to 1.28 trillion bits of information per second
In a conventional internetwork, information would be transmitted across great distances using twisted-pair copper wire, across a WAN or even a LAN. As useful and utilitarian as twisted-pair cabling and an electrical network have been, fiber-optics allow information to be transferred at immensely higher rates. In the past, when computers shared only brief conversations across the miles, electrical networks could handle the load. But now, as information is shared as it has never been shared before, there is a clear need for an upgrade in network capacities.
Comparing the bit rates in electrical networks to optical networks is like putting Woody Allen in a prison yard fist fight with Mike Tyson—there’s just no comparison. The greatest thing that optical networking has going for it is raw speed.
Illustration: 37
59 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Common WAN links that move across electrical networks are T-1 (1.544 Mbps) and T-3 (45 Mbps). On the LAN front, things get a little better. Most organizations use 10 or 100 Mbps Ethernet. The top-of-the-line Ethernet clocks in at 10 Gbps. However, once fi-ber-optics gets into the race, look out.
At their slowest, fiber-optic networks speed along much faster than a T-1 or a T-3. Once fiber shifts out of first gear, there ceases to be a comparison. When discussing optical networking speeds, you’ll hear the terminology change from T-1 or T-3 to OC. OC stands for opticalcarrier.OC takes over where T leaves off. Once the optical carrier gets involved, speeds not only reach 1 Gbps but even leave 1 Gbps in the rearview mirror. Table 4-1 shows how optical networking line speeds increase.
As you can see, the speed rates in optical networks (not to mention their development) are increasing at an amazing velocity. Thanks to dense wavelength division multiplexing (DWDM) optical bandwidth will only increase, because more than one stream of data can be introduced on a single run of fiber. More on that in a moment. Optical Technologies There are two prevalent technologies in the world of optical routing: SONET and DWDM. SONET is the oldest and most popular technology, while DWDM is somewhat of a new kid on the block, but supports capacities much greater than SONET. Let’s examine these technologies in a little more depth.
SONET -- The most basic and popular architecture for an optical network is the Synchronous Optical Network (SONET).
SONET is a standard for optical telecommunications transport developed by the Exchange Carriers Standards Association (ECSA) for the American National Standards Institute (ANSI), the body that sets industry standards in the U.S. for telecommunications and other industries. The comprehensive SONET standard is expected to provide the transport infrastructure for worldwide telecommunications for at least the next two or three decades.
NOTE: In Europe, SONET is known by another acronym, SDH, which is short for Synchronous Digital Hierarchy.
SONET is so speedy that you could transmit an entire 650MB CD-ROM from New York to Seattle in less than one second. Not only is SONET fast, but it’s also rather versatile. Voice calls from one office to another can be multiplexed along with data and fired out across the same fiber. Further, because of the generous bandwidth SONET affords, compression and encapsulation into Internet Protocol (IP) packets is unnecessary. For comparison’s sake, a single OC-3 connection can carry more than 2,000 simultaneous voice calls. Further, all types of data can be multiplexed alongside the calls. 60 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 SONET offers a top-end bandwidth of OC-192 (9.952 Gbps) and can carry a diverse range of information. In addition to high speeds, SONET features bit-error rates of one error in 10 billion bits. Compare this with copper transmission methods that have bit-error rates of one error in 1 million bits.
DWDM -- In its beginning, SONET delivered bandwidth that was previously unimaginable. At the time, delivering OC-3 levels (155.52 Mbps) provided more bandwidth than anyone knew what to do with. Of course, those were in the mid-1980s, a decade before the Internet and high-bandwidth applications. Technology kept delivering faster and faster optical carriers. After OC-3, there were OC-12, OC-48, and beyond.
OC-48 (2.5 Gbps) is a popular speed for SONET; however, the next level, OC-192 (10 Gbps) is about the best SONET will be able to deliver. Sure, ten years ago no one knew what a gigabit was, but now we do and we can’t get enough of them. The problem is that 10 Gbps is about Sonnet’s limit. The solution is to jump to DWDM.
DWDM is a technique in which multiple signals can traverse a single strand of optical fiber. The lasers used in optical networking can be tuned to different wavelengths (think of them as different colors). As such, it is possible to put multiple colors on a single fiber. When the receiving router sees the various colors, it knows which colors to separate out for which data streams.
Illustration: 38
Cisco’s Optical Offerings
Cisco utilizes both SONET and DWDM with its optical routers. For example, the Cisco ONS 15808 optical router supports DWDM technology. This carrier class router supports speeds between 2.5 and 10 Gbps and is capable of transmitting up to 2,000 kilometers. The router is able to be upgraded so that 160 channels are transmitted across the fiber with speeds up to 40 Gbps.
Cisco also provides a certain level of modularity with its devices. Rather than make a few models with a predetermined number of ports set up for Gigabit Ethernet and another amount dedicated to SONET or DWDM, the company has developed cards and modules that can be plugged into a router, making it customizable. That is, you can decide to load the router with SONET modules, DWDM modules, or any combination of optical and electrical you please.
Communicating with router
61 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Most users of internetworks don’t communicate with routers, they communicate through them. Network administrators, however, must deal directly with individual routers in order to install and manage them.
Routers are purpose-built computers dedicated to internetwork processing. They are important devices that individually serve hundreds or thousands of users—some serve even more. When a router goes down, or even just slows down, users howl and network managers jump. As you might imagine then, network administrators demand foolproof ways to gain access to the routers they manage in order to work on them. Routers don’t come with a monitor, keyboard, or mouse, so you must communicate with them in one of three other ways:
▼ From a terminal that’s in the same location as the router and is connected to it via a cable (the terminal is usually a PC or workstation running in terminal mode).
■ From a terminal that’s in a different location as the router and is connected to it via a modem that calls a modem connected to the router with a cable.
▲ Via the network on which the router sits.
In large networks, network administrators are often physically removed from routers and must access them via a network. However, if the router is unreachable due to a network problem, or if there’s no modem attached to the router itself, someone must go to its location and log directly into the router. The three ways to gain administrative access to routers are depicted in illustration: 39
Even when network administrators manage routers in the same building, they still prefer to access them by network. It doesn’t make sense to have a terminal hooked up to each router, especially when there are dozens of them stacked in a data closet or computer room. Also, it’s much more convenient to manage them all from a single PC or workstation.
Illustration: 40
62 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 There is several ways to communicate with a router each made possible by a particular communications protocol. The protocol, and how each is used.
Router Security
Routers aren’t very visible on internetworks, mainly because they usually don’t have addresses such as www.yahoo.com or www.amazon.com. Routers don’t need to have hu man-friendly addresses, because normal internetwork users never need to know that a router is there; they just need the connectivity it provides them.
The only people who ever need to log directly into a router are members of the network team responsible for managing it. In TCP/IP networks—the protocol on which most internetworks run—routers identify themselves to internetworks only with their IP addresses. For this reason, to log into a router you must first know that it exists and then what its IP address is. The network administrators responsible for the router will, of course, know this information.
The potential for abuse by hackers still exists. As you will learn in Chapter 14, routers constantly send messages to one another in order to update and manage the internetworks on which they operate. With the proper skills and enough determination, a hacker could discover a router’s IP address and then attempt to establish a Telnet connection to it. Given that routers are the links that stitch internetworks together, it’s easy to understand why Cisco and other internetwork equipment manufacturers design many security measures into their products. As shown in illustration: 41, security must restrict access to areas within an internetwork and to individual devices.
Illustration: 41
63 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 NOTE: Router passwords only control entry to the router devices themselves. Don’t confuse router passwords with passwords normal internetwork users must type in to enter certain Web sites or to gain admittance to intranets (private internetworks). Restrictions put on normal users are administered through firewalls and access lists
Router Passwords Router passwords aren’t intended only to keep out hackers. Password protection is administered on a router-by-router basis. Passwords to get into a router are stored inside the router itself in most cases. Large internetworks have dozens or even hundreds of routers— some more critical to network operations than others—so it’s a common practice for network managers to allow only select network team members access to certain routers, or even to command levels within routers. In illustration: 42 list router passwords and what they do.
llustration: 42
64 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 I n C i s c o routers, passwords are used to control access to
▼ The router device itself
■ The Privileged EXEC (enable mode) portion of the IOS software environment
▲ The use of specific IOS commands Line Passwords Line passwords are used to control who can log into a router. They are used to set password protection on the console terminal line, the AUX (auxiliary) line, and any or all of the five virtual terminal (VTY) lines.
You must set at least one password for the router’s VTY lines. If no Line password is set, when you attempt to log into the router via Telnet, you will be stopped by the error message “password required but none set.” Remember, anyone on the Internet can conceivably Telnet into any router, so setting Line passwords will stop all but the best hackers from getting a foothold. Here, IOS is prompting for a password:
User Access Verification
P a s s w o r d : Router>>
When you enter passwords into IOS, no asterisks appear to mask the letters typed— something to which most of us are accustomed. In the preceding example, at the prompt Router>> (the router’s host name in this example), the correct password was entered, the host router was successfully logged into, but no asterisks appear to the right of the password prompt. This might throw you off at first, but you’ll grow accustomed to it.
NOTE: You may have noticed that the password examples in this chapter are not made person-specific with usernames. While it is possible to have usernames with Enable and Enable Secret passwords, it is rarely done. This is because Enable and Enable Secret passwords are stored in router configuration files. Network managers find it more practical to simply issue generic passwords to avoid the administrative nightmare of maintaining 65 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 username/passwords across dozens or even hundreds of routers. Refer to Chapter 8 to find out how user accounts and passwords can be centrally maintained using TACACS+ and Cisco Secure. Enable and Enable Secret Password
Once you get past the Line password, you are logged into the router’s IOS software environment. IOS is divided into two privilege levels, EXEC and Privileged EXEC (which is usually called enable mode).
The EXEC level contains only basic, nondestructive commands. Being in enable mode provides access to more commands. EXEC-level commands basically allow you to view a router. Enable mode commands are more powerful in that they let you reconfigure the router’s settings. These commands are potentially destructive commands, the erase command being a good example.
Two types of passwords can be used to restrict access to Privileged EXEC (enable mode): the Enable password and the Enable Secret password. The idea of a “secret password” seems silly at first. Ofcourseall passwords are secret, or at least they should be. What the Cisco engineers are alluding to here is the level of encryption used to mask the password from unauthorized users.
The Privileged EXEC Level of IOS Enable and Enable Secret passwords both do the same thing: they restrict access to Privileged EXEC (enable mode). The difference between the two is in the level of encryption supported. Encryption is a technique used to scramble data, making it incomprehensible to those who don’t have a key to read it. Enable Secret passwords are scrambled using an advanced encryption algorithm based on 128 bits for which there is no known decoding technique. Encryption for the Enable password relies on a less powerful algorithm. Cisco strongly recommends using Enable Secret instead of the Enable password.
Enable Secret was introduced in 1997, so a fair amount of hardware and software that can support only Enable passwords is still in use, and servers storing backup IOS images frequently service both old and new routers. When both are set, the Enable Secret password always takes precedence over the Enable password. IOS will only put the Enable password to use when running an old version of IOS software.
IOS passwords are stored in the configuration file for a router. Configuration files routinely cross networks as routers are updated and backed up. Having an Enable Secret password means that a hacker using a protocol analyzer (a test device that can read packets) will have a tougher time decoding your password. The following sample configuration file illustrates this:
Version 11.2 service password-encryption service udp-small-servers service tcp-small-servers ! Hostname Router! Enable secret 5 $1$C/q2$ZhtujqzQIuJrRGqFwdwn71 enable password 7 0012000F
Note that the encryption mask of the Enable password on the last line is much shorter than the encryption mask of the Enable Secret password (on the second-to-last line).
66 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 The Service Password-Encryption Command Certain types of passwords, such as Line passwords, by default appear in clear text in the configuration file. You can use the service password-encryption command to make them more secure. Once this command is entered, each password configured is automatically encrypted and thus rendered illegible inside the configuration file (much as the Enable/Enable Secret passwords are). Securing Line passwords is doubly important in networks on which TFTP servers are used, because TFTP backup entails routinely moving config files across networks—and config files, of course, contain Line passwords.
Router Hardware and Memory
At first glance, routers seem a lot like a PC. They have a CPU, memory, and, on the back, ports and interfaces to hook up peripherals and various communications media. They sometimes even have a monitor to serve as a system console.
But there’s one defining difference from a PC: routers are diskless. They don’t even have floppy disks. If you think about it, this makes sense. A router exists to do just that: route. They don’t exist to create or display information or to store it, even temporarily. Routers have as their sole mission the task of filtering incoming packets and routing them outbound to their proper destinations.
Another difference is in the kind of add-on modules that can be plugged into routers. Whereas the typical PC contains cards for video, sound, graphics, or other purposes, the modules put into routers are strictly for networking (for obvious reasons). These are called interface modules, or just plain interfaces. When people or documents refer to a router interface, they mean an actual, physical printed circuit board that handles a particular networking protocol. EO and E1, for example, probably mean Ethernet interface numbers 1 and 2 inside a router. Interface modules are always layer-2 protocol specific. There is one protocol per interface.
Interfaces are added according to the network environment in which they will work. For example, a router might be configured with interface modules only for Ethernet. A router serving in a mixed-LAN environment, by contrast, would have interfaces for both the Ethernet and Token Ring protocols, and if that router were acting as a LAN-to-WAN juncture, it might also have an ISDN module.
There is one last difference between routers and general-purpose computers—a more subtle one. Computer product lines are almost always based on a common central processor (CPU) architecture, for example, Wintel PCs on the venerable Intel x86 architecture, Apple’s Motorola 68000 variants, Sun’s SPARC, and so on.
In contrast, Cisco routers use a variety of CPUs, each chosen to fit a particular mis- sion. Cisco SOHO 70 Series routers, for example, employ 50 MHz CPUs. Cisco probably made this selection because the 70 Series is designed for small office or home office use, where activity loads are light. The Motorola MPC 855T RISC chip is reliable; capable of handling the job; and, perhaps most important, inexpensive. Moving up the router product line, Cisco uses progressively more powerful general-purpose processors from Motorola, Silicon Graphics, and other chip makers.
Router Memory
67 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Routers use various kinds of memory to operate and manage themselves. Figure 4-9 depicts the layout of a motherboard in a Cisco 4500 router (a good example because it’s one of the most widely used routers in the world today). All Cisco router motherboards use four types of memory, each dedicated to performing specific roles.
Illustration: 43
Each Cisco router ships with at least a factory default minimum amount of DRAM and flash memory. Memory can be added at the factory or upgraded in the field. As a general rule, the amount of DRAM can be doubled or quadrupled (depending on the spe- cific model), and the amount of flash can be doubled. If traffic loads increase over time, DRAM can be upgraded to increase a router’s throughput capacity. RAM/DRAM RAM/DRAM stands for random access memory/dynamic random access memory. Also called working storage, RAM/DRAM is used by the router’s central processor to do its work, much like the memory in your PC. When a router is in operation, its RAM/DRAM contains an image of the Cisco IOS software, the running configuration file, the routing table, other tables (built by the router after it starts up), and the packet buffer.
Don’t be thrown by the two parts in RAM/DRAM. The acronym is a catch-all. Virtually all RAM/DRAM in Cisco routers is DRAM—dynamic random access memory. Nondynamic memory, also called static memory, became obsolete years ago. But the term RAM is still so widely used that it’s included in the literature to avoid confusion on the subject.
Cisco’s smallest router, the 70 Series, ships with a minimum of 16MB of DRAM. At the other end of the spectrum, the 12816-gigabit switch router, one of Cisco’s largest, supports up to 4GB.
NOTE: Shared memory (also called packet memory) is a specialized type of DRAM. Shared memory DRAM is dedicated to handling the router’s packet buffer. Cisco’s designers separate out shared memory to help assure I/O throughput. Shared memory is even physically nearer to the interface modules to further boost performance. NVRAM NVRAM stands for nonvolatile RAM. Nonvolatile means memory that will retain information after losing power. Cisco routers store a copy of the router’s configuration file in NVRAM (configuration files are covered later in this chapter). When the router is intentionally turned off, or if power is lost, NVRAM enables the router to restart in its proper configuration. Flash Memory 68 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Flash memory is also nonvolatile. It differs from NVRAM in that it can be erased and reprogrammed as needed. Originally developed by Intel, flash memory is in wide use in computers and other devices. In Cisco routers, flash memory is used to store one or more copies of the IOS software. This is an important feature because it enables network managers to stage new versions of IOS on routers throughout an internetwork and then upgrade them all at once to a new version from flash memory.
ROM
ROM stands for read-only memory. It, too, is nonvolatile. Cisco routers use ROM to hold a so-called bootstrapprogram, which is a file that can be used to boot to a minimum configuration state after a catastrophe. ROM is also referred to as ROMMON. In fact, when you boot from ROM, the first thing you’ll see is the rommon>> prompt. ROMMON (for ROM monitor) harks back to the early days of the UNIX operating system, which relied on ROMMON to reboot a computer to the point at which commands could at least be typed into the system console monitor. In smaller Cisco routers, ROM holds a bare-bones subset of the Cisco IOS software. ROM in some high-end Cisco routers holds a full copy of IOS. Router Ports and Modules
A router’s window to the internetwork is through its ports and modules. Without them, a router is a useless box. The ports and modules that are put into a router define what it can do.
Internetworking can be intimidating, with the seemingly endless combinations of products, protocols, media, feature sets, standards—you name it. The acronyms come so fast and so hard that it might seem hopeless to learn how to properly configure a router. But choosing the right router product can be boiled down to manageable proportions. Table 4-4 lays out five major requirement areas that, if met, will lead you to the best router solution.
Cisco obviously can’t manufacture a model of router to match every customer’s specific requirements. To make them more flexible to configure, routers come in two major parts: ▼ Chassis The actual box and basic components inside it, such as power supply, fans, rear and front faceplates, indicator lights, and slots ▲ Ports and modules the printed circuit boards that slide into the router box Cisco’s router product-line structure tries to steer you to a product—or at least to a reasonably focused selection of products—meeting all five requirement areas in Table
69 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Finding the right router for your needs is basically a three-step process. The following illustrates the process of selecting a router for a large branch office operation:]
Illustration: 44
First, Cisco’s routers are grouped into product families called series. Choosing a router product series is usually a matter of budget, because each series reflects a price/ performance tier. Models within series are generally based on the same chassis, which is the metal frame and basic components (power supply, fans, and so on) around which the router is built. We’ll select the Cisco 2000 Series because it fits both the purchase budget and performance requirements for our large branch office.
Illustration: 45
70 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 From the 2000 Series, we’ll take the Cisco 2600 Series. The 2600 chassis is versatile enough to fit a lot of situations, making it a popular brand of branch office router.
Illustration: 46
Third, we’ll select the Cisco 2650 because it has two Ethernet ports; and our imaginary branch office will operate two subnets, one for the customer service office and another for 71 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 the front office. The two Ethernet ports will let us separate the two departments, thereby isolating traffic.
NOTE: The term “port” can cause confusion if you’re not careful. When speaking of hardware, port means a physical connection through which I/O can pass(a serial port, for example), but there are also so-called ports at the transport layer of network protocols. These “ports” are actually port numbers used to identify what network application packets contain. These ports (port numbers) are also referred to as TCP ports or “listeners,” because they inform the receiver what’s inside the message. Example TCP-defined port numbers include Port 25 for Simple Mail Transfer Protocol and Port 80 for HTTP. Refer to Chapter 2’s section, “The Transport Layer,” for more on TCP ports. Router Packaging
Three major categories of modules can be configured into Cisco routers to support either LAN or WAN connectivity:
▼ Ethernet modules To support any of the many Ethernet LAN variants on the market, including Novell NetWare, Banyan VINES, and AppleTalk.
■ Token Ring modules IBM’s LAN technology, which is well established in banks, insurance companies, and other Fortune 1000 corporate environments. ▲ WAN connectivity modules To support a wide variety of WAN protocols, some old and some new. Example WAN technologies include newer protocols such as ISDN, Frame Relay, Asynchronous Transfer Mode (ATM), and legacy protocols such as SDLC and X.25. Configuration options depend mainly on the specific Cisco router: ▼ Lower-end routers tend to be “fixed configuration” in that the modules are factory integrated only (preconfigured). ■ Midrange routers, such as the Cisco 3600 Series, are “modular” in that they can accept a variety of modules, often packaging different protocols in the same box. Interface modules are plugged into this class of routers’ motherboards.
▲ High-end routers, the Cisco 7300 Series and Cisco 12000 Series, have buses (also called backplanes). Bus-based routers accept larger modules—usually referred to as blades or cards—that are effectively self-contained routers (they have their own CPUs, memory units, and so on).
In illustration 47, is a view of the back of a Cisco 4500 configured with two Token Ring modules (Ring A and Ring B) and four serial ports. Notice that an empty slot is available on the right. It’s a common practice to purchase a router model with room for adding an interface as network traffic grows.
Illustration: 47
72 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Media: PSTN
The Beginning of the PSTN The first voice transmission, sent by Alexander Graham Bell, was accomplished in 1876 through what is called a ring-down circuit. A ring-down circuit means that there was no dialing of numbers; Instead, a physical wire connected two devices. Basically, one person picked up the phone and another person was on the other end (no ringing was involved). Over time, this simple design evolved from a one-way voice transmission, by which only one user could speak, to a bi-directional voice transmission, whereby both users could speak. Moving the voices across the wire required a carbon microphone, a battery, an electromagnet, and an iron diaphragm. It also required a physical cable between each location that the user wanted to call. The concept of dialing a number to reach a destination, however, did not exist at this time. To further illustrate the beginnings of the PSTN, see the basic four-telephone network.
Illustration: 48
As you can see, a physical cable exists between each location.
73 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Place a physical cable between every household requiring access to a telephone, however, and you’ll see that such a setup is neither cost-effective nor feasible. To Determine how many lines you need to your house, think about everyone you call as a value of N and use the following equation: N× (N–1)/2. As such, if you want to call 10 people, you need 45 pairs of lines running into your house.
Illustration: 49
Due to the cost concerns and the impossibility of running a physical cable between 74 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Everyone on Earth who wanted access to a telephone, another mechanism was developed that could map any phone to another phone. With this device, called a switch , the telephone users needed only one cable to the centralized switch office, instead of seven. At first, a telephone operator acted as the switch. This operator asked callers where they wanted to dial and then manually connected the two voice paths. Shows how the four-phone network example would look today with a centralized operator to switch the calls.
Illustration: 50
Centralized Operator: The Human Switch
Now, skip ahead 100 years or so—the human switch is replaced by electronic switches. At this point, you can learn how the modern PSTN network is built.
Understanding PSTN Basics
Although it is difficult to explain every component of the PSTN, this section explains the most important pieces that make the PSTN work. The following sections discuss how your voice is transmitted across a digital network, basic circuit-switching concepts, and why your phone number is 10 digits long.
Analog and Digital Signaling
Everything you hear, including human speech, is in analog form. Until several decades ago, the telephony network was based on an analog infrastructure as well. Although analog communication is ideal for human interaction, it is neither robust nor efficient at recovering from line noise. ( Line noise is normally caused by the introduction of static into a voice network.) In the early telephony network, analog transmission was passed through amplifiers to boost the signal. But, this practice amplified not just the voice, but the line noise as well. This line noise resulted in an often unusable connection. Analog communication is a mix of time and amplitude.
Illustration: 51
Analog Waveform
75 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 If you were far away from the end office switch (which provides the physical cable to your home), an amplifier might be required to boost the analog transmission (your voice). Analog signals that receive line noise can distort the analog waveform and cause garbled reception. This is more obvious to the listener if many amplifiers are located between your home and the end office switch. In illustration 51, a show that an amplifier does not clean the signal as it amplifies, but simply amplifies the distorted signal. This process of going through several amplifiers with one voice signal is called accumulated noise.
Illustration: 52
Analog Line Distortion
In digital networks, line noise is less of an issue because repeaters not only amplify the signal, but clean it to its original condition. This is possible with digital communication because such communication is based on 1s and 0s.
Illustration: 52
Digital Line Distortion
76 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Therefore, when signals are repeated, a clean sound is maintained. When the benefits of this digital representation became evident, the telephony network migrated to pulse code modulation (PCM).
Digital Voice Signals
PCM is the most common method of encoding an analog voice signal into a digital stream of 1s and 0s. All sampling techniques use the Nyquist theorem , which basically states that if you sample at twice the highest frequency on a voice line, you achieve good-quality voice transmission.
The PCM process is as follows:
•Analog waveforms are put through a voice frequency filter to filter out anything greater than 4000 Hz. These frequencies are filtered to 4000 Hz to limit the amount of crosstalk in the voice network. Using the Nyquist theorem, you need to sample at 8000 samples per second to achieve good-quality voice transmission.
• The filtered analog signal is then sampled at a rate of 8000 times per second.
•After the waveform is sampled, it is converted into a discrete digital form. This sample is represented by a code that indicates the amplitude of the waveform at the instant the sample was taken. The telephony form of PCM uses eight bits for the code and a logarithm compression method that assigns more bits to lower-amplitude signals.
If you multiply the eight-bit words by 8000 times per second, you get 64,000 bits per second (bps). The basis for the telephone infrastructure is 64,000 bps (or 64 kbps).
Two basic variations of 64 kbps PCM are commonly used:ì-law, the standard used in North America; and a-law, the standard used in Europe. The methods are similar in that both use logarithmic compression to achieve from 12 to 13 bits of linear PCM quality in only eight-bit words, but they differ in relatively minor details. The ì-law method has a slight advantage over the a-law method in terms of low-level signal-to noise ratio performance, for instance.
Local Loops, Trunks, and Inters witch Communication
The telephone infrastructure starts with a simple pair of copper wires running to your home. This physical cabling is known as a local loop. The local loop physically connects your home 77 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 telephone to the central office switch (also known as a Class 5 switch or end office switch). The communication path between the central office switch and your home is known as the phone line, and it normally runs over the local loop. The communication path between several central office switches is known as a trunk . Just as it is not cost-effective to place a physical wire between your house and every other house you want to call, it is also not cost-effective to place a physical wire between every central office switch.
Illustration: 53
Meshed Network versus Hierarchical Network
Switches are currently deployed in hierarchies. End office switches (or central office switches) interconnect through trunks to tandem switches (also referred to as Class 4 switches). Higher layer tandem switches connect local tandem switches.
Illustration: 54
Circuit-Switching Hierarchy
78 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Central office switches often directly connect to each other. Where the direct connections occur between central office switches depends to a great extent on call patterns. If enough traffic occurs between two central office switches, a dedicated circuit is placed between the two switches to offload those calls from the local tandem switches. Some portions of the PSTN use as many as five levels of switching hierarchy. Now that you know how and why the PSTN is broken into a hierarchy of switches, you need to understand how they are physically connected, and how the network communicates.
PSTN Signaling
Generally, two types of signaling methods run over various transmission media. The signaling methods are broken into the following groups:
• User-to-network signaling—this is how an end user communicates with the PSTN. • Network-to-network signaling—this is generally how the switches in the PSTN intercommunicate. User-to-Network signaling generally, when using twisted copper pair as the transport,
User-to-Network Signaling Generally, when using twisted copper pair as the transport, a user connects to the PSTN through analog, Integrated Services Digital Network (ISDN), or through a T1 carrier.
The most common signaling method for user-to-network analog communication is Dual Tone Multi-Frequency (DTMF. DTMF is known as in-band signaling because the tones are carried through the voice path.
Illustration: 55
Dual Tone Multi-Frequency
79 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 When you pick up your telephone handset and press the digits the tone that passes from your phone to the central office switch to which you are connected tells the switch what number you want to call. ISDN uses another method of signaling known as out-of-band. With this method, the signaling is transported on a channel separate from the voice. The channel on which the voice is carried is called a bearer (or B channel) and is 64 kbps. The channel on which the signal is carried is called a data channel (D channel) and is 16 kbps.
Illustration: 56
Basic Rate Interface
Out-of-band signaling offers many benefits, including the following:
• Signaling is multiplexed (consolidated) into a common channel. • Glare is reduced (glare occurs when two people on the same circuit seize opposite ends of that circuit at the same time). • A lower post dialing delay. • Additional features, such as higher bandwidth, are realized. • Because setup messages are not subject to the same line noise as DTMF tones, call completion is greatly increased. In-band signaling suffers from a few problems, the largest of which is the possibility for lost tones. This occurs when signaling is carried across the voice path and it is a common reason why you can sometimes experience problems remotely accessing your voice mail.
Network-to-Network Signaling Network-to-network communication is normally carried across the following transmission media:
•T1/E1 carrier over twisted pair T1 is a 1.544-Mbps digital transmission link normally used in North America and Japan. E1 is a 2.048-Mbps digital transmission link normally used in Europe.
•T3/E3, T4 carrier over coaxial cable T3 carries 28 T1s or 672 64-kbps connections and is 44.736 Mbps. E3 carries 16 E1s or 512 64-kbps connections and is 34.368 Mbps. 80 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 T4 handles 168 T1 circuits or 4032 4-kbps connections and is 274.176 Mbps.
•T3, T4 carrier over a microwave link •Synchronous Optical Network (SONET) across fiber media
SONET is normally deployed in OC-3, OC-12, and OC-48 rates, which are 155.52 Mbps, 622.08 Mbps, and 2.488 Gbps, respectively.
Network-to-network signaling types include in-band signaling methods such as Multi- Frequency (MF) and Robbed Bit Signaling (RBS). These signaling types can also be used to network signaling methods.
Digital carrier systems (T1, T3) use A and B bits to indicate on/off hook supervision. The A/B bits are set to emulate Single Frequency (SF) tones (SF typically uses the presence or absence of a signal to signal A/B bit transitions). These bits might be robbed from the information channel or multiplexed in a common channel (the latter occurs mainly in Europe). More information on these signaling types is found in Chapter 3, “Basic Telephony Signaling.”
MF is similar to DTMF, but it utilizes a different set of frequencies. As with DTMF, MF tones are sent in-band. But, instead of signaling from a home to an end office switch, MF signals from switch to switch. Network-to-network signaling also uses an out-of-band signaling method known as Signaling System7 (SS7) (Or C7 in European countries).
NOTE: SS7 is beneficial because it is an out-of-band signaling method and it interconnects to the Intelligent Network (IN). Connection to the IN enables the PSTN to offer Custom Local Area Signaling Services (CLASS) services.
SS7 is a method of sending messages between switches for basic call control and for CLASS. These CLASS services still rely on the end-office switches and the SS7 network. SS7 is also used to connect switches and databases for network-based services (for example, 800-number services and Local Number Portability [LNP]). Some of the benefits of moving to an SS7 network are as follows:
•Reduced post-dialing delay There is no need to transmit DTMF tones on each hop of the PSTN. The SS7 network transmits all the digits in an initial setup message that includes the entire calling and called number. When using in-band signaling, each MF tone normally takes 50 ms to transmit. This means you have at least a .5-second post-dialing delay per PSTN hop. This number is based on 11-digit dialing (11 MF tones × 50 ms = 550 ms).
• Increased call completion SS7 is a packet-based, out-of-band signaling protocol, compared to the DTMF or MF in-band signaling types. Single packets containing all the necessary information (phone numbers, services, and so on) are transmitted faster than tones generated one at a time across an in-band network.
•Connection to the IN This connection provides new applications and services transparently across multiple vendors’ switching equipment as well as the capability to create new services and applications more quickly.
To further explain the PSTN, visualize a call from my house to my Grandma’s house 10 miles away. This call traverses an end office switch, the SS7 network (signaling only), and a second end office switch. 81 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Illustration: 58
PSTN Call Flow to Grandma’s House
To better explain the diagram let’s walk through the flow of the call:
1. I pick up the phone and send an off-hook indication to the end office switch. 2. The switch sends back a dial tone. 3. I dial the digits to call Grandma’s house (they are sent in-band through DTMF). 4. The switch interprets the digits and sends an Initial Address Message (IAM, or setup message) to the SS7 network. 5. The SS7 network reads the incoming IAM and sends a new IAM to Grandma’s switch. 6. Grandma’s switch sends a setup message to Grandma’s phone (it rings her phone). 7. An alerting message (alerting is the same as the phone ringing) is sent from Grandma’s switch (not from her phone) back to the SS7 network through an Address Complete Message (ACM). 8. The SS7 network reads the incoming ACM and generates an ACM to my switch. 9. I can hear a ringing sound and know that Grandma’s phone is ringing. (The ringing is not synchronized; your local switch normally generates the ringing when the ACM is received from the SS7 network.) 10 . Grandma picks up her phone, sending an off-hook indication to her switch. 11. Grandma’s switch sends an ANswer Message (ANM) that is read by the SS7, and a new ANM is generated to my switch. 12. A connect message is sent to my phone (only if it’s an ISDN phone) and a connect acknowledgment is sent back (again, only if it’s an ISDN phone). (If it is not an ISDN phone, then on-hook or off-hook representations signal the end office switch.) 13. I can now talk to Grandma until I hang up the phone (on-hook indication). 82 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 If Grandma’s phone was busy, I could use an IN feature by which I could park on her line and have the PSTN call me back after she got off the phone. Now that you have a basic understanding of how the PSTN functions, the next section discusses services and applications that are common in the PSTN.
New PSTN Network Infrastructure Model
As discussed in the previous sections, the new infrastructure will focus on the ability to separate the old stagnant infrastructure into a model by which multiple vendors can develop applications and features quickly for the consumer.
In the illustration clearly shows the relationship between all three layers as well as the relationship between these layers and the components that would be used in a live network. Carriers will enjoy this method, as it means they won’t be locked into a single solution for any of their layers. They will be able to mix and match all three layers to offer the services, functionality, and time-to-market that they need.
Illustration: 59
Illustration of Elements of Packet Telephony
Some carriers might be hesitant to utilize more than one equipment vendor to cut down on their integration timeframe, but many service providers will partner with a minimum of two vendors to ensure competition.
The reality of the illustration is that the bearers, connection plane, or media transport will be either IP gateways or ATM gateways, or a combination of both. Multiple vendors will be in this space initially, but most likely, they will consolidate to three to five major players.
The call-control plane is an extremely important piece of the new PSTN network infrastructure model, as it must gracefully coexist with both the connection plane and the service (application) layers. Many vendors are building MGC technology. In fact, the authors are working with approximately 15 vendors to ensure compatibility from the connection plane into the call-control and service/application plane. 83 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Many vendors will continue to be in the call-control plane, as service providers will more than likely use several vendors for this key technology, depending upon what service they decide to deploy. The onus on the Call Agent vendors will be to ensure compatibility from one Call Agent to another. Call Agent interoperability is one of the components that could keep service providers from using large-scale, packet based voice networks. The service or application plane is where the innovation in the network will happen. One major issue affecting the service plane is its reliance upon soft-switch vendors to open APIs that are useful enough to develop services. For this reason, you will see many application vendors attempting to develop Call Agent technology until APIs into the top Call Agent vendors are fully open and service-friendly. The service plane is where thousands of ISVs will converge to develop new and revenue enhancing applications. This is comparable to the client/server revolution in which Microsoft removed the barriers of having to code video drivers, and so on, and enabled ISVs to concentrate on applications. This same revolution is happening in the PSTN today and will change the way services and telephony/multimedia networks are designed, built, and deployed.
ISDN
A long time ago, the entire telephone network was analog. This was bad, because as a voice went farther down the line, and through more switches, the quality became worse and worse as noise crept in. And there was no way to eliminate the noise, no way to know what the signal was supposed to be. Digital encoding promised a way to encode the audio such that you'd know what the signal was supposed to be. As noise crept in, you could eliminate it throught the phone network, assuming it wasn't worse than the variation between different digital encoding levels.
With the transistor revolution, this theory became possible, and the phone companies began converting their own networks over to digital. Today, you have to search pretty hard to find a phone company switch that isn't digital. They call their network the Integrated Digital Network, or IDN.
This solved many of the phone company's problems. However for a variety of reasons, it has been attractive to make the phone network completely digital, from end to end. For computer users, this is ideal, because we can eliminate those clumsy modems, and will hopefully benefit from higher speed. For the phone companies, they can eliminate the last of the noise and loss from the audio data. And for dreamers, this will enable a wide variety of different services to be delivered to the customer over a single interface.
What is ISDN?
ISDN stands for Integrated Services Digital Network. It is a design for a completely digital telephone/telecommunications network. It is designed to carry voice, data, images, video, everything you could ever need. It is also designed to provide a single interface (in terms of both hardware and communication protocols) for hooking up your phone, your fax machine, your computer, your videophone, your video-on-demand system (someday), and your microwave. ISDN is about what the future phone network, and information superhighway, will look like (or would have looked like). ISDN was originally envisioned as a very fast service, but this was a long time ago when it was hoped to have fiber all the way to your house. It turned out that running all that fiber would be too expensive, so they designed ISDN to run on the copper wiring that you already 84 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 have. Unfortunately, that slowed things down considerably - too slow for quality video, for instance.
ISDN has been very slow in coming. The standards organizations have taken their time in coming up with the standards. In fact, many people consider them to be out of date already. But on the other side of the coin, the phone companies (especially in the U.S.) have been very slow at designing products and services, or marketing them with ISDN in mind.
Things are starting to pick up, but still very slowly. ISDN is available now in many places, but it is not widely used. Further most of the products and services that people have forecast for ISDN still aren't available. For this reason many people say that ISDN also stands for "It Still Does Nothing".
B-ISDN That brings is to B-ISDN. B-ISDN is Broadband ISDN. (The older ISDN is often called Narrowband ISDN.) This is not simply faster ISDN, or ISDN with the copper to your home finally upgraded to fiber. B-ISDN is a complete redesign. It is still capable of providing all the integrated services (voice, data, video, etc.) through a single interface just like ISDN was supposed to. But it will do it a lot faster than ISDN could. Of course, that copper to your house will still have to be replaced with fiber. But B-ISDN is still in development - it seems to be moving faster than ISDN, but it is still quite a ways off.
Fitting things together In order to understand what ISDN is, you have to understand a bit about modern telephony. You'll invariably find lots of buzzwords, or in most cases buzz-acronyms, that seem to overlap in a terribly complex way. That's because they do overlap considerably. Nevertheless you can generalize about how certain things fit together. Hopefully I can sum it all up in a few almost correct categories. There are two parts of a telephone network: the phone company's part, and the customer's part. The customer's part today is largely just the telephone, some house wiring, and some connectors. The phone company's part is lots more wire, fiber, switches, computers, and lots of expensive and complicated stuff.
ISDN is concerned (almost) entirely with the customer's part of the network. ISDN gets the data from you, to the phone company in a standard way. What they do with it in order to get it to its destination is entirely up to them. This is a very simple, important concept. If you understand this, then when someone says something like "SONET is the future of the modern telephone network" you'll know that they're talking (mostly) about what goes on inside the phone company, and between phone companies. They are probably right, but it is also true that "ISDN is the future of the modern telephone network" especially if you mean B-ISDN. They're just the future of different parts of the telephone network. Media: Kilo stream Kilo stream: Private Services are specially designed for businesses which rely heavily on communications. They provide permanently connected analogue and digital, voice and data circuits, between different sites, for the exclusive use of the business.
Speech Line and Key Line analogue circuits are used for straightforward voice or low-speed data applications. However, once you are regularly in touch with the same locations, making increased use of e-mail or exchanging larger and larger data files, then switching to KiloStream or the KiloStream N (the fastest KiloStream service for speech or data) digital services should result in substantial cost savings. In fact, because KiloStream circuits are leased for a fixed tariff, the more you use them, the more cost effective they become.
85 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 KiloStream comes in a range of different speeds, from 2.4kbit/s to 1,024kbit/s, to suit the needs and the budget of any business customer.
KiloStream services offer a resilient, high quality connection, and are available with a range of added-value packages to deliver an average performance target of 99.95%.
Key benefits of KiloStream include;
· Physical point-to-point connectivity - assuring high levels of security
· A state of the art network - providing very high levels of reliability and circuit availability
· Geographical coverage - extending over 99% of the UK
· 2-week provision
· Absence of modems - saving cost and adding reliability
· Connectivity applications, including data, voice and image; and, with suitable multiplexers, a mixture of all three.
Key features of KiloStream N include;
· Cost effectiveness where ordinary KiloStream is insufficient
· A smooth evolution path for network growth
· Easy accommodation of specialist applications such as CAD/CAM and video-conferencing
· High quality transmission, performance and reliability
· Resilience - both separation/diversity & disaster recovery service available
· Total Care support
· Nation-wide geographical coverage
· 6 week provision
The Private Service you choose will depend on the volume and kind of information you wish to communicate Analogue or digital circuits up to 64kbit/s are mainly used for low-speed voice or data applications, such as PC terminal users at branch offices who need on-line access to a host computer for electronic data interchange (EDI), file transfer or remote printing facilities.
At 64kbit/s, you can transmit voice and data, linking together local area networks (LANs) for order processing and stock control, or make Internet access more widely available. And at speeds of 128kbit/s and above, KiloStream N can be used for voice or data applications, to connect complete systems, for high speed faxing, or video conferences.
Finally, when you decide that you need more bandwidth, you'll find it simple to migrate to the MegaStream service, enabling your business to access even more applications as it grows. There is a Private Service to suit your precise geographical and traffic requirements. Whether
86 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 you work across the country or around the world, you will benefit from a single, seamless private network which is right for your business.
The cost of upgrading from analogue to digital private services, and from KiloStream to KiloStream N has reduced in real terms, making it more affordable for smaller businesses. There are a variety of discounts and a range of term-based contracts available to suit any business, and a bandwidth-based option with discount levels which increase in line with usage. All of which will help you to keep your costs down.
With the right Private Service, reliability comes as standard. With KiloStream you can expect a resilient and high quality connection, achieving an average network performance target of 99.95% error free seconds a year. There is even the option of KiloStream Assured Restore automatic back-up which offers very high levels of circuit availability. Moreover, with KiloStream, you get BT's Total Care maintenance service within tariff. That means for no extra cost, you will have the peace of mind of a guaranteed fault response time of 4 hours, any time, any day - or night. KiloStream coverage is global and seamless. You can be sure of cost-effective migration into even faster bandwidths when you want them. KiloStream is your fast track into the future of telecoms.
Lesson III: Internet Services
Electronic Mail
Electronic Mail every day, the citizens of the Internet send each other billions of e-mail messages. If you are online a lot, you yourself may send a dozen or more e-mails each day without even thinking about it. Obviously, e-mail has become an extremely popular communication tool.
Have you ever wondered how e-mail gets from your desktop to a friend halfway around the world? What is a POP3 server, and how does it hold your mail? The answers may surprise you, because it turns out that e-mail is an incredibly simple system at its core. In this article, we'll take an in-depth look at e-mail and how it works.
An E-mail Message According to Darwin Magazine: Prime Movers, the first e-mail message was sent in 1971 by an engineer named Ray Tomlinson. Prior to this, you could only send messages to users on a single machine. Tomlinson's breakthrough was the ability to send messages to other machines on the Internet, using the @ sign to designate the receiving machine.
An e-mail message has always been nothing more than a simple text message -- a piece of text sent to a recipient. In the beginning and even today, e-mail messages tend to be short pieces of text, although the ability to add attachments now makes many e-mail messages quite long. Even with attachments, however, e-mail messages continue to be text messages -- we'll see why when we get to the section on attachments.
E-mail Clients
87 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 You have probably already received several e-mail messages today. To look at them, you use some sort of e-mail client. Many people use well-known stand-alone clients like Microsoft Outlook, Outlook Express, Eudora or Pegasus. People who subscribe to free e-mail services like Hotmail or Yahoo use an e-mail client that appears in a Web page. If you are an AOL customer, you use AOL's e-mail reader. No matter which type of client you are using, it generally does four things:
· It shows you a list of all of the messages in your mailbox by displaying the message headers. The header shows you who sent the mail, the subject of the mail and may also show the time and date of the message and the message size. · It lets you select a message header and read the body of the e-mail message. · It lets you create new messages and send them. You type in the e-mail address of the recipient and the subject for the message, and then type the body of the message. · Most e-mail clients also let you add attachments to messages you send and save the attachments from messages you receive. Sophisticated e-mail clients may have all sorts of bells and whistles, but at the core, this is all that an e-mail client does.
A Simple E-mail Server
Given that you have an e-mail client on your machine, you are ready to send and receive e-mail. All that you need is an e-mail server for the client to connect to. Let's imagine what the simplest possible e-mail server would look like in order to get a basic understanding of the process. Then we will look at the real thing.
There are Web servers, FTP servers, telnet servers and e-mail servers running on millions of machines on the Internet right now. These applications run all the time on the server machine and they listen to specific ports, waiting for people or programs to attach to the port. The simplest possible e-mail server would work something like this:
· It would have a list of e-mail accounts, with one account for each person who can receive e-mail on the server. My account name might be mbrain; John Smith's might be jsmith, and so on. · It would have a text file for each account in the list. So the server would have a text file in its directory named MBRAIN.TXT, another named JSMITH.TXT, and so on. · If someone wanted to send me a message, the person would compose a text message ("Marshall, Can we have lunch Monday? John") in an e-mail client, and indicate that the message should go to mbrain. When the person presses the Send button, the e-mail client would connect to the e-mail server and pass to the server the name of the recipient (mbrain), the name of the sender (jsmith) and the body of the message. · The server would format those pieces of information and append them to the bottom of the MBRAIN.TXT file. The entry in the file might look like this: · From: jsmith · To: mbrain · Marshall, · Can we have lunch Monday? · John
88 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 There are several other pieces of information that the server might save into the file, like the time and date of receipt and a subject line; but overall, you can see that this is an extremely simple process.
As other people sent mail to mbrain, the server would simply append those messages to the bottom of the file in the order that they arrived. The text file would accumulate a series of five or 10 messages, and eventually I would log in to read them. When I wanted to look at my e-mail, my e-mail client would connect to the server machine. In the simplest possible system, it would:
· Ask the server to send a copy of the MBRAIN.TXT file · Ask the server to erase and reset the MBRAIN.TXT file · Save the MBRAIN.TXT file on my local machine · Parse the file into the separate messages (using the word "From:" as the separator) · Show me all of the message headers in a list
When I double-clicked on a message header, it would find that message in the text file and show me its body.
You have to admit that this is a very simple system. Surprisingly, the real e-mail system that you use every day is not much more complicated than this.
The Real E-mail System For the vast majority of people right now, the real e-mail system consists of two different servers running on a server machine. One is called the SMTP server, where SMTP stands for Simple Mail Transfer Protocol. The SMTP server handles outgoing mail. The other is either a POP3 server or an IMAP server, both of which handle incoming mail. POP stands for Post Office Protocol, and IMAP stands for Internet Mail Access Protocol. A typical e-mail server looks like this:
Illustration: 60
The SMTP server listens on well-known port number 25, POP3 listens on port 110 and IMAP uses port 143,
The SMTP Server Whenever you send a piece of e-mail, your e-mail client interacts with the SMTP server to 89 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 handle the sending. The SMTP server on your host may have conversations with other SMTP servers to actually deliver the e-mail.
Illustration: 61
Let's assume that I want to send a piece of e-mail. My e-mail ID is brain, and I have my account on howstuffworks.com. I want to send e-mail to [email protected]. I am using a stand-alone e-mail client like Outlook Express. When I set up my account at howstuffworks, I told Outlook Express the name of the mail server -- mail.howstuffworks.com. When I compose a message and press the Send button, here is what happens:
· Outlook Express connects to the SMTP server at mail.howstuffworks.com using port 25. · Outlook Express has a conversation with the SMTP server, telling the SMTP server the address of the sender and the address of the recipient, as well as the body of the message. · The SMTP server takes the "to" address ([email protected]) and breaks it into two parts: 1. The recipient name (jsmith) 2. The domain name (mindspring.com)
If the "to" address had been another user at howstuffworks.com, the SMTP server would simply hand the message to the POP3 server for howstuffworks.com (using a little program called the delivery agent). Since the recipient is at another domain, SMTP needs to communicate with that domain.
· The SMTP server has a conversation with a Domain Name Server. It says, "Can you give me the IP address of the SMTP server for mindspring.com?" The DNS replies with the one or more IP addresses for the SMTP server(s) that Mind spring operates. · The SMTP server at howstuffworks.com connects with the SMTP server at Mind spring using port 25. It has the same simple text conversation that my e-mail client had with the SMTP server for HowStuffWorks, and gives the message to the Mind spring server. The Mind spring server recognizes that the domain name for jsmith is at Mind spring, so it hands the message to Mind spring’s POP3 server, which puts the message in jsmith's mailbox.
90 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 If, for some reason, the SMTP server at HowStuffWorks cannot connect with the SMTP server at Mind spring, then the message goes into a queue. The SMTP server on most machines uses a program called send mail to do the actual sending, so this queue is called the send mail queue. Send mail will periodically try to resend the messages in its queue. For example, it might retry every 15 minutes. After four hours, it will usually send you a piece of mail that tells you there is some sort of problem. After five days, most send mail configurations give up and return the mail to you undelivered.
The actual conversation that an e-mail client has with an SMTP server is incredibly simple and human readable. It is specified in public documents called Requests For Comments (RFC), and a typical conversation looks something like this:
Hello test250 mx1.mindspring.com Hello abc.sample.com [220.57.69.37], pleased to meet you Mail from: [email protected] 250 2.1.0 [email protected]... Senders ok Rcpt to: [email protected] 250 2.1.5 jsmith... Recipient ok Data 354 Enter mail, end with "." on a line by itself from: [email protected] to:[email protected] Subject: testing John, I am testing.... 250 2.0.0 e1NMajH24604 Message accepted for delivery Quit 221 2.0.0 mx1.mindspring.com closing connection Connection closed by foreign host.
What the e-mail client says is in blue, and what the SMTP server replies is in green. The e-mail client introduces itself, indicates the "from" and "to" addresses, delivers the body of the message and then quits. You can, in fact, telnet to a mail server machine at port 25 and have one of these dialogs yourself -- this is how people "spoof" e-mail.
You can see that the SMTP server understands very simple text commands like HELO, MAIL, RCPT and DATA. The most common commands are:
· HELO - introduce yourself · EHLO - introduce yourself and request extended mode · MAIL FROM: - specify the sender · RCPT TO: - specify the recipient · DATA - specify the body of the message (To:, From: and Subject: should be the first three lines.) · RSET - reset · QUIT - quit the session · HELP - get help on commands · VRFY - verify an address · EXPN - expand an address · VERB - verbose
Newsgroup
Newsgroups are electronic meeting places where people with a similar interest have conversations. These conversations occur over a period of time - often days or weeks. Each newsgroup usually has a specific topic or focus that is reflected in its name. Users post questions with the hope that someone in the group has the knowledge and expertise to assist them and will reply. Messages posted on a subject that is not the focus of the group are
91 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 know as off-topic posts and are frowned upon by the membership. Newsgroups can be significant sources of information and access to other people’s knowledge. This is the reason a newsgroup reader is part of the Knowledge Workshop tool set. Conversations around a particular topic or question are called threads. They start with a single post and grow as others contribute messages. The reply to the first message appears below and indented from it in the displayed list. Other replies to the first message are listed directly below and in line with the first reply. Users may post a reply to a reply, creating an indented message beneath it, and so on.
Newsgroup messages are stored on a news server maintained by the organization that created the newsgroup. Many newsgroups are public but some have restricted access and require you to enter a username and password issued by the newsgroup host. The server may host many different newsgroups. To see the list of newsgroups you must first connect to the news server and request the list. You then choose the newsgroups you are interested in and subscribe to them. Once subscribed, you can open the newsgroup in Knowledge Workshop. The first time you open the newsgroup the most current messages (up to 300) are copied to your computer where you can read and reply to them.
FTP
FTP or File Transfer Protocol is used to connect two computers over the Internet so that the user of one computer can transfer files and perform file commands on the other computer.
Specifically, FTP is a commonly used protocol for exchanging files over any network that supports the TCP/IP protocol (such as the Internet or an intranet). There are two computers involved in an FTP transfer: a server and a client. The FTP server, running FTP server software, listens on the network for connection requests from other computers. The client computer, running FTP client software, initiates a connection to the server. Once connected, the client can do a number of file manipulation operations such as uploading files to the server, download files from the server, rename or delete files on the server and so on. Any software company or individual programmer is able to create FTP server or client software because the protocol is an open standard. Virtually every computer platform supports the FTP protocol. This allows any computer connected to a TCP/IP based network to manipulate files on another computer on that network regardless of which operating systems are involved (if the computers permit FTP access). There are many existing FTP client and server programs.
FTP runs exclusively over TCP. FTP servers by default listen on port 21 for incoming connections from FTP clients. A connection to this port from the FTP Client forms the control stream on which commands are passed to the FTP server from the FTP client and on occasion from the FTP server to the FTP client. For the actual file transfer to take place, a different 92 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 connection is required which is called the data stream. Depending on the transfer mode, the process of setting up the data stream is different.
In active mode, the FTP client opens a random port (> 1023), sends the FTP server the random port number on which it is listening over the control stream and waits for a connection from the FTP server. When the FTP server initiates the data connection to the FTP client it binds the source port to port 20 on the FTP server.
In passive mode, the FTP Server opens a random port (> 1023), sends the FTP client the port on which it is listening over the control stream and waits for a connection from the FTP client. In this case the FTP client binds the source port of the connection to a random port greater than 1023.
While data is being transferred via the data stream, the control stream sits idle. This can cause problems with large data transfers through firewalls which time out sessions after lengthy periods of idleness. While the file may well be successfully transferred, the control session can be disconnected by the firewall, causing an error to be generated.
When FTP is used in a UNIX environment, there is an often-ignored but valuable command; "reget" (meaning "get again") that will cause an interrupted "get" command to be continued, hopefully to completion, after a communications interruption. The principle is obvious—the receiving station has a record of what it got, so it can spool through the file at the sending station and re-start at the right place for a seamless splice. The converse would be "reput" but is not available. Again, the principle is obvious: The sending station does not know how much of the file was actually received, so it would not know where to start.
The objectives of FTP, as outlined by its RFC, are:
· To promote sharing of files (computer programs and/or data).
· To encourage indirect or implicit use of remote computers.
· To shield a user from variations in file storage systems among different hosts.
· To transfer data reliably, efficiently. Criticism of FTP
1. Passwords and file contents are sent in clear text, which can be intercepted by eavesdroppers. There are protocol enhancements that circumvent this.
2. Multiple TCP/IP connections are used, one for the control connection, and one for each download, upload, or directory listing. Firewall software needs additional logic to account for these connections.
3. It is hard to filter active mode FTP traffic on the client side by using a firewall, since the client must open an arbitrary port in order to receive the connection. This problem is largely resolved by using passive mode FTP.
4. It is possible to abuse the protocol's built-in proxy features to tell a server to send data to an arbitrary port of a third computer; see FXP.
5. FTP is a high latency protocol due to the number of commands needed to initiate a transfer.
93 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 6. No integrity check on the receiver side. If transfer is interrupted the receiver has no way to know if the received file is complete or not. It is necessary to manage this externally for example with MD5 sums or cyclic redundancy checking.
7. No error detection. FTP relies on the underlying TCP layer for error control, which uses a weak checksum by modern standards.
8. No date/timestamp attribute transfer. Uploaded files are given a new current timestamp, unlike other file transfer protocols such as SFTP, which allow attributes to be included. There is no way in the standard FTP protocol to set the time-last-modified (or time-created) date stamp that most modern file systems preserve. There is a draft of a proposed extension that adds new commands for this, but as of yet, most of the popular FTP servers do not support it.
Security problems
The original FTP specification is an inherently insecure method of transferring files because there is no method specified for transferring data in an encrypted fashion. This means that under most network configurations, user names, passwords, FTP commands and transferred files can be "sniffed" or viewed by anyone on the same network using a packet sniffer. This is a problem common to many Internet protocol specifications written prior to the creation of SSL such as HTTP, SMTP and Telnet. The common solution to this problem is to use either SFTP (SSH File Transfer Protocol), or FTPS (FTP over SSL), which adds SSL or TLS encryption to FTP as specified in RFC 4217
FTP return codes
FTP server return codes indicate their status by the digits within them. Brief explanations of various digits’ meanings are given below:
1. 1yz: Positive Preliminary reply. The action requested is being initiated but there will be another reply before it begins.
2. 2yz: Positive Completion reply. The action requested has been completed. The client may now issue a new command.
3. 3yz: Positive Intermediate reply. The command was successful, but a further command is required before the server can act upon the request.
4. 4yz: Transient Negative Completion reply. The command was not successful, but the client is free to try the command again as the failure is only temporary.
5. 5yz: Permanent Negative Completion reply. The command was not successful and the client should not attempt to repeat it again.
· x0z: The failure was due to a syntax error.
· x1z: This response is a reply to a request for information.
· x2z: This response is a reply relating to connection information.
· x3z: This response is a reply relating to accounting and authorization. 94 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · x4z: Unspecified as yet
· x5z: These responses indicate the status of the Server file system vis-a-vis the requested transfer or other file system action
Many sites that run FTP servers enable so-called "anonymous ftp". Under this arrangement, users do not need an account on the server. The user name for anonymous access is typically 'anonymous' or 'ftp'. This account does not need a password. Although users are commonly asked to send their email addresses as their passwords for authentication, usually there is trivial or no verification, depending on the FTP server and its configuration. Internet Gopher has been suggested as an alternative to anonymous FTP, as well as Trivial File Transfer Protocol.
Data format
While transferring data over the network, several data representations can be used. The two most common transfer modes are:
· ASCII mode
· Binary mode
The two types differ in the way they send the data. When a file is sent using an ASCII-type transfer, the individual letters, numbers, and characters are sent using their ASCII character codes. The receiving machine saves these in a text file in the appropriate format (for example, a Unix machine saves it in a Unix format, a Macintosh saves it in a Mac format). Hence if an ASCII transfer is used it can be assumed plain text is sent, which is stored by the receiving computer in its own format. Translating between text formats entails substituting the end of line and end of file characters used on the source platform with those on the destination platform, e.g. a Windows machine receiving a file from a Unix machine will replace the line feeds with carriage return-line feed pairs. ASCII transfer is also marginally faster, as the highest-order bit is dropped from each byte in the file.
Sending a file in binary mode is different. The sending machine sends each file bit for bit and as such the recipient stores the bit stream as it receives it. Any form of data that is not plain text will be corrupted if this mode is not used.
By default, most FTP clients use ASCII mode. Some clients try to determine the required transfer-mode by inspecting the file's name or contents.
The FTP specifications also list the following transfer modes:
1. EBCDIC mode
2. Local mode
In practice, these additional transfer modes are rarely used. They are however still used by some legacy mainframe systems.
FTP and web browsers
Most recent web browsers and file managers can connect to FTP servers, although they may lack the support for protocol extensions such as FTPS. This allows manipulation of remote files over FTP through an interface similar to that used for local files. This is done via an FTP URL, which takes the form ftp(s)://
FTP over SSH
FTP over SSH refers to the practice of tunneling a normal FTP session over an SSH connection.
Because FTP uses multiple TCP connections (unusual for a TCP/IP protocol that is still in use), it is particularly difficult to tunnel over SSH. With many SSH clients, attempting to set up a tunnel for the control channel (the initial client-to-server connection on port 21) will only protect that channel; when data is transferred, the FTP software at either end will set up new TCP connections (data channels) which will bypass the SSH connection, and thus have no confidentiality, integrity protection, etc.
If the FTP client is configured to use passive mode and to connect to a SOCKS server interface that many SSH clients can present for tunneling, it is possible to run all the FTP channels over the SSH connection.
Otherwise, it is necessary for the SSH client software to have specific knowledge of the FTP protocol, and monitor and rewrite FTP control channel messages and autonomously open new forwarding for FTP data channels. Version 3 of SSH Communications Security's software suite, and the GPL licensed FONC are two software packages that support this mode.
FTP over SSH is sometimes referred to as secure FTP; this should not be confused with other methods of securing FTP, such as with SSL/TLS (FTPS). Other methods of transferring files using SSH that are not related to FTP include SFTP and SCP; in each of these, the entire conversation (credentials and data) is always protected by the SSH protocol.
HTTP The Hypertext Transfer Protocol (HTTP) is the foundation protocol of the World Wide Web (WWW). The name is somewhat misleading in that HTTP is not a protocol for transferring hypertext; rather, it's a protocol for transmitting information with the efficiency necessary for making hypertext jumps. The data transferred by the protocol can be plain text, hypertext, audio, images, or any type of Internet-accessible information.HTTP is a transaction-oriented client/server protocol. The most typical use of HTTP is between a web browser and a web server. To provide reliability, HTTP makes use of TCP. Nevertheless, HTTP is a "stateless" protocol; each transaction is treated independently. A typical implementation creates a new TCP connection between client and server for each transaction and then terminates the connection as soon as the transaction completes, although the specification doesn't dictate this one-to-one relationship between transaction and connection lifetimes.The stateless nature of HTTP is well suited to its typical application. A normal session of a user with a web browser involves retrieving a sequence of web pages and documents. Ideally, the sequence is performed rapidly, and the locations of the various pages and documents may include a number of widely distributed servers.
Another important feature of HTTP is flexibility in the formats that it can handle. When a client issues a request to a server, it may include a prioritized list of formats that it can handle, and the server replies with the appropriate format. For example, a lynx browser can't handle images, so a web server need not transmit any images on web pages to this browser. 96 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 This arrangement prevents the transmission of unnecessary information and provides the basis for extending the set of formats with new standardized and proprietary specifications.
Illustration: 62
Illustrates three examples of HTTP operation.
Examples of HTTP operation.
The simplest case is one in which a user agent establishes a direct connection with an origin server. The user agent is the client that initiates the request, such as a web browser being run on behalf of an end user. The origin server is the server on which a resource of interest resides; an example is a web server at which a desired home page resides.
For this case, the client opens a TCP connection that's end-to-end between the client and the server. The client then issues an HTTP request. The request consists of a specific command (referred to as a method), a URL, and a message containing request parameters, information about the client, and perhaps some additional content information.When the server receives the request, it attempts to perform the requested action and then returns an HTTP response. The response includes status information, a success/error code, and a message containing information about the server, information about the response itself, and possibly body content. The TCP connection is then closed.The middle part of shows a case in which there is no end-to-end TCP connection between the user agent and the origin server. Instead, there are one or more intermediate systems with TCP connections between logically adjacent systems. Each intermediate system acts as a relay, so that a request initiated by the client is relayed through the intermediate systems to the server, and the response from the server is relayed back to the client.
Three forms of intermediate system are defined in the HTTP specification: proxy, gateway, and tunnel.
Illustration: 63 97 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Intermediate HTTP systems.
Proxy
A proxy acts on behalf of other clients, presenting requests from the other clients to a server. The proxy acts as a server in interacting with a client and as a client in interacting with a server. Several scenarios call for the use of a proxy:
1. Firewall. The client and server may be separated by a firewall, with the proxy on the client side of the firewall. Typically, the client is part of a network secured by the firewall and the server is external to the secured network. In this case, the server must authenticate itself to the firewall to set up a connection with the proxy. The proxy accepts responses after they have passed through the firewall.
2. Different versions of HTTP. If the client and server are running different versions of HTTP, the proxy can implement both versions and perform the required mapping.
In summary, a proxy is a forwarding agent—receiving a request for a URL object, modifying the request, and forwarding the request toward the server identified in the URL.
Gateway
A gateway is a server that appears to the client as if it were an origin server. It acts on behalf of other servers that may not be able to communicate directly with a client. There are several scenarios in which servers can be used:
· Firewall. The client and server may be separated by a firewall, with the gateway on the server side of the firewall. Typically, the server is connected to a network
98 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 protected by a firewall, with the client external to the network. In this case, the client must authenticate itself to the proxy, which can then pass the request to the server.
· Non–HTTP server. Web browsers have a built-in capacity to contact servers for protocols other than HTTP, such as FTP and Gopher servers. This capability can also be provided by a gateway. The client makes an HTTP request to a gateway server. The gateway server then contacts the relevant FTP or Gopher server to obtain the desired result. This result is then converted into a form suitable for HTTP and transmitted back to the client.
Tunnel
Unlike the proxy and the gateway, the tunnel performs no operations on HTTP requests and responses. Instead, a tunnel is simply a relay point between two TCP connections, and the HTTP messages are passed unchanged—as if there were a single HTTP connection between user agent and origin server. Tunnels are used when there must be an intermediary system between client and server but it's unnecessary for that system to understand the contents of any messages. An example is a firewall in which a client or server external to a protected network can establish an authenticated connection, and then maintain that connection for purposes of HTTP transactions.
Cache
Returning to the lowest portion of the figure shows an example of a cache. A cache is a facility that may store previous requests and responses for handling new requests. If a new request arrives that's the same as a stored request, the cache can supply the stored response rather than accessing the resource indicated in the URL. The cache can operate on a client or server or on an intermediate system other than a tunnel. In intermediary B has cached a request/response transaction, so that a corresponding new request from the client need not travel the entire chain to the origin server, but instead is handled by B.
Not all transactions can be cached, and a client or server can dictate that a certain transaction may be cached only for a given time limit.
Request Messages
A request message is sent by an agent to a server to request some action. These are the possible actions, called methods:
Method Description
OPTIONS A request for information about the options available.
GET A request to retrieve information.
HEAD Like a GET except that the server's response must not include an entity body; all of the header fields in the response are the same 99 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 as if the entity body were present. This enables a client to get information about a resource without transferring the entity body.
POST A request to accept the attached entity as a new subordinate to the identified URL.
PUT A request to accept the attached entity and store it under the supplied URL. This may be a new resource with a new URL, or a replacement of the contents of an existing resource with an existing URL.
DELETE Requests that the origin server delete a resource.
TRACE Requests that the server return whatever is received as the entity body of the response. This can be used for testing and diagnostic purposes.
Response Messages
A response message is returned by a server to an agent in response to a request message. It may include an entity body containing hypertext-based information. In addition, the response message must specify a status code, which indicates the action taken on the corresponding request. Status codes are organized into the following categories:
Category Description
Informational The request has been received and processing continues. No entity body accompanies this response.
Successful The request was successfully received, understood, and accepted.
Redirection Further action is required to complete the request.
Client Error The request contains a syntax error or the request cannot be fulfilled.
Server Error The server failed to fulfill an apparently valid request.
Internet Databases
WAIS
100 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 A wide area interoperability system (WAIS) interconnects communications systems over existing network infrastructure - allowing users of disparate radios, telephones, satellite phones, and dispatch centers to communicate with each other locally, Regionally, or in systems that span entire states.
Benefits
· Provides a wide area interoperability communications platform for critical incident command and control that is easily scalable.
· Allows efficient use of existing network resources with commercial- off-the-shelf (COTS) equipment and proven Radio over IP (RoIP)/ Voice over IP (VoIP) technology.
· Provides remote access from any point on the network. Authorized users may configure, control, and monitor unselected audio and communicate with any entity via selected audio.
· Allows disparate radio systems to be connected locally, regionally, statewide, or in a cross nation network.
· WAIS Controller software’s user-friendly interface presents clearly the state of the system and allows operators to make and break connections swiftly.
The JPSWAIS is a wide area interoperability solution that links multiple communication devices through an IP network. The system is managed via the WAIS Controller software, the system’s graphical user interface (GUI). AWAIS may range in size from just a few sites to an entire statewide system or larger. Fixed sites and mobile platforms can be integrated, provided they have an access point into the network. Any number of control points, equipped with the WAIS Controller, can be installed; each may control the entire system or any portion of it. Operators stationed at the control points can easily monitor communications and form or disband multiple user talk groups anywhere on the system. Each control point can have its own password-protected permission set, customized to include the specific sites or modules that they need to control. As a safety feature, system administrators may be given full authorization, allowing them to control the entire system or temporarily take over for another operator from any control station.
WAIS Architecture
Most WAIS end users interface the system via the industry standard radio interoperability gateway, the JPS ACU-1000. The ACU-1000 is the centerpiece for local interoperability; its modular design is customizable to accommodate disparate communication devices. Any number of these fully functional independent local systems, along with dispatch positions and other communications assets, are linked over a network to create a WAIS. The number of network audio links from each local interoperability system (LIS) may be adjusted to suit system requirements. Single, independent users, dispatchers, operators or radio Systems not part of an LIS can connect to the WAIS using JPS’s NXU-2A.
WAIS Controller Software
A PC application called WAIS Controller provides the GUI for the WAIS. It monitors all system elements to keep operators apprised on the state of the entire system in real time. The WAIS Controller software provides an Overview screen, which manages connections among any set of sites desired, or the operator can switch to the Local View at times when it is necessary to 101 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 focus on activity for a single site. Making and breaking connections are simple using the point and- click interface. All sites in the WAIS can have their own graphical icon, making it easy to identify each site at a glance. The WAIS Controller has a library of preset icons, but custom icons can be added.
Capabilities
· LAN, WAN, or the Internet can be used to link sites · Multiple vocoders available; allows optimization of bandwidth used versus features required · Distributed design approach eliminates single points of failure and Ensures reliable communications · New control points or system users can be added at any time to any point in the network · Local Interoperability Systems remain operational in the event of network failure · Pre-installed icon library to customize software interface · Software password protection for controlled access
Illustration: 64
Photo caption: Upper: A WAIS local interoperability site featuring the ACU-1000 Middle: A local view WAIS Controller GUI screen. Bottom:WAIS sample block diagram.
Illustration: 65
102 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Internet Databases: Archie
Archie was the first search engine ever invented, designed to index FTP archives, allowing people to find specific files. The original implementation was written in 1990 by Alan Emtage, Bill Heelan, and Peter J. Deutsch, then students at McGill University in Montreal.
The earliest versions of Archie simply contacted a list of FTP archives on a regular basis (contacting each roughly once a month, so as not to waste too much resources on the remote servers) and requested a listing. These listings were stored in local files to be searched using the UNIX grep command. Later, more efficient front- and back-ends were developed, and the system spread from a local tool, to a network-wide resource, to a popular service available from multiple sites around the Internet. Such archie servers could be accessed in multiple ways: using a local client (such as archie or xarchie); telneting to a server directly; sending queries by electronic mail; and later via World Wide Web interfaces.
The name derives from the word "archive", but is also associated with the comic book series of the same name. This was not originally intended, but it certainly acted as the inspiration for the names of Jughead and Veronica, both search systems for the Gopher protocol, named after other characters from the same comics.
Internet Databases: Gopher Gopher is a distributed document search and retrieval network protocol designed for the Internet. Its goal is to function as an improved form of Anonymous FTP, with features similar to that of the World Wide Web.
The Gopher protocol offers some features not natively supported by the Web and imposes a much stronger hierarchy on information stored on it. Its text menu interface is well-suited to 103 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 computing environments that rely heavily on remote computer terminals, common in universities at the time of its creation. Some consider it to be the superior protocol for storing and searching large repositories of information.
The World Wide Web was in its infancy in 1991, and Gopher services quickly became established. However, by the late 1990s, Gopher had almost disappeared. Insofar as information management is concerned, the progress from Gopher to the web as a standard can be seen simply as a natural progression from text-based to graphical interfaces. Several other factors contributed to the acceleration of Gopher's decline:
· In February of 1993, the University of Minnesota announced that it would charge licensing fees for the use of its implementation of the Gopher server. As a consequence of this some users suspected that a licensing fee would be also charged for independent implementations. In contrast, there was no such limitation on the World Wide Web. The University of Minnesota eventually re-licensed its Gopher software under the GNU GPL.
· Gopher's functionality was quickly duplicated by early Web browsers, such as Mosaic. Furthermore, the greater flexibility of the Web's HTML, and particularly its integration of text and graphics, encouraged the migration of content from Gopher to the World Wide Web.
· Gopher has an inflexible structure when compared to the free-form HTML of the Web. With Gopher, every document has a defined format and type, and the typical user must navigate through a single server-defined menu system to get to a particular document. Many people did not like the artificial distinction between menu and fixed document in the Gopher system, and found the Web's open-ended flexibility much more useful for constructing interrelated sets of documents and interactive applications.
Availability of Gopher today
As of 2006, there are fewer than 1000 gopher servers on the internet. Many of them are owned by universities in various parts the world. Most of them are neglected and rarely updated except for the ones run by enthusiasts of the protocol. A handful of new servers are set up every year by hobbyists - 25 have been set up and added to Flood gap’s list since 1999 and possibly some more that haven't been added. Today Gopher exists as an almost forgotten corner of the internet - one can publish email addresses in plaintext without having to worry about spam, and publish large amounts of files without the risk of the server's bandwidth becoming saturated, while at the same time people do still browse the gopher servers regularly.
Some have suggested that the bandwidth-sparing simple interface of Gopher would be a good match for mobile phones and Personal digital assistants (PDAs), but so far, the market prefers Wireless Markup Language (WML)/Wireless Application Protocol (WAP), DoCoMo i-mode, XHTML Basic or other adaptations of HTML and XML. The PyGopherd server, however, provides a built-in WML front-end to Gopher sites served with it.
The Microsoft Windows Vista operating system (2007) has dropped support for the Gopher protocol.
Gopher characteristics
104 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 A Gopher system consists of a series of hierarchical menus. The choice of menu items and titles is set by the administrator of the server.
Illustration: 66
The top level menu of a Gopher server. Selecting the "Fun and Games" menu item...
Illustration: 67
105 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327
... takes the user to the "Fun and Games" menu.
Similar to a file on a Web server, a file on a Gopher server can be linked to as a menu item from any other Gopher server. Many servers take advantage of this inter-server linking to provide a directory of other servers that the user can access.
Illustration: 68
Gopher support in Web browsers
106 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327
Mozilla Fire fox 1.5 displaying the top-level menu of the Floodgap gopher server Gopher support was disabled in Internet Explorer versions 5.* and 6 for Windows in June 2002 by a patch meant to fix a security vulnerability in the browser's Gopher protocol handler; however, it can be re-enabled by editing the Windows registry. In Internet Explorer 7, Gopher support was removed on the WinINET level. Internet Explorer for Mac (only on PowerPC architecture and in End-of-life) still supports Gopher.
Other browsers, including Mozilla and AOL, still support the protocol, but incompletely — the most obvious deficiency is that they cannot display the informational text found on many Gopher menus. Konqueror needs a plug-in to be installed for full Gopher support. Mozilla Firefox has full Gopher support as of release 1.5 and partial support in previous versions. The Sea Monkey Internet suite, successor of the Mozilla all-in-one suite, also supports Gopher fully, as does Camino, a browser based on Mozilla's engine. Such Mozilla based browsers are able to display embedded images from a gopher server on an HTTP-based HTML document and follow download links to a gopher server. However, the most extensive gopher support is offered in Lynx, a text based browser.
The Safari Web browser does not support Gopher at all while Opera requires the use of a proxy such as Squid.
UNIX Gopher client
A purpose-made open source gopher client for gopher exists. This is simply called gopher and is available in most Linux software repositories and source packages are available on the internet as well as some gopher servers. This client has the ability to fetch additional details about the files such as their size and their owner.
Gopher to HTTP gateways
Users of Web browsers that have incomplete or no support for Gopher can access content on Gopher servers via a server gateway that converts Gopher menus into HTML. One such server is at Floodgap.com. By default any Squid cache proxy server will act as a Gopher to HTTP gateway. Some Gopher servers, like PyGopherd, also have built-in Gopher to HTTP interfaces.
What is the “World Wide Web?” The World Wide Web is a global, seamless environment in which all information (text, images, audio, video, computational services) that is accessible from the Internet can be accessed in a consistent and simple way by using a standard set of naming and access conventions
Whew! Quite a statement, but it is true, and it exists today. You are on the Web now - consider.... You can access sites all over the world. You can connect from your desktop to thousands of Web servers simply by "clicking" on a selection (the underlined words), or by entering a specific address. You can connect to many different types of systems - and not be aware of the differences
107 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 You can access many different types of information - text, images (like the heading on this page, and the diagrams we'll use), audio, video, computational services - again, usually with no extra work on your part
You are using a single Web-browser to do this. One tool accessing many different types of systems, information across the world!
Already we can see some of the unique aspects of the Web that make it so popular:
· it is easy to use
· it is easy to move from place to place
· it combines words, graphics....even sound and movies - any data type!
· there are many tools (like this browser) that make the Web easy to use
· it is easy to publish information
· and there are millions of people using it now - and more every day!
The Web was initially conceived by Tim Berners-Lee and others at CERN. The scientists at CERN needed access to a wide variety of information on many different, distributed, computers. Berners-Lee had this idea of universal readership, which is that any client should be able to read any information. Berners-Lee developed the basis ideas, which others have since added to. Then those involved agreed to work by a common set of principles:
1. There would be no central control. The Web works because people work within the agreed-to guidelines. As part of this the Web ethic is that anyone can publish, and anyone (who is authorized) can read information.
2. All Web servers would use the same protocols/mechanisms....
1. http, a fast, stateless, extensible transport mechanism would be used to communicate within the Web
2. http, or http daemons, would be the base Web server - receiving messages and providing data as requested
3. URLs (Universal Resource Locator) would be used for network-wide addressing
4. all Web browsers would use the same basic language - Hypertext Markup Language HTML
3. And built into the mechanisms is support for format negotiation. Web clients tell servers what formats they can handle, and Web viewers allow basic browsers to use different formats
We'll look at those mechanisms in more detail in a few minutes. But one of the most important factors in the success of the Web is that it is built on the Internet, so next we'll look at the relationship between the Web and the Internet.
The Web and the Internet
108 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 The basis for the Web is the Internet. The Web is built on the Internet, and makes use of many of the mechanisms the Internet provides.
The Internet is the physical aspects - computers, networks, services. It allows us to connect to thousands of other computers across the world. But it doesn't mean that those systems users' can look at, and understand, the information there. The Web is an abstraction and common set of services on top of the Internet. It is the set of protocols and tools that let us share information with each other.
The Web was developed with the concept of "universal readership" any participating system should be able to read the information on any connected system using a common set of tools browsers servers/gateways addressing schemes common protocols format negotiation Illustration: 69
This is a generalized picture, but shows many aspects of the physical net. Your system, which has a unique number assigned to it (an IP address), is connected to an Internet Service Provider, possibly through a dial-in modem, or by a direct connection. There are currently about 2,300 of these ISPs in the States, most of which run local networks of their own with multiple Points of Presences (POPs), allowing you to dial in to a local number, even if your ISP is not located nearby.
The ISP in turn is connected to other providers, and eventually to one of the big carriers, who have huge networks that use fiber optic cables running at 45 mb/second (these are referred to as T3s). At the other end there is probably an ISP who gets a request you made to a specific web address, and who will route it to the server at that address, which then does its magic, and sends you back what you asked for - again using the various pieces of the network we just went through. How the Web is Used Today
109 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 The Web is used in many creative and interesting ways today, and new uses are being introduced nearly daily. This page contains links to some examples of how the Web is used. Many of these pages are available for you to use at any t ime, as a part of Web Central:
· By companies all over the World
· for external communication...
· to share product information
· to learn about the marketplace
· to share in process work with business partners
· and for internal sharing of information....
· by business people....
· for education at all levels....
· by technical people....
· for reference Information
· for online news...
· to provide financial information...
· by lawyers....
· for library information...
· for government information...
· for personal services...
· and for information about the Web itself!
· ..and for some unusual reasons...
Illustration: 70
110 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Let's look at that simplified view in a little more detail. This has the same basic components, but we've now added a new server path, which can be one that goes through a firewall, and/or which accesses applications.
A firewall is a mechanism to control access to and from Web servers. Most companies have firewalls set up to prevent access to their internal servers from external clients.
Illustration: 71
111 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 There are many different clients, such as Netscape or Microsoft's Explorer. The Web client usually sends an http message, but as the diagram shows it can send any Internet message (e.g. ftp, file, gopher, wais). The servers can be behind a firewall, which is a way to prevent access to a server. Servers can include gateways which allow them to "talk" to applications. These gateways can be in any computing language, the most common being perl and TCL, but C and other languages are also used. Finally, the components know how to find documents because they all use a standard addressing scheme, or URLs.
Note that there is no central control. Anyone can create a Web server, and for the most part anyone can read what is online. The reason it all works is because everyone is using the same set of "standards".
Illustration: 72
112 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 World Wide Web Servers
The phrase "World-Wide Web" is often used to refer to the collective network of servers speaking HTTP as well as the global body of information available using the protocol. In a May 1996 survey Netcraft found 193,150 servers on the Web (and who knows how many more exist behind corporate firewalls?).
A Web client (or browser) sends requests to a Web server.
Every retrievable piece of information on the Web is identified by a URL, which includes the name of the object, where it is located, and the protocol used to get it.
Only information on a server (or your local system) is part of the Web. You need to "publish it" (i.e. put it on a Web server) to make it accessible.
The Web server is responsible for document storage and retrieval. It sends the document requested (or an error message) back to the requesting client. The client interprets and presents the document. The client is responsible for document presentation.
The language that Web clients and servers use to communicate with each other is called the Hypertext Transfer Protocol (HTTP). All Web clients and servers must be able to speak 113 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 HTTP in order to send and receive hypermedia documents. For this reason, Web servers are often called HTTP servers, or HTTP Daemons (HTTPD).
A Sample Web Request · In this example you are using a browser (Netscape, or Mosaic, or some other browser) and you click on a reference to Webmaster Magazine Online. The browser is able to figure out that what you really want (in Web terminology) is the object the Web knows as http://www.cio.com/WebMaster/wmhome.html - the Web address (or URL) for Webmaster Magazine Online.
Illustration: 73
A typical transaction between Web servers and clients.
· The browser sends your request to the right server. How it figures out what that machine is, and where it is, is beyond this discussion, but gets us into looking at name servers and other tools which keep track of domain names, address names, and phys ical locations. Let's trust that some bit of magic occurs, and the browser can send your request off to the machine that handles requests for www.cio.com.
In the message it sends there is a lot of information you don't need to see, like the method to be used, the URL, possibly parameters (used when you want to pass a search string, for example), and other information.
· When it gets to the server the Web server, also known as the http (which stands for http daemon) takes over. It knows where it stores Web objects, and it tracks down the one you asked for. It might pass your request off to another process (like a sea rich engine or an application), and wait for a reply. When it gets what it asked for, it sends the object back to your browser.
114 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 If the document contains several Web objects (for example, this document includes several different graphic files as well as the HTML you are reading, each of which is a separate object), the server will send each of these objects back individually.
· Your browser collects together the different pieces you requested. A Web page can have references to objects all over the net, and may include objects that are on your local system (for example objects that it cached earlier). It pulls them all together , and presents the finished product to you.
Lesson IV
1. Electronic Mail
Electronic mail, also known as e-mail, is probably the most common method of using the Internet. You must have access to the Internet to use e-mail. Many teachers can access the Internet through their home institution. This access appears to be "free," but someone, somewhere is paying for the connection. You can also have access from free net systems (similar to a public broadcasting operation where user donations support the system) or commercial providers (e.g., CompuServe, AOL, Prodigy). Many of the latter exist, and you need to shop wisely to get the most service and access for your money.
Once you have access, you also need software so that your computer can talk to other computers; this involves sending commands, receiving and sending mail, and any other general communication functions computers do. You need to install the software, set the parameters, and begin sending and receiving mail. Several e-mail programs exist, and the way they function differs, so it is best to (1) read the instructions and, if they make little or no sense to you, then (2) get someone who already knows how to do this to help you set it up.
How does e-mail work? You can send and receive original messages, include parts of messages in other messages, reply to messages, forward messages, and save your messages to a file. These functions are all done by commands, which differ from system to system. It is, therefore, very important to understand how your system works so you do not forward a message to someplace you wish you hadn't. If you hit the "reply" button, make sure you know to what address the message is headed. E-mail addresses are a bit like a teeter-totter with the "@" sign as the fulcrum:
Other Internet Clients: [File Transfer Protocol]
ABOUT FTP
FTP is short for File Transfer Protocol, this page contains additional information about the FTP command and help using that command in Unix and MS-DOS (Windows).
Windows FTP
From the MS-DOS prompt or shell type in FTP, once typed in you will have access to the FTP command line. In this command line type: open ftp.address.domain
115 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Where address is the name of the server and the domain is the domain such as .COM, .NET... In addition, the IP address can be typed in, such as 255.255.255.0.
Once connected you will be asked for a username and password; if done successfully, you will have access to transfer files between computers.
FTP Commands Depending upon the version of FTP and the Operating System being used, each of the below commands may or may not work. Generally typing -help or a ? will list the commands available to you. Command Information ! Using this command you will have the capability of toggling back and forth between the operating system and ftp. Once back in the Operating System generally typing exit will take you back to the FTP command line. ? Access the Help screen. abor Abort Transfer append Append text to a local file. ascii Switch to ASCII transfer mode bell Turns bell mode on / off. binary Switches to binary transfer mode. bye Exits from FTP. cd Changes directory. cdup Change to parent directory on remote system close Exits from FTP. cwd Change working directory on remote system dele Delete file on remote system delete Deletes a file. debug Sets debugging on / off. dir Lists files if connected.
dir -C = Will list the files in wide format. dir -1 = Lists the files in bare format in alphabetic order dir -r = Lists directory in reverse alphabetic order. dir -R = Lists all files in current directory and sub directories. dir -S = Lists files in bare format in alphabetic order. disconnect Exits from FTP. get Get file from the computer connected to. glob Sets globbing on / off. hash Sets hash mark printing on / off help Access the Help screen and displays information about command if command typed after help. lcd Displays local directory or if path typed after lcd will change local directory. list Send a list of file names in the current directory on the remote system on the data connection. literal Sends command line ls Lists files if connected.
116 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 mdelete Multiple delete mdir Lists contents of multiple remote directories mget Get multiple files mkd Make directory. mkdir Make directory. mls Lists contents of multiple remote directories. mode Specifies the transfer mode. Available parameters are generally S, B or C. mput Sent multiple files nlst Send a full directory listing of the current directory on the remote system on the data connection. open Opens address. pass Supplies a user password. port Specify the client port number. prompt Enables/disables prompt. put Send one file pwd Print working directory quit Exits from FTP. quote Send arbitrary ftp command recv Receive file retr Get file from remote system. remotehelp Get help from remote server rename Renames a file rmdir Removes a directory send Send single file status Shows status of currently enabled / disabled options trace Toggles packet tracing type Set file transfer type user Send new user information verbose Sets verbose on / off.
Newsgroup
117 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Although most of the hype and attention that the Internet gets today is about e-commerce and business, there are two main reasons that most of us use it: communication and information. We rely on the Internet to send e-mail and instant messages, and search through the World Wide Web to find information for work or play.
Illustration: 74
Microsoft's Outlook Express contains a newsgroup client.
One source of both information and communication is newsgroups. A newsgroup is a continuous public discussion about a particular topic. You can join a newsgroup at any time to become part of a huge conversation between hundreds or even thousands of people.
Newsgroups originated in North Carolina back in 1979. That's when a couple of Duke University students hooked a few computers together to start an exchange of information with other UNIX users. Just down the road at the University of North Carolina in Chapel Hill, another student was writing software that could be used to distribute the information. Eventually, the work of these three students became the first bastion of newsgroups, termed Usenet.
Talk Amongst Yourselves Along with e-mail, newsgroups are one of the oldest communication methods on the Internet. But there are many ways to communicate on the Web. You probably use more than one method, depending on your needs. Let's take a look at the different methods and when you might use them: · E-mail - By far the most popular means of communicating over the Internet, e-mail allows you to send a message directly to another person or group of people. Messages can range from short to long and may include quotes or attached files. You can learn more about e-mail in the article How E-mail Works.
E-mail is most effective when:
118 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · You don't need an immediate answer. · You are communicating with a single person or specific group of people. · You know with whom you wish to communicate. · You need to attach a file or provide extensive information.
· Chat - Chat is a conversation between two or more people that takes place in a chat room. The chat room software allows a group of people to type in messages that are seen by everyone in the "room." Chat rooms can be found all over the Internet, including on the America Online service and the Web site TalkCity.com.
Chat is most effective when:
· You need an immediate answer. · You want to communicate with more than one person. · You can communicate in brief messages the information you need to know or wish to provide. · You want to meet new people.
· Instant messages - Instant messaging is something of a cross between chat and e-mail. It allows you to maintain a list of people that you wish to interact with. You can send messages to any of the people in your list, as long as that person is online. Sending a message opens up a small window where you and your friend can type in messages that each of you can see.
Instant messages are most effective when:
· You need an immediate answer. · You only need to communicate with a single person or small group. · You know with whom you wish to communicate. · You need to communicate in real time.
· Newsgroup - As stated earlier, a newsgroup is a continuous public discussion about a particular topic. Newsgroups are decentralized, which means that the messages are not maintained on a single server, but are replicated to hundreds of servers around the world.
Newsgroups are most effective when:
· You don't need an immediate answer. · You want to communicate with more than one person.
· You want to communicate with a group of people interested in the same topic. · You need or want to provide extensive information about that topic.
· Forum/Discussion Board - Forums and discussion boards are very similar to newsgroups, with one major difference: Most forums and discussions boards are kept on a single server maintained by the owner or originator of the forum or discussion board.
Forums or discussion boards are most effective when:
· You don't need an immediate answer. · You want to participate in a community that is discussing a particular topic. 119 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · You want to communicate with a group of people interested in the same topic. · You need or want to provide extensive information about that topic.
· Listserv - Most of us probably belong to one listserv or another. Every time you register for a newsletter, such as the free HowStuffWorks newsletter, you are placed on a listserv. Basically, this is a type of broadcast e-mail. Information on a listserv is sent to everyone who is listed in the e-mail group on the server. The biggest difference between a listserv and a newsgroup is that listservs are not interactive.
Listservs are most effective when:
· You don't need an immediate answer. · You want or need regularly updated information about a particular topic. · You want to receive information from a group of people interested in the same topic.
· Conferencing - Conferencing is like a chat room on steroids. The conference software, such as Microsoft Netmeeting, allows you to have a real-time chat with one or more other users. It also allows you to do such things as share an electronic whiteboard or a software application. Most conferencing-software packages provide several means of communication, including text only, audio and even video.
Conferencing is most effective when:
· You need an immediate answer. · You want to communicate with more than one person. · You can communicate in brief messages the information you need to know or wish to provide. · You want to attach files or use the whiteboard function in addition to sending and receiving traditional text messages.
· Video - Some users take advantage of a fast connection, such as a cable modem or DSL, in conjunction with a Webcam to communicate by way of video. This method of communication is not common yet, but will probably increase in popularity as the number of users with high-speed connections increases.
Video is most effective when:
· You need an immediate answer. · You want to visually demonstrate or display information. · You and the people you are connecting to have fast connections. · IP Telephony - Another emerging communications technology is IP telephony, which uses the Internet in much the same way that a regular telephone uses a phone line. Most IP telephony software requires that each person who wishes to talk have a computer, Internet connection and special software. But some companies, such as Net2Phone, provide software that allows you to call someone directly over the Internet and connect to that person's normal phone.
IP telephony is most effective when:
· You need an immediate answer.
120 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · You have the appropriate equipment (speakers, sound card and microphone). · The information you are providing or requesting is easier to convey through talking than through text messaging.
Most of the methods described above require some type of client software on your computer. In most cases, the client software is either free or integrated with another software application. For example, Netscape Navigator and Microsoft's Internet Explorer and Outlook Express each includes a newsgroup reader client that you can use to subscribe to newsgroups. We'll talk more about subscribing to newsgroups later.
The News Newsgroups use a lot of special terms to describe the newsgroup process: · Usenet - The primary exchange and listing of newsgroups · Feed or Newsfeed - The group of messages that make up a single newsgroup, sent from one server to another server or to a subscriber · Posting - Entering a message into a newsgroup · Posts or Articles - The messages that are entered into a newsgroup
Illustration: 75
Similar to most e-mail programs, you normally can click on the title of a newsgroup post in order to read the message.
· Thread or Threaded discussion - A post and the series of messages replying to it · Hierarchies - Category information provided in the name of the newsgroup
Newsgroups are categorized according to interest. The name of the newsgroup provides the category information, going from general to specific (left to right). For example, comp.lang.java.programmer is a newsgroup for Java programmers, in the Java section of the language category, which is part of the overall computer category.
121 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · Big Eight - Usenet's original eight newsgroup categories
Now, there are thousands of newsgroups in hundreds of categories, but Usenet originally divided newsgroups into one of eight major categories:
· comp (computers) · humanities (arts and culture) · misc (miscellaneous) · news (news and current events) · rec (recreational) · sci (science) · soc (social) · talk (general discussion)
Ironically, alt (alternate), which is now the biggest general newsgroup category, was not part of the original eight.
· Moderated - A newsgroup that has a person or persons who read all messages before they are posted to the general group and reserve the right to reject a message that they deem inappropriate for the newsgroup · Unmoderated - A newsgroup in which any message posted is immediately incorporated into the newsgroup (more common) · Flame - A criticism of someone else's post · News server - A server that maintains an archive of the messages posted to a newsgroup or series of newsgroups · Network News Transfer Protocol (NNTP) - The protocol typically used to transmit newsgroup messages over the Internet · UNIX-to-UNIX Copy Protocol (UUCP) - A protocol occasionally used for direct connections between some UNIX servers · Newsreader - The client software used to read the posts in a newsgroup, often combined with other software · Online - Type of newsreader that maintains a live connection to the news server while it is open · Offline - Type of newsreader that connects to the news server just long enough to download the new messages in any newsgroups you subscribe to and then disconnects, reconnecting when you're ready to send new messages
Illustration: 76
122 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Forte' Incorporated's FreeAgent newsreader is a good example of an offline newsgroup client.
The Newsgroup Process A newsgroup begins on a single news server, but is eventually replicated to hundreds or thousands of other servers. News servers provide the infrastructure that makes newsgroups work. Each news server has special software that maintains a file for each newsgroup serviced by that server.
Here's what happens when you access a newsgroup:
· Your newsreader, using NNTP, connects to the news server designated in your configuration. Typically, the news server's connection information is provided to you by your Internet Service Provider (ISP). If your ISP does not have a news server, you can refer to a list of publicly accessible news servers. · Once the connection is established, your newsreader downloads all of the new messages posted in the newsgroups that you are subscribed to. · You read through the messages and decide to reply to a couple. You also decide to start a new thread with a post of your own.
123 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Illustration: 77
Replying to a post in a newsgroup is as easy as replying to an e-mail.
· Your newsreader sends your messages to the news server. (If it's an offline newsreader, it must first reconnect to the news server via NNTP). · The news server saves your messages in the file for that newsgroup. Newsgroup files are large text files, meaning that each new message is simply appended to the end of the text file. As the file reaches a certain size, or after a certain length of time, the messages at the beginning of the file are removed and placed in a newsgroup-archive text file. · The news server connects to one or more other news servers using NNTP (or UUCP) and sends the updated information. Each news server compares its own file for the newsgroup with the files it receives for that same newsgroup. It adds any differences that it finds -- this is important, because if the news server simply saved the received file over the one it already had, it would lose any messages posted to it during the update. By comparing the files, it can extract the new messages and add them to the file it has, without losing any new postings. The news server then sends the combined file to the other news servers. · The newsgroup changes are replicated to each news server until all of them have the updated information. This process is ongoing, and most large newsgroups change so quickly that the updating is virtually continuous. · Other subscribers read your messages, plus all the others posted since the last time they looked at the newsgroup, and reply. · You see their replies and new messages, and the process repeats.
Other Internet Clients: Telnet 124 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 All Windows versions include Telnet Client and Telnet Server components. Using Telnet Client and Server, you can create a remote command console session on a host. You can run command line programs, shell commands, and scripts in a remote command console session just as though you were locally logged on to the host and using a local command prompt window.
Windows Server 2003 Telnet Client and Server are well suited for troubleshooting and configuring remote computers, especially in mixed environments that require interoperability between different operating systems. For example, you can use Telnet Client to connect to a Telnet server that is running on another operating system such as UNIX. Likewise, you can use a Telnet client that is running on UNIX to connect to a computer running Telnet Server. Windows Server 2003 Telnet Client and Server are also ideal in situations where memory and processor resources are minimal on a client or host or where network bandwidth is limited. This is because computers running Telnet clients and servers use less memory and processor time than other remote management tools, and Telnet clients and servers transmit only plaintext (unencrypted characters) across the network.
Understanding Telnet
Before using the Windows Server 2003 Telnet tools, you should consider the following:
2. Windows Server 2003 Telnet Client and Server are based on the Telnet protocol, which specifies a method for transmitting and receiving unencrypted ASCII characters (plaintext) across a network. Understanding how the protocol works, and how Telnet clients and servers use the Telnet protocol, helps you manage Telnet connections.
3. The Windows Server 2003 Telnet tools have several inherent limitations that affect the types of remote management tasks you can perform and the level of security that is in effect when you perform those tasks. Understanding these limitations helps you determine when and when not to use the Telnet tools.
4. You can configure Telnet Server settings by using the Windows Server 2003 Telnet administration tool (Tlntadmn.exe) and the registry editor (Regedit.exe). Although the default Telnet Server settings are sufficient for most Telnet client connections, you might need to change the default settings to better suit your organization. Examples of Telnet Server settings include: authentication type, default port assignment for Telnet connections, maximum number of client connections, and maximum number of failed logon attempts.
5. By default, members of the local administrators group can log on to a Telnet server. However, you might not want all Telnet users to have full administrative control of the host they log on to. In this case, you can use a Telnet clients group to grant users Telnet logon rights without granting them any administrative rights on the host. To configure these user rights from the graphical user interface, you must use the Active Directory Users and Groups snap-in or the Local Users and Groups snap-in. You can also use the Net User and Net Group commands to configure user rights from the command line.
6. You can configure several optional settings when you use Telnet Client to establish a Telnet session on a host. Depending on the type of Telnet server you are logging on to, and how the Telnet server is configured, you might need to enable or change some of these optional settings. Examples of Windows Server 2003 Telnet Client
125 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 settings include: client-side logging, terminal type, port assignment, and alternate user name for logon.
Note: The information in this document refers to the Telnet Client and Telnet Server components that are installed by default with Windows Server 2003 and Windows XP Professional operating systems.
Telnet Architecture
Most network operating systems provide a Telnet client and a Telnet server. Telnet clients and servers are small executable programs that allow a local computer (a client) to access services and programs on a remote computer (a host). Telnet clients and servers, including Windows Server 2003 Telnet Client and Telnet Server, are based on the Telnet protocol, which is a subset of the TCP/IP suite and is described in RFC 854. The Telnet protocol specifies two general mechanisms: how Telnet clients and servers establish a connection across a network and how they transmit and receive information across a network.
You usually run a Telnet client program on a local computer: for example, a workstation that you are logged on to. You usually run a Telnet server program on a remote computer: for example, a host you want to administer. Telnet client programs initiate connections with Telnet servers. Telnet servers run in the background on a host, listening for Telnet clients to request a connection.
Common Telnet Features
Because Telnet clients and servers are based on the same standard protocol, all Telnet clients and servers have several features in common. These common features are what make Telnet clients and servers well suited for performing remote administration tasks in environments that require interoperability among disparate operating systems. In short, the Telnet protocol makes it possible for you to connect a computer running Windows Server 2003 Telnet Client to a UNIX Telnet server. The key features that make this interoperability possible include the following:
Common communication protocols
All Telnet clients and servers use TCP/IP as the underlying communication protocol. This makes Telnet clients and servers particularly useful for remotely administering computers across the Internet or within wide area networks (WANs) that are connected to the Internet. If your network does not support TCP/IP, you will not be able to use a Telnet client or server.
Common communication ports
TCP port 23 is reserved for Telnet client and server communication. By default, most Telnet clients initiate communication on port 23, and most Telnet servers listen on port 23 for connection requests. You can change the default port assignments with some Telnet client and server software, such as Windows Server 2003 Telnet Client and Server, but port 23 is the universally accepted port for Telnet communication.
Common character set for communication
126 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 All Telnet clients and servers are capable of transmitting and receiving a predefined character set consisting of standard ASCII character codes and ASCII control codes. All Telnet clients and servers transmit the ASCII codes across a network in unencrypted form (plaintext).
Common implementation of the Network Virtual Terminal
All Telnet clients and servers implement a network virtual terminal (NVT). The NVT is responsible for translating operating system-specific instructions (keyboard codes or display codes) into a consistent set of codes that all Telnet clients and servers can transmit and receive. The NVT is what makes Telnet clients and servers capable of communicating with each other regardless of which operating system they are using.
Creating a Telnet Connection
To create a Telnet connection between a Telnet client and a Telnet server, you must do the following:
· Start the Telnet Server program on the host. On Windows Server 2003, Telnet Server (Tlntsvr.exe) runs as a service. You can start the service manually every time you want to connect to a host, or you can configure the service so that it starts every time your computer starts. Telnet clients cannot connect to a host unless a Telnet server program (or service) is running and listening for connection requests.
· Run the Telnet Client program on the local computer. When you run Windows Server 2003 Telnet Client (Telnet.exe), you must specify the host to which you want to connect. You can also configure several optional connection settings and features.
When you run a Telnet client, it makes a connection request to the host. If a Telnet server responds to the request, the Telnet client and server negotiate the details of the connection, such as flow control settings, window size, and terminal type. After the connection details are successfully negotiated, and logon credentials are validated, the Telnet server program creates a Telnet command console session.
On Windows Server 2003, each Telnet command console session consists of two processes: Tlntsess.exe and Cmd.exe. Tlntsess.exe is responsible for managing the Telnet session. Cmd.exe is the command interpreter, or shell program, that runs commands, programs, or scripts on the host.
Note: Cmd.exe is the default command interpreter for a Windows Server 2003 Telnet command console session. However, you can configure the Windows Server 2003 Telnet Server program to use as a default any command interpreter or shell program that is installed on the host.
Running Programs Remotely Using a Telnet Connection
After you establish a Telnet connection with Telnet Server, the following message appears in the command prompt window on the client:
127 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 *======
Welcome to Microsoft Telnet Server.
*======
This message indicates that your credentials are valid and that you have an active Telnet session with Telnet Server. Assuming you have the appropriate administrative user rights, you can use this session to remotely run command-line programs, shell commands, and scripts on a host. Telnet client and server processes rely on the Telnet network virtual terminal (NVT) to translate operating system-specific keyboard and display codes to Telnet character codes that all Telnet clients and servers can understand.
Telnet Limitations
Telnet connections have several limitations. You can address many of these limitations by changing Windows Server 2003 security and Group Policy settings, but the following limitation cannot be eliminated or modified.
You cannot run GUI tools over a Telnet connection
Telnet is a character-based communication protocol. It is not designed to transmit cursor movements or graphical user interface information. Because of this, you can only run command line programs, shell commands, scripts, and batch files over a Telnet connection. Some editing programs, such as vi and Edit, can be run over a Telnet connection; however, these interactive programs are not true GUI programs because cursor movement is controlled by the keyboard, not the mouse.
Lesson V: Media & Active Content
7. Object & Active Content
The growing popularity of the internet for a wide variety of information exchange and the features that have been developed to meet the demand for increased browser-based functionality have led to growing concern within organizations about their vulnerability to worms, viruses and Trojan horses. In particular, there is concern that these and other types of malicious code can be delivered covertly in programs that contain everything they need to run without requiring the intervention of the user. This type of program has come to be defined as ‘active content’, a term that refers to an object’s ability to act upon and change the way in which a user’s computer operates.
The threat
In the past there was always a distinction between static data and the programs by which it was called. Today many data objects such as web pages, mail and documents can interleave data and code, allowing dynamic execution of the program and giving external computers a way into the user’s computer. The transparency of the execution and the fact that the program is often called by a browser from a server on a remote website are seen as major potential security risks. Current concern centres round the security question of what happens if the code is malicious. Fears exist about the ability of active content to deliver, for instance,
128 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Trojans that lie dormant and undetected until triggered to cause widespread damage to an organization’s networks and business credibility, perhaps by stealing passwords, sending email, transferring money and so on – and destroying any information about where they came from. The threat can be summarized as follows:
– I send you a program and you run it to get my message. – The running program may be able to do other things with your rights.
Users’ fears are exacerbated by the claims of some vendors using terms like ‘the silent killer’ and other hyperbole to talk up the risks associated with code that carries a material threat and exploits other security vulnerabilities of operating systems and weak network design.
Delivery of active content
The two main technologies associated with the term ‘active content’ are ActiveX controls and Java which provide additional functionality to web pages, both of which are described later in this paper. In fact, the concept of active content goes back much further than either of these technologies. One of the earliest scripting languages, the page description language, PostScript, goes back in its current form to 1982 and is a classic example of active content, offloading the processing and interpretation of the presentation of documents to the printer. The commands are language statements in ASCII text that are translated into the printer’s machine language by a PostScript interpreter built into the printer. In an early example of the inappropriate use of active content, an attacker sent codes that altered the passwords on the receiving printer. Any subsequent jobs not using the password could not print. However, it is through the increased functionality of the browser – the client for web and other Internet servers – that active content has been able to flourish.
Browsers and active content
Browsers allow code to run in one of three ways.
1 Browser-initiated code
In some cases a file can be started by the browser to be executed by the operating system. In this case the browser might download a .DOC file and automatically run Word to process it without the user knowing, unless the “Confirm open after download” has been selected in Windows Explorer, under Options.
129 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 2 Browser-extending codes
Some programs are installed on to the hard disk to extend the browser. These include ActiveX controls and Netscape Navigator plug-ins which enable active content by allowing browsers to support different types of content such as audio, video and interactive animation. Some programs exist as both plug-ins and ActiveX controls and this seems likely to increase as Internet Explorer for Windows, versions 5.5 SP 2 and 6.0 no longer supports plug-ins.
3 Browser-interpreted code
This third type of browser-using code operates functionally within the browser and includes Java applets, HTML interpreters and interpreters for scripting languages. There is much focus on this area as web pages, which are written in HTML – and increasingly in more powerful XML-based languages, can have embedded within them programs written in scripting languages such as VBScript or JavaScript. Although such languages do not themselves have full access to the system’s resources, they can create a vulnerability as they can invoke, or attempt to invoke, ActiveX controls and/or Java applets as well as other software components such as plug-ins, document macros and other executable files. Because they execute on the browser’s side of the connection instead of the server’s they “move the security risk squarely from the server to the client, thus bringing the problem of security right down to the user’s desktop. In addition, the close binding of the browser to related functionality such as email capability or the underlying operating system means that the security that one might have expected in confining scripting languages to the boundaries of the web browser is not realized. Many people disable “Active scripting”, ie the ability for web pages to run scripts, but in doing so the tradeoff between security and functionality must be carefully weighed.
130 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Similar security can be achieved by turning off the Windows Scripting Host unless it is specifically needed for running stand-alone scripts. ActiveX ActiveX, announced in March 1996, is built on Microsoft’s COM (Component Object Model) and is a development of its OLE (Object Linking and Embedding) technology for transferring and sharing information among applications. ActiveX allows for the automation of many background tasks and consists of a set of technologies that enable software components to interact with one another in a networked environment, regardless of the language in which the they were created. Although ActiveX can be used in desktop applications and other programs, its primary aim is to allow desktop applications to interact with Microsoft’s Internet Explorer web browser. Netscape Navigator users can also use ActiveX controls but only via plug-ins. ActiveX ‘controls’ are reusable software components based on ActiveX technology. They are executable programs, compiled separately for each target operating system – which are installed on the user’s computer to extend the functionality of the browser by allowing applications to communicate with each other. ActiveX controls can be written in several different languages including Java, C++ and Visual Basic and run in an application program that uses the Component Object Model program interfaces. The controls perform many functions and can be invoked by web pages. They can be downloaded over the web, but many are pre-installed with Windows. When an ActiveX control is called by a web script, email or document, the browser checks to see if it is already installed on the user’s computer. If it is not, the embedded script calls the remote server on which the control is stored in order to download a copy. The browser can be configured to prevent ActiveX controls from being downloaded. However, if the downloading is allowed, the add-on program code is stored locally on the user’s machine and can then be used automatically by local programs or by an HTML page on that machine. It then remains on the client machine. Once an ActiveX control is installed on a user’s system, it has full access to more or less all the system resources, including those relating to the hardware. That a document, usually via an embedded script, can call upon a remote site for its program which is then rendered on the user’s computer causes legitimate worries about security. Germany’s Chaos Computer Club alleged that they had demonstrated a control that could check for the presence of a popular financial package and transfer money from an unknowing user’s bank account to another in Switzerland.
Digital signatures
ActiveX places no restrictions on what a control can do. Rather it relies on a number of security features built in to the browser and works on the basis that as long as users are as careful about installing ActiveX controls as they should be with .EXE files, the ActiveX should not endanger their system. Web browsers come with a pre-installed list of certifying
131 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 authorities that they trust, the most well-known being VeriSign. The certificates in the list can be viewed, activated/deactivated, installed and deleted.
Internet Explorer’s Authenticode feature, used in conjunction with VeriSign and other certification authorities, allows vendors of ActiveX controls to attach digital signatures to their products. Since version 4.0 which allowed customization, users of Internet Explorer can automatically check whether the certificate has been revoked and can choose to download unsigned software. A company creating an ActiveX control registers with VeriSign or another approved certification authority. After checking the credentials of the company, VeriSign issues a unique certificate to the company using a private key supplied by that company. When the company creates the program it wishes to sign, it activates the signing process by using the key and then appending the certificate. When a browser attempts to invoke the program, it first checks that the program is signed, and if it is, then checks that it comes from a ‘trusted’ company. Recent versions of Internet Explorer allow customization of what ActiveX controls can do, such as preventing scripting languages from interacting with them, and allow users to download controls automatically, disable them or be prompted with a dialog box, based on whether or not they are signed. Indeed, as stated above, the browser can be set to refuse all ActiveX controls.
132 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 The system of digital signatures was introduced so that users can feel certain that the software they are installing is safe by letting them know the publisher and underwriter. This remains, however, essentially a trust model and there remains a risk even if the ActiveX control is digitally signed. The certification process ensures that the control cannot be anonymous (allowing users to trace it back to the owner) and cannot be tampered with by third parties after its publication. However, it does not certify that it is safe. Even if it is issued, owned and used in good faith, just because it is signed does not mean that it does not have a virus. The Exploder ActiveX control developed – and later withdrawn – by Fred McLain was certified but performs a clean shutdown of any Windows 95 machine that downloads it. The weakness in the system is that although additional cryptographic information can be added to a digital signature to add confidence, the company which creates the ActiveX control is the person who initially signs it and sends it for authorisation, and controls can be fraudulently signed. In March 2001, VeriSign revoked two digital certificates that it had issued two months earlier to someone falsely claiming to be a representative of Microsoft. “In this scenario, it is possible that the fraudulent party could create a destru ctive program or ActiveX control, then sign it using either certificate and host it on a web site or distribute it to other web sites.
Safe for scripting
A further measure of security can be imposed on ActiveX controls to prevent unsafe controls from being initialized or scripted through a feature called ‘safe for scripting’. Controls not marked ‘safe for scripting’ cannot be launched, linked to or invoked from unsafe places like web pages. It was asserted in May 2001 that “of the thousand or so registered controls only 50 to 100 have the marked designation as safe for scripting” ‘Safe for scripting’ allows an ActiveX object to be given a discretionary marker to indicate to the operating system that code such as scripts from other people’s web pages can call it because it does not do anything Permanent, dangerous or dubious. As with digital signatures, not only is the user trusting the author not to have any ill intent, but is also assuming that the code cannot be subverted by others. Another problem is that Microsoft themselves ship their operating system with important and pre-installed ActiveX objects, which users have little choice but to trust. Some ActiveX objects implement an interface – a set of services that can be called from another script or program – with tens or hundreds of functions which users can invoke. Some of these have sub services, such as creating a pop up window, and all the functions need to be safe for the control itself to be safe. It is not surprising that some of the more obscure ones might be overlooked and later abused. This situation of controls believed to be secure and signed as such, but in fact turning out not to be represents a real threat. The VBS/Kakworm virus took advantage of just such a mistake, using a security hole in two Microsoft installed ActiveX controls (scriptlet.typelib and Eyedog) to write an infected file into the Windows startup folder so that it runs on starting Windows. The other danger is that someone can create some 133 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 malicious code, sign it as secure and incorporate a dialog box which will trick users into using it by relying on the well-proven fact that when presented with an OK button as the default option, many users will ignore the ‘Warning’ message and simply click on OK.
Indeed, this is how many Word macro viruses have been successfully spread – relying on what Russel Sanders has referred to as “the blind indifference with which end users treat dialogue boxes that appear to inform them of impending doom.”[2] This is echoed by Chess and Morar who state that “years of security experience show that users are all too ready to push ‘OK’ on a prompt that they do not understand, just to get it out of the way so they can get on with their jobs.
Java
Java is a cross-platform programming language created by Sun Microsystems in 1995, the ability of its applications to run on many different types of hardware platform being an example of so-called “mobile code”. Java programs are compiled into byte code which does not rely on platform specific instructions (as ActiveX does). They run in a special interpreted software environment called the Java Virtual Machine. Java applications are generally large, full-blown programs which, like other programs, can access any system resources allowed by an organization’s security. Java applets, however, are small programs which are stored on the server side of the connection. It is these which are usually called by web pages, referenced with the
Approaches to blocking malicious code
Although the threats arrive from the outside, the vulnerability is at the desktop and this is where the security mechanisms are focused. The dilemma is how to achieve maximum browser-based, ie client-side, functionality, while protecting the systems from security vulnerabilities.
At the browser
In reality, in looking at browser security we are talking about the security available through Internet Explorer (although Netscape Navigator provides very similar mechanisms)*. Security models in leading browsers and in the operating systems themselves can provide layers of protection, both in terms of physical security and by assigning/denying access right to particular users or groups of users. Different security settings in the browser give network and systems administrators a great deal of control over what can be received by whom. One caveat is that browsers assume the privileges of whoever logged in – so if this was an administrator, a script – malicious or otherwise – will run with the administrator’s privileges.
135 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Questions of the security mechanisms, such as digital signatures, built into active content have been dealt with above. These mechanisms are designed to trap any content which might be suspect and to prevent it accessing and compromising the system. Some vendors of content scanning packages believe that this is not sufficient and advocate that the only way that an organization can be secure against malicious code is to install software which will examine the behavior of all traffic on its networks. The argument is that as code such as ActiveX controls can be referenced in a stream of content in web pages, mail and news, the downloaded code should be monitored in real time and the program blocked if a security policy has been violated. While this behavioural analysis approach does have legitimacy, it is also wellknown to lead to completely safe files being falsely identified as malicious and vice-versa. It also brings greater overhead, and slows down performance. There are many other ways in which companies can protect their networks, not least of which is applying operating system patches immediately they are issued by Microsoft. Scanning email at the gateway will prevent malicious code from entering an organization’s network as will blocking certain file types or all files with double extensions, eg by writing a script at the email server, buying specific software which will do it, or requiring one’s ISP (Internet Service Provider) to do it. Installing and keeping up to date anti-virus software at the desktop will provide complete protection at the point of access. Viruses and Trojans, for instance, will each have their own identity and as soon as an attempt is made to run them, the program will be prevented from running. Some vendors raise the concern that PE (Portable Executable) compression utilities such as UPX and Petite can compress files, including of course malicious code, which when unpacked runs directly in memory, thus bypassing anti-virus software. Furthermore, when unpacked the resulting file is not an exact copy of the original, making detection even more difficult. Nevertheless each manifestation will have its own identity – in exactly the same way that variants of other viruses have their own identities – and will be prevented from running as soon as an attempt is made to launch it. A full description of other means by which systems can be protected against malicious code can be found in.
136 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 ------
8. What are the types of Browser Plug-ins
The Web Browser Plug-in section of Web Tools contains direct links to information about the most widely used plug-ins and helper applications. A list of online resources is also provided which contains detailed information on these and other plug-ins and helper applications not specifically addressed in this section.
Browser Watch’s Plug-In Plaza! Netscape Inline Plug-ins
The Web Browser Plug-in section of Web Tools contains direct links to information about the most widely used plug-ins and helper applications. A list of online resources is also provided which contains detailed information on these and other plug-ins and helper applications not specifically addressed in this section.
Plug-ins
Below is a compiled list of the most widely used web browser plug-ins. Each direct link contains detailed information specific to each of the plug-ins. Download information is also provided at these sites.
------
Beatnik Beatnik delivers high-quality interactive sound from websites. It is provided by Headspace, Inc. and is available for Netscape Navigator and Communicator on both Macintosh PowerPC and Windows 95/NT. The Headspace website offers detailed information on Beatnik, as well as an array of sites that showcase the plug-in's capabilities.
Quick Time
QuickTime, a product of Apple, Inc., is capable of delivering multimedia such as movies, audio, MIDI soundtracks, 3D animation, and virtual reality. It is available to Macintosh and Windows 3.x/95/NT. The QuickTime package contains a plug-in and helper applications. The 137 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 QuickTime Plug-in allows QuickTime and QuickTime VR content to be viewed directly within a browser. The Movie Player and Picture Viewer, helper applications, allow all QuickTime multimedia to be played (file creation and editing can be completed with QuickTime.
RealPlayer
RealPlayer is a live and on-demand RealAudio and Real Video player which functions without download delays. It is provided by Real Networks, Inc. and is available for Macintosh, Unix, and Windows 3.1/95/NT as both a plug-in and helper application. To test your RealPlayer plug-in, visit any of the sites listed in their showcase. The plug-in is compatible with many popular browsers. For a complete listing visit the RealPlayer system requirements page.
Shockwave
The Shockwave plug-in, provided by Macromedia, Inc., allows multimedia files created using Macromedia's Director, Author ware, and Flash to be viewed directly in your web browser. The plug-in is compatible with Netscape Navigator 2.0 or later and Internet Explorer 3.0 or later on Macintosh and Windows 3.1/95/NT platforms. The Macromedia site also provides detailed information about Shockwave, including a section called Shock zone where users can view Shockwave in action.
Vivo Active Player
Vivo Active Player delivers on-demand video and audio from any website offering Vivo Active content. This product, provided by Vivo Software, Inc., is available for Netscape Navigator and Microsoft Internet Explorer browsers on Power Macintosh and Windows 3.x/95/NT platforms. Learn more about the Vivo Active Player and test its capabilities at the Vivo Software website.
9. Additional file format
------
10. Image file format
Image File Formats Which to use?
138 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Briefly, the three most common image file formats, the most important for general purposes today, are TIF, JPG and GIF. I propose we also consider the new PNG format too.
Best file types for these general purposes:
Photographic Images Graphics, including Logos or Line art
Properties Continuous tones, 24 bit color or 8 Solid colors, up to 256 colors, with bit Gray, no text, few lines and text or lines and sharp edges edges
Best Quality for TIF or PNG PNG or GIF or TIF Archived Master (no JPG artifacts) (no JPG artifacts)
Smallest File Size JPG with a higher Quality factor can TIF LZW or GIF or PNG be decent (JPG is questionable (graphics/logos usually permit quality for archiving master copies) reducing to 2 to 16 colors for smallest file size)
M a x i m u m TIF or JPG TIF without LZW Compatibility (the simplest programs may not or GIF (PC, Mac, Unix) read TIF LZW)
Worst Choice 256 color GIF is very limited color, JPG compression adds artifacts, and is a larger file than 24 bit JPG smears text and lines and edges
These are not the only choices, but they are good and reasonable choices.
TIF file format is the undisputed leader when best quality is required. TIF is very commonly used in commercial printing or professional environments.
Web pages require JPG or GIF or PNG image types, because that is all that browsers can show. On the web, JPG is the best choice (smallest file) for photo images, and GIF is most common for graphic images.
A common question is "How do I make my image files smaller?".
The JPG section following attempts to explain why the wonderfully small JPG files are NOT the best choice to be the master copy of your important image. However JPG cannot be beat for emailing photographs to friends, and for web page use. The JPG file format is the smallest by far, and a JPG copy should be used for such purposes (when file size is all important). For Line art and graphic files (as opposed to photographic images), then GIF files have historically been best, both for smallest size and for best quality.
But note that lowering scan resolution to reasonable values for the purpose is often the best file size improvement you can make.
139 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 The Next button will browse through the descriptions on the next pages, or you can use these shortcut links directly:
PNG - Portable Network Graphics
(.PNG file extension, the pronunciation 'Ping' is specifically mentioned in the PNG Specification). PNG needs to be mentioned. PNG is not the number one file format, but you will want to know about it. PNG is not so popular yet, but it's appeal is growing as people discover what it can do.
PNG was designed recently, with the experience advantage of knowing all that went before. The original purpose of PNG was to be a royalty-free GIF and LZW replacement (see LZW next page). However PNG supports a large set of technical features, including superior lossless compression from LZ77. Compression in PNG is called the ZIP method, and is like the 'deflate" method in PKZIP (and is royalty free).
But the big deal is that PNG incorporates special preprocessing filters that can greatly improve the lossless compression efficiency, especially for typical gradient data found in 24 bit photographic images. This filter preprocessing causes PNG to be a little slower than other formats when reading or writing the file (but all types of compression require processing time).
Photoshop 7 and Elements 2.0 correct this now, but earlier Adobe versions did not store or read the ppi number to scale print size in PNG files (Adobe previously treated PNG like GIF in this respect, indicated 72 ppi regardless). The ppi number never matters on the video screen or web, but it was a serious usability flaw for printing purposes. Without that stored ppi number, we must scale the image again every time we print it. If we understand this, it should be no big deal, and at home, we probably automatically do that anyway (digital cameras do the same thing with their JPG files). But sending a potentially unsized image to a commercial printer is a mistake, and so TIF files should be used in that regard.
Most other programs do store and use the correct scaled resolution value in PNG files. PNG stores resolution internally as pixels per meter, so when calculating back to pixels per inch, some programs may show excessive decimal digits, perhaps 299.999 ppi instead of 300 ppi (no big deal).
PNG has additional unique features, like an Alpha channel for a variable transparency mask (any RGB or Grayscale pixel can be say 79% transparent and other pixels may individually have other transparency values). If indexed color, palette values may have similar variable transparency values. PNG files may also contain an embedded Gamma value so the image brightness can be viewed properly on both Windows and Macintosh screens. These should be wonderful features, but in many cases these extra features are not implemented properly (if at all) in many programs, and so these unique features must be ignored for web pages. However, this does not interfere with using the standard features, specifically for the effective and lossless compression.
Netscape 4.04 and MS IE 4.0 browsers added support for PNG files on web pages, not to replace JPG, but to replace GIF for graphics. For non-web and non-graphic use, PNG would compete with TIF. Most image programs support PNG, so basic compatibility is not an issue. You may really like PNG.
PNG may be of great interest, because it's lossless compression is well suited for master copy data, and because PNG is a noticeably smaller file than LZW TIF. Perhaps about 25% smaller
140 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 than TIF LZW for 24 bit files, and perhaps about 10% to 30% smaller than GIF files for indexed data.
Different images will have varying compression sizes, but PNG is an excellent replacement for GIF and 24 bit TIFF LZW files. PNG does define 48 bit files, but I don't know of any programs that support 48 bit PNG (not too many support 48 bit in any form).
Here are some representative file sizes for a 9.9 megabyte 1943x1702 24-bit RGB color image:
File type File size
TIFF 9 . 9 megs
TIFF 8 . 4 LZW megs
PNG 6 . 5 megs
JPG 1 . 0 (1.0 / 9.9) is 10% file size megs
BMP 9 . 9 megs
Seems to me that PNG is an excellent replacement for TIFF too.
TIFF - Tag Image File Format
(.TIF file extension, pronounced Tif) TIFF is the format of choice for archiving important images. TIFF is THE leading commercial and professional image standard. TIFF is the most universal and most widely supported format across all platforms, Mac, Windows, Unix. Data up to 48 bits is supported.
TIFF supports most color spaces, RGB, CMYK, YCbCr, etc. TIFF is a flexible format with many options. The data contains tags to declare what type of data follows. New types are easy to invent, and this versatility can cause incompatibly, but about any program anywhere will handle the standard TIFF types that we might encounter. TIFF can store data with bytes in either PC or Mac order (Intel or Motorola CPU chips differ in this way). This choice improves efficiency (speed), but all major programs today can read TIFF either way, and TIFF files can be exchanged without problem.
Several compression formats are used with TIF. TIF with G3 compression is the universal standard for fax and multi-page line art documents.
141 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 TIFF image files optionally use LZW lossless compression. Lossless means there is no quality loss due to compression. Lossless guarantees that you can always read back exactly what you thought you saved, bit-for-bit identical, without data corruption. This is a critical factor for archiving master copies of important images. Most image compression formats are lossless, with JPG and Kodak PhotoCD PCD files being the main exceptions.
Compression works by recognizing repeated identical strings in the data, and replacing the many instances with one instance, in a way that allows unambiguous decoding without loss. This is fairly intensive work, and any compression method makes files slower to save or open.
LZW is most effective when compressing solid indexed colors (graphics), and is less effective for 24 bit continuous photo images. Featureless areas compress better than detailed areas. LZW is more effective for grayscale images than color. It is often hardly effective at all for 48 bit images (VueScan 48 bit TIF LZW is an exception to this, using an efficient data type that not all others use ).
LZW is Lempel-Ziv-Welch, named for Israeli researchers Abraham Lempel and Jacob Zif who published IEEE papers in 1977 and 1978 (now called LZ77 and LZ78) which were the basis for most later work in compression. Terry Welch built on this, and published and patented a compression technique that is called LZW now. This is the 1984 Unisys patent (now Sperry) involved in TIF LZW and GIF (and V.42bis for modems). There was much controversy about a royalty for LZW for GIF, but royalty was always paid for LZW for TIF files and for v.42bis modems. International patents recently expired in mid-2004.
Image programs of any stature will provide LZW, but simple or free programs often do not pay LZW patent royalty to provide LZW, and then its absence can cause an incompatibility for compressed files.
It is not necessary to say much about TIF. It works, it's important, it's great, it's practical, it's the standard universal format for high quality images, it simply does the best job the best way. Give TIF very major consideration, both for photos and documents, especially for archiving anything where quality is important.
But TIF files for photo images are generally pretty large. Uncompressed TIFF files are about the same size in bytes as the image size in memory. Regardless of the novice view, this size is a plus, not a disadvantage. Large means lots of detail, and it's a good thing. 24 bit RGB image data is 3 bytes per pixel. That is simply how large the image data is, and TIF LZW stores it with recoverable full quality in a lossless format (and again, that's a good thing). $200 today buys BOTH a 320 GB 7200 RPM disk and 512 MB of memory so it is quite easy to plan for and deal with the size.
There are situations for less serious purposes when the full quality may not always be important or necessary. JPEG files are much smaller, and are suitable for non-archival purposes, like photos for read-only email and web page use, when small file size may be more important than maximum quality. JPG has its important uses, but be aware of the large price in quality that you must pay for the small size of JPG, it is not without cost.
JPEG - Joint Photographic Experts Group
(.JPG file extension, pronounced Jay Peg). This is the right format for those photo images which must be very small files, for example, for web sites or for email. JPG is often used on digital camera memory cards, but RAW or TIF format may be offered too, to avoid it. The JPG file is wonderfully small, often compressed to perhaps only 1/10 of the size of the original data, which is a good thing when modems are involved. However, this fantastic compression 142 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 efficiency comes with a high price. JPG uses lossy compression (lossy meaning "with losses to quality"). Lossy means that some image quality is lost when the JPG data is compressed and saved, and this quality can never be recovered.
File compression methods for most other file formats are lossless, and lossless means "fully recoverable". Lossless compression always returns the original data, bit-for-bit identical without any question about differences (losses). We are used to saving data to a file, and getting it all back when we next open that file. Our Word and Excel documents, our Quicken data, any data at all, we cannot imagine NOT getting back exactly the original data. TIF, PNG, GIF, BMP and most other image file formats are lossless too. This integrity requirement does limit efficiency, limiting compression of photo image data to maybe only 10% to 40% reduction in practice (graphics can be smaller). But most compression methods have full lossless recoverability as the first requirement.
JPG files don't work that way. JPG is a big exception. JPG compression is not lossless. JPG compression is lossy. Lossy means "with losses" to image quality. JPG compression has very high efficiency (relatively tiny files) because it is intentionally designed to be lossy, designed to give very small files without the requirement for full recoverability. JPG modifies the image pixel data (color values) to be more convenient for its compression method. Tiny detail that doesn't compress well (minor color changes) can be ignored (not retained). This allows amazing size reductions on the remainder, but when we open the file and expand the data to access it again, it is no longer the same data as before. This lost data is like lost purity or integrity. It can vary in degree, it can be fairly good, but it is always unrecoverable corruption of the data. This makes JPG be quite different from all the other usual file format choices. This will sound preachy, but if your use is critical, you need a really good reason to use JPG.
There are times and places this compromise is an advantage. Web pages and email files need to be very small, to be fast through the modem, and some uses may not need maximum quality. In some cases, we are willing to compromise quality for size, sacrificing for the better good. And this is the purpose of JPG. There is no magic answer providing both high compression and high quality. We don't get something for nothing, and the small size has a cost in quality. Still, mild quality losses may sometimes be acceptable for less critical purposes. The sample JPG images on next page show the kind of problem to expect from excessive compression.
Even worse, more quality is lost every time the JPG file is compressed and saved again, so ever editing and saving a JPG image again is a questionable decision. You should instead just discard the old JPG file and start over from your archived lossless TIF master, saving that change as the new JPG copy you need. JPG compression can be selected to be better quality in a larger file, or to be lesser quality in a smaller file. When you save a JPG file, your FILE - SAVE AS dialog box should have an option for the degree of file compression.
Many programs (Photoshop, Elements, PhotoImpact, PhotoDeluxe) call this setting JPG Quality. Other programs (Paint Shop Pro and Corel) call it JPG Compression, which is the same thing, except Quality runs numerically the opposite direction from Compression. High Quality corresponds to Low Compression. Typical values might be 85 Quality, or 15 Compression. These numbers are relative and have no absolute meaning. Compression in one program will vary from another even at the same number. The number is also not a percentage of anything, and Quality 100 does NOT mean no compression, it is just an arbitrary starting point. JPG will always compress, and Quality 90 is not so different from Quality 100 in practice. There's very little improvement over 95.
Digital cameras also offer JPG quality choices too. Large image files do fill memory cards fast. You can buy more and larger cards, or you can compromise by sacrificing image quality for small file size (but I hope you won't go overboard with this). The camera menu will have two 143 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 kinds of resizing choices. One size choice actually creates a smaller image size (pixels), resampled smaller from the original standard size of the CCD chip, for example perhaps to half size in pixel dimensions. The correct image size in pixels is related to your goal for using the image. For example you may need enough pixels to print 8x10 inches on paper (6 megapixels), or you may only want a small image for video screen viewing (1 megapixel).
Regardless of that selected image size in pixels, the camera menu will also offer a smaller file size choice in bytes, related to quality, via JPG file compression. This menu will offer a best quality setting which is the largest file, and maybe intermediate sizes, and a smallest but worst quality choice. My Nikon D70 offers three JPG file size choices of Fine (about 1/4 size in bytes), Norm (about 1/8 size in bytes), or Basic (about 1/16 size in bytes), comparing compressed file size to the uncompressed size. The best (largest) JPG file size will still contain JPG artifacts, but very mild, essentially undetectable, vastly better than the smallest file choice. Even better, some cameras also offer a RAW or TIF format to bypass JPG problems all together. These images may be large, but memory cards are becoming less expensive ($100 for 1 GB), and larger or multiple cards are by far the best quality solution.
With either scanner or camera images, individual image JPG file sizes will vary a little, because detail in the individual image greatly affects compressibility. Large featureless areas (skies, walls, etc.) compress much better (smaller) than images containing much tiny detail all over (a tree full of leaves). Therefore images of the same size in pixels and using the same JPG quality setting, but with differing image content, will vary a little in JPG file size, with extremes perhaps over a 2 to 1 range around the average size.
Since each image varies a little, the file size is only a crude indicator of JPG quality, however it is a rough guide. For ordinary color images (24 bit RGB), the uncompressed image size when opened in memory is always 3 bytes per pixel. For example, an image size of 3000x2000 pixels is 6 mega pixels, and therefore by definition, when uncompressed (when opened), this memory size is 3X that in bytes, or 18 MB. That is simply how large the 24 bit data is. The compressed JPG file size will be smaller (same pixels, but fewer bytes). A High quality JPG file size might be compressed to 50% to 25% uncompressed size (bytes). A JPG file size only 10% of that image's size in memory would the general ballpark for a fair tradeoff of quality vs. file size for color images of web page quality (but not best quality).
The 10% size is not very precise, but of course only refers roughly to the average image size, since each individual image varies a little. Color compresses better than grayscale files, so grayscale doesn't decrease as much. These are very rough guidelines, your image, your photo program, your purpose, and your personal criteria or tolerance will all be a little different.
It is difficult to describe the JPG quality losses, except by seeing an example image (next page). JPG does not discard pixels. Instead it changes the color detail of some pixels in an abstract mathematical way. JPG is mathematically complex and requires considerable CPU processing power to decompress an image. JPG also allows several parameters, and programs don't all use the same JPG rules. Programs vary, some programs take shortcuts to load JPG faster but with less quality (browsers for example), and other programs load JPG slower with better quality. Final image quality can depend on the image details, on the degree of compression, on the method used by the compressing JPG program, and on the method used by the viewing JPG program.
144 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 JPEG Artifacts
I am sent JPG images with questions why they appear "bad" and very "unclear", when the only problem is excessive JPG compression, too much, or saved too many times, or both. Too little Quality or too much Compression will affect JPG image quality visibly. It can be awful.
You can and should learn to recognize this bad effect yourself. It's easy to recognize, almost trivial, after you've seen it once. Then you will know if it is acceptable or not, and you will know that you have choices.
How to see JPG artifacts the first time
If using a digital camera, shoot the same scene as the best file format you have, a RAW or TIF file, or at least at the best JPG quality setting (largest file). Also shoot exactly the same scene at the lowest JPG quality setting (smallest file). If using a scanner, scan a new photo image and save it as a TIF file. Then, after that, also save the same image as a JPG file using more compression than usual, a lower JPG quality setting this one time to help learn to see it. Then close that JPG image, and reopen that file to be sure you are viewing the contents of the file image you actually wrote (instead of the uncompressed image on the screen that you thought you wrote).
Then examine both large and small file images side by side on the same screen, by zooming in to about 4 times size (400%, huge) on both. You will have to scroll around on them, but the 400% is to help you learn to recognize the artifacts this first time. The differences you see are the JPG artifacts of compression.
The visible signs of excessive JPG compression are JPG artifacts, which include:
· Areas of similar color (generally featureless areas like skies or walls) become "blocky", with tile-like squares that are 8x8 pixel blocks of one color (compresses well), or which may contain horizontal or vertical patterns at extremes. You see the 8x8 squares, like big pixels, but at 400% zoom, you can recognize the size difference in the pixels (jaggies) and the 8x8 pixel blocks.
· The sharp contrasting edges in the image will have "vague dark smudges" surrounding the sharp edges. This is normally the worst effect in most images, it obsures detail. Excessive JPG compression is tough on all sharp edges in the image (especially so for text and graphic images). This is very counterproductive to apparent sharpness and fine detail.
· Some false color or color changes.
Here are samples below, zoomed large, but same size and side by side for comparison, to better learn to recognize the JPG artifacts (that is a cookie jar at left, with a little reflection on it)
145 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Original TIF image ABOVE, zoomed to 300% size
Low Quality JPG image BELOW, zoomed to 300% size
146 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Not that it seems necessary, but I have marked a few of the 8x8 pixel blocks seen in areas of similar color, and also some of the smudging around sharp edges (all edges). You can find more almost any place in the image. Notice the missing detail at the ridged knob marked in blue. The two power cords behind the fruit are nearly obscured by JPG artifacts. The tiny jaggies in the TIF are the original pixels zoomed to 300%. Their size is a guide to show the larger JPG 8x8 pixel blocks are clearly a different artifact.
Zooming to 300% size is not the normal thing to do, but it does help to recognize these JPG artifacts the first time. After you learn what you are looking for, then you can recognize them at 100% size too.
This sample shows what is meant by "lossy compression" in JPG - the losses are from deteriorating image quality due to the JPG artifacts. Using a Higher JPG Quality setting to produce a larger JPG file improves the JPG image quality. Using a Lower JPG Quality setting to produce a smaller file makes it worse. That seems straight-forward. A High JPG Quality setting of 8 or 9 (assuming a 1-10 scale, but 80 or 90 on a 1-100 scale is used by some software - same thing) is normally a pretty good image, normally not much problem for read-only uses, like viewing or printing (both zoomed images above are in a High quality JPG now for web presentation). The problem then is when you want to edit the image and save it yet again as JPG (additional JPG artifacts accumultate each time). We should only save a JPG image ONE TIME.
Note that most other types of image file compression (for example PNG or TIF LZW) are lossless compression, meaning that there is absolutely no loss of quality due to compression (zero loss), so that then file quality is simply not an isssue at all, and the most critical user 147 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 need not ever worry about it. The TIF file above used LZW compression. However lossless file compression is less effective, meaning that it can not produce files so drastically small as JPG. The lossless file size is closer to the actual size of the color data, perhaps 70% or 80% instead of 5% to 20%.
I want to call this JPG an extreme example, and it is poor, but it is not extreme. You ought to see some of the images that people send me asking why their images are so poor. This JPG was done in Photoshop, and Adobe's lowest quality settings are conservative, and won't let us make them as extreme and poor as some other programs will. The JPG quality numbers like 8 or 9 are NOT absolute values, instead they are relative to the JPG properties that each program chooses to individually use.
JPG artifacts do vary, and this will be of more concern when you do this same test on your own images. Once you realize what you are looking for, then JPG artifacts are easy to see and recognize at 100% size. Some people are more critical than others, asking "How good can I make it?" instead of "How small can I make it"? Your priorities are your own, but after you are able to recognize JPG artifacts, you will be able to judge how much of this you want in your own images.
Use a higher JPG Quality factor to minimize these effects (or don't use JPG at all if maximum quality is important). Less JPG Quality is more JPG compression, a smaller file, but worse artifacts. Normally you may detect some artifacts even at high Quality factors, and you can learn to recognize this easier if you zoom in to about 400% size. But the image can still be very usable size if the compression is mild. Now you know what to look for, and how to look for it, and how to judge if you want it or not.
The JPG artifacts become part of the image data, and it cannot be removed. Sharpening again after JPG compression (next time) will emphasize these JPG artifacts, so be careful with that. Actually, very slight intentional softening or blurring before JPG compression will help minimize the effect of the JPG artifacts (and will slightly reduce JPG file size too).
JPG normally should not be used for text or graphic images. It obscures the sharp edges too much, and the results are typically poor. TIF LZW, PNG, and GIF are vastly better for line art or graphic images, and these will normally compress text and graphics effectively. The way to make a scanned text document PDF file smaller is to scan in Line art mode and use compression. Acrobat will use G4 compression which is very small for line art, but LZW is good too. For text, line art will be smaller and better result than a JPG grayscale file. JPG cannot handle line art or indexed color anyway, JPG requires 8 bit grayscale or 24 bit color. However for continuous tone photo images, as opposed to text or graphics, then files with high JPG Quality (low compression) are normally acceptable for viewing (read-only purposes), and the small file size is extremely desirable for modems.
Due to the quality concerns, JPG compression is generally NOT suitable for archiving the important master copy of your image. With only mild compression, it might view OK, but you should grit your teeth, hold your breath, and cross your fingers for luck, if you ever have the need to modify and save a JPG file again. Because this will lower the quality of that image even more, every time you save the file. By "save", I mean to select the FILE - SAVE or FILE - SAVE AS or FILE - SAVE FOR WEB menu with JPG format from an image program. That SAVE step does the JPG compression again.
Note that downloading, or copying, or opening and viewing JPG files is no problem at all. This does not save the file again, and it does not alter the file in any way, so it does not create more artifacts. You can open and view a JPG file a jillion times without any concern (a web page for example). The artifacts are created only when the data is compressed for saving it
148 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 as a JPG file. This happens only at the menus FILE - SAVE or FILE - SAVE AS or FILE - SAVE FOR WEB when you select JPG format.
Archive your important master copy images in a non-lossy format (TIF LZW and PNG are very good, and compress moderately), and then also make a JPG copy if needed for view-only purposes that need it to be smaller, like email and web pages. You won't gain any quality by converting JPG to TIF now, because that image copy will still contain the JPG artifacts it had before. It is part of the image now, there is no way to improve it again. However, if you do need to edit a JPG, then saving it as TIF will prevent adding more artifacts by not doing another JPG Save, so TIF would be a good plan then.
If you find that you must edit the JPG image and must save it as JPG again, at least try to use the same program and same value of Quality or Compression every time you save the file. Using different values will use different parameters that will aggravate the damage due the lossy compression. I am certainly not suggesting repeated saving of JPG files with the same parameters is a good thing, but only that there are even worse ways to do it.
JPEG Lossless Rotation
Sometimes for viewing, we need to rotate an image beween horizontal or vertical (Landscape or Portrait), without needing any other editing operations. But if you save that rotated image as JPG, that is another SAVE operation and additional JPG artifacts. But there are image programs that can rotate a JPG file image exactly 90 or 180 degrees, and then save it again without additional loss, by rotating it without uncompressing and recompressing it first, thus preventing any additional JPG artifacts. This option merely rearranges the row and column data, without uncompressing that data. So while the JPG file itself is still lossy, the lossless rotation does not create additional artifacts.
Jpegclub.org has a list of programs that support this lossless rotation feature. Standard photo editor programs typically do not provide this option, but a few of the better known programs that can do this are
IrfanView Free, Windows GraphicConverter Trail available, Macintosh ThumbsPlus Trial available, Windows
IrfranView needs its free extra JPG plugin package to provide lossless rotation, which is then found at menu Options - Lossless JPG Operations when the JPG file is open. Irfanview also shows the JPG EXIF information well (Exchangeable Image File Format) - extra file data added by cameras showing information like camera name, date, f-stop, and shutter speed, exposure compensation, metering and flash used, etc.
More Preaching, but it is important
JPG format has a magic status for us. JPG is wonderful when the purpose is right, but we need to understand we pay a cost in quality. I know you surely will consider using JPG for master copies; everyone does at first, because the JPG file is so small, and the idea is frightfully appealing. But it's a high price paid in lost quality, and you will eventually come around to appreciate the quality of TIF files. I hope that happens before you have damaged important images that you cannot scan again. Some people argue that high quality JPG masters are not so bad. That's OK with me, it's your file, but the file size is one property of an image, and quality is another. You can choose either way, and I hope you choose Highest Quality, at least for your master copies. My goal is that you understand the effects of your choice in regard to your valuable master copies.
149 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 If you simply must use JPG for your master copy (if file size is the most important property of this image), then at least use the best Quality you can. Aim for a average JPG file size at least as large as 25% of its true memory size. For web images, file size is indeed important, and absolute quality is less important, and a JPG file size 10% of uncompressed size is often a good tradeoff of quality for size for color images, but that is too extreme for any notion of a master copy. Grayscale images do not compress as much as color at the same settings. Grayscale is already 1/3 the size of color, and grayscale JPG files won't compress as much.
Then keep a master copy that you never rewrite. For any purpose, always start with a copy of that JPG master and never modify that JPG master itself. Saving that JPG master image again causes more artifacts, even more loss, permanent loss, so simply don't do it. If you simply need to rotate to vertical, or to scale for printing, consider not saving that simple change at all if it is trivial to do again. Save any change to a different file, and never make your one master copy worse. Or save the change to a TIF file, so as to not create more JPG artifacts. You could of course just use a TIF master instead, and sleep better at night. And be aware that lowering scan resolution to reasonable values for the purpose is often the best size reduction you can make.
Judge your own common sense periodically. Why can't we see that a decent JPG image compressed to be only 25% of original size (1/4 size, still amazingly small, and still with mild artifacts), is more useful, and a much better deal, than one at 5% size that is simply too awful to use? I know, the macho types are saying "but I can get it down to 3%". Then go for it man!
For emailing and for web pages, JPG file format is the smallest by far, and the best goal by far (for photo images). JPG is acceptable for such read-only "viewing" use, and a JPG copy normally is used for such purposes. JPG is NOT very acceptable for "editing" use, when you may need to edit and save the image again (we always do). JPG is best only for photographic images. For line art and graphic files (characterized by containing graphic sharp edges and very few but solid colors), then TIF LZW or PNG is good, or GIF files too, which have historically been used for graphics on web pages.
The process of saving a JPG file to disk first compresses the image data to make the file be much smaller. This JPG compression affects the image quality, because JPG compression uses a very different technique than does PKZIP or TIF LZW or PNG file compression (lossy vs. lossless). JPG compression is extremely effective, because it does not attempt to be 100% faithful to the original image data. If parts of the image data are a problem for it, the JPG compression simply modifies that data to make it conform better. That means when we read that JPG file later, we don't get back the same original image data (that is the meaning of lossy, losses of image quality). Also the software opening that JPG image does a similar thing when it reads it. It does not necessarily show exactly what is in the file, it can also take similar liberties. Web browsers for example are not the best JPG image viewers, but they are the fastest JPG viewers.
Every time we save a JPG file (menu FILE - SAVE AS - JPG), it compresses again, and image quality is lost. This loss is variable, depending on the JPG Quality factor used, but it is never zero. If we edit the JPG file image and save it again, then quality is lost again. Even if we save it at a much higher quality setting the second time, it is still a second compression and another loss. Every time we compress a JPG file, we add more JPG artifacts and lose a little more quality. Artifacts are always a one way trip, there is no recovery. The damage is done.
Most other file formats (say TIF or PNG) use lossless compression. These files are larger than JPG because they use milder, fully recoverable (lossless) compression to carefully preserve all of the original image data. These file formats remain full quality at all times, no matter how
150 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 many times we might save them to a file, which is of course exactly what is needed for a master copy.
The small JPG file size is great, but it has a big price of reduced quality. There are proper times and places one would use it, and also major reasons one would not. There's nothing wrong with creating a JPG image using a moderate to high Quality factor to put a photo image on a web page or to send it via email. It's the only practical way. However there is an additional quality loss when we try to edit and save that JPG file a second time, so JPG is usually inappropriate for important archived master copies. The risk if you make this mistake now is that you cannot undo it in the future, so now is the best time to understand the situation. Progressive JPG is a web option that quickly shows the entire image in very low quality, and the quality fills in and improves as the file download completes.
JPEG 2000
A new JPEG 2000 file format is available now in many newer program versions, with various file extensions, normally .jp2 (or .jpg, with option extensions). JPEG 2000 uses a wavelet compression method. It has a lossless option (to perhaps 65% size). Otherwise it is lossy, size comparable to regular JPG files, but different... slower then JPG, but arguably perhaps better quality. Extremes of compression have few detectable artifacts; however JPEG 2000 images become noticeably soft with greater compression.
Paint Shop Pro 8 and Photo Impact 8 started including JPEG 2000. Adobe Elements 3.0 includes JPEG 2000. Elements 2.0 have an update to add JPEG 2000 - at the How To palette, Select a Recipe, Download New Adobe Recipes, then (very important) the BACK button and you will see it. Photoshop has an optional JPEG 2000 plug-in from CD; see JPEG 2000 at the CS or CS2 Help menu.
Note that web browsers do NOT support JPEG 2000 yet, and so compatibility is still a very significant issue. Therefore it seems important to stay with the original JPG format for now, at least if you want others to be able to open your files.
More Preaching, but it is important
JPG format has a magic status for us. JPG is wonderful when the purpose is right, but we need to understand we pay a cost in quality. I know you surely will consider using JPG for master copies, everyone does at first, because the JPG file is so small, and the idea is frightfully appealing. But it's a high price paid in lost quality, and you will eventually come around to appreciate the quality of TIF files. I hope that happens before you have damaged important images that you cannot scan again. Some people argue that high quality JPG masters are not so bad. That's OK with me, it's your file, but the file size is one property of an image, and quality is another. You can choose either way, and I hope you choose Highest Quality, at least for your master copies. My goal is that you understand the effects of your choice in regard to your valuable master copies.
If you simply must use JPG for your master copy (if file size is the most important property of this image), then at least use the best Quality you can. Aim for a average JPG file size at least as large as 25% of its true memory size. For web images, file size is indeed important, and absolute quality is less important, and a JPG file size 10% of uncompressed size is often a good tradeoff of quality for size for color images, but that is too extreme for any notion of a master copy. Grayscale images do not compress as much as color at the same settings. Grayscale is already 1/3 the size of color, and grayscale JPG files won't compress as much.
Then keep a master copy that you never rewrite. For any purpose, always start with a copy of that JPG master and never modify that JPG master itself. Saving that JPG master image again 151 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 causes more artifacts, even more loss, permanent loss, so simply don't do it. If you simply need to rotate to vertical, or to scale for printing, consider not saving that simple change at all if it is trivial to do again. Save any change to a different file, and never make your one master copy worse. Or save the change to a TIF file, so as to not create more JPG artifacts. You could of course just use a TIF master instead, and sleep better at night. And be aware that lowering scan resolution to reasonable values for the purpose is often the best size reduction you can make.
Judge your own common sense periodically. Why can't we see that a decent JPG image compressed to be only 25% of original size (1/4 size, still amazingly small, and still with mild artifacts), is more useful, and a much better deal, than one at 5% size that is simply too awful to use? I know, the macho types are saying "but I can get it down to 3%". Then go for it man!
For emailing and for web pages, JPG file format is the smallest by far, and the best goal by far (for photo images). JPG is acceptable for such read-only "viewing" use, and a JPG copy normally is used for such purposes. JPG is NOT very acceptable for "editing" use, when you may need to edit and save the image again (we always do). JPG is best only for photographic images. For line art and graphic files (characterized by containing graphic sharp edges and very few but solid colors), then TIF LZW or PNG is good, or GIF files too, which have historically been used for graphics on web pages.
The process of saving a JPG file to disk first compresses the image data to make the file be much smaller. This JPG compression affects the image quality, because JPG compression uses a very different technique than does PKZIP or TIF LZW or PNG file compression (lossy vs. lossless). JPG compression is extremely effective, because it does not attempt to be 100% faithful to the original image data. If parts of the image data are a problem for it, the JPG compression simply modifies that data to make it conform better. That means when we read that JPG file later, we don't get back the same original image data (that is the meaning of lossy, losses of image quality). Also the software opening that JPG image does a similar thing when it reads it. It does not necessarily show exactly what is in the file, it can also take similar liberties. Web browsers for example are not the best JPG image viewers, but they are the fastest JPG viewers.
Every time we save a JPG file (menu FILE - SAVE AS - JPG), it compresses again, and image quality is lost. This loss is variable, depending on the JPG Quality factor used, but it is never zero. If we edit the JPG file image and save it again, then quality is lost again. Even if we save it at a much higher quality setting the second time, it is still a second compression and another loss. Every time we compress a JPG file, we add more JPG artifacts and lose a little more quality. Artifacts are always a one way trip, there is no recovery. The damage is done.
Most other file formats (say TIF or PNG) use lossless compression. These files are larger than JPG because they use milder, fully recoverable (lossless) compression to carefully preserve all of the original image data. These file formats remain full quality at all times, no matter how many times we might save them to a file, which is of course exactly what is needed for a master copy.
The small JPG file size is great, but it has a big price of reduced quality. There are proper times and places one would use it, and also major reasons one would not. There's nothing wrong with creating a JPG image using a moderate to high Quality factor to put a photo image on a web page or to send it via email. It's the only practical way. However there is an additional quality loss when we try to edit and save that JPG file a second time, so JPG is usually inappropriate for important archived master copies. The risk if you make this mistake now is that you cannot undo it in the future, so now is the best time to understand the situation. 152 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Progressive JPG is a web option that quickly shows the entire image in very low quality, and the quality fills in and improves as the file download completes.
JPEG 2000
A new JPEG 2000 file format is available now in many newer program versions, with various file extensions, normally .jp2 (or .jpx, with option extensions). JPEG 2000 uses a wavelet compression method. It has a lossless option (to perhaps 65% size). Otherwise it is lossy, size comparable to regular JPG files, but different... slower then JPG, but arguably perhaps better quality. Extremes of compression have few detectable artifacts; however JPEG 2000 images become noticeably soft with greater compression.
Paint Shop Pro 8 and Photo Impact 8 started including JPEG 2000. Adobe Elements 3.0 includes JPEG 2000. Elements 2.0 have an update to add JPEG 2000 - at the How To palette, Select a Recipe, Download New Adobe Recipes, then (very important) the BACK button, and you will see it. Photoshop has an optional JPEG 2000 plug-in from CD, see JPEG 2000 at the CS or CS2 Help menu.
Note that web browsers do NOT support JPEG 2000 yet, and so compatibility is still a very significant issue. Therefore it seems important to stay with the original JPG format for now, at least if you want others to be able to open your files.
Graphic Interchange Format (GIF)
(.GIF file extension) There have been raging debates about the pronunciation. The designers of GIF say it is correctly pronounced to sound like Jiff. But that seems counter-intuitive, and up in my hills, we say it sounding like Gift (without the t).
GIF was developed by CompuServe to show images online (in 1987 for 8 bit video boards, before JPG and 24 bit color was in use). GIF uses indexed color, which is limited to a palette of only 256 colors (next page). GIF was a great match for the old 8 bit 256 color video boards, but is inappropriate for today's 24 bit photo images.
GIF files do NOT store the image's scaled resolution ppi number, so scaling is necessary every time one is printed. This is of no importance for screen or web images. GIF file format was designed for CompuServe screens, and screens don't use ppi for any purpose. Our printers didn't print images in 1987, so it was useless information, and CompuServe simply didn't bother to store the printing resolution in GIF files.
GIF is still an excellent format for graphics, and this is its purpose today, especially on the web. Graphic images (like logos or dialog boxes) use few colors. Being limited to 256 colors is not important for a 3 color logo. A 16 color GIF is a very small file, much smaller, and clearer than any JPG, and ideal for graphics on the web.
Graphics generally use solid colors instead of graduated shades, which limits their color count drastically, which is ideal for GIF's indexed color. GIF uses lossless LZW compression for relatively small file size, as compared to uncompressed data. GIF files offer optimum compression (smallest files) for solid color graphics, because objects of one exact color compress very efficiently in LZW. The LZW compression is lossless, but of course the conversion to only 256 colors may be a great loss. JPG is much better for 24 bit photographic images on the web. For those continuous tone images, the JPG file is also very much smaller (although lossy). But for graphics, GIF files will be smaller, and better quality, and (assuming no dithering) pure and clear without JPG artifacts.
153 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 If GIF is used for continuous tone photo images, the limited color can be poor, and the 256 color file is quite large as compared to JPG compression, even though it is 8 bit data instead of 24 bits. Photos might typically contain 100,000 different color values, so the image quality of photos is normally rather poor when limited to 256 colors. 24 bit JPG is a much better choice today. The GIF format may not even be offered as a save choice until you have reduced the image to 256 colors or less.
So for graphic art or screen captures or line art, GIF is the format of choice for graphic images on the web. Images like a company logo or screen shots of a dialog box should be reduced to 16 colors if possible and saved as a GIF for smallest size on the web. A complex graphics image that may look bad at 16 colors might look very good at say 48 colors (or it may require 256 colors if photo-like). But often 16 colors is fine for graphics, with the significance that the fewer number of colors, the smaller the file, which is extremely important for web pages.
GIF optionally offers transparent backgrounds, where one palette color is declared transparent, so that the background can show through it. The GIF File - Save As dialog box usually has an Option Button to specify which one GIF palette index color is to be transparent. Interlacing is an option that quickly shows the entire image in low quality, and the quality sharpens as the file download completes. Good for web images, but it makes the file slightly larger. GIF files use a palette of indexed colors, and if you thought 24 bit RGB color was kinda complicated, then you isn’t seen nothing' yet.
For GIF files, a 24 bit RGB image requires conversion to indexed color. More specifically, this means conversion to 256 colors, or less. Indexed Color can only have 256 colors maximum. There are however selections of different ways to convert to 256 colors.
Indexed Color and Palettes
What's an Index? What's a Palette?
This topic of Indexed Color is NOT unique to GIF. TIF and PNG files can save Indexed color too, but GIF requires it, GIF has no other mode. We speak of GIF files here, but it could be TIF or PNG too. File formats like TIF and JPG store a 24 bit RGB value for each of the millions of image pixels. But GIF files only store a 4 or 8 bit index at each pixel, so that the image data is 1/6 or 1/3 the size of 24 bits.
Indexed Color is limited to 256 colors, which can be any 256 from the set of 16.7 million 24 bit colors. Each color used is a 24 bit RGB value. Each such image file contains its own color palette, which is a list of the selected 256 colors (or 16 colors in a smaller palette). Images are called indexed color because the actual image color data for each pixel is the index into this palette. Each pixel's data is a number that specifies one of the palette colors, like maybe "color number 82", where 82 is the index into the palette, the 82nd color in the palette list of colors. We have to go to the palette to see what color is there. The palette is stored in the file with the image. The index is typically a 4 bit value (16 colors) or 8 bit value (256 colors) for each pixel, the idea being that this is much smaller than storing 24 bits for every pixel. But an 8 bit number can only contain a numerical value of 0 to 255, so only 256 colors can be in the palette of possible colors. The size of most graphics files can be limited to use 16 colors, which only uses 4 bit indexes, making the file smaller yet, half the 8 bit size for the index for each pixel.
The file also contains the palette too, which is the table of the selected 24 bit colors, or 3 bytes of RGB overhead for each color in the palette (768 bytes for 256 colors). The first RGB
154 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 color in the table is index 0, the second RGB color is index 1, etc. There can be at most only 256 colors in the palette.
So indexed files have 24 bits stored for each palette color, but not for each pixel. Each pixel only stores either a 4 bit or 8 bit index to specify which palette color is used. Image programs will show the palette for indexed images, and can modify palette colors. There are various ways to create the palette, to choose the possible color choices that it will contain. This palette choice affects the image greatly.
Converting to 16 or 256 colors
There are several ways to convert to indexed color. Two choices are required, to specify a palette of colors, and also a choice how to dither or show colors not in that limited palette.
The dithering choices might be:
Dithering - Dithering means representing a color by combinations of dots of other colors. Dots are pretty much the definition. If the one exact shade of pink is not in the palette, then dots of other palette colors are mixed to simulate the color. Dithering often causes a visible and objectionable dotted or speckled image. But that simulated color may be much closer than a nearest color approximation.
Nearest Color - This selection is the same as disabling dithering, or no dots. If the exact shade of pink is not in the palette, then the closest color in the palette is used, which might not be very pink at all. Sometimes exact color is not important. No dithering means no visible dots.
Choices for a palette often have many names, but are of two main types, Standard or Adaptive:
Adaptive or Optimized or Perceptual Palette
These custom palettes are created from the 256 most representative colors within the one specific current image. The gradient image below is red graduated to white, originally over 5000 shades of pink and white. It is pretty much all red.
The Photo Impact optimized palette at right for this image includes 8 system colors, including black, but this palette specifically has 248 shades of pink to be appropriate for this specific image. That is enough shades of pink that the quality of the GIF image does not show any banding or dithering in this case (one color). However only 256 colors is often pretty bad for normal photos (too few indexed colors for photos, which typically need up to maybe 100,000 colors, not 256). Since most colors in this graphic image are red, this optimized palette appropriately contains mostly red, so it is a much easier case than a normal photo. Optimized 155 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 and Adaptive (two words for same thing) means optimized for the specific image, instead of being optimized for the old Windows 8 bit Palette Manager.
Standard Web Browser Palette
The Standard Palette (often called the Netscape palette or Web palette) always contains the same colors for any image. It is sometimes called 6-6-6, because it contains six standard evenly spaced colors for each of Red, Green, and Blue. Those 6x6x6 color combinations create 6x6x6 = 216 standard color combinations, which are independent of any specific image (used for any image). The remaining 40 colors are standard colors reserved for the Windows or Macintosh desktop.
The Standard Palette always contains combinations of the following 6 tones for each of the Red, Green, Blue primaries:
6 colors, Hexadecimal (0-ff) 6 colors, Decimal (0-255)
00 33 66 99 CC FF 0 51 102 153 204 255
The 216 combinations of these 18 colors above (six shades of the three RGB colors, 6x6x6 = 216) produce the Standard web-safe palette below (this one is the standard "Web" palette from Elements 2.0)
The 6x6x6 standard colors are intended to be somewhat suitable for ANY generic image, but of course, it is not correct (not precise) for any normal image. For example, this very GIF
156 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 image uses that same palette, and notice that 6 shades of blue are insufficient to show the Title bar without banding.
This standard palette was very important for old 8 bit video cards. The situation used to be (in historic times, ten years ago) that 8 bit video card drivers had only one Windows video Palette Manager, requiring ALL images on that desktop to share ONE palette simultaneously. We used to see bizarre psychedelic colors when the palette of a new image reloaded that palette and affected all other images on the desktop. Perhaps palette Index 82 changed from pink to green, correct for the one image, but very wrong for all other images on the desktop. When the WWW started showing web pages with multiple images on the one web page, it was awful. Netscape standardized the 216 color palette as a workaround. Using one standard palette for all GIF images was better than the alternative, even if that generic palette was far from optimum for any image. This standard 216 color palette is that palette, one generic palette for all indexed images, even if not best for any image. But this concept of a Standard Palette or web-safe palette is obsolete today, now that we all have 24 bit video cards.
Today, using the Standard Netscape 216 color web-safe palette only helps those still using ancient 8 bit video cards. Using the standard 216 colors means old 8 bit video systems can show them without adding additional dithering (dotted color). This used to be important several years ago when we all had 8 bit video boards.
There are two schools of thought about this
· the belief that this might somehow still be important, and
· Why bother anymore? Why be artificially limited for no current reason?
My opinion is that the so-called web-safe palette is an obsolete concept, no longer useful today, and is instead outright detrimental today. There is no Windows Palette Manager anymore (except in drivers for obsolete 8 bit video boards). Today's 24 bit video boards can show any possible palette color, even when in 256 color mode (easy to test). We don't see dithered colors (speckled dotted colors) in 256 color mode anymore, unless the images were created that way in the first place by using a so-called web-safe palette. The GIF problems today are mostly all self-inflicted due to still trying to be limited to obsolete concepts (the use of non-optimum palettes). There is no magic in 216 colors on today's computers with 24 bit video.
Adaptive palettes (instead of web-safe 216 color palettes) give better images, and makes creating poor images unnecessary in indexed color. A non-dithered Adaptive palette is hard to beat. Most of the GIF graphic images on this site (screen captured dialogs, etc) use a 16 color Adaptive palette from Nearest Color. The exact shade of color was not very important, but the small file size for the web was important (however where there was embedded photo content of many colors, the image may have been sliced, mostly GIF portions with a little JPG portion).
The few users still using ten year old 8 bit boards, if any, are frankly used to poor color, it's nothing they haven't seen before, and they can upgrade if they wish. Why make the overwhelming majority, those with the now standard 24 bit boards capable of good color, see poor color because it used to be that not everyone could? Personally, I ignore web-safe today, and I go out of my way to use the optimized or adaptive palettes for GIF files. The images are much better, without dithering dots to ruin them.
157 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 For the other side of it, Microsoft has a good write-up about Safety Palettes, about dithering and the standard 216 color web-safe browser palette. Note it is dated 1996, and it was still important then, but modern 24 bit video cards have changed the world now.
A little more, to be sure you understand Adaptive palettes:
GIF, particularly 16 colors GIF, is ideal for web pages for logos and similar simple graphics without continuous tones. The files can be very small (assuming solid color graphics).
The next example shows how a 16 color GIF file with Adaptive palette can be better than a 256 color GIF file with Standard palette. The 16 color Optimized GIF file enlarged 4 times, and its palette.
OK, it's a special case, but since most of this image's colors are red, the optimized palette contains mostly red also. Since we only have 16 colors, and since 16 colors is insufficient for this graduated image, then many of the pinks are combined into the same few colors, and we see some dots (pixels, this is shown at 4x size).
In the image below with the Standard palette (above was Adaptive Palette), we do have the standard 216 colors available, but very few of them are the Reds that we need for THIS image, so the results are not even as good as the 16 color Optimized palette in this case.
This graduated image was ill-suited for 16 colors, but 16 colors of Red is still more than 6 colors of Red. This graduated image would be better as a 256 color GIF, or a JPG. The JPG file would be smaller, but not likely better than 256 color Adaptive in THIS case, because 24 bit color only has 256 shades of Red too, and red is about all we need here.
Most graphics (logos, dialog boxes, etc) don't have more than 16 colors anyway. In those cases, 16 bit GIF files can be much smaller than 256 color GIF files. ------
LESSON VI: Internetworking Servers
158 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 11. Server Implementation
DHCP Server General Implementation and Management Issues
DHCP is a client/server protocol, relying on both server and client to fulfill certain responsibilities. Of the two device roles, the DHCP server is arguably the more important, because it is in the server that most of the functionality of DHCP is actually implemented. The server maintains the configuration database, keeps track of address ranges and manages leases. For this reason, DHCP servers are also typically much more complex than DHCP clients.
In essence, without a DHCP server, there really is no DHCP. Thus, deciding how to implement DHCP servers is a large part of implementing the protocol. This overall chapter is about describing the function of protocols like DHCP and not getting into details of how to implement them. However, I feel it is useful to look at some of the general issues related to how DHCP servers are set up and used, to help put into perspective how the protocol really works. DHCP Server Implementations A “classical” DHCP server consists of DHCP server software running on a server hardware platform of one sort or another. A DHCP server usually will not be a dedicated computer except on very large networks. It is more common for a hardware server to provide DHCP services along with performing other functions, such as acting as an application server, general database server, providing DNS services and so forth. So, a DHCP server need not be a special computer; any device that can run a DHCP server implementation can act as a server.
In fact, the DHCP server may not even need to be a host computer at all. Today, many routers include DHCP functionality. Programming a router to act as a DHCP server allows clients that connect to the router to be automatically assigned IP addresses. This provides numerous potential advantages in an environment where a limited number of public IP addresses is shared amongst multiple clients, or where IP Network Address Translation (NAT) is used to dynamically share a small number of addresses. Since DHCP requires a database, a router that acts as a DHCP server requires some form of permanent storage. This is often implemented using flash memory on routers, while “true” servers of course use hard disk storage.
Virtually all modern operating systems include support for DHCP, including most variants of UNIX, Linux, newer versions of Microsoft Windows, Novell NetWare and others. In some cases, you may need to run the “server version” of the operating system to have a host act as a DHCP server. For example, while Microsoft Windows XP supports DHCP, I don't believe that a DHCP server comes in “Windows XP Home”, the “home user” version. (Of course, you could install one yourself!)
DHCP Server Software Features
In most networks you will choose the operating system based on a large number of factors. The choice of OS will then dictate what options you have for selecting DHCP server software. Most common operating systems have a number of options available for software. While all will implement the core DHCP protocol, they will differ in terms of the usual software attributes: cost, performance, ease of use and so. They may also differ in terms of their features, such as the following:
· How they allow address ranges (scopes) to be defined.
159 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · How clients can be grouped and managed.
· The level of control an administrator has over parameters returned to a client.
· The level of control an administrator has over general operation of the protocol, such as specification of the T1 and T2 timers and other variables, and how leases are allocated and renewals handled.
· Security features.
· Ability to interact with DNS to support dynamic device naming.
· Optional features such as BOOTP support, conflict detection, and automatic private IP addressing. Choosing the Number of Servers In setting up DHCP for a network, there are a number of important factors to consider and decisions to be made. One of the most critical is the number of servers you want to have. In theory, each network requires only one DHCP server; in practice, this is often not a great idea. Servers sometimes experience hardware or software failures, or have to be taken down for maintenance. If there is only one server and clients can't reach it, no DHCP clients will be able to get addresses. For this reason, two or more servers are often used.
If you do use more than one server, you have to carefully plan how you will configure each one. One of the first decisions you will need to make is which servers will be responsible for which addresses and clients. You have to determine whether you want the servers to have distinct or overlapping address pools, as discussed in the topic on DHCP address ranges. Distinct pools ensure that addresses remain unique but result in unallocatable addresses if a server fails; overlapping addresses are more flexible, but risk address conflicts unless a feature like conflict detection is used.
Server Placement, Setup and Maintenance
Once you know how many servers you want, you have to determine on which part of the network you want to place them. If you have many physical networks, you may also need to use DHCP relaying to allow all clients to reach a server. Of course, the structure of the network may affect the number of servers you use, so many of these decisions are interrelated.
You must make policy decisions related to all the DHCP operating parameters we have seen earlier. The two biggies are deciding on the size and structure of the address pool, and making lease policy decisions such as lease length and the settings for the T1 and T2 timers. You also must decide what clients will be dynamically allocated addresses and how manually-configured clients will be handled.
Finally, it's essential for the administrator to remember that an organization's DHCP server is a database server and must be treated accordingly. Like any database server, it must be maintained and managed carefully. Administrative policies must be put into place to ensure the security and efficient operation of the server. Also, unlike certain other types of database systems, the DHCP database is not automatically replicated; the server database should therefore be routinely backed up, and using RAID storage is also a good idea.
160 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 12. Content server
Content Server
Business users in today’s organizations require timely and relevant information to make effective decisions on a daily basis. Inmagic, Inc. has offered content and information management tools and applications for information-intensive organizations for close to two decades. Unlike more complex content management systems that often require extensive support from internal technical staff as well as a significant commitment of both time and money, Inmagic® Content Server offers a self-managed system that can be deployed quickly and cost-effectively, with only minimal ongoing support from IT. With Inmagic Content Server YOU are in the driver’s seat. Content Server enables you to:
• Create content as well as integrate existing content from internal and external sources • Deploy that content to Web pages on intranets, extranets and the Internet • Organize and manage the information flow and have the databases you create drive the content that end users access. Inmagic Content Server [CS] combines the advantages of a robust and flexible database management environment with high speed search and categorization, making finding relevant information fast, easy and precise. Web publishing capabilities are built into Inmagic CS, offering a better way to publish, access, and maintain information on corporate intranets and the Internet.
Scalable and Extensible Architecture
Content Server is built on the Microsoft® SQL Server™ database, ensuring that Inmagic can offer you a range of content management solutions; from the single department or workgroup to global deployment of an enterprise-wide system. From Content Server Workgroup to Content Server Enterprise, you are able to take advantage of the unparalleled scalability and reliability of this industry-standard database. Inmagic CS makes extensive use of other industry and open standards, including XML. Content Server easily integrates internal, external and licensed content providing a single access point for your users. Content from internal data repositories can be combined with dynamic content from Web sites to provide up-to the- minute information on competitive intelligence, business intelligence and other critical content management tasks. Content Server for the Web uses XML and SOAP to enable you to: • Publish content via a Web browser • Add, edit and delete content directly from the browser • Create interactive Web forms using standard third-party tools • Tightly integrate content from your Content
Server databases with other applications.
Powerful Database Capabilities Combined With Search Accuracy
Inmagic Content Server includes a Windows client – CS/Text Works – that allows you to set up and modify databases, independent of direct support from the IT group. While using SQL Server as its data store, Content Server includes a Text base Engine that greatly enhances the database and search capabilities of the standard SQL database. This allows you to create new databases easily and quickly without requiring support from a DBA or Systems Administrator. The Text base Engine provides features such as support for fields and records with no size limitations – an entire document can be stored in a record without utilizing large objects. Repeating fields – such as multiple authors or descriptive terms – are supported without the user having to design additional tables. And, you have complete control over
161 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 indexing fields for fast retrieval. These capabilities make Inmagic Content Server ideally suited to manage unstructured content with maximum flexibility and complete local control.
CS/Text Works also supports a drag-and-drop forms designer to create reports. Full data entry capabilities include a built-in spell checker and thesaurus as well as validation controls. Password protection at the database, field or record level provide inherent security and full-text searching – including keyword, term, phrase, Boolean, proximity and comparison searches – complement the database design capabilities of Inmagic Content Server.
The Importance of Local Control
Research conducted by Wallace and Washburn Inc. Marketing of Boston clearly demonstrates that business users look to content management systems to provide the right people with the right information. They want such systems to make accessing information, navigating varying information resources and updating content easy and fast. They also want to have greater control of the content at the local level — in their business units and departments — without requiring extensive integration or support from a vendor or IT personnel. Inmagic Content Server meets these requirements for local control and self-management — databases can be set up and modified independent of support from a database administrator. The product’s non programming database design and development environment allow rapid applications development to ensure quick turnaround of product that meets the unique needs of the business group. Inmagic often works with customers to develop specialized content management solutions in a matter of weeks or months. And, even if your organization has invested in an enterprise-wide portal or information system, standards such as XML and SOAP allow Content Server to complement and interface with other corporate systems. You have a local solution that meets your distinct content and information requirements while still integrating effectively within the enterprise-wide system.
For DB/TextWorks Customers
Inmagic Content Server is an extension of Inmagic’s DB/Text® product line, currently installed in over 7,000 organizations around the world. Because of its proprietary database architecture, DB/Text Works® is used primarily in small or mid-size information intensive departments such as corporate libraries and document centers, museums, archives, etc. DB/ Text Works is used to manage content collections ranging from litigation support documents to Technical reports, digital and image libraries and all manner of Web-based resources. By upgrading to Inmagic Content Server, current DB/TextWorks customers can expand their information and content management solutions more widely within their enterprises. The extensive compatibility between DB/ Text Works and Inmagic CS ensure that all databases currently built using DB/TextWorks as well as all Web sites developed using the DB/Text® Web Publisher products will run seamlessly in the Inmagic Content Server environment, requiring only that the text base records be exported from DB/Text Works and imported into Content Server. And, because the Windows and Web user interfaces are nearly identical, your users will be able to use Inmagic CS without additional training.
About Inmagic
162 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Inmagic is a global provider of content and information management software and services that organize and deliver enterprise content, seamlessly integrate both internal and external content sources, and deploy business-critical information to corporate portals, intranets, extranets, and the Web. Specific applications include market, business, and competitive intelligence, library automation, litigation support, and Web publishing. Inmagic’s information management solutions are installed in more than 7,000 organizations in over 50 countries.
13. Performance server
The What, Why, and Whether
If part of your job is measuring your Web server's performance-or measuring prospective servers against each other-you may want to turn to Web server benchmarking tools to gather the data you need. This article talks about benchmarks: what tools are available, what people are saying about them, and why, if your site handles moderate traffic, you may not need them at all.
What is a Benchmark? A benchmark is simply a way to measure system performance. Benchmarks aren't new. Engineers have used them for years. Designing benchmarks to measure chip performance, for example, is a science unto itself. The idea behind benchmarking is simple enough. Perform a process that is typical of what the system you're testing will be expected to perform. Execute and time the process, then perform the exact same test on different systems and measure your results. The devil, in this case, is in the details. As an analogy, say Zolo the Great and Omar are competing knife throwers at the carnival. As Webmaster, er, ringmaster, you have to decide who has the better act, so you decide to benchmark their performance. The first step is to make sure test conditions are identical. Are the models of the same height, in the same position, and standing perfectly still? If they're not standing perfectly still, is the amount of wincing and squirming the same for both throwers? Are the number and position of balloons (or other markers) constant? Is the lighting the same (Omar might just cry foul, for example, if he had to test in the afternoon when the sun was in his eyes.)
Once you've set up and run the tests (more than once, of course, to allow for statistical variations) the job is far from over. You now have to interpret the results. Which elements of the performance are the most important? Omar may claim precision should be the determinant, while Zolo the Great argues number of knives thrown per minute is the true measure of skill. Third parties, such as the model, may have different opinions altogether.
How Do We Measure Server Performance?
When we turn to analyzing Web server performance, deciding what to measure is certainly the first step in benchmarking. Right now the leading benchmarks tend to focus on two measurements of performance: throughput-the rate at which the server can process HTTP requests; and response time-the time a server spends processing a single request. A typical approach is to configure the benchmarking software to simulate a large number of clients, and then to request a set of pages of varying lengths, so you can see results for both small and large files.
163 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 To give you an idea of what kind of results you'd be looking at, just below are SPECweb96's benchmark results for Digital's Alpha server 4000 5/400 running Zeus 1.1.5. Full details of this performance evaluation are at open.specbench.org/osg/web/results/res96q4/web96-961125- 01539.html (see below for a description of SPECweb96).
Throughput Response (Ops/sec) (msec) 117 9.0 234 8.9 351 8.9 468 9.4 586 10.5 703 11.5 820 13.1 937 15.4 1054 19.8 1157 30.2
There are a number of Web server benchmarking programs available. One of the main ways in which they differ lies in the ability of the user running the test to configure the test parameters. Silicon Graphic Inc.'s WebStone, the first benchmarking software specifically developed to measure HTTP performance, is highly configurable. The idea is that the results are more meaningful if they are tailored to a specific configuration. Why measure retrieval time for a generic 10 KB HTML document when you can measure the time for the actual document itself? The flip side of this argument is that standard tests better allow administrators to analyze comparison data. Standard Performance Evaluation Corp. (SPEC) calls this "apples-to-apples" comparison.
Comparing Benchmarks The descriptions below summarize and compare the benchmarking resources scouted for this article. Availability ranges from free (WebStone is an open, non-proprietary benchmark you can download source code for) to none (the Web66 GStone benchmarks results are available, but not the software itself).
SGI's WebStone WebStone, from SGI, was the first available benchmarking software for the Web. It measures throughput and latency for HTTP GET requests, and simulates the load from a varying number of clients. It gives you either statistical output or (if requested) data for each transaction. WebStone also reports transaction failures, and touts itself as being highly customizable, so you can get results that measure how your particular setup will perform. WebStone is an open (some say de facto-Ed.) standard and is available free for download.
Ziff-Davis' Web Bench WebBench was developed by the Ziff-Davis Benchmark Operation for use in Ziff-Davis publications' evaluations. It produces two overall scores, one for requests per second, and one for throughput as measured in bytes per second. WebBench includes an additional, dynamic test suite which uses CGI operations, but these tests are platform-dependent. Output is in the form of an Excel workbook, and shows both overall statistics and data for the individual clients. Clients for WebBench must run on Windows 95 or Windows NT systems.
164 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 The software is licensed-but free for download, or you can order the CD for a nominal cost ($5, ZDBOp, 1001 Aviation Parkway, Suite 400, Morrisville, NC 27560).
The SPEC's SPECweb96 SPECweb96 is the benchmark for The Standard Performance Evaluation Corporation (SPEC), a non-profit body that develops standards for workstation performance measurements. Unlike WebStone, SPECweb96 doesn't let you customize the workload configuration. The workloads were established by studying representative Internet sites and include a range of file sizes from 1 KB to 1 MB. People who are shopping for a potential server can study the benchmark results (www.specbench.org/osg/web96/results/) submitted by member companies and know they're getting an even comparison. If you want to buy the software to run the test yourself, the cost is $800.
14. Database server - - - -
Database is simply an organized set of information. Strictly speaking, there's no need for it to even be in computerized form - address books are often used as a simple example of a database, since they contain large volumes of information organized into categories (name, address, and phone number). Despite this, most people now take the term 'database' to refer to information stored electronically.
Note also that the general definition of a database given here could encompass many common PC applications, such as electronic mail (which can be organized by recipient or sender) or information in a spreadsheet (which tends to be structured mathematically). In practice, the term 'database' is most commonly used to refer to highly structured information (examples might include order forms or medical information). Most business databases will contain a carefully planned set of information that can be analyzed to indicate overall trends, as well as providing a historical record of past transactions and activities.
A basic database system simply allows you to enter and search for information (a process often known as querying). Most modern databases also support the development of specific applications that run on top of the database, which enable you to access the features you need without having to worry about all the complexities of the system. For instance, within one company general staff might have access to an order entry system, while financial staff have access to automatic reporting systems that provide sales summaries and other information. Both use a database server to store information, but the means of entering this, and the ability to change it after entry, will vary between the different applications.
What is a database server?
165 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 A single-person business could run database software on just one PC quite easily. However, in most businesses data will need to be accessed by multiple staff across a network, often simultaneously. Database server software handles this task, making databases available to all relevant employees. This is not simply a matter of creating network connections. Database server software must ensure that information isn't modified by multiple staff members at once, index and sort the information that is entered so it can be easily searched, and allow analysis of the information stored in the database to produce reports. It also allows different applications to use the same centralised storage mechanism. With appropriate hardware, a database server can handle as many of these options as are needed. Staff may use different applications to access this information, but the server will handle the core data storage. Since databases were one of the earliest computer applications to be developed, they have become highly sophisticated over the years. Modern database servers can handle huge volumes of information and present it in a variety of formats, including automatically outputting Web pages from stored data and allowing the placing of orders and other electronic commerce tasks. For many businesses, a database server in some form will lie at the heart of most daily activities. A note about the word server is in order here. Database server software is distinct both from server hardware (which is the physical machine or machines used to run a network) and the server operating system (which runs on the server hardware and provides an interface between it and specific applications, of which a database server is one example). Despite this distinction, in practice all three will work together closely, and your choice of database server will influence your server OS and hardware choices.
The types of databases available As database software has evolved, a number of different approaches to storing and linking information have emerged. Some of the more common include: Relational databases. A relational database creates formal definitions of all the included items in a database, setting them out in tables, and defines the relationship between them. For instance, a typical business database would include tables for defining both customers and orders. Using ids or keys, the two tables can be related together. Such databases are called 'relational' because they explicitly define these connections (an order form can look up customer details from the customer table rather than having to store the information twice). Most relational databases now make use of SQL to handle queries (discussed in more detail below). Currently these are the most common form of database. Object databases. Object databases store data in discreet, self-contained units - objects. These objects have specific data, attributes and behaviours associated with them. An
166 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 extremely simple example might be a product database with a shirt object, which has attributes such as size, colour, and price. In practical use, the main difference between object and relational databases is the way in which data is accessed. Programmers use object-oriented programming languages to access the data objects from the object database by calling methods in their code. This takes much of the information that would have resided in the application code and transfers that information to the object database. Thus the application code is simplified. However, at the same time the fact that the database and application are tightly entwined can make accessing the data outside of the application more complex. Object-relational databases. Object-relational databases attempt to combine object and relational approaches. This allows the benefits of using objects where necessary to be tied to the strengths of relational databases. Hierarchical databases. While relational databases arrange data in tabular format, hierarchical databases arrange them in a tree format, with a parent node leading to further child nodes (which in turn may have further nodes of their own). The model is very similar to the way in which a program such as Windows Explorer displays the contents of a hard drive (double-clicking on a parent directory leads down the tree to further information, and so on down the directory tree). This allows for multiple types of subsidiary data, but makes it difficult to identify complex multiple relationships between individual data items (just as there is no obvious link between two subdirectories on a hard drive).
Until recently, hierarchical databases have been more common in computer science fields than in real-world applications. However, hierarchical methods have become more popular with the emergence of XML (Extensible Markup Language), which uses a hierarchical structure, as a common data exchange format.
What do I need to know about SQL? SQL (Structured Query Language) is used by relational databases to define queries and help generate reports. First developed in 1976, it provides a standardized means of sending queries to relational databases. SQL also defines more fundamental elements of databases, such as data types. SQL has become a dominant standard in the world of database development, since it allows developers to use the same basic constructions to query data from a wide variety of systems. The central functions of SQL have been defined by international standards organizations: originally the American National Standards Institute (ANSI), and subsequently the International Standards Organization (ISO). Like most computer-based standards, SQL also comes in a number of flavors. The two most recent iterations are SQL-99 (also sometimes known as SQL-3) and SQL-92, both named for the
167 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 years in which they were first released. (SQL-99 had been originally planned for release in 1996). While SQL-92 defined three individual levels of compliance (basic, intermediate and advanced), SQL-99 reverted to the model used in previous versions of providing just one large feature list. More importantly, SQL-99 also added some basic support for object features, extending SQL's usefulness and blurring the distinction between relational and object approaches to databases.
It is comparatively rare for vendors to implement the precise standards laid down for SQL, which is a complex standard running into thousands of pages. Several companies choose not to implement every aspect of the existing standards, arguing that the functions in question are rarely if ever needed by developers or users. Simultaneously, many provide additional, functions (known as extensions) to make particular tasks easier. Despite what you might suspect, many companies will simultaneously add their own extensions while ignoring some aspects of the basic standard. Whether these additions and exclusions are important to your business will depend on the exact mix of applications you wish to run or develop, and what existing applications you already have in place.
What other features should I look for in a database? When choosing a database server, the most important consideration will be whether it can deliver the specific application functions you require. These may be supplied in the form of pre-packaged software, or you may choose to develop your own (or more likely hire a consultant to do so). This may involve considerable expenditure, but this needs to be balanced against improved staff productivity and the ability to more accurately analyze your business.
Standards compliance. As discussed above, SQL support varies widely between database server suppliers. Databases which comply with SQL should allow relatively straightforward data exchange, so SQL compliance is important, especially in environments running more than one operating system. Security systems. Databases often store highly valuable and sensitive commercial information, so it's important that there is some security system in place, even if this is only a basic username/password system. Most database servers will provide audit trails, allowing you to see who has entered, accessed or modified information. If your database server is going to be exposed to the Internet, then security mechanisms will need to be more robust and you will need to consider whether other mechanisms (such as encryption) are necessary as well.
168 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Performance features. Databases are generally critical applications, and even a brief outage can be harmful to your business. To help prevent this, modern database servers have borrowed many features from the world of general network operating systems, including fault tolerance (systems to keep the server running in the event of unexpected errors) and load balancing (which allows database queries on high-volume systems to be handled by multiple servers, improving performance and response times). These are unlikely to be needed if you're just running a single database server, but as your needs expand they are likely to become more crucial considerations. If your database server is used for e-commerce applications, these availability issues will take on a heightened degree of importance. While you may not require all these features immediately, you should consider future needs as well as your current plans. Retrofitting these features to your system is likely to be more difficult than installing a database server that supports them from the beginning, even if it takes time for you to actively deploy them.
XML: If you will be doing data interchange now or in the future you may want to look for a database that allows extraction of data as XML. This may make using XML as a data interchange format simpler. the database players
Reflecting its long heritage, literally dozens of companies offer database server solutions. Some are tailored to individual markets; some are designed to work closely with existing application and OS software; some are aimed at people building their own specific applications, while others have many commercial add-on applications available. A partial, alphabetized list of major companies in the space would include Borland, IBM, Informix, Microsoft, My SQL, Oracle, Red Hat, SAP and Sybase. Many other companies supply applications that will work with database servers from these providers.
What hardware will I need? There's no single answer to this question. You will need to work out how many staff members are likely to connect to the database, and how many transactions (looking up existing data, modifying it or entering new information) will take place. These will provide you with broad parameters for selecting an appropriate database server, and in turn give you a good indication of what your server requirements will be. As with many computing tasks, the more memory, processing power and disk storage you can provide on the server, the better. Don't neglect network connection speeds, either; if you have a high-powered server but a poorly configured network, you'll lose most of the advantage waiting for data to be sent back to you.
169 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 How much should I expect to pay? This is also a difficult question. Database server prices vary hugely. At one end of the spectrum, open source database solutions are available for no up-front cost. At the other extreme, specialized database solutions can cost hundreds of thousands of dollars. Two broad parameters will guide cost: the number of supported features, and the number of connected users. A system that is designed to serve just a few dozen staff and with minimal security features should be cheaper than one that handles thousands of simultaneous transactions and also powers a number of Web sites.
The other important factor to consider is the degree to which your database accessing application will be customized. If you can make use of an off-the-shelf application, this will generally be cheaper than having a developer build a custom system for your business. Even a 'free' database system using readily available software will still require an investment in training for your staff, however.
15. Mirrored server - - - -
16. Popular server products
Popular Server Products
The Netserver X 1000 in Brief
There’s little that can beat Giacom’s NetServer XL 1000 dedicated server in terms if price and performance. If these you’re first dedicated server or you’re searching for the most cost-effective solution, then this is the server for you.
Prices start from only £99 per month, and there are no set-up fees. You get access to the desktop of your own Windows 2003 Server for your online use. Simply connect to the server from your PC or Mac, and you’ll have complete control over the vast array of on-board features such as the unlimited domain web server, FTP server, full unlimited mail server, stats server, and a place to launch your online applications.
And exclusive to Giacom, all NetServers are individually protected by their own hardware firewall providing second-to-none security and protection at no extra cost.
170 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Netserver XL 1000 specification
Processor : Intel Core 2 Duo E6300 Processor HDD : 250 GB SATAII Hard Disk Memory : 512MB DDRII RAM O/S : Windows 2003 Server- Web edition (Upgrade to Windows 2003 Standard edition available. Server : 1 static IP address : Full remote administration access : Giacom managed DNS : No domain import / export charges Load balanced network Domains : Free domains imports Free domain exports Giacom managed DNS Network : 24x7 network monitoring 100% network SLA Load balanced datacenter Training : Server training
Pricing
THE NETSERVER XL1000
Bandwidth (GB) 10 15 20 30 40 60 Per month £99 £149 £199 £269 £339 £469 Setup Free Free Free Free Free Free
THE NETSERVER XL1400 So you’re looking for server performance? The NetServer XL1400 definitely comes with excellent credentials – centered around the fast Core 2 Duo E6600 processor, this is Giacom’s most popular power server.
Prices start from only £139 per month, with only £99 set-up. You get access to the desktop of your own Windows 2003 Server for your online use. Simply connect to the server from your PC or Mac, and you’ll have complete control over the vast array of on-board features such as the unlimited domain web server, FTP server, full unlimited mail server, stats server, and a place to launch your online applications.
And exclusive to Giacom, all NetServers are individually protected by their own hardware firewall providing second-to-none security and protection at no extra cost.
Netserver XL 1400 Specification Processor : Processor
HDD : 320GB SATAII hard disk
Memory : 1 GB DDRII RAM 171 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 O/S : Windows 2003 Server – Web edition : (Upgrade to Windows 2003 Server edition available.) Server : Free VPN hardware Firewall : 1 static IP address : Full remote administration access : Giacom managed DNS : No domain import / export charges : Load balanced network Domains : Free domain imports : Free domain exports : Giacom managed DNS Network : 24x7 network monitoring : 100% network SLA : Load balanced datacenter
Support : Giacom dedicated support : Free priority pager service Training : Server Training
PRICING
Bandwidth (GB) 10 15 20 30 40 60 Per month £139 £189 £239 £309 £379 £509 Setup £99 £99 £99 £99 £99 £99
THE NETSERVER XL1600
Giacom’s flagship dedicated server, the NetServer XL1600 provides ultra fast performance using its Intel Core 2 Extreme processor backed up by twin RAID1 duplicated hard drives.
Prices start from only £199 per month, with only £599 set-up. You get access to the desktop of your own Windows 2003 Server for your online use. Simply connect to the server from your PC or Mac, and you’ll have complete control over the vast array of on-board features such as the unlimited domain web server, FTP server, full unlimited mail server, stats server, and a place to launch your online applications.
And exclusive to Giacom, all NetServers are individually protected by their own hardware firewall providing second-to-none security and protection at no extra cost. Netserver XL 1600 specification
Processor : Intel Core 2 Extreme 2.6Ghz (Quad Core) Memory : 400Gb SATAII hard disk x 2 RAID 1 172 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 O/S : 2GB DDRII 800 RAM Server : Windows 2003 Server – Web edition (Upgrade to Windows 2003 Standard edition available.) Domains : RAID 1 : Free VPN hardware Firewall : 1 static IP address : Full remote administration access : Giacom managed DNS : No domain import / export charges : Load balanced network Network : 24x7 network monitoring : 100% network SLA : Load balanced datacenter Support : Giacom dedicated support : Free priority pager service Training : Server training
PRICING
Bandwidth (GB) 10 15 20 30 40 60 Per month £199 £249 £299 £369 £439 £569 Setup £599 £599 £599 £599 £599 £599 UNIVERSAL SERVER FEATURES
Microsoft Windows 2003 Server OS Microsoft IIS web and ftp server SmarterMail - mail server SmarterStats - website statistics SmarterTicket - website support system F-Prot anti-virus Dedicated Hardware Firewall Domain name management 100% network Service Level Agreement Fully managed DNS Hardware fault management Priority support Secure remote desktop administration YOUR SERVER WILL BE PROTECTED BY ITS OWN DEDICATED VPN FIREWALL
We believe all dedicated servers should be protected fully when exposed to the Internet, that's why we only supply our servers with an accompanying Dedicated Hardware Firewall as standard. This Stateful Packet Inspection Firewall sits between your server and the rest of our network and, of course, the Internet. You have full individual control over the firewall settings allowing specific rules to be set should you need extra protection. What's more, the firewall is installed as standard with no extra cost to you.
CONNECT USING A VIRTUAL PRIVATE NETWORK (VPN)
173 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Communication across the Internet can be insecure, so you can connect to your Dedicated Server using the on-board VPN. From your network or PC, using high-level encryption, you'll create a data-tunnel protecting the information you pass to and from your server.
FIREWALL SPECIFICATION
Firewall High performance hardware VPN firewall Type Stateful Packet Inspection 2x VPN (IPSec) endpoints and pass-through support VPN DES (56 bit) and 3DES (168 bit) encryption Auto Internet Key Exchange (IKE) Remote management via Server desktop only Management Menu-driven user interface for easy server management System performance and status monitoring NAT One-to-One
Lesson VII: Web Server and Databases
· Databases
What is a Database?
Wondering what a database is? Well, this is the right place to be. Basically, a database is a collection of data organized for easy storage and access. Data is a single piece of information, and can include, but is not limited to, text, images, numbers, and media clips! Databases help to organize data in a coherent way to help us in our everyday lives. Every single day, we can accumulate tons of data. Without a way to organize all of this, the world would be a complete mess of little pieces of scattered information!
You might not know it, but you frequently use databases in your everyday lives. Have you ever checked a dictionary for the spelling of some unknown word? Well, dictionaries are large databases of words! They organize words and their definitions in such a way to make it easy for us to use! Your local library is also an example of a database. They store hundreds upon hundreds of books in a logical order. There's the children's section and the adult's section, the nonfiction and fiction sections, the paperback and hardcover sections, and so on. Within these sections, books are placed in alphabetical order or in the order denoted by the Dewey Decimal System. All of this is a coherent way to organize books!
Libraries and dictionaries are both paper-based databases. This means that these databases are not computerized, but instead are written down on paper. Paper-based databases have often presented dozens of problems. They are cumbersome and hard to transport from one place to another. It is also quite easy to misplace a page or a portion of the database! That is why some brilliant scientists have introduced computerized databases. These are databases
174 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 that rely on computers to organize and store data. They are fast, compact, durable, and extremely reliable!
Ever since computerized databases have been introduced, they have played an integral part in record keeping and in the storage of data. These databases are often used by the government, by universities and colleges, and by businesses to keep track of data. Most of these databases are private and of no interest to the general public. However, there is also a variety of databases, both commercial and non-profit, for the public. Anybody could access these databases, providing that they had the proper software and hardware.
As the years have progressed, databases have begun to play an important role in the Internet. They are vital components of many websites, especially on-line stores, like Amazon or Barnes and Noble. Databases are also vital components of search engines like Yahoo and Google. They enable the search engines to keep track of the millions of websites so that you can access them quickly and efficiently. Also, there is what is called an on-line database. These are databases that are available to anybody that has a modem, computer, and a telephone line- basically, anybody with Internet access! The user merely connects to the database, type in the data it wants to access, and voila, the data is displayed on the computer screen
Parts of a Database
Now that you know what a database is, you might be wondering how exactly these things work. Databases are divided into three main parts:
1. Data: Of course, as databases are specially designed to organize data, data is one of the key parts of a database! As you probably already know, data is a small bit of information. It could be in the form of pictures, text, numeric values, media, or audio samples. Any information that you gather can be considered data. For example, pretend that you are growing sunflowers in science class. Your teacher has told you to measure the plant to see how much it has grown. Your measurements are a type of data! The data is the information that is placed in a database. Here is a database that we are all familiar with- an address book. Pretend that you have an address book that is set up to resemble a table: Name Address Phone Number E-mail John Cook 255 Main St. 543-7885 [email protected] Matthew Smith 43 First Ave. 423-4567 [email protected] Emily Coffey 2 Meadow Ln. 978-2346 [email protected] Mary France 43 Strawberry Rd. 231-5926 [email protected]
All of the names, addresses, phone numbers, and e-mail addresses are called data.
2. Field: Fields are where the data goes when it is entered into a database. Recall the address book that we mentioned above. Each of the columns would be called a field. The words in bold are called Field Names, because they describe the information that is to 175 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 be contained in the field. All fields can contain different types of information. For example, the "Name" field above contains the names of the people, while the "Address" field contains the people's addresses. Some fields can contain text, some numeric values, some images, some a short movie clip, some an audio file- the list goes on and on. Sometimes, you can control the type of data that can be placed in a certain field. For example, say you want to limit the "Name" field in the above address book to only letters. There are "attribute" tags you can add to each field that restrict the kind of data that is to be placed in the field. For example, there are "attribute" tags that might say that you can only place letters in a field and other "attribute" tags that might say you can only place numbers in the field.
There is also another type of attribute tag that can limit the number of characters that you can place in a field. For example, a certain attribute tag might say that you can only type in a maximum of 10 letters in a certain field.
Finally, there is a type of attribute tag that is called "required, optional, or calculated." These attribute tags control the amount of data to be placed in a certain field. If a field is given the "required" attribute tag, that means that the field is mandatory- meaning that you must put data in the field. You cannot leave it empty. On the other hand, a field with an "optional" attribute tag does not have to be filled in. Finally, the "calculated" attribute tag means that the user is not supposed to fill it in. The computer will later fill in this field after it has performed the necessary calculations.
3. Records Just as fields would be the columns in the address book above, the records would be the rows. If you take a look at the address book that we mentioned above, a record would be a row of data. However, keep in mind that the first row is NOT called a record because it contains the field names. Thus, records begin with the second row.
Database Files:
As you might know, computer files are collections of data. For example, the word document I typed this article on would be a text file. The picture of my cat that I scanned onto my computer would be saved as a graphic file, while the MP3 that I downloaded off the Internet would be stored in an MP3 file. There are also special types of files that are used to store the data in databases. These are called database files.
Database files are composed of all of the fields, records, and data that are found in a typical database. The more files that are found in a single database, the larger the database will be!
Types of Databases
As with just about everything else in the world, there are many different types of databases. There are also different ways to classify these types of databases. Two popular ways to
176 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 differentiate between databases is by the function of the database and by the data model of the database.
Function: First, we shall take a look at classifying by the function of the database. There are different databases that are used for different tasks and jobs. The two categories here are analytic databases and operational databases:
Analytic Databases: Analytic databases, also called On Line Analytical Processing (OLAP), are databases that are primarily used to keep track of statistics. Usually, they are read-only, meaning that you can only retrieve data, but you can't modify the data in any way. They are often used by stores as inventory catalogs. They keep track of the company's sales, and then can be read and analyzed to determine how and when more products are sold. These databases can hold all sorts of descriptive information about the goods stored in a company's inventory.
Operational Databases:
Operational databases, also called On Line Transaction Processing (OLTP), are databases that have a completely different job. These databases, unlike the analytic databases, let you actually change and manipulate the data. While analytic databases only let you view the data, these databases let you modify the data in any way you would like- you can add data, delete data, or even change the data.
Data Model: Another way of classifying databases is by their data model. So, what's a data model? A data model is the intangible form in which data is stored. It is kind of like the structure of a database, but data models are only a theoretical idea; they are abstract concepts that you cannot touch. Data models are used to describe how the data is stored and retrieved in a database. Now, we will discuss a few of the types of data models.
Flat-file Database Model: The flat-file data model is generally used by the old paper-based databases. In this system, data was stored in numerous files. However, the files were not linked, so often, data might be repeated in more than one file. This caused everything to be quite redundant. The original "database," flat-file databases inspired scientists to find a way to link files so that they would not be repetitive.
Hierarchical Database Model:
The hierarchical database model took steps to get rid of the repetitiveness of the flat-file database model, but although it was somewhat successful, it did not completely succeed. There is still a level of redundant data in hierarchical databases.
A hierarchical database consists of a series of databases that are grouped together to resemble a family tree:
177 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Each of the boxes in the diagram represents one database. The top database in the hierarchical model is called the "parent" database. The databases under it are called "child" databases. One "parent" can have many "children," but a "child" can only have one "parent." The child databases are all connected to the parent database via links called "pointers."
To get to a child database in the hierarchical database model, you must first go through the parent database, and then through the levels above it. If you have Microsoft Windows, you might realize that this is how Windows Explorer works. First, you open up a file- usually it's "My Computer." Under "My Computer," you can then choose from a list of drives. Pretend you clicked on the "Disk Drive C" icon. Then, under this, you can choose from a series of folders. After opening one folder, you can open another folder, and another, until you reach the file that you want.
Notice in the diagram above how the child databases on the same level are not connected. This presents a problem in the hierarchical database model and makes searching for data extremely difficult. Another problem is that data cannot be entered into the child databases until that field has been added to the parent database. This method was quite inefficient. Thus, although the hierarchical database model reduced some repetitiveness of data, it also presented many new problems.
Network Database Model:
The network database model was designed to help resolve some of the hierarchical database model's problems. For one thing, it allowed for links between the child databases. This no only reduces the chance of redundant data, but also makes searching for data much easier!
Another improvement of the network database model over the hierarchical model is that while in the hierarchical model a child database can only have one parent, in the network model, a child database can have more than one parent!
However, the network database model still had its share of problems. For one thing, it was difficult to execute and maintain. Only database experts could successfully use these databases. It was difficult for the general public to use network databases for real-life applications.
Relational Database Model:
The relational database model came in at full swing during the 1980s. Modeled after the work of Dr. E. F. Codd of IBM, the relational database model is extremely popular because it solved many of the problems displayed in the hierarchical and network database models.
178 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 The relational database model is different from the hierarchical and network database models in that there are no "parent" and "child" databases. Rather, all of the databases in the relational database model are equal.
Data can be stored in any number of separate databases. Then, these databases are connected by a "key" field. A key field is a field (in case you don't remember, a field is the columns in the database where the data is stored- see "Parts of a Database" for more details) that is found in all of the databases that are being linked together. All of the databases can be used to hold different types of data. For example, let's pretend that we have an address book, which is a paper-based database. If we were to use the relational database model, then we would store all of the data in separate databases. One database could hold a person's address; another could hold the person's phone number, etc. However, all of the databases might have one field that is the same, like, for example, the person's name.
This makes it easy to search for and extract data from the databases. It is also very efficient and easy to use. No wonder why this database model is so popular!
Object-oriented Databases:
You might realize that databases can not only store text and numeric values, but that there are special databases that can also store photos, sounds, videos, and all sorts of graphics. How is this possible? Well, I can tell you one thing, the old hierarchical, network, and relational databases couldn't store all of these types of data! As a matter of fact, this is where the object-oriented database models come in.
Object-oriented database models let databases store and manipulate not only text, but also sounds, images, and all sorts of media clips! They are extremely useful, but unfortunately, they are large in size and are extremely expensive. Thus, they are only found in large commercial or governmental organizations.
Client/Server Databases
As you probably know, the Internet is one of the most popular usages of a computer. Client/server databases are the databases that you will use for the Internet and for the World Wide Web. The database is left on 24 hours a day, 7 days a week. This is so that users, or clients, can access the web sites or whatever other data is contained in the database at any time they want. This is especially necessary if you want to make the applications available to anybody, anywhere in the world. There is a special type of interface that lets the clients submit certain data requests from the database. Then, the database will handle and process the requests!
How Databases Work
In previous sections, we have described what databases are, what the parts of a database are, and what the types of databases are. But how do you interact with a database? How do you add, change, or delete data? There are two ways to connect to a database.
179 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 1. Command Shell: The first way to connect to a database is by using a command shell. These are devices that gives you access to the data in a database. First, you need to log on, after which you will be able manipulate data and receive the output of your actions. Command-shells come in many different forms. Some can be quite simple, while others are quite fancy, complete with graphics and icons!
2. Network Sockets: The second way to connect to a database is through sockets over a network, like the Internet. This way lets you connect to a database by using a local-area network (LAN) or a wide-area network (WAN). Thus, you can access a database in Seattle, Washington while at a business conference across the country in Philadelphia, Pennsylvania!
SQL:
As with computers, people, and animals, databases have a type of language. The most common language used by relational databases is SQL, or Structured Query Language.
As you may or may not know, databases are arranged in table-like structures. The columns are called fields, while the rows are called records. (See Parts of a Database for more details.) It is in these fields and records that data is stored. You can use commands in the SQL language to access and manipulate the data in the fields and records of a database.
How does SQL keep track of all of these tables? Well, SQL databases have "data dictionaries." These are merely tables that keep track of all of the data tables! You will type in the name of the table where the data you want access to is held, and the database will search through its data dictionary until it comes up with your table.
When you type in a command to an SQL database, you first must type in the name of the table you want. Then, you can give the database the specific details that you want. The database will search through its data dictionaries until it finds the correct piece of data. The database will then produce a "view," or result, of the information that you specified! It's as simple as that.
· Introduction to database gateways for web servers
The Recital Universal Application Server comes complete with the Recital Relational Database Management System. It can also be extended via database gateways to give full access to other data sources and SQL based RDBMSs. Gateways are available to ODBC data sources, such as Microsoft Access or SQL Server, to JDBC data sources via third party JDBC drivers, to MySQL, PostgreSQL, Informix, Ingres and Oracle and to Recital itself.
180 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Once a connection is established to the server, there are two ways for the Universal Application Server (UAS to access the data:
1. ‘Pass Through’ technology. This allows full access to all data related commands provided by the server. The UAS passes SQL statements through to the server without parsing them, so the SQL statements issued must be valid for that particular server. The Remote Data Connectivity Functions can be used to provide automatic cursor creation and fetching when using Pass Through SQL. Please see the SQL Reference Manual for details.
2. Recital/4GL. A subset of the Recital work surfaces and 4GL commands and functions can be used against the data source.
The SET GATEWAY and LOGIN commands can each be used to establish a connection to a gateway server.
SET GATEWAY
SET GATEWAY TO [
181 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 // Connect to Ingres server set gateway to “ing@sun5:inguser1/ingpass1-accounts.tcpip” For server-specific variations, please see below: JDBC server@machinename:username/password-driver path;url
// Connect to Recital JDBC Driver set gateway to “jdbc@linux:jdbuser1/jdbpass1-/usr/java/lib/RecitalJDBC/Recital/sql/RecitalDriver;” + ; “jdbc:Recital:SERVERNAME=linux1;DIRECTORY=/usr/recital/ud/demo”
ODBC server@machinename:username/password-datasource
the datasource name as specified in the ODBC Data Source Administrator. // short format for local ODBC data source set gateway to “odbc:Northwind” // long format set gateway to “odbc@host1:user1/pass1-Northwind” ORACLE server@machinename:username/password-service
182 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 // Connect to Oracle server set gateway to “ora@host2:scott/tiger-orasamples”
LOGIN
LOGIN [
Example // Recital Gateway login “recital”, “hp5”, “hpuser1”, “hppass1”, “/usr/recital/data/southwind” // Recital JDBC Driver server = “jdbc” nodename = “host2” username = “user1” password = “pass1” database = “/usr/java/lib/RecitalJDBC/Recital/sql/RecitalDriver;" + ; “jdbc:Recital:SERVERNAME=host2;DIRECTORY=/usr/recital/ud/demo” 183 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 login server,nodename,username,password,database // Oracle Gateway login “oracle”, “orahost”, “scott”, “tiger”, “orasamples”
------NOT YET - - - - -
· Common Gateways Interface
The Common Gateway Interface (CGI) is a standard for interfacing external applications with information servers, such as HTTP or Web servers. A plain HTML document that the Web daemon retrieves is static, which means it exists in a constant state: a text file that doesn't change. A CGI program, on the other hand, is executed in real-time, so that it can output dynamic information. For example, let's say that you wanted to "hook up" your Unix database to the World Wide Web, to allow people from all over the world to query it. Basically, you need to create a CGI program that the Web daemon will execute to transmit information to the database engine, and receive the results back again and display them to the client. This is an example of a gateway, and this is where CGI, currently version 1.1, got its origins.
The database example is a simple idea, but most of the time rather difficult to implement. There really is no limit as to what you can hook up to the Web. The only thing you need to remember is that whatever your CGI program does, it should not take too long to process. Otherwise, the user will just be staring at their browser waiting for something to happen.
Specifics
Since a CGI program is executable, it is basically the equivalent of letting the world run a program on your system, which isn't the safest thing to do. Therefore, there are some security precautions that need to be implemented when it comes to using CGI programs. Probably the one that will affect the typical Web user the most is the fact that CGI programs need to reside in a special directory, so that the Web server knows to execute the program rather than just display it to the browser. This directory is usually under direct control of the webmaster, prohibiting the average user from creating CGI programs. There are other ways to allow access to CGI scripts, but it is up to your webmaster to set these up for you. At this point, you may want to contact them about the feasibility of allowing CGI access. If you have a version of the NCSA HTTPd server distribution, you will see a directory called /cgi-bin. This is the special directory mentioned above where all of your CGI programs currently reside. A CGI program can be written in any language that allows it to be executed on the system, such as:
· C/C++
· Fortran
· PERL
· TCL
· Any Unix shell
184 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · Visual Basic
· AppleScript It just depends what you have available on your system. If you use a programming language like C or Fortran, you know that you must compile the program before it will run. If you look in the /cgi-src directory that came with the server distribution, you will find the source code for some of the CGI programs in the /cgi-bin directory. If, however, you use one of the scripting languages instead, such as PERL, TCL, or a Unix shell, the script itself only needs to reside in the /cgi-bin directory, since there is no associated source code. Many people prefer to write CGI scripts instead of programs, since they are easier to debug, modify, and maintain than a typical compiled program.
How do I get information from the server? Each time a client requests the URL corresponding to your CGI program, the server will execute it in real-time. The output of your program will go more or less directly to the client. A common misconception about CGI is that you can send command-line options and arguments to your program, such as command% myprog -qa blorf CGI uses the command line for other purposes and thus this is not directly possible. Instead, CGI uses environment variables to send your program its parameters. The two major environment variables you will use for this purpose are: · QUERY_STRING
QUERY_STRING is defined as anything which follows the first ? in the URL. This information could be added either by an ISINDEX document, or by an HTML form (with the GET action). It could also be manually embedded in an HTML anchor which references your gateway. This string will usually be an information query, i.e. what the user wants to search for in the archie databases, or perhaps the encoded results of your feedback GET form.
This string is encoded in the standard URL format of changing spaces to +, and encoding special characters with %xx hexadecimal encoding. You will need to decode it in order to use it.
If your gateway is not decoding results from a FORM, you will also get the query string decoded for you onto the command line. This means that each word of the query string will be in a different section of ARGV. For example, the query string "forms rule" would be given to your program with argv[1]="forms" and argv[2]="rule". If you choose to use this, you do not need to do any processing on the data before using it.
· PATH_INFO
CGI allows for extra information to be embedded in the URL for your gateway which can be used to transmit extra context-specific information to the scripts. This information is usually made available as "extra" information after the path of your gateway in the URL. This information is not encoded by the server in any way.
The most useful example of PATH_INFO is transmitting file locations to the CGI program. To illustrate this, let's say I have a CGI program on my server called /cgi-bin/foobar that can process files residing in the DocumentRoot of the server. I 185 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 need to be able to tell foobar which file to process. By including extra path information to the end of the URL, foobar will know the location of the document relative to the DocumentRoot via the PATH_INFO environment variable, or the actual path to the document via the PATH_TRANSLATED environment variable which the server generates for you.
How do I send my document back to the client?
I have found that the most common error in beginners' CGI programs is not properly formatting the output so the server can understand it. CGI programs can return a myriad of document types. They can send back an image to the client, and HTML document, a plaintext document, or perhaps even an audio clip. They can also return references to other documents. The client must know what kind of document you're sending it so it can present it accordingly. In order for the client to know this, your CGI program must tell the server what type of document it is returning.
In order to tell the server what kind of document you are sending back, whether it be a full document or a reference to one, CGI requires you to place a short header on your output. This header is ASCII text, consisting of lines separated by either linefeeds or carriage returns (or both) followed by a single blank line. The output body then follows in whatever native format.
· A full document with a corresponding MIME type
In this case, you must tell the server what kind of document you will be outputting via a MIME type. Common MIME types are things such as text/html for HTML, and text/plain for straight ASCII text.
For example, to send back HTML to the client, your output should read: Content-type: text/html
Sample output
What do you think of this? · A reference to another documentInstead of outputting the document, you can just tell the browser where to get the new one, or have the server automatically output the new one for you.
For example, say you want to reference a file on your Gopher server. In this case, you should know the full URL of what you want to reference and output something like: Content-type: text/html Location: gopher://httprules.foobar.org/0
Go to gopher instead
186 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Now available at a new location On our gopher server. However, today's browsers are smart enough to automatically throw you to the new document, without ever seeing the above since. If you get lazy and don't want to output the above HTML, NCSA HTTP will output a default one for you to support older browsers. If you want to reference another file (not protected by access authentication) on your own server, you don't have to do nearly as much work. Just output a partial (virtual) URL, such as the following: Location: /dir1/dir2/myfile.htmlThe server will act as if the client had not requested your script, but instead requestedhttp://yourserver/dir1/dir2/myfile.html. It will take care of most everything, such as looking up the file type and sending the appropriate headers. Just be sure that you output the second blank line. If you do want to reference a document that is protected by access authentication, you will need to have a full URL in the Location:, since the client and the server need to re-transact to establish that you access to the referenced document. Advanced usage: If you would like to output headers such as Expires or Content-encoding, you can if your server is compatible with CGI/1.1. Just output them along with Location or Content-type and they will be sent back to the client.
Writing secure CGI scripts
Any time that a program is interacting with a networked client, there is the possibility of that client attacking the program to gain unauthorized access. Even the most innocent looking script can be very dangerous to the integrity of your system. With that in mind, we would like to present a few guidelines to making sure your program does not come under attack.
· Beware the eval statement
Languages like PERL and the Bourne shell provide an eval command which allow you to construct a string and have the interpreter execute that string. This can be very dangerous. Observe the following statement in the Bourne shell:
eval `echo $QUERY_STRING | awk 'BEGIN{RS="&"} {printf "QS_%s\n",$1}' `
This clever little snippet takes the query string, and convents it into a set of variable set commands. Unfortunately, this script can be attacked by sending it a query string which starts with a ;. See what I mean about innocent-looking scripts being dangerous?
· Do not trust the client to do anything
A well-behaved client will escape any characters which have special meaning to the Bourne shell in a query string and thus avoid problems with your script misinterpreting the characters. A mischievous client may use special characters to confuse your script and gain unauthorized access. 187 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 · Be careful with popen and system.
If you use any data from the client to construct a command line for a call to popen() or system(), be sure to place backslashes before any characters that have special meaning to the Bourne shell before calling the function. This can be achieved easily with a short C function.
· Turn off server-side includes
If your server is unfortunate enough to support server-side includes, turn them off for your script directories!!!. The server-side includes can be abused by clients which prey on scripts which directly output things they have been sent.
· Server Application Programming Interfaces (SAPIs)
· JavaScript
JavaScript is a scripting language used in many websites. A scripting language is a language, which is easy and fast to learn. So is this reference. A scripting language is interpreted in run-time. It is not compiled like other languages as C++, C#, VB.NET etc. JavaScript is a client side language and it runs on the client browser. Netscape developed it and because of its simplicity it is one of the most known scripting languages. However JavaScript can be also used on the server-side. JavaScript can be used on all most known browsers. It can be easily used to interact with HTML elements. You can validate text fields, disable buttons, validate forms, or change the background color of your page. All this is possible with JavaScript. Like each programming language, it contains variables, arrays, functions, operators, objects and much more which can be help you to create better scripts for your pages. On the server side you can use JavaScript for example to manage your database entry. JavaScript code can be inserted directly in the HTML or you can place it in a separate file with the .js extension and link the web page with the .js file.
Is JavaScript a lighter version of Java?
Many who haven’t worked before with java or scripting languages think that JavaScript is the same as Java or a lighter version of it. This is not true. Java is a different language developed by Sun Micro Systems. Java is much more complex then JavaScript. In Java you have to declare each variable with the type, in JavaScript you don’t need to do that. All variables are declared when you first time use them. Furthermore in Java you have to declare all variables, functions and classes. In JavaScript you don’t even need to think about these things. Java is compiled to byte codes on the server and the result is send to the client. JavaScript is interpreted on the client side it doesn’t require any compilation.
Usage of JavaScript
Usually web-designers design pages and coders code applications. However with JavaScript a designer has the possibility to create a client side application with very less efforts. He can easily create some kind of dynamic pages – i.e.: you can easily show a prompt box and asks the user to enter his name whenever the page loads for the first time. He can then use the entered value to create a welcome string. These procedures are called events. Events can be used and called when something occurs – like loading the page for the first time. You could also write another event which is called whenever the page is closed. JavaScript is also mostly used to validate text fields. For example in asp.net you have the possibility to validate 188 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 your controls with some validator controls. These validator controls are basically nothing more then a JavaScript file. This validation can be easily written with JavaScript to verify if a text field is empty or not.
JavaScript Functions
Introduction Functions are an important part in any programming language. A function is a block of commands, which can be executed again and again, by the calling the name. JavaScript contains some inbuilt functions, which can be used freely and you can create your own functions. A function contains of a name and a body. Each function starts with the keyword function and then the name of the function. After that you can put your required commands in the body part which must be enclosed with the {} brackets. Once you have created your function you can call it with the name of the function.
JavaScript Simple Function
189 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Output
JavaScript Functions with Parameters
The above example displays a simple function, however you can also pass arguments to a function. For that you have to define the parameter in the in the brackets of the function name. Once you have declared your parameter you can simply pass the value when calling the function. The following example demonstrates it.
Example
…function CalculcatePI(number){ document.write("This is a function with parameters.
") var PI = 3.14 document.write(number*PI+"
")} // Call function and pass 5 as a parameter Calculate(5)…
Output
190 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 JavaScript Functions with Return Values
Functions can also return the value instead of displaying it. This can be useful if you need to do some more processing with the variable returned by the function. The following example demonstrates it.
Example
…function ReturnPIValue(number){ document.write("This is a function with parameters and a return value.
") var PI = 3.14 return number*PI// Call function, which will return a value.var x = ReturnPIValue(5) document.write(x)…
Output
· ASP
Active Server Pages or ASP is a technology developed by Microsoft to further the ability to write applications for the web. Active Server Pages can be written in Visual Basic Script or Javascript and contain the ability to be dynamic. What this means is that a few lines of code can create pages that can change minute to minute. For instance, many applications that access and manipulate data in databases are now being written in ASP and as the database is updated the web pages are also updated. In this tutorial we will focus on the Visual Basic Scripting side of ASP and demonstrate how a few functions work within this technology.
In order to properly utilize Active Server Pages technology you must either have a Windows NT 4.0 Server or greater with IIS 3.0 or greater or Windows NT 4.0 Workstation or greater with PWS installed. Once this service is installed and running we must find the root web directory. For ease we will the default web site that is normally installed. Multiple websites can later be added to this. The default directory for these pages is c:\inetpub\wwwroot\. When we place files in this directory they should be able to be accessed via this directory. As an example place a simple text file in this directory and then using the server console try to access this file by typing http://localhost/filename.txt where filename.txt is the file that you
191 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 placed in this directory. If this succeeds you have placed the file in the proper location and have the server software correctly installed.
We want to now create dynamic web pages. Let's start with the basics. First off all ASP 3.0 pages are designated with the file extension of .asp. This tells the server to access the asp.dll library to process the page. Secondly all ASP code must fall between these two designators:
<% %>The <%%> tells theserverto process thatinformation asASP. The next line after the initial <% needs to be:
@Language=VBScript This tells the server that the language that will be used to create these pages is VBScript. We can now Visual Basic Scripting code within these brackets. An easy and useful element to begin with is the write method of the response object. Response.write has the ability to display on the page either a string or a value from a variable. For instance:
Response. Write ("Hello!") This would output:
Hello! At this point it would obviously be faster to just create an HTML page to do the same thing. However, perhaps we want to display Hello! five times in a row. By using a for-next loop the ASP code would be three lines:
For i = 1 to 5 response.write ("Hello!
") Next i Whereas the HTML code would be five lines:
Hello!
Hello!
Hello!
Hello!
Hello!
This is a very simplistic and unstructured example, but it shows where a working knowledge of ASP can cut down on our coding and make our lives easier. In the previous examples a variable of "i" is used to move through the for-next loop. All variables in ASP should be declared by using a "Dim" statement:
Dim i This declares the "i" variable as a variant type. A variant type of variable means that "i" can contain an integer, float, string, etc. All constants must be declared using the "Const" statement: Const t=5 To make sure all variables are declared an "Option Explicit" should be one of the first lines in the code. By the way, many programmers do not include this statement and create variables on the fly. The previous for-next loop example would work without first declaring "i" but it is considered bad, unstructured programming. In the rest of the examples it will be assumed that "Option Explicit" proceeds the example as does the variable declarations. Loop-the-Loop 192 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Active Server Pages contains various loops that can be used to process data. These loops include:
If-Then-Else Loop For-Next Loop For-Each-Next Loop Do-While/Until Loop While-Wend Loop
We will now discuss each of these loop types individually.
If-Then-Else Loop
This loop allows for checking of code, if a statement is found then certain processing occurs, otherwise something else is processed.
If name="Joe" then Response.write ("Hello Joe!") else Response.write("I don't think we have met.") End If In this example if the variable "name" is equal to the value "Joe" then the "Hello Joe!" string is outputted. Otherwise, the script will output "I don't think we have met."
For-Next Loop
The For-Next loop allows for code to be processed until a specific goal is reached. We have already seen one example of the for-next loop. Here is another example:
For i=1 to 20 response.write (i) Next This example outputs the value of "i" 20 times, each time incrementing it once. This would produce: 1234567891011121314151617181920. This allows for code to be run a multiple of times very easily. At any time in a For-Next Loop the loop can exit using an "Exit For" statement. y=7 for i=1 to 20 response.write (i) if i=y then exit for next In this example when "i" equals "y" the For-Next Loop will not continue and the rest of the page will process.
For-Each-Next Loop
The For-Each-Next Loop is specifically designed for arrays, collections, or dictionaries. This allows for processing to occur on each value in one of these types of situations. If you had an array, for example, you could process for each value in the array.
Dim x (20) For Each g in x 193 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 response.write (g) Next The "Exit For" statement works the same way in this situation as it does in the previous For-Next Loop.
Do-While/Until Loop
The Do-While/Until Loop allows for bits of code to be run Until a goal is attained or While a goal is not attended. x=1 Do While x<10 Response.write ("Hi There!
") x=x+1 Loop This outputs Hi There!
to the page each time incrementing "x". Once x has reached the value of "10" the loop is broken and the page continues processing. Do-Until loops basically work the same way, but instead of proceeding through the loop while a value remains true, it procedures through the loop until something becomes true.
At any time in a Do-While/Until Loop the loop can exit using an "Exit Do" statement. x=10 y=7 Do Until x<5 response.write("hello") x=x-1 if x=y then Exit Do Loop In this example when "x" equals "y" the loop will not continue and the rest of the page will process.
While-Wend Loop
The final loop type we will discuss is the While-Wend. The While-Wend loop will process given statements until a goal or condition is achieved. This type of loop is very similar to the Do-While Loop and generally not thought of as structured as the Do-While Loop. x=1 While x<10 Response.write ("Hi There!
") x=x+1 Wend This outputs Hi There!
and increments "x". Once x has reached the value of "10" the While-Wend loop is broken and the page continues processing. Make a Statement! Now that we have gone over the common loops found in ASP we need to talk about a couple of other useful statements.
Select-Case Statement
194 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 First of, the Select-Case statement is a very effective means of executing statements based on the value of a given variable. Job Type="Programmer" Select Case Job Type Select "Graphics Designer" Response.write ("I am a Graphics Designer") Select "Network Engineer" Response.write ("I am a Network Engineer") Select "Programmer" Response.write ("I am a Programmer")
End Select In this example the statement "I am a Programmer" would be outputted to the page. A "Case Else" statement can be added so if the condition is not met a selected set of statements are then executed. The Select-Case statement is useful for a greater degree of control, ease of use, and ease of reading than several If-Then-Else statements.
With Statement
The With statement enables several statements to be performed on an object without the object being requalified every time.
With Color Scheme .Trim="Green" .Main Color="Red" .PinStrips="Yellow" End With Nesting Statements
All the previous loops and statements have the ability to be nested within each other and within themselves. For instance:
If x=y then do until x=8 x=x+1 loop End If In this instance if "x" is equal to "y" a Do-Until Loop is executed until "x" is equal to "8". Where's the Function? Of course, one of the major strengths of any language is the ability to create functions for tasks and then call on those functions whenever necessary. In Visual Basic Script a function is declared by simply calling "Function" a name and any variable names that will get passed into the function. g=total amount (1,5) Function Total Amount(x,y) Total Amount=x+y End Function In this function the values x and y are passed into the function, added and then final result is passed back to the original call. This was an example of a function that returned a value. 195 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Functions can just as easily process a few lines and then return without sending back any results.
Call Write This Function Write This Response. Write ("Hello!") Response. Write ("This is a function!") End Function The function "WriteTshis" was called using the Call statement, it processed the two response. write statements and then returned. No variable was needed since it did not need any values from outside the function.
This is not SUB Par!
Subs are very close to Functions with one major exception, Subs cannot return any values. This means values can be sent to them, they can process statements within them, they can call other functions or subs, but they cannot send back any results. Remember that "g=total amount(1,5)" that called the total amount function and made the variable "g" equal to the result. That cannot occur with Subs. An error would occur. Subs can only be called by using the Call Statement, like the second function example previously.
Call WhatColorisThis (8) Sub WhatColorisThis(x) Select Case x Case 2 Response.write ("Yellow") Case 4 Response.write ("Blue") Case 6 Response.write ("Red") Case 8 Response.write ("Green") End Sub By calling this sub the response of "Green" is outputted to the page and the rest of the processing of the Visual Basic Script occurs.
I am ready to write a script!
Now that we have gone over the basics we can write a full script.
We will create a page and designate it default. asp. Create the file in the c:\inetpub\wwwroot\ directory. Open the file using any text editor, my editor of choice is notepad because of the fact it is installed on every Microsoft Windows machine in the world. In this file place the following code:
<% @Language=VBScript Option Explicit
Dim Final Result Dim Number1 Dim Number2
196 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 Final Result=NumberstoMultiply (5, 6)
Function NumberstoMultiply (Number1, Number2) NumberstoMultiply=5*6 End Function
Response. Write ("Your Final Result is” & Final Result & ".")
%> Save the file and then open a web browser on the server and go to the URL: http://localhost/default.asp, or replace local host with the ip address or name of the server. This should process your page. With any luck the output of the page would be:
Your Final Result is 30.
As you can see from the previous example, the response. Write method outputted a string and the variable result followed by. This is done by using the apersand (&) in between the string declaration and variable.
I want more!
This example is simplistic example of what we can accomplish using ASP. With our arsenal of statements and loops that we discussed we can begin to create much more complex ASP scripts that will make our web lives easier. We can pull data from a database, perform computations, even pull data from other websites. Feel free to try out each loop, it takes practice and experience to learn what loop and statement needs to be used where. ASP is designed to be easy and robust at the same time.
· PHP
What is PHP?
PHP, which stands for "Hypertext Preprocessor", is a server-side, HTML embedded scripting language used to create dynamic Web pages. Much of its syntax is borrowed from C, Java and Perl with some unique features thrown in. The goal of the language is to allow Web developers to write dynamically generated pages quickly.
In an HTML page, PHP code is enclosed within special PHP tags. When a visitor opens the page, the server processes the PHP code and then sends the output (not the PHP code itself) to the visitor's browser. It means that, unlike JavaScript, you don't have to worry that someone can steal your PHP script.
PHP offers excellent connectivity to many databases including MySQL, Informix, Oracle, Sybase, Solid, PostgreSQL, and Generic ODBC. The popular PHP-MySQL combination (both are open-source products) is available on almost every UNIX host. Being web-oriented, PHP also contains all the functions to do things on the Internet - connecting to remote servers, checking email via POP3 or IMAP, url encoding, setting cookies, redirecting, etc.
What do PHP code look like?
PHP is a rather simple language. Much of its syntax is borrowed from C except for dealing with the types of variables. You don't need to think of the types of variables at all - you just 197 F.C Ledesma Avenue, San Carlos City, Negros Occidental Tel. #: (034) 312-6189/(034) 729-4327 work with their values, not their types. And you don't have to declare variables before you use them.
Basic Syntax
· F i l e n a m e : You should save your file with the extension .php (earlier versions used the extensions .php3 and .phtml).
· C o m m e n t s : //This comment extends to the end of the line. /* This is a multi-line comment */
· Escaping from HTML: A PHP code block starts with "". A PHP code block can be placed anywhere in the HTML document.
· Instruction separation: Each separate instruction must end with a semicolon. The PHP closing tag (?>) also implies the end of the instruction.
Here's a small PHP example... < h t m l >