USOO8700765B2
(12) United States Patent (10) Patent N0.: US 8,700,765 B2 Yumerefendi et a]. (45) Date of Patent: Apr. 15, 2014
(54) METHODS AND COMPUTER PROGRAM 2006/0098577 A1 5/2006 MeLampy et a1. PRODUCTS FOR MONITORING AND 2007/0156900 A1 7/2007 ChIOII REPORTING NETWORK APPLICATION 58888888318 21 188888 .411. PERFORMANCE 2010/0132024 A1 5/2010 Ben-Natan etal. 2010/0145925 A1* 6/2010 Flinta et a1...... 707/709 (75) Inventors: Aydan R. Yumerefendi, Raleigh, NC 2010/0226282 A1 9/2010 Aitken et al. (US); Patrick A. Reynolds, Chapel Hill, 2010/0332646 A1 * 12/2010 Balasubramanian et 31. 709/224 NC (U S); John B. Bley, Durham, NC OTHER PUBLICATIONS (Us) Trie [online], Sep. 18, 2010 [retrieved on Dec. 21, 2010]. Retrieved (73) Assignee: Blue Stripe Software, Inc., Morrisville, from the Internet:
Related US. Application Data Primary Examiner * Hua Fan (60) Provisional application No. 61/234,049, ?led on Aug. (74) Attorney, Agent, or Firm * Myers Bigel Sibley & 14, 2009. Sajovec, PA.
(51) Int. Cl. (57) ABSTRACT G06F 15/173 (2006.01) Provided are methods and computer program products for (52) US. Cl. monitoring and reporting network application performance in USPC ...... 709/224; 370/255; 370/329 a networked device. Methods may include collecting perfor (58) Field of Classi?cation Search mance data corresponding to at least one application running USPC ...... 709/224 on the networked device in substantially real time using at See application ?le for complete search history. least one kernel space driver interface; generating multiple kernel level metrics and multiple user level metrics based on (56) References Cited the collected performance data; aggregating the kernel level metrics and the user level metrics that were generated during U.S. PATENT DOCUMENTS a prede?ned time interval; and generating, responsive to aggregating the kernel level metrics and the user level met 5,958,010 A * 9/1999 Agarwal et al...... 709/224 2003/0036964 A1* 2/2003 Boyden et al. .. 705/26 rics, an event incorporating the aggregated metrics. 2004/0066759 A1 * 4/2004 Molteni et al. .. 370/329 2004/0225530 A1* 11/2004 Bell et al...... 705/2 17 Claims, 12 Drawing Sheets
—1 Establish hooks on a networked device to the Aggregate the kernel level and user level metrics operating system's internal network protocol kemel generated in the most recent 15-seccnd time interval interface, and to an application-oriented system call interface to a tmnsport network stack m Process the aggregated kernel level and user level metrics to remove redundant data and reconcile Collect via the hooks performance data inconsistent data corresponding to at least one network ' ' m running on the networked device
Perform a reverse DNS lookup of each IP address if included in the aggregated kernel level and user level metrics to determine the DNS name associated with Generate kernel level and user level metrics based the l? address on the collected pertorrnance data £12 M 1 1 Generate an event incorporating the kernel level and Aggregate the kernel level and user level metrics by user level metrics and thaietermined DNS name(s) application (e.g.. by local IP address, local port, and process ID) US. Patent Apr. 15, 2014 Sheet 1 0f 12 US 8,700,765 B2
@5622 66260 66260 22:82
@5624 66260 3%£60:63806 66260 92622 /mchmE?sts 66260 4/, @562260: 66260 \l66260 US. Patent Apr. 15, 2014 Sheet 2 0f 12 US 8,700,765 B2
owmnmpmo
coswo=nq A:.QE wr Ew=0 US. Patent Apr. 15, 2014 Sheet 4 0f 12 US 8,700,765 B2 @250 ow? NNP \ 280 :62 50:62 mowr\\\\ 0: 83mm UP.9“ Lowwmooi£22 2252o: gmEmEu @925 US. Patent Apr. 15, 2014 Sheet 9 0f 12 US 8,700,765 B2 859:_m>m_ucm_w>m_6mem5BmmmhmgEm: EEBEmE:9603-339:m55E39@290ch ma Em m.QE Emumocmczotmq960;@>“8:009: cowmuinm{oz/Hm:“may20528360pm96 w2>ouvmvroEm:@555:9:Co % An:$805 am“ |l||4 US. Patent Apr. 15, 2014 Sheet 10 0f 12 US 8,700,765 B2 $5.23. 35% o Hanuwa?uV H W, US 8,700,765 B2 1 2 METHODS AND COMPUTER PROGRAM features or essential features of this disclosure, nor is it PRODUCTS FOR MONITORING AND intended to limit the scope of the invention. REPORTING NETWORK APPLICATION Some embodiments of the present invention are directed to PERFORMANCE a method for monitoring application performance in a net worked device. Methods may include collecting performance RELATED APPLICATIONS data using at least one kernel space driver interface, the col lected performance data corresponding to at least one appli This non-provisional patent application claims priority to cation running on the networked device. Multiple kernel level US. Provisional Application No. 61/234,049, ?led Aug. 14, metrics and/or multiple user level metrics based on the col 2009 and entitled Network Monitoring Methods, Systems lected performance data may be generated, and an event and Computer Program Products, the disclosure of which is incorporating at least one of the kernel level metrics and at hereby incorporated herein by reference as if set forth fully least one of the user level metrics may be generated. herein. In some embodiments, ones of the kernel level metrics and/or ones of the user level metrics that were generated FIELD OF INVENTION during a prede?ned time interval may be aggregated. The event generated may incorporate the aggregated metrics gen The present invention relates to computer networks and, erated during the prede?ned time interval. more particularly, to network performance monitoring meth Some embodiments provide that the at least one applica ods, devices, and computer program products. 20 tion includes multiple applications, and ones of the kernel level metrics and/or ones of the user level metrics that corre BACKGROUND spond to a speci?ed one of the applications may be aggre gated. The event generated may incorporate the aggregated The growing presence of computer networks such as intra metrics corresponding to the speci?ed one of the applications. nets and extranets has brought about the development of 25 In some embodiments, a ?rst portion of performance data applications in e-commerce, education, manufacturing, and may be selectively collected from an operating system inter other areas. Organizations increasingly rely on such applica nal kernel interface that provides an interface between a net tions to carry out their business, production, or other objec work protocol and a network protocol client. A second por tives, and devote considerable resources to ensuring that the tion of performance data may be selectively collected from an applications perform as expected. To this end, various appli 30 application oriented system call interface to a transport net cation management, monitoring, and analysis techniques work stack. Some embodiments provide that the at least one have been developed. application includes multiple applications. The ?rst portion One approach for managing an application involves moni and the second portion of performance data may be aggre toring the application, generating data regarding application gated where the ?rst portion and second portion correspond to performance, and analyzing the data to determine application 35 a speci?ed one of the applications. The event generated may health. Some system management products analyze a large incorporate aggregated performance data corresponding to number of data streams to try to determine a normal and the speci?ed one of the applications. abnormal application state. Large numbers of data streams In some embodiments, the collected performance data may are often analyzed because the system management products be processed to remove redundant portions thereof and/ or to may not have a semantic understanding of the data being 40 reconcile inconsistent data therein. analyzed. Accordingly, when an unhealthy application state Some embodiments provide that the collected performance occurs, many data streams may have abnormal data values data may indicate detection of network link establishment, because the data streams are causally related to one another. detection of extant network links, detection of network link Because the system management products may lack a seman failure, detection of network link termination, detection of tic understanding of the data, they may not be able to assist the 45 network port binding, detection of extant bound network user in determining either the ultimate source or cause of a ports, detection of network port unbinding, detection of pend problem. Additionally, these application management sys ing reads, detection of completed reads, detection of stalled tems may not know whether a change in data indicates an reads, network bytes sent, network bytes received, process application is actually unhealthy or not. identi?cation, process start time, process exit time, local Current application management approaches may include 50 Internet Protocol (IP) address, local port, remote IP address, monitoring techniques such as deep packet inspection (DPI), and/or remote port, among others. which may be performed as a packet passes an inspection In some embodiments, respective ones of the kernel level point and may include collecting statistical information, metrics may include server response time, read wait time, among others. Such monitoring techniques can be data-inten number of pending reads, number of completed reads, aver sive and may be ineffective in providing substantively real 55 age read wait time, number of stalled reads, number of com time health information regarding network applications. pleted responses, total read wait time, total bytes sent, total Additionally, packet trace information may be lost and appli response time, and/ or number of responses, among others. cation-speci?c code may be required. Some embodiments provide that respective ones of the user Embodiments of the present invention are, therefore, level metrics may include aggregate central processing unit directed towards solving these and other related problems. 60 percentage, aggregate memory percentage, total network bytes sent, total network bytes received, number of page SUMMARY faults, number of pages input, number of pages output, queue length, number of bytes read from logical disk, number of It should be appreciated that this Summary is provided to bytes written to logical disk, number of completed read introduce a selection of concepts in a simpli?ed form, the 65 requests on logical disk, number of completed write requests concepts being further described below in the Detailed on logical disk, total read wait times, and/or total write wait Description. This Summary is not intended to identify key times, among others. US 8,700,765 B2 3 4 In some embodiments, the collected performance data may BRIEF DESCRIPTION OF THE DRAWINGS include an Internet Protocol (IP) address. A reverse Domain Name System (DNS) lookup of the IP address may be per The present invention will now be described in more detail formed to determine a DNS name associated with the IP in relation to the enclosed drawings, in which: address. The event generated may incorporate the determined FIGS. 1a-1d are block diagrams illustrating exemplary DNS name. networks in which operations for monitoring network appli Some embodiments provide a method for generating a cation performance may be performed according to some model of network application health. Activity data corre embodiments of the present invention, sponding to activities of multiple applications executing on at FIG. 2 is a block diagram illustrating an architecture of a least one networked device is received and combined to computing device as discussed above regarding FIGS. lc and remove redundant portions thereof and/or to reconcile incon 1d, FIG. 3 is a block diagram illustrating operations and/or sistencies therein. Based on the received activity data, ones of functions of a collector application as described above the multiple applications are identi?ed, and relationships regarding FIG. 111, between the applications are determined. A model, including FIG. 4 is a diagram illustrating determining a read wait the identi?ed applications and the relationships between the time corresponding to a user transaction according to some applications, is generated, and a representation thereof is embodiments of the present invention, displayed. FIG. 5 is a block diagram illustrating a kernel level archi Some embodiments provide that the at, least one net tecture of a collector application to explain kernel level met worked device includes multiple networked devices. Receiv 20 rics according to some embodiments of the present invention, ing activity data includes receiving activity data from mul FIG. 6 is a ?owchart illustrating exemplary operations tiple collector applications that are each operable to execute carried out by a collector application in monitoring and on respective ones of multiple networked devices. In some reporting network application performance according to embodiments, the existence of a second application for which some embodiments of the present invention, corresponding activity data was not received may be inferred 25 FIG. 7 is a screen shot of a graphical user interface (GUI) based on identi?cation of a ?rst application for which corre including a model generated by a health data processing sponding activity data was received, and the relationship application according to some embodiments of the present between the ?rst and second applications may be inferred invention, based on the identi?cation of the ?rst application. FIG. 8 is a ?owchart illustrating exemplary operations Some embodiments provide that the received activity data 30 carried out by a health data processing application in gener is stored along with associated data indicating when the activ ating and displaying a real-time model of network application ity data was received. Identi?cation of applications is based health according to some embodiments of the present inven on stored activity data received during a speci?ed time inter tion, and val. FIG. 9 is a ?owchart illustrating exemplary operations In some embodiments, the at least one network device 35 carried out by a health data processing application in gener includes a virtual machine, and receiving activity data ating and displaying an historical model of network applica includes receiving activity data from a collector application tion health according to some embodiments of the present executing within the virtual machine. In other embodiments, invention. the at least one network device includes a host machine that is operable to execute at least one virtual machine, and receiving 40 DETAILED DESCRIPTION activity data includes receiving activity data from a collector application executing on the host machine, with the collector In the following description, for purposes of explanation application being external to the at least one virtual machine. and not limitation, speci?c details are set forth such as par Some embodiments provide that identifying the ones of ticular architectures, interfaces, techniques, etc. in order to multiple applications and the relationships between the appli 45 provide a thorough understanding of the present invention. cations includes correlating the received activity data with a However, it will be apparent to those skilled in the art that the prede?ned telecommunications standard. In some embodi present invention may be practiced in other embodiments that ments, the prede?ned telecommunications standard includes depart from these speci?c details. In other instances, detailed the port numbers list maintained by the Internet Assigned descriptions of well known devices, circuits, and methods are Numbers Authority (IANA). 50 omitted so as not to obscure the description of the present Some embodiments provide that generating a model invention with unnecessary detail. While various modi?ca includes dynamically generating a real -time or near-real-time tions and alternative forms of the embodiments described representation of the activities of the multiple applications. herein may be made, speci?c embodiments are shown by way In some embodiments, a computer program product of example in the drawings and will herein be described in including a non-transitory computer usable storage medium 55 detail. It should be understood, however, that there is no intent having computer-readable program code embodied in the to limit the invention to the particular forms disclosed, but on medium is provided. The computer-readable program code is the contrary, the invention is to cover all modi?cations, con?gured to perform operations corresponding to methods equivalents, and alternatives falling within the spirit and described herein. scope of the invention as de?ned by the claims. Like reference Other methods, devices, and/or computer program prod 60 numbers signify like elements throughout the description of ucts according to exemplary embodiments will be or become the ?gures. apparent to one with skill in the art upon review of the fol As used herein, the singular forms “a,” “an,” and “the” are lowing drawings and detailed description. It is intended that intended to include the plural forms as well, unless expressly all such additional methods, devices, and/or computer pro stated otherwise. It should be further understood that the gram products be included within this description, be within 65 terms “comprises” and/or “comprising” when used in this the scope of the present invention, and be protected by the speci?cation are taken to specify the presence of stated fea accompanying claims. tures, steps, operations, elements, and/or components, but do US 8,700,765 B2 5 6 not preclude the presence or addition of one or more other contain, store, or transport the program for use by or in con features, steps, operations, elements, components, and/or nection with the instruction execution system, apparatus, or groups thereof. It will be understood that when an element is device. referred to as being “connected” or “coupled” to another The computer-usable or computer-readable medium may element, it can be directly connected or coupled to the other be, for example but not limited to, an electronic, magnetic, element or intervening elements may be present. Further optical, electromagnetic, infrared, or semiconductor system, more, “connected” or “coupled” as used herein may include apparatus, or device. More speci?c examples (a non-exhaus wirelessly connected or coupled. As used herein, the term tive list) of the computer-readable medium would include the “and/or” includes any and all combinations of one or more of following: a portable computer diskette, a random access the associated listed items, and may be abbreviated as “/”. memory (RAM), a read-only memory (ROM), an erasable Unless otherwise de?ned, all terms (including technical programmable read-only memory (EPROM or Flash memory), and a portable compact disc read-only memory and scienti?c terms) used herein have the same meaning as (CD-ROM). commonly understood by one of ordinary skill in the art. It Computer program code for carrying out operations of data will be further understood that terms, such as those de?ned in processing systems discussed herein may be written in a commonly used dictionaries, should be interpreted as having high-level programming language, such as C, C++, or Java, a meaning that is consistent with their meaning in the context for development convenience. In addition, computer program of the relevant art, and will not be interpreted in an idealized code for carrying out operations of exemplary embodiments or overly formal sense unless expressly so de?ned herein. may also be written in other programming languages, such as, It will be understood that, although the terms ?rst, second, 20 but not limited to, interpreted languages. Some modules or etc. may be used herein to describe various elements, these routines may be written in assembly language or even micro elements should not be limited by these terms. These terms code to enhance performance and/or memory usage. How are only used to distinguish one element from another. ever, embodiments are not limited to a particular program Exemplary embodiments are described below with refer ming language. It will be further appreciated that the ence to block diagrams and/or ?owchart illustrations of meth 25 functionality of any or all of the program modules may also be ods, apparatus (systems and/or devices), and/or computer implemented using discrete hardware components, one or program products. It is understood that a block of the block more application speci?c integrated circuits (ASle), or a diagrams and/or ?owchart illustrations, and combinations of programmed digital signal processor or microcontroller. blocks in the block diagrams and/or ?owchart illustrations, It should also be noted that in some alternate implementa can be implemented by computer program instructions. 30 tions, the functions/ acts noted in the blocks may occur out of the order noted in the ?owcharts. For example, two blocks These computer program instructions may be provided to a shown in succession may in fact be executed substantially processor of a general purpose computer, special purpose concurrently or the blocks may sometimes be executed in the computer, and/ or other programmable data processing appa reverse order, depending upon the functionality/acts ratus to produce a machine, such that the instructions, which 35 involved. Moreover, the functionality of a given block of the execute via the processor of the computer and/or other pro ?owcharts and/ or block diagrams may be separated into mul grammable data processing apparatus, create means (func tiple blocks and/or the functionality of two or more blocks of tionality) and/or structure for implementing the functions/ the ?owcharts and/ or block diagrams may be at least partially acts speci?ed in the block diagrams and/or ?owchart block or integrated. blocks. 40 Reference is made to FIGS. la-ld, which are block dia These computer program instructions may also be stored in grams illustrating exemplary networks in which operations a computer-readable memory that can direct a computer or for monitoring and reporting network application perfor other programmable data processing apparatus to function in mance may be performed according to some embodiments of a particular manner, such that the instructions stored in the the present invention. computer-readable memory produce an article of manufac 45 Computing Network ture including instructions which implement the functions/ Referring to FIG. 1a, a network 10 according to some acts speci?ed in the block diagrams and/or ?owchart block or embodiments herein may include a health data processing blocks. application 100 and a plurality of network devices 20, 24, and The computer program instructions may also be loaded 26 that may each include respective collector applications onto a computer or other programmable data processing 50 200. It is to be understood that a “network device” as dis apparatus to cause a series of operational steps to be per cussed herein may include physical (as opposed to virtual) formed on the computer or other programmable apparatus to machines 20; host machines 24, each of which may be a produce a computer-implemented process, such that the physical machine on which one or more virtual machines may instructions, which execute on the computer or other pro execute; and/ or virtual machines 26 executing on host grammable apparatus, provide steps for implementing the 55 machines 24. It is to be further understood that an “applica functions/acts speci?ed in the block diagrams and/or ?ow tion” as discussed herein refers to an instance of executable chart block or blocks. software operable to execute on respective ones of the net Accordingly, exemplary embodiments may be imple work devices. The terms “application” and “network appli mented in hardware and/or in software (including ?rmware, cation” may be used interchangeably herein, regardless of resident software, micro-code, etc.). Furthermore, exemplary 60 whether the referenced application is operable to access net embodiments may take the form of a computer program prod work resources. uct on a non-transitory computer-usable or computer-read Collector applications 200 may collect data related to the able storage medium having computer-usable or computer performance of network applications executing on respective readable program code embodied in the medium for use by or network devices. For instance, a collector application execut in connection with an instruction execution system. In the 65 ing on a physical machine may collect performance data context of this document, a non-transitory computer-usable related to network applications executing on that physical or computer-readable medium may be any medium that can machine. A collector application executing on a ho st machine US 8,700,765 B2 7 8 and external to any virtual machines hosted by that host input/output devices 13011-13019 (generally referred to using machine may collect performance data related to network reference numeral 130), and a cache memory 140 in commu applications executing on that host machine, while a collector nication with the central processing unit 101. application executing on a virtual machine may collect per The central processing unit 101 is any logic circuitry that formance data related to network applications executing responds to and processes instructions fetched from the main within that virtual machine. memory unit 122. In many embodiments, the central process The health data processing application 100 may be on a ing unit 101 is provided by a microprocessor unit, such as: network device that exists within the network 10 or on an those manufactured by Intel Corporation of Mountain View, external device that is coupled to the network 10. Accord Calif.; those manufactured by Motorola Corporation of ingly, in some embodiments, the network device on which the Schaumburg, Ill.; the POWER processor, those manufactured health data processing application 100 may reside may be one by International Business Machines of White Plains, N.Y.; of the plurality of machines 20 or 24 or virtual machines 26. and/or those manufactured by Advanced Micro Devices of Communications between various ones of the network Sunnyvale, Calif. The computing device 121 may be based on devices may be accomplished using one or more communi any of these processors, and/or any other processor capable of cations and/or network protocols that may provide a set of operating as described herein. standard rules for data representation, signaling, authentica Main memory unit 122 may be one or more memory chips tion and/ or error detection that may be used to send informa capable of storing data and allowing any storage location to tion over communications channels therebetween. In some be directly accessed by the microprocessor 101, such as Static embodiments, exemplary network protocols may include random access memory (SRAM), Burst SRAM or Synch HTTP, TDS, and/or LDAP, among others. 20 Burst SRAM (BSRAM), Dynamic random access memory Referring to FIG. 1b, an exemplary network 10 may (DRAM), Fast Page Mode DRAM (FPM DRAM), Enhanced include a web server 12, one or more application servers 14 DRAM (EDRAM), Extended Data Output RAM (EDO and one or more database servers 16.Although not illustrated, RAM), Extended Data Output DRAM (EDO DRAM), Burst a network 10 as used herein may include directory servers, Extended Data Output DRAM (BEDO DRAM), Enhanced security servers, and/ or transaction monitors, among others. 25 DRAM (EDRAM), synchronous DRAM (SDRAM), JEDEC The web server 12 may be a computer and/or a computer SRAM, PC100 SDRAM, Double Data Rate SDRAM (DDR program that is responsible for accepting HTTP requests SDRAM), Enhanced SDRAM (ESDRAM), SyncLink from clients 18 (e.g., user agents such as web browsers) and DRAM (SLDRAM), Direct Rambus DRAM (DRDRAM), or serving them HTTP responses along with optional data con Ferroelectric RAM (FRAM), among others. The main tent, which may be, for example, web pages such as HTML 30 memory 122 may be based on any of the above described documents and linked objects (images, etc.). An application memory chips, or any other available memory chips capable server 14 may include a service, hardware, and/or software of operating as described herein. In some embodiments, the framework that may be operable to provide one or more processor 101 communicates with main memory 122 via a programming applications to clients in a network. Applica system bus 150 (described in more detail below). In some tion servers 14 may be coupled to one or more web servers 12, 35 embodiments of a computing device 121, the processor 101 database servers 16, and/ or other application servers 14, may communicate directly with main memory 122 via a among others. Some embodiments provide that a database memory port 103. Some embodiments provide that the main server 16 may include a computer and/ or a computer program memory 122 may be DRDRAM. that provides database services to other computer programs FIG. 1d depicts some embodiments in which the main and/ or computers as may be de?ned, for example by a client 40 processor 101 communicates directly with cache memory server model, among others. In some embodiments, database 140 via a secondary bus, sometimes referred to as a backside management systems may provide database server function bus. In some other embodiments, the main processor 101 may ality. communicate with cache memory 140 using the system bus Some embodiments provide that the collector applications 150. Cache memory 140 typically has a faster response time 200 and the health data processing application 100 described 45 than main memory 122 and may be typically provided by above with respect to FIG. 111 may reside on ones of the web SRAM, BSRAM, or EDRAM. In some embodiments, the server(s) 12, application servers 14 and/or database servers processor 101 communicates with various I/O devices 130 16, among others. In some embodiments, the health data via a local system bus 150. Various busses may be used to processing application 100 may reside in a dedicated com connect the central processing unit 101 to any of the I/O puting device that is coupled to the network 10. The collector 50 devices 130, including aVESA VL bus, an ISA bus, an EISA applications 200 may reside on one, some or all of the above bus, a MicroChannel Architecture (MCA) bus, a PCI bus, a listed network devices and provide network application per PCI-X bus, a PCI-Express bus, and/or a NuBus, among oth formance data to the health data processing application 100. ers. For embodiments in which the I/O device is a video Computing Device display 124, the processor 101 may use an Advanced Graph Web server(s) 12, application servers 14 and/or database 55 ics Port (AGP) to communicate with the display 124. FIG. 1d servers 16 may be deployed as and/or executed on any type depicts some embodiments of a computer 100 in which the and form of computing device, such as a computer, network main processor 101 communicates directly with I/O device device, or appliance capable of communicating on any type 130 via HyperTransport, Rapid I/O, or In?niBand. FIG. 1d and form of network and performing the operations described also depicts some embodiments in which local busses and herein. FIGS. lc and 1d depict block diagrams of a computing 60 direct communication are mixed: the processor 101 commu device 121 useful for practicing some embodiments nicates with I/O device 130a using a local interconnect bus described herein. Referring to FIGS. lc and 1d, a computing while communicating with I/O device 1301) directly. device 121 may include a central processing unit 101 and a The computing device 121 may support any suitable instal main memory unit 122. A computing device 100 may include lation device 116, such as a ?oppy disk drive for receiving a visual display device 124, a keyboard 126, and/ or a pointing 65 ?oppy disks such as 3.5-inch, 5.25-inch disks, or ZIP disks, a device 127, such as a mouse. Each computing device 121 may CD-ROM drive, a CD-R/RW drive, a DVD-ROM drive, tape also include additional optional elements, such as one or more drives of various formats, USB device, hard disk drive US 8,700,765 B2 9 10 (HDD), solid-state drive (SSD), or any other device suitable device 12411 for the computing device 121. One ordinarily for installing software and programs such as any client agent skilled in the art will recognize and appreciate the various 120, or portion thereof. The computing device 121 may fur ways and embodiments that a computing device 121 may be ther comprise a storage device 128, such as one or more hard con?gured to have multiple display devices 12411-12411. disk drives or solid-state drives or redundant arrays of inde In further embodiments, an I/O device 130 may be a bridge pendent disks, for storing an operating system and other 170 between the system bus 150 and an external communi related software, and for storing application software pro cation bus, such as a USB bus, an Apple Desktop Bus, an grams such as any program related to the client agent 120. RS-232 serial connection, a SCSI bus, a FireWire bus, a Optionally, any of the installation devices 116 could also be FireWire 800 bus, an Ethernet bus, an AppleTalk bus, a Giga used as the storage device 128. Additionally, the operating bit Ethernet bus, an Asynchronous Transfer Mode bus, a system and the software can be run from a bootable medium, HIPPI bus, a Super HIPPI bus, a SerialPlus bus, a SCI/LAMP for example, a bootable CD, such as KNOPPIX®, a bootable bus, a FibreChannel bus, and/or a Serial Attached small com CD for GNU/Linux that is available as a GNU/Linux distri puter system interface bus, among others. bution from knoppix.net. A computing device 121 of the sort depicted in FIGS. lc Furthermore, the computing device 121 may include a and 1d may typically operate under the control of operating network interface 118 to interface to a Local Area Network systems, which control scheduling of tasks and access to (LAN), Wide Area Network (WAN) or the Internet through a system resources. The computing device 121 can be running variety of connections including, but not limited to, standard any operating system such as any of the versions of the telephone lines, LAN or WAN links (e.g., T1, T3, 56 kb, Microsoft® Windows operating systems, any of the different X.25), broadband connections (e.g., ISDN, Frame Relay, 20 releases of the Unix and Linux operating systems, any version ATM), wireless connections (e.g., IEEE 802.11), or some of the Mac OS® for Macintosh computers, any embedded combination of any or all of the above. The network interface operating system, any real-time operating system, any open 118 may comprise a built-in network adapter, network inter source operating system, any proprietary operating system, face card, PCMCIA network card, card bus network adapter, any operating systems for mobile computing devices, and/or wireless network adapter, USB network adapter, modem, or 25 any other operating system capable of running on a comput any other device suitable for interfacing the computing device ing device and performing the operations described herein. 121 to any type of network capable of communication and Typical operating systems include: WINDOWS 3.x, WIN performing the operations described herein. A wide variety of DOWS 95, WINDOWS 98, WINDOWS 2000, WINDOWS I/O devices 13011-13011 may be present in the computing NT 3.51, WINDOWS NT 4.0, WINDOWS CE, WINDOWS device 121. Input devices include keyboards, mice, track 30 XP, WINDOWS VISTA, WINDOWS 7.0, WINDOWS pads, trackballs, microphones, and drawing tablets, among SERVER 2003, and/or WINDOWS SERVER 2008, all of others. Output devices include video displays, speakers, ink which are manufactured by Microsoft Corporation of Red jet printers, laser printers, and dye-sublimation printers, mond, Wash.; MacOS, manufactured by Apple Computer of among others. The I/O devices 130 may be controlled by an Cupertino, Calif.; OS/2, manufactured by International Busi I/O controller 123 as shown in FIG. lc. The I/O controller 35 ness Machines of Armonk, N.Y.; and Linux, a freely-avail may control one or more I/O devices such as a keyboard 126 able operating system distributed by Red Hat of Raleigh, and a pointing device 127, e.g., a mouse or optical pen. NC, among others, or any type and/or form of a Unix oper Furthermore, an I/O device may also provide storage 128 ating system, among others. and/ or an installation medium 116 for the computing device In some embodiments, the computing device 121 may have 100. In still other embodiments, the computing device 121 40 different processors, operating systems, and input devices may provide USB connections to receive handheld USB stor consistent with the device. For example, in one embodiment age devices such USB ?ash drives. the computing device 121 is a Treo 180,270, 1060, 600 or 650 In some embodiments, the computing device 121 may smart phone manufactured by Palm, Inc. In this embodiment, comprise or be connected to multiple display devices 124a the Treo smart phone is operated under the control of the 124n, which each may be of the same or different type and/or 45 PalmOS operating system and includes a stylus input device form. As such, any of the I/O devices 13011-13011 and/or the as well as a ?ve-way navigator device. Moreover, the com I/O controller 123 may comprise any type and/or form of puting device 121 can be any workstation, desktop computer, suitable hardware, software, or combination of hardware and laptop, or notebook computer, server, handheld computer, software to support, enable, or provide for the connection and mobile telephone, any other computer, or other form of com use of multiple display devices 12411-12411 by the computing 50 puting or telecommunications device that is capable of com device 121. For example, the computing device 121 may munication and that has su?icient processor power and include any type and/or form of video adapter, video card, memory capacity to perform the operations described herein. driver, and/or library to interface, communicate, connect or Architecture otherwise use the display devices 12411-12411. In some Reference is now made to FIG. 2, which is a block diagram embodiments, a video adapter may comprise multiple con 55 illustrating an architecture of a computing device 121 as nectors to interface to multiple display devices 12411-12411. In discussed above regarding FIGS. lc and 1d. The architecture some other embodiments, the computing device 121 may of the computing device 121 is provided by way of illustration include multiple video adapters, with each video adapter con only and is not intended to be limiting. The architecture of nected to one or more of the display devices 12411-12411. In computing device 121 may include a hardware layer 206 and some embodiments, any portion of the operating system of 60 a software layer divided into a user space 202 and a kernel the computing device 100 may be con?gured for using mul space 204. tiple displays 12411-12411. In some embodiments, one or more Hardware layer 206 may provide the hardware elements of the display devices 12411-12411 may be provided by one or upon which programs and services within kernel space 204 more other computing devices connected to the computing and user space 202 are executed. Hardware layer 206 also device 121, for example, via a network. Such embodiments 65 provides the structures and elements that allow programs and may include any type of software designed and constructed to services within kernel space 204 and user space 202 to com use another computer’s display device as a second display municate data both internally and externally with respect to US 8,700,765 B2 11 12 computing device 121. The hardware layer 206 may include In some embodiments, any portion of the components 240 a processing unit 262 for executing software programs and and 236 may run or operate in the kernel space 204, while services, a memory 264 for storing software and data, and other portions of these components 240 and 236 may run or network ports 266 for transmitting and receiving data over a operate in user space 202. In some embodiments, the com network. Additionally, the hardware layer 206 may include puting device 121 uses a kemel-level data structure providing multiple processors for the processing unit 262. For example, access to any portion of one or more network packets, for in some embodiments, the computing device 121 may include example, a network packet comprising a request from a client a ?rst processor 262 and a second processor 262'. In some or a response from a server. In some embodiments, the kemel embodiments, the processor 262 or 262' includes a multi-core level data structure may be obtained by the packet engine 240 processor. The processor 262 may include any of the proces via a transport layer driver interface (TDI) or ?lter to the sors 101 described above in connection with FIGS. lc and 1d. network stack 267. The kemel-level data structure may Although the hardware layer 206 of computing device 121 include any interface and/or data accessible via the kernel is illustrated with certain elements in FIG. 2, the hardware space 204 related to the network stack 267, network traf?c, or portions or components of computing device 121 may packets received or transmitted by the network stack 267. In include any type and form of elements, hardware or software, some embodiments, the kernel-level data structure may be of a computing device, such as the computing device 121 used by any of the components or processes 240 and 236 to illustrated and discussed herein in conjunction with FIGS. lc perform the desired operation of the component or process. and Id. In some embodiments, the computing device 121 may Some embodiments provide that a component 240 and 236 is comprise a server, gateway, router, switch, bridge, or other running in kernel mode 204 when using the kemel-level data type of computing or network device, and have any hardware 20 structure, while in some other embodiments, the component and/ or software elements associated therewith. 240 and 236 is running in user mode when using the kernel The operating system of computing device 121 allocates, level data structure. In some embodiments, the kernel-level manages, or otherwise segregates the available system data structure may be copied or passed to a second kemel memory into kernel space 204 and user space 202. As dis level data structure, or any desired user-level data structure. cussed above, in the exemplary software architecture, the 25 A policy engine 236 may include, for example, an intelli operating system may be any type and/or form of various ones gent statistical engine or other programmable application(s). of different operating systems capable of running on the In some embodiments, the policy engine 236 provides a con computing device 121 and performing the operations ?guration mechanism to allow a user to identify, specify, described herein. de?ne or con?gure a caching policy. Policy engine 236, in The kernel space 204 may be reserved for running the 30 some embodiments, also has access to memory to support kernel 230, including any device drivers, kernel extensions, data structures such as lookup tables or hash tables to enable and/ or other kernel related software. As known to those user-selected caching policy decisions. In some embodi skilled in the art, the kernel 230 is the core of the operating ments, the policy engine 236 may include any logic, rules, system, and provides access, control, and management of functions or operations to determine and provide access, con resources and hardware-related elements of the applications. 35 trol and management of objects, data or content being cached In accordance with some embodiments of the computing by the computing device 121 in addition to access, control and device 121, the kernel space 204 also includes a number of management of security, network traf?c, network access, network services or processes working in conjunction with a compression, and/ or any other function or operation per cache manager sometimes also referred to as the integrated formed by the computing device 121. cache. Additionally, some embodiments of the kernel 230 40 High speed layer 2-7 integrated packet engine 240, also will depend on embodiments of the operating system generally referred to as a packet processing engine or packet installed, con?gured, or otherwise used by the device 121. engine, is responsible for managing the kemel-level process In some embodiments, the device 121 includes one net ing of packets received and transmitted by computing device work stack 267, such as a TCP/IP based stack, for communi 121 via network ports 266. The high speed layer 2-7 inte cating with a client and/ or a server. In other embodiments, the 45 grated packet engine 240 may include a buffer for queuing device 121 may include multiple network stacks. In some one or more network packets during processing, such as for embodiments, the network stack 267 includes a buffer 243 for receipt of a network packet or transmission of a network queuing one or more network packets for transmission by the packer. Additionally, the high speed layer 2-7 integrated computing device 121. packet engine 240 is in communication with one or more As shown in FIG. 2, the kernel space 204 includes a high 50 network stacks 267 to send and receive network packets via speed layer 2-7 integrated packet engine 240 and a policy network ports 266. The high speed layer 2-7 integrated packet engine 236. Running these components or processes 240 and engine 240 may work in conjunction with policy engine 236. 236 in kernel space 204 or kernel mode instead of the user In particular, policy engine 236 is con?gured to perform space 202 improves the performance of each of these com functions related to tra?ic management such as request-level ponents, alone and in combination. Kernel operation means 55 content switching and request-level cache redirection. that these components or processes 240 and 236 run in the The high speed layer 2-7 integrated packet engine 240 core address space of the operating system of the device 121. includes a packet processing timer 242. In some embodi For example, data obtained in kernel mode may not need to be ments, the packet processing timer 242 provides one or more passed or copied to a process or thread running in user mode, time intervals to trigger the processing of incoming (i.e., such as from a kernel level data structure to a user level data 60 received) or outgoing (i.e., transmitted) network packets. In structure. In this regard, such data may be dif?cult to deter some embodiments, the high speed layer 2-7 integrated mine for purposes of network application performance moni packet engine 240 processes network packets responsive to toring. In another aspect, the number of context switches the timer 242. The packet processing timer 242 provides any between kernel mode and user mode are also reduced. Addi type and form of signal to the packet engine 240 to notify, tionally, synchronization of and communications between 65 trigger, or communicate a time related event, interval or any of the components or processes 240 and 236 can be occurrence. In many embodiments, the packet processing performed more e?iciently in the kernel space 204. timer 242 operates in the order of milliseconds, such as for US 8,700,765 B2 13 14 example 100 ms, 50 ms, or 25 ms. For example, in some intercept read operations and the time of their duration. Some embodiments, the packet processing timer 242 provides time operating systems may include a kernel mode driver other intervals or otherwise causes a network packet to be pro than the AFD. In this regard, operations described herein may cessed by the high speed layer 2-7 integrated packet engine be used with other such kernel mode drivers to intercept 240 at a 10 ms time interval, while in other embodiments, at application operational data. a 5 ms time interval, and still yet in further embodiments, as The raw data related to the occurrence of and attributes of short as a 3, 2, or 1 ms time interval. The high speed layer 2-7 transactions between network applications may be generally integrated packet engine 240 may be interfaced, integrated referred to as performance data. The raw data may have value and/or in communication with the policy engine 236 during for diagnosing network application performance issues and/ operation. As such, any of the logic, functions, or operations or for identifying and understanding the structure of the net of the policy engine 236 may be performed responsive to the work applications. The measurements or aggregations of per packet processing timer 242 and/or the packet engine 240. formance data may be generally referred to as metrics. Therefore, any of the logic, functions, and/or operations of Performance data and the metrics generated therefrom may the policy engine 236 may be performed at the granularity of be temporally relevantii.e., the performance data and the time intervals provided via the packet processing timer 242, metrics may be directly related to and/or indicative of the for example, at a time interval of less than or equal to 10 ms. health of the network at the time the performance data is In contrast to kernel space 204, user space 202 is the collected. Performance data may be collected, and metrics memory area or portion of the operating system used by user based thereon may be generated, on a client side and/or a mode applications or programs otherwise running in user server side of an interaction. Some embodiments provide that mode. Generally, a user mode application may not access 20 performance data is collected in substantially real-time. In kernel space 204 directly, and instead must use service calls in this context, “substantially real-time” means that perfor order to access kernel services. As shown in FIG. 2, user space mance data is collected immediately subsequent to the occur 202 of computing device 121 includes a graphical user inter rence of the related network activity, subject to the delays face (GUI) 210, a command line interface (CLI) 212, shell inherent in the operation of the computing device and/ or the services 214, and daemon services 218. Using GUI 210 and/ 25 network and in the method of collection. The performance or CLI 212, a system administrator or other user may interact data collected and/ or the metrics generated may correspond with and control the operation of computing device 121. The to a prede?ned time interval. For example, a time interval may GUI 210 may be any type and form of graphical user interface be de?ned according to the dynamics of the network and may and may be presented via text, graphical or otherwise, by any include exemplary period lengths of less than 1, l, 5, 10, 15, type of program or application, such as a browser. The CLI 30 20, 30, and/or 60, seconds, among others. 212 may be any type and form of command line or text-based Exemplary client side metrics may be aggregated accord interface, such as a command line provided by the operating ing to one or more applications or processes. For example, the system. For example, the CLI 212 may comprise a shell, client side metrics may be aggregated according to destina which is a tool to enable users to interact with the operating tion address, port number, and a local process identi?er (PID). system. In some embodiments, the CLI 212 may be provided 35 A PID may be a number used by some operating system via a bash, csh, tcsh, and/ or ksh type shell. The shell services kernels to uniquely identify a process. This number may be 214 may include the programs, services, tasks, processes used as a parameter in various function calls allowing pro and/ or executable instructions to support interaction with the cesses to be manipulated, such as adjusting the process’s computing device 121 or operating system by a user via the priority and/or terminating the process. In this manner, mul GUI 210 and/or CLI 212. 40 tiple connections from the same application or process to the Daemon services 218 are programs that run continuously same remote service may be aggregated. or in the background and handle periodic service requests Similarly, server side metrics may be aggregated according received by computing device 121. In some embodiments, a to the same application or service regardless of the client. For daemon service may forward the requests to other programs example, some embodiments provide that server side metrics or processes, such as another daemon service 218 as appro 45 may be aggregated according to local address, port number, priate. As known to those skilled in the art, a daemon service and PID. Respective ones of the client side and server side 218 may run unattended to perform continuous and/or peri metrics may be collected from the kernel space and/or user odic system wide functions, such as network control, or to space. perform any desired task. In some embodiments, one or more The kernel space module 310 may include a kernel events daemon services 218 run in the user space 202, while in other 50 sender 316 that is con?gured to receive performance data embodiments, one or more daemon services 218 run in the from the AFD ?lter 312 and/or the TDI ?lter 314, and gener kernel space. ate metrics based on the performance data for receipt by a Collector Application kernel events receiver 322 in the user space module 320. In Reference is now made to FIG. 3, which is a block diagram the user space module 320, metrics data received by the illustrating operations and/or functions of a collector appli 55 kernel event receiver 322 may be processed by a reverse cation 200 as described above regarding FIG. 1a. The collec domain name system (DNS) resolver 325 to map an observed tor application 200 includes a kernel space module 310 and a network address to a more user-friendly DNS name. Addi user space module 320. The kernel space module 310 may tionally, metrics data received by the kernel events receiver generally operate to intercept network activities as they occur. 322 may be used by a process resolver 326 to determine the Some embodiments provide that the kernel space module 3 1 0 60 processes and/ or applications corresponding to the collected may use a kernel mode interface in the operating system, such kernel metrics data. as, for example, Microsoft Windows transport data interface The user space module 320 may include a machine infor (TDI). The kernel space module 310 may include a TDI ?lter mation collector 324 that is operable to determine static 314 that is con?gured to monitor and/ or intercept interactions machine information, such as, for example, CPU speed, between applications. Additionally, some embodiments pro 65 memory capacity, and/or operating system version, among vide that the kernel space module 310 may include an ancil others. As the performance data is collected corresponding to lary functions driver (AFD) ?lter 312 that is con?gured to applications and/or processes, the machine information may