USOO7444347B1
(12) United States Patent (10) Patent No.: US 7.444,347 B1 Alley et al. (45) Date of Patent: Oct. 28, 2008
(54) SYSTEMS, METHODS AND COMPUTER 2005/0261891 A1* 11/2005 Chan et al...... TO4/9 PRODUCTS FOR COMPRESSION OF 2007/004401.0 A1* 2, 2007 Sull et al...... 715,500.1 HERARCHICAL IDENTIFIERS 2007/0128899 A1* 6/2007 Mayer ...... 439,152 (75) Inventors: Charles L. Alley, Durham, NC (US); FOREIGN PATENT DOCUMENTS Anthony J Bybell, Carrboro, NC (US); WO WOO135216 5, 2001 Jason M. Sullivan, Raleigh, NC (US) Naming Issues in the Design of Transparently Distributed Operating (73) Assignee: International Business Machines Systems, 1987, Robert J. Stroud. (Item 1 from file 35) 1010738 Corporation, Armonk, NY (US) Order No. AADDX-80202)) online; retrieved on Oct. 26, 2007: retrieved from the Internet
2004/011774.0 A1* 6, 2004 Chen et al...... 715,531 by the prefix pointer and concatenated onto the string pointed 2005. O157217 A1* 7, 2005 Hendricks ...... 348,734 to by the name pointer; and thus a full name is created. 2005/0198070 A1* 9/2005 Lowry ...... TO7 104.1 2005/0216421 A1* 9/2005 Barry et al...... TO5/64 4 Claims, 2 Drawing Sheets COMPRESSION OFHER-ARCICALSyd IDENTIFEERSMETHOD 70 C0MPRESSNAMESBYPERFOR&N8 SUBOPERATIONS -
SORTINGNAMESINLEXICAL, OREERSUBERATION - SPLETTING FULLNAMESNTO A FERARCHY STAND 74 SPLITTING FULL NAMES INTO AREMANDERSTsuB Y OPERATIONS - - - CQMRESSINGPREFEXES OF NAMESNERARCYLis8tje PERATION
COMPRESSN3PREFXES OF NAMES INNAMELIST RUNNING OUTPUT OF COMPRESSE pREEixes of NAMESN HERARCHYLISTAND RUNNRNG dUTPUT OF COMPRESSED PREFXES OF NAMESNNAMELISTSUBOPERATIONS
DECOMPRESSINGNAMESSYPERFORMINGSL3 OpERATION3 RNNDNG COMPRESSEDHERARCELISTANINAMES THROUGHECOMPRESSRSOFTWAREACKAE
READINGWAUEATSTART OF AN ENTRY COPYNg QUANY OF Byres FROx1FRECEDG ENTRY
APPENING FOLLOWINGSTRING SETTING ONER STEPPING THROUGHNAME ENTRIES CREATINKstriNg PoNTED TO ANDREATING Full NAME - 8s 38 - U.S. Patent Oct. 28, 2008 Sheet 1 of 2 US 7.444,347 B1
COMPRESSION OF HIERARCHICAL IDENTIFIERS METHOD 70
1. 72 COMPRESS NAMES BY PERFORMING SUB OPERATIONS 1. 73 SORTING NAMES IN LEXICAE ORDER SUB OPERATION 1.
SPLITTING FULL NAMES INTO A HIERARCHY LIST AND 74 SPLITTING FULL NAMES INTO AREMAINDER LIST SUB 1. OPERATIONS
COMPRESSING PREFIXES OF NAMES IN HIERARCHY LIST SUB 1. 75 OPERATION 76 COMPRESSING PREFIXES OF NAMES IN NAME LIST l1
RUNNING OUTPUT OF COMPRESSED PREFIXES OF NAMES IN 77 HIERARCHY LIST AND RUNNING OUTPUT OF COMPRESSED 1. PREFIXES OF NAMES IN NAME LIST SUB OPERATIONS 78 DECOMPRESSING NAMES BY PERFORMING SUB OPERATIONS 1.
RUNNING COMPRESSED HIERARCH LIST AND NAME LIST 1. 79 THROUGH DECOMPRESSOR SOFTWARE PACKAGE
80 READING VALUE AT START OF AN ENTRY 1. COPYING OUANTY OF BYTES FROM PRECEDING ENTRY - 81 82 APPENDING FOLLOWING STRING -1
SETTING POINTER - 83
STEPPING THROUGH NAME ENTRIES a-a- 84
CREATING STRING POINTED TO AND CREATING FULL NAME 85
86 U.S. Patent Oct. 28, 2008 Sheet 2 of 2 US 7.444,347 B1
COMPRESSION OF HIERARCHICAL IDENTIFIERS SYSTEM 20 COMPUTER N. WORKSTATION PROCESSOR 22
HERARCHICAL LIST 23 REPOSITORY 26 MEMORY 24
N
nth NAME LIST Lin
PREFIX LIST LA
NAME LIST L3 PROGRAM UNIT 40
REMAINDER LIST 2 PROGRAM 41 HERARCHY LIST LI
MEMORY CONTROLLER Algorithm
An ALGORTHM UNIT 30 1. 27
NETWORK INTERFACE 28 OUTPUT DEVICE 34 MOUSE 29
KEYBOARD 60
PLURALITY OF NETWORKED COMPUTER DEVICES 51
FIG. 2 US 7,444,347 B1 1. 2 SYSTEMS, METHODS AND COMPUTER string starting at the String beginning at +k bytes. Compress PRODUCTS FOR COMPRESSION OF ing prefixes of names in the name list is performed by running HERARCHICAL IDENTIFIERS an output of compressing of prefixes of names in the hierar chy list and running an output of the compressing of prefixes TRADEMARKS of names in the name list through a standard compressor Software application package, where the standard compressor IBM(R) is a registered trademark of the International Busi Software application package includes bzip2, which produces ness Machines Corporation, Armonk, N.Y., U.S.A. Other a compressed hierarchy list in conjunction with operations names used herein may be trademarks and registered trade compressing prefixes of names in a name list. In the exem marks, or trade or product names of International Business 10 plary embodiment, decompressing of names uses the inverse Machines Corporation or other companies. of the prefix compressor algorithm and is performed by run ning the compressed hierarchy list and the name list through TECHNICAL FIELD a standard decompressor Software application package, and where the standard decompressor Software application pack This invention relates generally to networked computer 15 age includes bzip2, then the Sub operations of reading a value system administration and management, software services, at a start of an entry; copying a quantity of bytes from a application solutions and data replication management preceding entry; and appending a string that follows; then directed to data handling programs. Such as event monitoring setting a pointer at a beginning of the prefix list and setting a in Software management applications, data mining, pattern pointer at a beginning of the name list. This operation is recognition, data analysis, data transcoding and data conver followed by stepping through the name entries one by one sion operating in various operating system environments until no names exist, when the flag emitted when a full name’s including UNIX operating systems, as well as implementa hierarchy changes from the preceding name is encountered, tions in hardware directed to keeping memory Small. More then this flag is used to indicate that the prefix pointer is particularly, this invention relates to storage management of moved forward by one entry for the next element in the hierarchical identifiers in simulation models and netlists, in 25 hierarchy list. These operations create a string pointed to by data compression and decompression applications. This the prefix pointer and concatenated on to the string pointed to invention can be applied to any system running any service by the name pointer, and thus a full name is created. provider application and/or operating system. BRIEF DESCRIPTION OF THE DRAWINGS BACKGROUND 30 The Subject matter that is regarded as the invention is Known general purpose algorithms. Such as ZipfgZip are particularly pointed out and distinctly claimed in the claims at used for reducing large storage requirements for large models the conclusion of the specification. The foregoing and other and netlists. In general, for large models and netlists, the objects, features, and advantages of the invention are apparent storage requirements are concomitantly large and the com 35 from the following detailed description taken in conjunction pression and decompression operations perform very slowly. with the accompanying drawings, which are meant to be Therefore, the need exists for methods and systems that exemplary, and not limiting, wherein: reduce the storage requirements and increase performance FIG. 1 illustrates operations of the method of storage man speeds for large models and netlists compression and decom agement of hierarchical identifiers. pression operations. In addition, the need exists for improving 40 FIG. 2 illustrates the system implementing operations of the compressibility of data. the method illustrated in FIG. 1. SUMMARY OF THE INVENTION DETAILED DESCRIPTION
A method of storage management of hierarchical identifi 45 The disclosed exemplary embodiments are intended to be ers in simulation models and netlists is accomplished using a illustrative only, since numerous modifications and variations prefix compressor algorithm running on a general purpose therein will be apparent to those of ordinary skill in the art. In computer processor executing computer executable program reference to the drawings, like numbers will indicate like code that causes the general purpose computer processor to parts continuously throughout the view. Further, the terms reduce storage requirements. Full names are split into a hier 50 “a”, “an”, “first”, “second' and “third herein do not denote archy part and a remainder part. The hierarchy data that is limitations of quantity, but rather denote the presence of one split off is prefix compressed then fed to a general purpose or more of the referenced item(s). computer processor compressor. In addition, the remainder A compression of hierarchical identifiers method 70 data that is split off is prefix compressed and also fed to a (herein referred to as “method 70') and a compression of general purpose computer processor compressor. Keeping 55 hierarchical identifiers system 20 (herein referred to as “sys the hierarchy and remainderparts separate improves the com tem 20') for storage management of hierarchical identifiers in pressibility of the data. The method includes compressing simulation models and netlists using a prefix compressor names by performing Sub operations of sorting names in algorithm running on a general purpose computer processor lexical order, then splitting full names into a hierarchy listand executing computer executable program code that causes the a remainder list, where any duplicate entries are not allowed 60 general purpose computer processor to reduce storage to be emitted to the hierarchy list. Next, when a full name's requirements are illustrated in FIGS. 1 and 2 respectively. hierarchy changes from a preceding name, a flag is emitted to Referring to FIG. 2, system 20 includes computer work a name list indicating that a next element in the hierarchy list station processor 22, which contains memory 24. Algorithm is to be used. Compressing prefixes of names in the hierarchy unit 30 resides in memory 24 and contains a plurality of list is performed by comparing a previous entry to a current 65 algorithms including first algorithm A31, second algorithm entry, where a value is emitted indicating the number of equal A32 and third algorithm A33 up to nth algorithm An. Also, bytes (k) at the beginning of a string and emitting a remaining residing in System 20 is program unit 40, containing program US 7,444,347 B1 3 4 41. Memory 24 also contains a hierarchical list repository 26, compression algorithm) causes system 20 to perform the which contains a plurality of repository entry locations R91, operation of compressing names by performing the following R92, R93, R94 and up to Rn, which hold and/or store a Sub operations: At Sub operation 73, program 41 executing plurality of hierarchy lists, remainder lists, name lists and method 70 causes system 20 to sort names in lexical order, prefix list L1, L2, L3 and L4 respectively up to nth NAME where sorting is done by looking at the lexical order of the first LIST Ln. character in each item, then the second character until each In the exemplary embodiment, system 20 includes a com character is reviewed. At sub operation 74, program 41 bination of controllers including display controller 23, executing method 70 causes system 20 to split full names into memory controller 25 and input/output (I/O) controller 27 a hierarchy list L1 and a remainder list L2, where duplicate and a combination of computer peripheral devices commu 10 entries are not emitted to the hierarchy list L1, and where a nicatively coupled to System 20 including display 21, a set of flag is emitted to a name list L3 indicating that a next element input devices including keyboard 60 and mouse 29, network in the hierarchy list L1 is used, when a full name's hierarchy interface 28, and output device 34, via standard interface changes from a preceding name. At Sub operation 75, pro connectivity. Network interface 28 cooperatively couples gram 41 executing method 70 causes system 20 to compress computer workstation processor 22 via network 50 (where 15 prefixes of names in the hierarchy list L1, where compressing network 50 can be a wide area network such as the Internet or prefixes is performed by comparing a previous entry to a a local area network, including an intranet or an extranet) to a current entry, where a value is emitted indicating the number plurality of networked computer devices 51. The plurality of of equal bytes (k) at the beginning of the String and emitting networked computer devices 51 includes a plurality of stor the remaining string starting at a string beginning at +k bytes. age devices represented as SD 1, SD 2, SD 3 up to SD n, At sub operation 76, program 41 calls third algorithm A33 associated with a plurality of sites, including site1, site 2, site (the prefix compressor algorithm) to compress prefixes of 3 up to site n, as illustrated in FIG. 2. In the exemplary names in the name list L3, where the name list L3 is accessed embodiment SD 1 is a networked storage device associated from the repository entry location R93 in the hierarchical list with one of the networked computer devices of the plurality of repository 26. At sub operation 77, program 41 calls another networked computer devices 51 and SD1 is physically and/or 25 algorithm from the plurality of algorithms in algorithm unit logically located and/or connected at site 1. SD 2 is another 30 and causes system 20 to run an output of compressing networked storage device associated with the plurality of prefixes of names in the hierarchy list L1 and run an output of networked computer devices 51 and SD 2 is located and/or compressing prefixes of names in the name list L2, where the logically connected at site 2: SD 3 is a third associated with another algorithm contains a standard compressor Software the plurality of networked computer devices 51 and SD 3 is 30 application package including bzip2, which produces a hier located and/or logically connected at site 3. SD n is an nth archy list L1 that is compressed and name list L3. storage device and is associated with site n. Referring to FIGS. 1 and 2, at operation 78, program 41, Referring to FIG. 2, display 21 displays indications of the executed by System 20, and calling the second algorithm A32 progress, state and/or status of storage management opera (the decompression algorithm, where the decompression tions of hierarchical identifiers in simulation models and 35 algorithm uses the inverse of third algorithm A33, and has a netlists. In the exemplary embodiment display 21 can display standard decompressor Software application package includ administration and management solutions directed to data ing bzip2, i.e., the prefix compressor algorithm) causes sys handling programs, including event monitoring in Software tem 20 to perform the operation of decompressing names, by management applications and data handling programs, performing the following sub operations: At Sub operation 79. directed to different site locations. 40 program 41, calling an other algorithm from the plurality of Referring to FIG. 1 and FIG. 2, at operation start 71 of algorithms residing in algorithm unit 30, causes system 20 to method 70, system 20 receives a signal from an operator/user run the hierarchy list L1 which is compressed in operation 72 input device including mouse 29 or keyboard 60, which acti and run the name list L3 from operation 76, through a stan vates and initiates program 41, where method 70 is stored as dard decompressor Software package. At Sub operation 80, executable program code on a computer executable medium. 45 program 41 calling an other algorithm from the plurality of Once activated, program 41 performs other selections in algorithms from algorithm unit 30 causing system 20 to read method 70 from selection signals received from keyboard 60 a value at a start of an entry. At Sub operation 81, program 41 or mouse 29, causing program 41 to be executed by computer causes system 20 to copy a quantity of bytes from a preceding workstation processor 22 to perform the operations of method entry. At Sub operation 82, program 41 causes system 20 to 70, thereby calling first algorithm A31, second algorithm 50 append a string that follows. At Sub operation 83, program 41 A32, a third algorithm A33 up to nth algorithm An to perform causes system 20 to set a pointerata beginning of a prefix list operations and Sub operations of method 70 of storage man L4 and further causes system 20 to set a pointerata beginning agement of hierarchical identifiers in simulation models and of the name list L2. At Sub operation 84, program 41 causes netlists using compression and decompression algorithms. In system 20 to step through the name entries one by one until no the exemplary embodiment, the first algorithm A31 is a com 55 names exist, wherein when the flag emitted (i.e., the flag for pression algorithm; the second algorithm A32 is a decom the name list L2 when the full name's hierarchy changes from pression algorithm; the third algorithm A33 is a prefix com the preceding name) is encountered, this flag indicates that pressor algorithm. Therefore, any number of compression/ the prefix pointer is moved forward by one entry for the next decompression solutions can be dynamically generated when element in the hierarchy list L1. At Sub operation 85, program operators/users want to view images of their data manage 60 41 causes system 20 to create a string pointed to by the prefix ment solution states. Thus, at start 71 the method is initiated pointer and concatenate onto the String pointed to by the name for storage management of hierarchical identifiers in simula pointer and create the full name. Thus, full name compression tion models and netlists using a general purpose computer and decompression is effected and can be provided over the processor including computer workstation processor 22 to network 50 to any of the storage devices SD1, SD2, SD3 up to reduce storage requirements. 65 SDn of the plurality of networked computer devices 51. In Referring to FIGS. 1 and 2, at operation 72, program 41, regard to memory access, for Some applications, keeping the executed by system 20, and calling first algorithm A31 (the hierarchies and names prefix compressed in memory may US 7,444,347 B1 5 6 prove advantageous for large models as the speed of decom What is claimed is: pression is offset by the delay incurred by translation misses 1. A method of storage management of hierarchical iden and page faults due to the Smaller size of the prefix com tifiers in simulation models and netlists using a general pur pressed data as compared to the expanded data. This depends pose computer processor executing computer executable pro gram code that causes the general purpose computer on the access pattern of the data. processor to reduce storage requirements, the method com At operation return/end 86, method 70 can be directed by prising: program 41 to return to any of the above operations and/or Sub compressing names by using an algorithm of a plurality of operations to continue iteratively processing and performing algorithms stored in an algorithm unit of the general said operations and Sub operations for a plurality of compres 10 purpose computer processor that when executed causes sion/decompression solutions, or program 41 can direct the general purpose computer processor to perform Sub method 70 to end. operations of The following Tables provide examples of compressability Sorting names in lexical order, using the above algorithms, operations and Sub operations of 1 splitting full names into a hierarchy list and a remainder 5 list, the exemplary embodiment of method 20: wherein the hierarchy list is stored in a hierarchical list repository entry location and the remainder list TABLE 1. is stored in a remainder list repository entry loca tion included in a memory of the general purpose proc netlist computer processor, for the following "+ bz2 indicates the size of that specific data if wherein duplicate entries are not emitted to the hier compressed directly with standard file compressor bzip2.: archy list, flattened name size: 114536886 bytes (+ bz2 = 2484974)number of names: 1376109 levels of hierarchy: wherein a flag is emitted to a name list indicating that 431771 a next element in the hierarchy list is used, prefix compressed: 7619002 (+ bz2 = 369098) 25 when a full name's hierarchy changes from a preced hier split: 34760857 (+ bz2 = 8001.45)name split: 90.91544 (+ bz2 = 322236) ing name, hier prefix compressed: 2320351 (+ bz2 = 127161)name prefix wherein the name list is stored in a name list reposi compressed: 6827041 (+ bz2 = 160695)(*) compressed to 287856 tory entry location, also included in the memory (397x, 0.21 bytes/name) and 30 wherein the name list repository entry location is one of a plurality of name list repository entry locations included in the memory; TABLE 2 compressing prefixes of names in the hierarchy list, 3 way processor RTL model wherein compressing prefixes is performed by com 35 paring a previous entry to a current entry, flattened name size: 20114828 (+ bz2 = 857610)number of names: 320457levels of hierarchy: 38978 wherein a value is emitted indicating the number of prefix compressed: 2832756 (+ bz2 = 349559) equal bytes (k) at the beginning of the string and hier split: 2609205 (+ bz2 = 90601)name split: 3906283 (+ bz2 = emitting the remaining string starting at a string begin 443489) hier prefix compressed: 265012 (+ bz2 = 18786)name prefix ning +k bytes; compressed: 2707349 (+ bz2 = 267731)(*) compressed to 2865.17 40 compressing prefixes of names in the name list; (70X, 0.89 bytes/name) running an output of compressing prefixes of names in the hierarchy list and running an output of compress ing prefixes of names in the name list through a com pressor Software application package, TABLE 3 45 which produces a compressed hierarchy list in con Trace logic RTL model junction with compressing prefixes of names in the name list; and flattened name size: 1381816 (+bz2 = 43269)number of names: decompressing names by performing Sub operations of 23181 levels of hierarchy: 3932 prefix compressed: 159007 (+ bz2 = 11536) running the compressed hierarchy list and the name list hier split: 273670 (+ bz2 = 6151)name split: 200315 (+ bz2 = 11474) 50 through a decompressor Software application pack hier prefix compressed: 32494 (+ bz2 = 2023)name prefix compressed: age. 136959 (+ bz2 = 7447)(*) compressed to 9470 (145x, 0.41 bytes/name) reading a value at a start of an entry; copying a quantity of bytes from a preceding entry; and While the disclosure has been described with reference to appending a string that follows: 55 setting a pointer at a beginning of the prefix list and exemplary embodiments, it will be understood by those setting a pointer at a beginning of the name list; skilled in the art that various changes may be made and stepping through the name entries one by one until no equivalents may be substituted for elements thereof without names exist, departing from the scope of the disclosure. In addition, many wherein when the flag emitted for the full name's modifications may be made to adapt a particular situation or 60 hierarchy changes from the preceding name of the material to the teachings of the disclosure without departing name list is encountered, from the essential scope thereof. Therefore, it is intended that then the next element in the hierarchy list is used and the disclosure not be limited to the particular exemplary the prefix pointer is moved forward by one entry; embodiment disclosed as the best mode contemplated for and carrying out this disclosure, but that the disclosure will 65 creating a string pointed to by the prefix pointer and include all embodiments falling within the scope of the concatenated on to the string pointed to by the name appended claims. pointer, and creating the full name. US 7,444,347 B1 7 8 2. The method according to claim 1, wherein, in large when a full name's hierarchy changes from a preced models, when the hierarchy list and the remainder list are ing name, stored in the hierarchical list repository included in the wherein the name list is stored in a name list reposi memory of the general purpose computer processor, speed of tory entry location, also included in the memory decompression is offset by delays incurred in translation 5 and misses and page faults due to a smaller size of prefix com wherein the name list repository entry location is one pressed data. of a plurality of name list repository entry locations 3. A system of storage management of hierarchical identi included in the memory; fiers in simulation models and netlists, the system compris compressing prefixes of names in the hierarchy list, ing: 10 wherein compressing prefixes is performed by com a computer workstation processor, paring a previous entry to a current entry, a combination of computer peripheral devices connected to wherein a value is emitted indicating the number of the computer workstation processor, equal bytes (k) at the beginning of the string and where the combination of computer peripheral devices emitting the remaining string starting at a string includes a display, a set of input devices including a 15 beginning +k bytes; keyboard and a mouse, an output device, and a net compressing prefixes of names in the name list; work interface, running an output of compressing prefixes of names in wherein the network interface is communicatively con the hierarchy list and running an output of compress nected to a network, ing prefixes of names in the name list through a com wherein the network is communicatively connected to a pressor Software application package, plurality of networked computer devices containing a which produces a compressed hierarchy list in con plurality of storage devices; junction with compressing prefixes of names in the a combination of controllers residing in the computer name list; and workstation processor, decompressing names by performing Sub operations of wherein the combination of controllers includes a dis 25 running the compressed hierarchy list and the name list play controller, a memory controller and an input/ through a decompressor Software application pack output controller; age. a memory, a program unit and an algorithm unit residing in reading a value at a start of an entry; the computer workstation processor, copying a quantity of bytes from a preceding entry; and wherein the memory contains a hierarchical list reposi 30 appending a string that follows: tory storing a plurality of lists in a plurality of reposi setting a pointer at a beginning of the prefix list and tory entry locations in the hierarchical list repository; setting a pointer at a beginning of the name list; the algorithm unit contains a plurality of algorithms and the stepping through the name entries one by one until no program unit contains a program that when executed by names exist, the computer workstation processor, causes the com 35 wherein when the flag emitted for the full name's hierarchy puter workstation processor to perform operations and changes from the preceding name of the name list is Sub operations of compressing names by performing encountered, Sub operations of: then the next element in the hierarchy list is used and the Sorting names in lexical order, prefix pointer is moved forward by one entry; and splitting full names into a hierarchy list and a remainder 40 creating a string pointed to by the prefix pointer and lists concatenated on to the string pointed to by the name wherein the hierarchy list is stored in a hierarchical pointer, and creating the full name. list repository entry location and the remainder list 4. The system according to claim 3, wherein, in large is stored in a remainder list repository entry loca models, when the hierarchy list and the remainder list are tion included in a memory of the general purpose 45 stored in the hierarchical list repository included in the computer processor, memory of the general purpose computer processor, speed of wherein duplicate entries are not emitted to the hier decompression is offset by delays incurred in translation archy list, misses and page faults due to a smaller size of prefix com wherein a flag is emitted to a name list indicating that pressed data. a next element in the hierarchy list is used,