US 20090248.614A1 (19) United States (12) Patent Application Publication (10) Pub. No.: US 2009/0248.614 A1 Bhagwan et al. (43) Pub. Date: Oct. 1, 2009
(54) SYSTEMAND METHOD FOR Related U.S. Application Data CONSTRUCTING TARGETED RANKING FROMMULTIPLE INFORMATION SOURCES (60) Provisional application No. 61/041,128, filed on Mar. 31, 2008. (75) Inventors: Varun Bhagwan, San Jose, CA O O (US); Tyrone Wilberforce Andre Publication Classification Grandison, San Jose, CA (US); (51) Int. Cl. Daniel Frederick Gruhl, San Jose, G06F 7/30 (2006.01) CA (US); Jan Hendrik Pieper, San Jose, CA (US) (52) U.S. Cl...... 707/1: 707/E17.005: 707/E17.134 Correspondence Address: (57) ABSTRACT VAN N. NGUY IBM CORPORATION ALMADEN RESEARCH Embodiments of the invention provide a system and method CENTER 9 for determining preferences from information mashups and, INTELLECTUAL PROPERTY LAW DEPT in particular, a system and method for constructing a ranked C4TAAU2B 650 HARRY ROAD list from multiple sources. In an exemplary embodiment, the SAN JOSE CA 95120-6099 (US) system and method tunably combines multiple ranked lists by 9 computing a score for each item within the list, wherein the (73) Assignee: INTERNATIONAL BUSINESS score is a function of the associated rank of the item within the MACHINES CORPORATION list. In one exemplary embodiment, the function is equal to Armonk, NY (US) s 1/(n(1/p)), where p is a tuning parameter that enables selec s tion between responsiveness in the combined ranking to one (21) Appl. No.: 12/195,098 candidate ranked highly in one source versus responsiveness in the combined ranking to a candidate with lower but broader (22) Filed: Aug. 20, 2008 Support among the various sources ranking the candidates.
1
Jerry Michael
Matthew Larry Larry Larry Larry Jerry Betty e
Jerry
Matthew
Narrow support Broad support Patent Application Publication Oct. 1, 2009 Sheet 1 of 5 US 2009/0248.614 A1
FIGURE 1A
3 Robert Timothy I Betty
6 William Sandra Kathleen
8 Richard Larry Pamela 9 Charles James Frank
FIGURE 1B
Je
Narrow support Broad support Patent Application Publication Oct. 1, 2009 Sheet 2 of 5 US 2009/0248.614 A1
i : : : X- & : in rank order) Patent Application Publication Oct. 1, 2009 Sheet 3 of 5 US 2009/0248.614 A1
302 FIGURE 3A FIGURE 3B FIGURE 3C score 101520253013540.4550 score 101520l25.303540.45150 score 10152025.30|35140|4550 il Rihanna i Eminen #1 Erninem Eminen Rihanna i2 The Beatles Simple Plan The Beatles #3 Rihanna 4. The Beatles Britney Spears #4 Lincoln Park Timbaland Simple Plan #5 Green Day Britney Spears iG Linkin Park is Radiohead In Flames Green Day #7 Britney Spears #8 Belle and Sebastian Cascade #8 Simple Plan Cascada is Tinbaland i9 Fall Out Boy O Green Day O My Chemical Romance O My Chemical Romance #11 Linkin Park # Fall Out Boy 1 Avenged Sevenfold it 12 Miley Cyrus 2 50 Cent i2 50 Cet #3 Paramore 13 Avenged Sevenfold #3 Snoop Dogg i4 Metallica 4. Radiohead #14 Cascada i5 50 Cent is In Flames #1S Timbaland 16 My Chemical Romance fié Metallica i16 Johnny Cash #17 Akon #17 Paramore #17 Hilary Duff i8 R.E.M. #8 Nickelback i18 Cassie #9 Fogging Molly it 9 Leona Lewis #19 Amy Winehouse i2O Fall Out Boy #20 Snoop Dogg #20 Aerosmith W 2 Bon Jovi #2 Hilary Duff #21 Bloc Party #22 Leona Lewis i Bon Jovi i22 Rage Against the Machine #23 Coldplay #23 Avenged Sevenfold #23 Amy Winehouse #24 Nickelback i24 Cassie #24 Lostprophets i25 Billy Joel 25 Belle and Sebastian #25 Kate Nash #26 We the Kings #26 Rage Against the Machine #26 Metalica #27 Rage Against the Machine #27 Slipknot #27. Killswitch Engage i28 AC/DC #28 Akon i28 System of a Down i29 The Medic Droid i29 Aerosmith #29 Arctic Monkeys #30 Slipknot i30 Kate Nash #30 Led Zeppelin #31 3. #31 30 Seconds to Mars Amy Winehouse Miley Cyrus #32 Nickelback i32 Avril Lavigne #32 Coldplay 33 Audioslave f33 Arctic Monkeys #33 Gorillaz #34 The Rolling Stones #34 R.E.M. i;34. Korn i35 #35 Paranore #35 Hilary Duff System of a Down #36 KT Tunstall #36 Van Morrison i36 Led Zeppelin #37 #37 Deep Purple #37 Nirvana Deep Purple #38 Leona Lewis i38 Cassie #38. Flogging Molly #39 #39 Kylie Minogue Deep Purple Johnny Cash fia.0 Iron Maiden 40 40 30 Seconds to Mars Snoop Dogg i41 i4 Megadeth 4. Gorillaz Lily Allen Patent Application Publication Oct. 1, 2009 Sheet 4 of 5 US 2009/0248.614 A1
46 44
Processor
Main memory 50 Display interface Display unit Communication Infrastructure (BUS) Secondary memory Hard disk drive 56 60 Removable Removable storage drive StOrage unit 58 64 Removable Interface storage unit
62 68
Communicationinterface Communication path
FIGURE 4 Patent Application Publication Oct. 1, 2009 Sheet 5 of 5 US 2009/0248.614 A1
- EE st d SERVER STORAGE CLIENT
108 FIGURE 5 US 2009/0248.614 A1 Oct. 1, 2009
SYSTEMAND METHOD FOR 0006. Also provided is a user interface for tunably com CONSTRUCTING TARGETED RANKING bining multiple ranked lists, the user interface including a FROM MULTIPLE INFORMATION SOURCES display to display a combined list of items from multiple ranked lists, wherein each item in a list of the multiple ranked CROSS-REFERENCE TO RELATED lists has an associated rank, n, within the list, and wherein the APPLICATIONS combined list is based on combining the items in the multiple lists, the combining including assigning a value to a variable 0001. This application claims the benefit of and incorpo p, the value being neither 0 nor negative; for each list in the rates by reference in its entirety U.S. provisional application multiple lists, computing a score for each item within the list, No. 61/041,128, which was filed on Mar. 31, 2008. wherein the score is a function of the associated rank, and wherein the function is equal to 1/(n(1/p)); for each item, FIELD OF INVENTION Summing corresponding scores to produce a total score for 0002 The present invention relates to information mash the item; and ranking each item based on the corresponding ups, and in particular to a system and method for constructing total score of each item. The user interface also includes a tool targeted rankings from multiple information Sources. to receive input indicating a new value of p, the tool config ured to transmit the input to trigger construction of a new BACKGROUND combined list based upon the new value of p. 0007. A computer program product fortunably combining 0003. Through the advances of technology, today's world multiple ranked lists is also provided. The computer program has become inundated with information. One continuing product includes a computer readable medium; first program technological and Societal challenge is finding methods and instructions to receive multiple lists, wherein each list systems to extract and combine useful data, knowledge, and includes items each having an associated rank, n, within the understanding from a pool of information that is constantly list; and second program instructions to combine the items in growing in quantity and increasing in granularity. the multiple lists into a combined list, wherein the combining 0004. Even when we narrow our analysis to one domain of includes assigning a value to a variable p, the value being interest, e.g. ranking wines, how do we combine all the infor neither 0 nor negative; for each list in the multiple lists, mation indicating preferences within the domain when the computing a score for each item within the list, wherein the information is available from multiples sources and the score is a function of the associated rank, and wherein the sources differ in modality? For example, how do we combine function is equal to 1/(n(1/p)); for each item, Summing cor multiple lists of preferences, e.g., from different online com responding scores to produce a total score for the item; rank munities, sales numbers from different stores, etc.? How do ing each item based on the corresponding total score of each we combine the information in a manner that will reveal the item; and outputting the combined list based on the ranking, aspects of that information that are important, valuable, sig wherein the first and second program instructions are stored nificant to an entity (e.g., a machine, business, customer, on the computer readable media. end-user, etc.) requesting the results'? And how do we enable 0008. A method for providing a combined ranked list from tuning of the outcome, e.g., at the touch of a button, to target multiple ranked lists is also provided. The method includes certain characteristics and elevate those characteristics to the receiving an identification of a domain of interest; receiving forefront? information describing a business objective; gathering mul tiple ranked lists on the domain of interest from multiple SUMMARY OF THE INVENTION Sources, wherein each list includes items each having an associated rank, n, within the list; combining the items in the 0005. A computer-implemented method fortunably com multiple lists into a combined list, wherein the combining bining multiple ranked lists is provided. The method includes includes automatically computing a score for each item receiving multiple lists, wherein each list includes items each within each of the multiple lists using a function equal to having an associated rank, n, within the list. The method 1/(n(1/p)), wherein p is a variable and the value of p is further includes combining the items in the multiple lists into determined based on the business objective and is neither 0 a combined list, wherein the combining includes assigning a nor negative; and transmitting the combined list. The multiple value to a variable p, the value being neither 0 nor negative; Sources may have differing modalities. Gathering the mul for each list in the multiple lists, computing a score for each tiple ranked lists may include gathering unstructured data item within the list, wherein the score is a function of the indicating preferences for items within the domain of interest; associated rank, and wherein the function is equal to 1/(n(1/ and constructing a ranked list from the unstructured data. The p)); for each item, Summing corresponding scores to produce method may further include receiving further information a total score for the item; ranking each item based on the describing the business objective; tuning the value of the corresponding total score of each item; and outputting the variable p based on the further information; repeating the combined list based on the ranking. Assigning a value to a combining based on the tuning; and repeating the transmit variable p may include assigning a value of 2.5 to the variable ting based on the repeating of the combining. p.The method may further include increasing the value of the variable p to increase ranking in the combined list of items BRIEF DESCRIPTION OF THE FIGURES appearing in a majority of the multiple lists. The method may further include decreasing the value of the variable p to 0009 FIG. 1A shows a table of three ranked lists A, B, increase ranking in the combined list of items having higher and C. than average associated ranking in a minority of the multiple 0010 FIG. 1B shows a table of various combined lists lists. The method may further include calculating the associ when different values for the tuning parameter pare used. ated rank n for each item based on a position of the item with 0011 FIG. 2 shows a graph of effects on ranking when the the corresponding list in the multiple lists. value of the tuning parameter p is changed. US 2009/0248.614 A1 Oct. 1, 2009
0012 FIGS. 3A-3C are each screenshots of a website out cation. In another use, the first system and method may be putting a combined list when certain values of pare used. used separately from the latter. 0013 FIG. 4 is a high level block diagram showing an 0019 Plurality type voting systems are those that add information processing system useful for implementing one together the number of votes from each Source and simply embodiment of the present invention. adjudicate the winner based on whomever or whichever can 0014 FIG.5 represents an exemplary distributed data pro didate has the most votes. Plurality type Voting systems cessing system in which aspects of the illustrative embodi include systems in which votes are weighted. Plurality type ments may be implemented. Voting systems have deficiencies when combining informa tion gathered from multiple sources with differing modalities. DESCRIPTION This can occur, for example, when there are large differences in the numbers returned by sources or when the values mea Overview sured to derive those numbers indicate very different things. 0020 For example, say we want to combine different on 00.15 Embodiments of the present invention provide a line data to generate a list of wines. One source of preferences system and method for constructing targeted rankings from may be generated from sales numbers of wines. Another multiple information sources. In an exemplary embodiment, Source may be a list generated from wine tasters. Yet another the system and method tunably combines multiple ranked may be generated by professionals at a wine magazine. Yet lists by computing a score for each item within the list, another may be generated from counts of comments users wherein the score is a function of the associated rank of the post on a wine aficionado site. There are many more sales of item within the list. In one exemplary embodiment, the func wines than posts on a website. Many people buy wines tion is equal to 1/(n(1/p)), where p is a tuning parameter that whereas composing a review takes more time and may indi enables selection between responsiveness to one candidate cate more interest in a particular vintage. Ultimately, the goal (sometimes referred hereinto as an item) ranked highly in one is to combine these multiples sources that indicate interest in Source versus responsiveness to a candidate with lower but all the same underlying Subject matter without allowing one broader Support among the various sources ranking the can Source to unduly influence the combined/consensus list. didates. Accordingly, by providing a tunable parameter, the 0021. In more traditional information integration sce items ranked in the single, merged list may be ranked in a narios, Systems compare things with identical modalities, targeted fashion, elevating and revealing items with certain Such as number of sales from different sources. In this case characteristics (e.g., broad appeal) that are inline with, for using a plurality type voting system makes sense. However example, a business reason for constructing the merged list. there are many domains of interests (e.g., patient preferences, Thus, tuning of the ranking may be driven by or based on, for drugs for certain medical conditions, cars, wine, financial example, predetermined business objectives. products (stocks, bonds, etc.), consumer goods, cameras, 0016 For example, in certain situations, strong minority computers, books, etc.) where information is available from Support may be considered more valuable, e.g., when there is many different modalities (e.g., comments, passive listens, a threshold support before certain action is taken. One such sales, hits on a website, creation of new website, views on situation may be, for example, when a business is more inter television, etc.). In the domain of books, the following infor ested in determining which television show has strong Sup mation may be available for determining book preferences: port, even when only a minority of the Voters has demon book sales and returns, lists of books read, library checkouts, strated that sentiment, before they release a DVD of that comments on books read (e.g., online, in newspapers, in television show's season. magazines, on television or radio), etc. How can one combine 0017. In other situations, weaker but broader support may the data from the various sources when they are heteroge be considered more valuable, e.g., when more general popu neous in modality? Comparing different modalities is akin to larity is more appropriate towards meeting a business objec comparing apples and oranges. How does one determine tive. One such situation may be, for example, when one wants overall rankings for certain wines, for example, based on the to reward an all time best book, even though the chances of combination of data on sales, written reviews, returns, web Such a book appearing first in a list from any one source (e.g., site polls, etc.? How do you combine data indicating that the a voter) may be slim because of the number of books available reviewer loves a certain wine (glowing reviews), but the pub from which to choose. Each source (e.g., a web site, a New lic hates it (e.g., by ranking it low on wine.com or low sales)? York Times top seller's list, a survey, etc.) is treated in Do we decide that ten times as many posts on a website reflect embodiments of this invention as a “voter” in applying vote ten times as much interest in an event or item? There is a fair counting/computing methods, Voting theory, and voting sys amount of Subjectivity in how these combinations occur and tems. The systems and methods fortunably combining mul it is not typically clear how to combine all these sources. tiple ranked lists described in detail herein are particularly 0022. The present disclosure differs from traditional work advantageous in scenarios in which the number of candidates in the field in several ways. For example, the disclosure is significantly larger (for example, orders of magnitudes addresses situations where, as noted, people are providing larger) than the number of voters. preferences in non-uniform ways (complaints, purchase, 00.18 Embodiments of the system and method described opinion posted, time, etc.). In Such situations, ad hoc weights in detail herein compliments the system and method for deter don’t work well because ad hoc weights can only adjust for mining preferences from information mashups described in the deficiencies that exist at a simple point in time. Consider co-pending U.S. patent application Ser. No. (having the use of ad hoc weights in combining top-10 lists from attorney docket number ARC92008.0029US3), and filed on Amazon.com and Barnes & Noble in 1995. In that year, . Accordingly, in one use, the system or method Amazon.com ranks should be weighted lower (having less described in detail herein may be used in conjunction with the weight in the over scheme of the analysis) than Barnes & system or method described in detail in the co-pending appli Noble because Amazon.com opened its online store in July US 2009/0248.614 A1 Oct. 1, 2009
1995. If the top-10 lists were compared today, the weights 931st place and 932nd place. These factors and more play into would differ. Thus, although ad hocs weights are useful when the need to create new Voting counting methods that better combining lists of preferences at one point in time, they need address this domain. adjusting each time new data from the Sources are recom bined to account for, e.g., changes in the market, business Features of Exemplary Embodiment cycles, seasons, time of day, new product releases (which 0026. An exemplary embodiment of the present invention could, for example, skew statistics for a few days), blitz provides a method to combine multiple ranked lists. The marketing campaigns, events (e.g., Olympics or Superbowl), method can be tuned between elevating the ranks of items in etc. These real world changes have the potential of causing the lists that have broad appeal and elevating the ranks of dramatic shifts in the rankings being reported. The ad hoc items in the list that have narrow (but strong) support. The weights adjustments are time-dependent. If we calculate the ranked lists may originate from sources of the same or differ rankings at a different point in time, the weights would be ent domains. For example, one source list may be based on a reconsidered and changed, tuned each time we calculate the number of hits to specific website while another list may be rankings. This can be particularly onerous depending on how based on processing user posts from a social networking site, often the combined rankings are calculated (in real time, e.g., by employing a series of unstructured information man daily, weekly, monthly, quarterly, annually, etc.) particularly agement architecture (UIMA) annotators driven off of entity if the tuning is done without the assistance of any computer spotting. implemented algorithms. 0027. In one embodiment, an item is considered to have 0023. Additionally, the present disclosure differs from tra broad appeal when it appears in a majority of the lists, even ditional work in the field because we are generally looking at though it may have an average or near average ranking in each domains with orders of magnitude more “candidates' than list. For example, the item may appear in fifth place in eight “voters', the reverse of conventional elections. An example of out of the ten lists being combined. In one embodiment, an item is considered to have narrow Support when it has higher a conventional election is a U.S. presidential election, where than average associated ranking in a minority of the lists. For the millions of people (“voters') are choosing between essen example, the item may appear in first place in two out often tially two nominees (“candidates'). In the domain of conven lists being combined. From one perspective, an item having tional elections, the number of voters>>the number of candi narrow support can be considered to be a favorite candidate of dates. Accordingly, conventional Voting techniques do not a few Voters, whereas an item having broad Support can be examine scenarios in which the number of “candidates' is considered to have above average or average Support from orders of magnitude more than the number “voters. For many Voters, though none of the Voters may consider the example, the Borda function is intended for use in situations candidate their favorite. when there are a large number of voters and a small number of 0028. Table 2 below shows two ranked lists to illustrate candidates, such as in a presidential election. how multiple “voters' can return their favorite candidates in 0024. In contrast, the domain addressed by exemplary rank order. embodiments of the present invention covers scenarios in which the number of voters is smaller than the number of TABLE 2 choices, such that the number of voters
0039. As seen in the pseudo code, an item in the list (ex including anti-spam heuristics. In the wine example, the heu pressed in the pseudo code as a candidate c in the list l) is ristic may, for example, require that a wine has been men given a Score that is based upon a function of rank rank(c.). tioned for at least three weeks in a row before it will be As seen in the pseudo code, the rank (rank(c)) is a function of considered for inclusion in the combined list. the candidate c, which is related to the candidate's position in 0044. In one embodiment, a system is provided that per the list1,. In the case above, the rank of each candidate in the combined list places the candidate with the highest score first. mits viewing of the effects of changing the value of p on the The combined list may also place the candidate with the rankings of the items in the combined list. For example, a user highest score last, so as to provide the list in descending order, can adjust the value of p (e.g. by entering a value or moving a and still remain in accordance with embodiments of the scrollbar tool in a computer user interface). In response, the invention. system recombines the various lists based on the new p value 0040 FIG. 2 shows a graph of effects of changing the and outputs the results (a new single, merged list) onto a value of the tuning parameter p on rank. The X-axis of the display screen. Accordingly, the system provides a user inter graph is the rank of an item in the list and the y-axis of the face for tunably combining multiple ranked lists. The user graph is the score given to that item. As discussed above, the interface includes a display to show a single, merged list of rank of the item is a function of that item's position in the list. items from multiple ranked lists. In this manner, a user may As seen in FIG. 2, the lower the value of p, the greater the see the effects of changing the value of p on the outcome of difference between the score given to the higher ranked items the rankings in the combined list, and tune as appropriate to (e.g., 1’, 2", etc.) and the score given to lower ranked items. elevate the rankings of items which have characteristics For example, in the graph, when p=1, the score of the item which the user values. For example, the user may increase the ranked 1 is 1 and the score of the item ranked 20 is approxi value of p to elevate in the rankings of items having diffuse mately 0.05. When p=3, the graph shows that the score of the general popular Support. The user may decrease the value of item ranked 1 is also 1, but the score of the item ranked 20 is p to elevate in the rankings of items having strong concen almost 0.4 (or 8x greater than the score for same item when trated individual/minority support. Methods implemented by p=1). Accordingly, the higher the value of p, the smaller the Such a system are in accordance with embodiments of the difference is between the score given to the higher ranked invention. items and the score given to lower ranked items. 004.5 FIGS. 3A-3C are screenshots of a website allowing 0041. In addition, as seen in FIG. 2, for each p, when the a user to view various outputted combined lists depending on number of candidates that one is interested in viewing is the value of p. In FIG.3A, p equals 1. In FIG.3B, p equals 2.5. Smaller (e.g., the top-5), and one wants to favor narrow, strong In FIG. 3C, p equals 5. As can be seen from FIGS. 3A-3C, the support, a lower value of p should be selected. If one is value of p can be represented in a manner that is more user interested in viewing more candidate (e.g., the top-100), even friendly, Such as, for example, by representingap value of 2.5 when one wants to favor narrow, strong Support, a higher as 25. Moreover, without revealing the algorithm used to value of p could selected than the value one selects when one determine the ranking, the tool 302, which allows the user to is interested in viewing just the top-5. selectap value, also communicates to the user that a higher p 0042 Effects of ranking in accordance with embodiments value (e.g., by indicating score 10152025.303540.14550) of the present invention include the following. First, the rank tends to take into account more candidates, and therefore Such ing method makes the difference between the top ranked a selection would tend to reflect more diffuse support in the candidates (sometimes referred to as the “top ranks') more rankings, than a lowerp selection (e.g., indicated by score 10 significant than the difference between lower ranked candi 152025.303514O455O). The user may perceive the tool as dates (sometimes referred to as the “lower ranks'). (Depend indicating that more candidates are being scored when a ing on the value of p, the “top ranks' could be the top few or higher number is selected. This provides the user with an top hundreds candidates or items.) Second, as p is increased, intuitive reason for the changes in the ranking caused by the the ranking method has the effect of allowing several votes for tuning even though the algorithm used by the tool is not a lower ranked candidate to “count more than a single vote scoring more or fewer candidates (as the user may believe), for a higher ranked candidate. For example, if p is larger, but instead scoring all candidates in various source lists in a several votes of second or third place can count more than a novel manner. In FIGS. 3A-3C, the tool includes a set of single vote of first place. This allows the combined score for hyperlinks (seen in FIGS. 3A-3C as underlined numbers) a candidate in second or third place in Source lists to be higher from among which the user can select. In other embodiments, than the combined score for a candidate for first place in one the tool may include alternatively or additionally a scrollbar source list. This allows for elevation within the combined list or other graphical user interface tools. of a candidate that was a second or third place candidate in 0046 Embodiments of the invention can take the form of multiple lists over a first place candidate in a single list. Stated an entirely hardware embodiment, an entirely software another way, this allows broad Support for a particular artist to embodiment, or an embodiment containing both hardware “count more than a single source ranking an artist number and software elements. In a preferred embodiment, the inven one, thereby reducing the ranking methods sensitivity to tion is implemented in software, which includes but is not single source outliers. limited to firmware, resident software, and microcode. 0043. In one application, this ranking method is used in 0047. Furthermore, the invention can take the form of a collaboration with other vote counting frameworks. For computer program product accessible from a computer-us example, this ranking method may be “plugged into a Borda able or computer-readable medium providing program code Count-like framework. In this way, the ranking method can be for use by or in connection with a computer or any instruction cascaded with other variations. Such variations may include execution system. For the purposes of this description, a allocating delegates to each Voter based on the size/impor computer-usable or computer-readable medium can be any tance of the population being represented, or other heuristics apparatus that can contain, store, communicate, propagate, or US 2009/0248.614 A1 Oct. 1, 2009 transport the program for use by or in connection with the system. Such means may include, for example, a removable instruction execution system, apparatus, or device. storage unit 62 and an interface 64. Examples of such means 0048. The medium can be, for example, an electronic, may include a program cartridge and cartridge interface (Such magnetic, optical, electromagnetic, infrared, or semiconduc as that found in video game devices), a removable memory tor System (or apparatus or device). Examples of a computer chip (such as an EPROM, or PROM) and associated socket, readable medium include a semiconductor or Solid State and other removable storage units 62 and interfaces 64 which memory, magnetic tape, a removable computer diskette, a allow software and data to be transferred from the removable random access memory (RAM), a read-only memory (ROM), storage unit 62 to the computer system. a rigid magnetic disk, or an optical disk. Current examples of 0055. The computer system may also include a communi optical disks include compact disk read-only memory (CD cations interface 66. Communications interface 66 allows ROM), compact disk read/write (CD-R/W), and DVD. software and data to be transferred between the computer 0049. A data processing system suitable for storing and/or system and external devices. Examples of communications executing program code will include at least one processor interface 66 may include a modem, a network interface (Such coupled directly or indirectly to memory elements through a as an Ethernet card), a communications port, or a PCMCIA system bus. The memory elements can include local memory slot and card, etc. Software and data transferred via commu employed during actual execution of the program code, bulk nications interface 66 are in the form of signals which may be, storage, and cache memories which provide temporary stor for example, electronic, electromagnetic, optical, or other age of at least some program code in order to reduce the signals capable of being received by communications inter number of times code must be retrieved from bulk storage face 66. These signals are provided to communications inter during execution. face 66 via a communications path (i.e., channel) 68. This 0050 Input/output or I/O devices (including but not lim channel 68 carries signals and may be implemented using ited to keyboards, displays, pointing devices) can be coupled wire or cable, fiber optics, a phone line, a cellular phone link, to the system either directly or through intervening I/O con an RF link, and/or other communications channels. trollers. 0056. In this document, the terms “computer program 0051 Network adapters may also be coupled to the system medium.” “computer usable medium, and “computer read to enable the data processing system to become coupled to able medium' are used to generally refer to media Such as other data processing systems or remote printers or storage main memory 52 and secondary memory 54, removable stor devices through intervening private or public networks. age drive 58, and a hard disk installed in hard disk drive 56. Modems, cable modems and Ethernet cards are just a few of 0057 Computer programs (also called computer control the currently available types of network adapters. logic) are stored in main memory 52 and/or secondary 0052 FIG. 4 is a high level block diagram showing an memory 54. Computer programs may also be received via information processing system useful for implementing one communications interface 66. Such computer programs, embodiment of the present invention. The computer system when executed, enable the computer system to perform the includes one or more processors, such as processor 44. The features of the present invention as discussed herein. In par processor 44 is connected to a communication infrastructure ticular, the computer programs, when executed, enable the 46 (e.g., a communications bus, cross-over bar, or network). processor 44 to perform the features of the computer system. Various software embodiments are described in terms of this Accordingly, such computer programs represent controllers exemplary computer system. After reading this description, it of the computer system. will become apparent to a person of ordinary skill in the 0.058 FIG.5 represents an exemplary distributed data pro relevant art(s) how to implement the invention using other cessing system in which aspects of the illustrative embodi computer systems and/or computer architectures. ments may be implemented. Distributed data processing sys 0053. The computer system can include a display interface tem 100 may include a network of computers in which aspects 48 that forwards graphics, text, and other data from the com of the illustrative embodiments may be implemented. The munication infrastructure 46 (or from a frame buffer not distributed data processing system 100 contains at least one shown) for display on a display unit 50. The display interface network 102, which is the medium used to provide commu may forward, for example, the graphics and text shown in nication links between various devices and computers con FIG. 3A. The computer system also includes a main memory nected together within distributed data processing system 52, preferably random access memory (RAM), and may also 100. The network 102 may include connections, such as wire, include a secondary memory 54. The secondary memory 54 wireless communication links, or fiber optic cables. may include, for example, a hard disk drive 56 and/or a 0059. In the depicted example, server 104 and server 106 removable storage drive 58, representing, for example, a are connected to network 102 along with storage unit 108. In floppy disk drive, a magnetic tape drive, or an optical disk addition, clients 110, 112, and 114 are also connected to drive. The removable storage drive 58 reads from and/or network 102. These clients 110, 112, and 114 may be, for writes to a removable storage unit 60 in a manner well known example, personal computers, network computers, or the like. to those having ordinary skill in the art. Removable storage In the depicted example, server 104 provides data, such as unit 60 represents, for example, a floppy disk, a compact disc, boot files, operating system images, and applications to cli a magnetic tape, or an optical disk, etc. which is read by and ents 110, 112, and 114. Clients 110, 112, and 114 are clients written to by removable storage drive 58. As will be appreci to server 104 in the depicted example. Distributed data pro ated, the removable storage unit 60 includes a computer read cessing system 100 may include additional servers, clients, able medium having stored therein computer software and/or and other devices not shown. data. 0060. In the depicted example, distributed data processing 0054. In alternative embodiments, the secondary memory system 100 is the Internet with network 102 representing a 54 may include other similar means for allowing computer worldwide collection of networks and gateways that use the programs or other instructions to be loaded into the computer Transmission Control Protocol/Internet Protocol (TCP/IP) US 2009/0248.614 A1 Oct. 1, 2009
Suite of protocols to communicate with one another. At the assigning a value to a variable p, the value being neither heart of the Internet is a backbone of high-speed data com 0 nor negative; munication lines between major nodes or host computers, for each list in the multiple lists, computing a score for consisting of thousands of commercial, governmental, edu each item within the list, wherein the score is a func cational and other computer systems that route data and mes tion of the associated rank, and wherein the function is sages. Of course, the distributed data processing system 100 equal to 1/(n(1/p)); may also be implemented to include a number of different for each item, Summing corresponding scores to pro types of networks, such as for example, an intranet, a local duce a total score for the item; area network (LAN), a wide area network (WAN), or the like. ranking each item based on the corresponding total score As stated above, FIG. 5 is intended as an example, not as an of each item; and architectural limitation for different embodiments of the outputting the combined list based on the ranking. present invention, and therefore, the particular elements 2. The computer-implemented method of claim 1, wherein shown in FIG.5 should not be considered limiting with regard assigning a value to a variable p comprises assigning a value to the environments in which the illustrative embodiments of of 2.5 to the variable p. the present invention may be implemented. 3. The computer-implemented method of claim 2, further 0061. In one use, as an example, clients 110 and 112 comprising: collect information (e.g., from user input) and provides it to server 104. Server 104 stores the information in storage 108. increasing the value of the variable p to increase ranking in Server 106 contains hardware devices and software tools to the combined list of items appearing in a majority of the combine the information (e.g., into information mashups and/ multiple lists; and or combined/consensus lists) according to the present inven repeating the combining. tion. Server 106 transmits the combined information to server 4. The computer-implemented method of claim 1, further 104 and/or clients 110, 112, and/or 114, for example. comprising: 0062. In use, client 114 may transmit a change in the decreasing the value of the variable p to increase ranking in tunable parameter p to the server. In response, the server the combined list of items having higher than average recombines information based on the parameter and outputs a associated ranking in a minority of the multiple lists; and different the list to the client. This list may take the form of a repeating the combining. webpage, e.g., seen in FIGS. 3A-3C. 5. The computer-implemented method of claim 1, further 0063 References in the claims to an element in the singu comprising: lar is not intended to mean “one and only unless explicitly so receiving a new value for the variable p; and stated, but rather "one or more.” All structural and functional repeating the combining using the new value as the value equivalents to the elements of the above-described exemplary for the variable p. embodiment that are currently known or later come to be 6. The computer-implemented method of claim 1, further known to those of ordinary skill in the art are intended to be comprising: encompassed by the present claims. No claim element herein is to be construed under the provisions of 35 U.S.C. section assigning a new value to the variable p to adjust the ranking 112, sixth paragraph, unless the element is expressly recited in the combined list of items; and using the phrase “means for or “step for.” repeating the combining. 0064 Reference throughout this specification to “one 7. The computer-implemented method of claim 1, wherein embodiment' or “an embodiment’ means that a particular assigning the value to the variable p comprises automatically feature, structure, or characteristic described in connection assigning the value to the variable p based on a predetermined with the embodiment is included in at least one embodiment business objective. of the present invention. Thus, appearances of the phrases "in 8. The computer-implemented method of claim 1, further one embodiment' or “in an embodiment” in various places comprising: throughout this specification are not necessarily referring to calculating the associated rank n for each item based on a the same embodiment. position of the item with the corresponding list in the 0065. Thus, a system and method for determining prefer multiple lists. ences from information mashups and, in particular, for con 9. A user interface fortunably combining multiple ranked structing targeted rankings from multiple information lists, the user interface comprising: sources is disclosed. While the preferred embodiments of the a display to display a combined list of items from multiple present invention have been described, it will be understood ranked lists, wherein each item in a list of the multiple that modifications and adaptations to the embodiments shown ranked lists has an associated rank, n, within the list, and may occur to one of ordinary skill in the art without departing wherein the combined list is based on combining the from the scope of the present invention as set forth in the items in the multiple lists, the combining comprising: claims. Thus, the scope of this invention is to be construed assigning a value to a variable p, the value being neither according to the claims and not limited by the specific details 0 nor negative; disclosed in the exemplary embodiments. for each list in the multiple lists, computing a score for We claim: each item within the list, wherein the score is a func 1. A computer-implemented method for tunably combin tion of the associated rank, and ing multiple ranked lists, the method comprising: wherein the function is equal to 1/(n(1/p)); receiving multiple lists, wherein each list comprises items for each item, Summing corresponding scores to pro each having an associated rank, n, within the list; and duce a total score for the item; and combining the items in the multiple lists into a combined ranking each item based on the corresponding total score list, wherein the combining comprises: of each item; and US 2009/0248.614 A1 Oct. 1, 2009
a tool to receive input indicating a new value of p, the tool 15. A method for providing a combined ranked list from configured to transmit the input to trigger construction multiple ranked lists, the method comprising: of a new combined list based upon the new value of p. receiving an identification of a domain of interest; 10. The user interface of claim 9, wherein the tool is receiving information describing a business objective; selected from the group consisting of a scrollbar tool, a gathering multiple ranked lists on the domain of interest hyperlink, and a keyboard. from multiple sources, wherein each list comprises 11. The user interface of claim 9, wherein the display is items each having an associated rank, n, within the list; further to display near the combined list selectable numbers combining the items in the multiple lists into a combined based upon predetermined values of p. list, wherein the combining comprises: 12. A computer program product for tunably combining automatically computing a score for each item within multiple ranked lists, said computer program product com each of the multiple lists using a function equal to prising: 1/(n(1/p)), a computer readable medium; wherein p is a variable and the value of p is determined first program instructions to receive multiple lists, wherein based on the business objective and is neither 0 nor each list comprises items each having an associated negative; and rank, n, within the list; and transmitting the combined list. second program instructions to combine the items in the multiple lists into a combined list, wherein the combin 16. The method of claim 15 wherein the multiple sources ing comprises: have differing modalities. assigning a value to a variable p, the value being neither 17. The method of claim 16 wherein gathering the multiple 0 nor negative; ranked lists comprises: for each list in the multiple lists, computing a score for gathering unstructured data indicating preferences for each item within the list, wherein the score is a func items within the domain of interest; and tion of the associated rank, and wherein the function is constructing a ranked list from the unstructured data. equal to 1/(n(1/p)); 18. The method of claim 15 where the value of p is higher for each item, Summing corresponding scores to pro when the business objective indicates interest in ranking duce a total score for the item; items with broad appeal higher than items with narrow appeal ranking each item based on the corresponding total score than when the business objective indicates interest in ranking of each item; and items with narrow appeal higher than items with broad outputting the combined list based on the ranking, appeal. wherein said first and second program instructions are 19. The method of claim 15, further comprising: stored on said computer readable media. calculating the associated rank n for each item within each 13. The computer program product of claim 12, further list based on a position of the item with the correspond comprising: ing list in the multiple lists. third program instructions to assign a new value to the 20. The method of claim 15, further comprising: variable p to adjust the ranking in the combined list of receiving further information describing the business items; and objective; fourth program instructions to repeat the combining, tuning the value of the variable p based on the further wherein said third and fourth program instructions are information; stored on said computer readable media. repeating the combining based on the tuning; and 14. The computer program product of claim 12, wherein repeating the transmitting based on the repeating of the assigning the value to the variable p comprises automatically combining. assigning the value to the variable p based on a predetermined business objective.