Analogical Modeling: An Update David Eddington and Deryle Lonsdale Department of Linguistics & English Language Brigham Young University Provo, UT, USA 84602 {eddington,lonz}@byu.edu Abstract accessed. A search is conducted for the stored ex- emplars that are most similar to the one whose be- Analogical modeling is a supervised havior is being predicted. The behavior of highly exemplar-based approach that has been similar stored entities generally predicts the behav- widely applied to predict linguistic behavior. ior of the one in question, although less similar ones The paradigm has been well documented in have a small chance of applying as well. the linguistics and cognition literature, but AM has been implemented in a succession of is less well known to the machine learning computer implementations that allow users to spec- community. This paper sets out some of the ify an exemplar base and then to test unseen in- basics of the approach, including a simpli- stances of language behavior against this accumu- fied example of the fundamental algorithm’s lated store to predict (and quantify) the relevant and operation. It then surveys some of the recent possible outcome(s). analogical modeling language applications, and sketches how the computational system A comprehensive examination of the system is has been enhanced lately to offer users beyond the scope of this paper1. Instead, an at- increased flexibility and processing power. tempt will be made to situate AM within the space of Some comparisons and contrasts are drawn exemplar-based modeling systems and, to a certain between analogical modeling and other extent, within the larger context of machine learning language modeling and machine learning systems and cognitive modeling systems in general. approaches. The paper concludes with a This paper begins by giving an informal overview discussion of ongoing issues that still con- of the algorithm used to perform AM. To accomplish front developers and users of the analogical this it traces in schematic form an early but typi- modeling framework. cal application of the AM approach: Spanish gen- der guessing. The subsequent section discusses AM 1 Introduction with respect to other types of modeling systems, and Analogical modeling (AM) is an exemplar-based gives an overview of the types of linguistic phenom- modeling approach designed to predict linguistic be- ena modeled by AM. The paper then discusses sev- havior on the basis of stored memory tokens (Sk- eral items of recent and ongoing work with the AM ousen, 1989; Skousen, 1992; Skousen, 1995; Sk- program. The conclusion mentions possible areas ousen, 1998). It is founded on the premise that for future improvement and investigation. previous linguistic experience is stored in the men- tal lexicon. When the need arises to determine 1For further information including a bibliography some linguistic behavior (pronunciation, morpho- and software downloads see the project webpage at: logical relationship, word, etc.), the lexicon itself is http://humanities.byu.edu/am Subcontexts Members of the Subcontext Pointers # of Disagreements pan none none 0 ¯pan plan M plan M > plan M 0 p¯an none none 0 pa¯n paz F, par M paz F > paz F 2 par M > par M *paz F > par M *par M > paz F ¯pa¯n cal F cal F > cal F 0 p¯an¯ tren M, ron M tren M > tren M 0 ron M > ron M tren M > ron M ron M > tren M p¯a¯n none none 0 p¯a¯n¯ rey M rey M > rey M 0 Table 1: Subcontexts and their disagreements. 2 The AM algorithm are marked with asterisks in Table 1. A disagree- ment occurs when words that are equally similar to Perhaps the best way to understand the AM algo- the given context, exhibit different behaviors, in this rithm is with a concrete illustration. Predictions are case, different genders. The number of disagree- always made in terms of specific exemplars, there- ments is determined by pairing all members of a sub- fore, for the purposes of the example, the following context with every other member, including itself, seven monosyllabic Spanish nouns and their corre- 2 by means of unidirectional pointers, and counting sponding gender will be considered : the number of times the members of the pair have paz F different behaviors. In this example, the only sub- plan M context containing any disagreement is pa¯n. tren M cal F Subcontexts are then arranged into more compre- ron M hensive groups called supracontexts as shown in Ta- par M ble 2. In Table 2, a hyphen indicates a wildcard. rey M In the subcontextual analysis a tally of the all The task will be to predict the gender of pan of the subcontextual disagreements is made. The ’bread’ on the basis of these seven items as the supracontextual analysis consists of analyzing all of set of exemplars. The three variables used are the the words that appear in a given supracontext, and phoneme or phoneme cluster of the onset, nucleus, again tallying disagreements (Table 3). and rhyme. First, all possible exemplar items are as- In the supracontexutal analysis, words that have signed to a series of subcontexts which are defined more than one variable in common with pan appear in terms of the given context pan. This yields eight in more that one supracontext. This is AM’s way of subcontexts (Table 1). A bar over a variable indi- allowing the gender of words which are more similar cates that any value of the variable except the barred to pan to influence the gender assignment of pan to value is permitted. a greater extent. By assigning members to subcontexts it is pos- The purpose of AM’s algorithm is to determine sible to determine disagreements. Disagreements which members of the exemplar base are most likely 2This example is a simplification; see (Eddington, 2002) for to affect the gender assignment of pan, and also to a complete AM treatment of this phenomenon. calculate the extent of analogical influence exerted. Supracontext Subcontexts in Supracontext # Subcontextual Disagreements p a n pan 0 p a – pan, *pa¯n 2 p – n pan, p¯an 0 – a n pan, ¯pan 0 p – – pan, p¯an, *pa¯n, p¯a¯n 2 – a – pan, ¯pan, *pa¯n, ¯pa¯n 2 – – n pan, ¯pan, p¯an, ¯p¯an 0 – – – pan, ¯pan, p¯an, *pa¯n, ¯pa¯n, ¯p¯an, p¯a¯n, ¯p¯a¯n 2 Table 2: Subcontextual analysis. This is accomplished by calculating heterogeneity. the influence of the analogical set on the behavior Heterogeneity is determined by comparing the num- of the given context. One is to assign the most fre- ber of disagreements in the supracontextual and sub- quently occurring behavior to the given context (se- contextual analyses. If there are more disagreements lection by plurality). Of the 18 pointers in the set, in the supracontextual analysis, the supracontext is 14 point to masculine. Therefore, selection by plu- heterogenous, and its members are eliminated from rality would assign masculine gender to pan. The consideration as possible analogs. If the number of other method (random selection) involves randomly disagreements does not increase, the supracontext is selecting a pointer, and assigning the behavior of the homogenous. word indicated by the pointer to the given context. Words belonging to homogenous supracontexts In this case, the probability of masculine gender as- comprise the analogical set (Table 4). In the example signment would be 77.78% (14/18). under consideration, disagreements increase in the supracontexts – – –, and – a –. Therefore, their mem- 3 Situating AM bers are eliminated from consideration. Cal, and rey The AM theoretical construct has been implemented appear exclusively in these heterogenous supracon- computationally through several generations of ap- texts. As a result, they do not form part of the ana- plication programs. Pascal code for the first version logical set. The word plan is also a member of both was listed as an appendix in (Skousen, 1989). A – – –, and – a –, however, it is also a member of the subsequent version was implemented in C. In recent homogenous supracontexts – a n, and – – n, so it will years the system was reimplemented as a Perl mod- still be available to influence pan. ule; extensive details on how to use the current ver- It should not be surprising that rey would be elim- sion are available in (Skousen et al., 2002). inated through heterogeneity; it has no phonemes in The system assumes a priori labeled input in- common with pan. However, consider the words ron stances, and thus only works in supervised learn- and cal. Both share only one feature with pan, yet ing mode. Each exemplar must be represented as heterogeneity eliminates only cal, and not ron. This a fixed-length vector comprised of features that may is due to the fact that ron appears in the supracontext or may not eventually be relevant to the phenomenon – – n, and all of the members of that supracontext being modeled. The features themselves may be are masculine, therefore, there is no disagreement. of fixed or variable length, as long as they are ap- Cal F, on the other hand, competes with masculine propriately delimited. An outcome is specified for words in all of the supracontexts in which it appears. each exemplar instance vector. Vector features are The analogical set contains all of the exemplar purely symbolic and nominal, not admitting contin- items that can possibly influence the gender assign- uous values. When the use of continuous-valued ment of pan. There are two methods for calculating features is needed (e.g. integers or real numbers), Supracontexts Words in Pointers # of Subcontextual Supracontext Disagreements p a n none none 0 p a – par M, paz F paz F > paz F 2 par M > par M *paz F > par M *par M > paz F p – n none none 0 – a n plan M plan M > plan M 0 p – – par M, paz F paz F > paz F 2 par M > par M *paz F > par M *par M > paz F – a – plan M, cal F, par M, paz F plan M > plan M 6 *plan M > cal F plan M > par M *plan M > paz F cal F > cal F *cal F > plan M *cal F > par M cal F > paz F *paz F > plan M *paz F > par M (not all shown; all the rest involve agreement) – – n plan M, tren M, ron M (not shown) 0 – – – plan M, tren M, ron M, cal F, (not shown) 12 paz F, par M, rey M Table 3: Supracontextual analysis.