NSA’s Review of Emerging Technologies

Letter from the Editor

The Next Wave 71:  LV NQRZQ IRU LWV FRYHUDJH RI FRPSXWHU VFLHQFH DQG

HQJLQHHULQJ,QWKLVLVVXHKRZHYHUZHWDNHDVOLJKWO\GLIIHUHQWSDWKWRSUHVHQW UHVHDUFKRQJRLQJDWWKH8QLYHUVLW\RI0DU\ODQG&HQWHUIRU$GYDQFHG6WXG\RI /DQJXDJH &$6/ 

,Q16$&66ZDVGHVLJQDWHG([HFXWLYH$JHQWIRUWKH8QLYHUVLW\$IÀOLDWHG 5HVHDUFK&HQWHU 8$5& DWWKH8QLYHUVLW\RI0DU\ODQG&HQWHUIRU$GYDQFHG 6WXG\ RI /DQJXDJH &$6/ ZDV FUHDWHG DV WKH ÀUVW DQG RQO\ 'HSDUWPHQW RI

'HIHQVH 'R'  8$5& GHGLFDWHG WR UHVHDUFK LQ KXPDQ VNLOOV³ODQJXDJH FRJQLWLRQ DQG FXOWXUH 7KLV XQLTXH VWDWXV FDPH DERXW EHFDXVH 'R' DQG WKH ,QWHOOLJHQFH &RPPXQLW\ ,&  UHFRJQL]HG WKH QHHG IRU D VXVWDLQHG IRFXV RQ ODQJXDJHUHDGLQHVV

,Q  &$6/ ZDV LQWHJUDWHG LQWR WKH 16$&66 5HVHDUFK 'LUHFWRUDWH7KLV LQWHJUDWLRQ LV H[SHFWHG WR SURYLGH &$6/ ZLWK PXOWLGLVFLSOLQDU\ NQRZOHGJH KHOSLQJ&$6/DGGUHVVDOOGLPHQVLRQVRIWKHODQJXDJHDQDO\VW·VWDVNV

&$6/·V PLVVLRQ LV WR SURYLGH HPSLULFDO GDWD IURP VROLG UHVHDUFK WR VXSSRUW PDQDJHPHQWGHFLVLRQVLQYROYLQJODQJXDJHDQDO\VWV)RUH[DPSOHKRZFDQZH LGHQWLI\SHRSOHZKRFDQOHDUQODQJXDJHVPRUHHDVLO\"+RZFDQZHLQFUHDVHD ODQJXDJHDQDO\VW·VDELOLW\WRSUREOHPVROYH"+RZFDQZHXVHWHFKQRORJ\WRDVVLVW ODQJXDJHDQDO\VWVLQÀQGLQJXQIDPLOLDUZRUGV"7KHVHTXHVWLRQVDQGPRUHDUH DGGUHVVHGLQ&$6/·VUHVHDUFKDVSUHVHQWHGLQWKLVLVVXHRI71:

The Next Wave is published to disseminate technical advancements and research activities in telecommunications and information technologies. Mentions of company names or commercial products do not imply endorsement by the US Government. For more information, please contact us at [email protected]

CONTENTS

INTRODUCTION 4 The Mind of the Language Analyst: A Peek Inside the Black Box

FEATURES 6 A Working Memory Workout for Language Practitioners 14 “Thinking Out of the Box” Through Cognitive Neuroscience 20 For a Better Dictionary, Build a Better Parser 33 A Talent for Language

FOCUS 42 What’s the Bottom Line? The Mind of the Language Analyst: A Peek Inside the Black Box

As Jane the language analyst (LA) listens to voice material and composes her report, she is faced with a cognitively, linguistically, and technologically challenging job. Although we can hear the voice material and read her report, we can’t see what’s going on inside Jane’s head as she works. The University of Maryland Center for Advanced Study of Language (CASL) is devoted to finding out what’s going on inside Jane’s head and to giving her the linguistic, cognitive, and technological support she needs to do her job more effectively and efficiently. Considerable resources have boosted the Intelligence Community’s capabilities to capture information, and technologically advanced language tools have been created. However, optimizing the performance of LAs facing challenging tasks also requires considering insights and methods from the behavioral and social sciences, particularly the cognitive, language, and brain sciences. CASL’s multidisciplinary research brings together scientists from these diverse fields to help Jane, the human in the equation, work more productively. CASL scientists begin with complicated tasks such as text comprehension or translation and decompose them into more elementary cognitive functions and more detailed information processing mechanisms such as Ê UÊ ÜœÀŽˆ˜}ʓi“œÀÞ]Ê Ê UÊ `ˆÛiÀ}i˜ÌÊÌ inking and creative problem solving, Ê UÊ ˆ˜vœÀ“>̈œ˜ÊiÛ>Õ>̈œ˜Ê>˜`ÊÃiiV̈œ˜] Ê UÊ >˜>ÞÈÃʜvÊܜÀ`ʓi>˜ˆ˜}Ê>˜`ʈÌÃÊÀi>̈œ˜Ê̜ÊܜÀ`ÊvœÀ“]Ê>˜`Ê Ê UÊ Ã«iiV ÉÜÕnd discrimination and its contribution to general language aptitude. The articles in this issue of The Next Wave describe CASL research on these five processes in more detail. The articles illustrate how CASL scientists decompose Jane’s mental processes to improve language analyst selection, training, and on-the-job performance while highlighting the importance of the partnership between LAs and CASL researchers.

A working memory workout “Thinking out of the box” Because Jane needs to hold many bits and Jane is also faced with the problem of linking pieces of information in mind as she listens, she the information that she hears with other knowledge needs a good working memory. Working memory that she has. In other words, she needs to make is the cognitive mechanism that allows us to hold connections between what she hears and what she disparate information in mind. It is dedicated to the knows. Henk Haarmann and Jared Novick discuss temporary processing, maintenance, and integration how insights and methods from cognitive science of information. Michael Bunting and his collaborators may be used to improve the divergent thinking of have worked to alleviate some working memory LAs and all-source analysts so that they can more limitations for Jane. Because limitations in working easily connect information. Divergent thinking PHPRU\DUHUHODWHGWRODQJXDJHSURFHVVLQJGLIÀFXOW\ is the ability to think of many useful solutions to and misinterpretation, training that improves open-ended problems—not just common solutions, working memory should improve memory capacity. but also creative ones. For example, LAs are often Additionally, such training should also generalize required to think of possible causes, consequences, to broader sets of cognitive tasks and processes that and conceptualizations of actors and events they rely on working memory—for example, resolving report on. Divergent thinking can be enhanced by ambiguity, overcoming misleading context, and changing Jane’s neurophysiological and cognitive ÀOOLQJLQPLVVLQJRUFRUUXSWHGLQIRUPDWLRQ states. Providing analysts with approaches from

4 The Mind of the Language Analyst Introduction

cognitive neuroscience to improve their divergent A talent for language thinking is important for improving their ability On the other hand, if Jane is an expert, to generate multiple plausible interpretations for experienced LA, she will not have many of the texts with missing information, massive linguistic problems faced by junior LAs. How can we identify ambiguity, and active distortion or intended those LAs who have the potential to become deception. expert LAs? Cathy Doughty and her colleagues What's the bottom line? are developing an aptitude battery to assist with the selection and hiring of future top analysts. Jane and her supervisor need to ensure The battery aims to identify individuals with the quality of Jane's report. As part of a quality high language aptitude so that training and hiring evaluation, Jane's supervisor needs to assess the resources can be focused on those who are most difficulties Jane faced with the material and to likely to succeed. Currently, used language aptitude evaluate it along critical dimensions. Erica Michael tests predict success in the early lower-level stages and her colleagues have identified key component of language learning. A unique feature of the new cognitive processes of what's going on inside Jane's battery-Hi-LAB-is that it predicts high-level head as she accomplishes her complex language language learning success. The Hi-LAB cognitive analysis task. Based on this cognitive task analysis, behavioral tasks decompose language learning, a CASL has developed a related quality control multi faceted and complicated cognitive language (QC) tool for assessing the quality of the summary task, into component processes that support it, such product. The QC tool's evaluation dimensions (e.g., as verbal working memory capacity, processing completeness, significance, accuracy) identify speed, and sound discrimination, in order to assess important elements for evaluation, engender a candidate's aptitude. Doughty and her team thorough quality assessment, and provide a common currently are validating the Hi-LAB battery for language for supervisors and LAs to discuss the operational use. reports. So what's going on inside Jane's head-the black box-is neither simple nor transparent. We For a better dictionary, build a often think of language analysis as simple, because better parser using language in our everyday lives proceeds When analysts have less-than-perfect so easily without conscious thought. However, language skills, finding words in the dictionary research shows that the problems faced by Jane on can be difficult due to the complicated structure, or the job are complex and difficult. CASL's research morphology, of many word forms, as well as the is intended to alleviate some of those problems and complexities of spelling in some languages. As a make Jane's job a little easier. ~ result, junior analysts working in morphologically complex languages like Pashto and Urdu may be spending too much time looking up unfamiliar words, making it difficult for them to arrive at a thorough understanding of the overall text. Anne David and her associates describe work that makes dictionary search both faster and more accurate by means of a computational tool called a morphological parser. Their innovative approach offers two improvements over traditional parser­ building methodologies: (1) it safeguards against software obsolescence thanks to grammatical formalisms that are written at a linguistic, rather than a computational, level; and (2) it provides a template for developing grammars - and hence parsers-that is easily portable to other languages, thus addressing a 30-year-old limitation.

The Next Wave • Vol 18 No 1 • 2009 5 A Working Memory Workout for Language Practitioners

he brain stops developing after childhood. At least that was the commonly held belief­ until recently. A person may continue to learn throughout life, but core cognitive abilities-such as the speed with which information can be processed, the ability to focus Tattention, the capacity for remembering and tracking information when distracted-supposedly remain fixed and even decline with age.

Scientific research is changing that perception, however, suggesting now that certain types of mental workouts, also known as "brain training," can actually improve core mental abilities and protect against cognitive decline. Just as exercising a muscle increases physical strength, exercising the mind can increase mental fitness in terms of how much information can be processed and maintained at once, and the ability to focus attention on a task. A thriving brain training industry is growing around this research. Brain training software is now available for video game systems and even cell phones [1 ]. The possibility of improving mental abilities into adulthood and even old age is indeed exciting, as is the prospect of mitigating the effects of clinical disorders such as Alzheimer's disease and attention deficit hyperactivity disorder-ADHD. But normal populations stand to benefit from cognitive training as well. Researchers at CASL are currently testing how intensive mental workouts benefit healthy participants and who is likely to gain most from such interventions. Within the Intelligence Community (IC), language analysts have been identified as likely candidates for brain training. Understanding language in one's first and foreign languages requires the temporary maintenance of linguistic input because words, phrases, and sentences are not spoken (and therefore not heard) all at once. Instead, they unfold sound by sound, word by word. The same is true for written material: we make successive eye fixations across text on a page as we read, rather than viewing an entire sentence all at once. Until additional information reaches the ears or the eyes, a listener, reader, or translator must hold provisional interpretations in mind so that newly encountered input can be updated and integrated into one's developing representations of sentence meaning. Thus, comprehension proceeds incrementally. Even under ideal listening or reading conditions, focusing attention is crucial for an accurate interpretation. Just a brief mental lapse can cause important information to be missed. Mental focus is even more important when interpreting material that is either incomplete or ambiguous, as this article explains later. And when details of a story unfold over long periods of time, or when the full account comes from multiple sources bit by bit, mental focus is all the more important. People differ naturally in their core cognitive abilities, and these differences have an impact on language processing and comprehension tasks. What is not clear, however, is whether performance on language tasks can be improved through a relevant brain training regimen that targets general cognitive abilities and, if so, under what conditions and by how much. Research at the University of Maryland Center for Advanced Study of Language [CASL) has targeted understanding core cognitive abilities. This research, particularly in memory and attentional control, applies to language practitioners and their efforts to understand a foreign language.

6 A Working Memory Workout for Language Practitioners FEATURE

Which mental resources perform a certain task (e.g., riding a bi- tral, limited-capacity resource that can be enable quick and efficient cycle) are all saved in long-term memory, voluntarily applied to holding and manip- information processing? not working memory. ulating information in memory [2-5]. The central aspect of attention indicates that The limitations of working ,Q RUGHU WR TXLFNO\ DQG HIÀFLHQWO\ the resource is shared between all mo- process information, we rely on the work- memory dalities (vision, hearing, etc.) and types ing memory system—that is, the atten- As we know from the experience of information coding (phonological, tion and memory systems that enable of our own memory limitations, working orthographic, spatial, etc.). The limited- what we commonly refer to as “multi- memory is time and capacity limited. For capacity aspect indicates that one type of tasking.” Working memory can also be example, many people have had the expe- manipulation or storage can be increased thought of as our mental workspace: the rience of almost immediately forgetting only at the expense of other types. small amount of memory that holds and the name of a new acquaintance. Clearly, The allocation of attention can be manipulates information for ongoing use. there are constraints on how much infor- This dynamic working memory system mation can be processed, manipulated, PRGLÀHGYROXQWDULO\DVLVLQGLFDWHGZKHQ participants adjust their attention alloca- guides behavior and allows us to main- and integrated effectively all at once. Al- tion according to variable instructions or tain conscious awareness of goal-relevant though everyone has had the experience payoffs. However, attention is not com- information. of losing (i.e. forgetting) information they At the same time, the working mem- had been holding in working memory SOHWHO\ YROXQWDU\ ÁDVKLQJ OLJKWV ORXG noises, motion (or “pop-ups”) on a web- ory system prevents potentially irrelevant (like your new acquaintance’s name), in- site, and sudden movements can invol- or distracting information from gaining dividuals vary in their working memory untarily grab attention away from an in- access to our consciousness. By deliber- capacity and in how susceptible they are tended task. Such a distraction can cause ately focusing or dividing attention, we to short-term forgetting. Furthermore, a lapse in attention and, by extension, a can pay attention, make and maintain people are not always aware of how divid- failure to remember critical components plans, and engage in goal-directed behav- ed attention affects their ability to process of the task that was being completed (e.g., ior. information from one task to another. For the name of your new acquaintance; the example, when a conversation is abruptly The working memory system is dif- context of the newspaper article you were interrupted by another event (a waiter ferent from the long-term memory system, just reading). which is the repository of information that asking to take your order), many people can be stored for days, weeks, and years. experience an inability to remember ex- The neuroscience of working Information such as our personal experi- actly what they had intended to say. memory ences (e.g., what we ate for lunch yester- Attentional control is important to The working memory system is day), facts (e.g., the date the Declaration the function of the working memory sys- located diffusely throughout the brain and of Independence was signed), and how to WHP+HUHZHGHÀQHDWWHQWLRQDVDFHQ- appears to rely mainly on the same brain

Testing attentional control: Flashing lights grab people’s attention, and advertisers often exploit this as a way to get us to look at their ads. It is also Pro-saccade Anti-saccade

used in laboratory tasks to measure attentional control. The anti-saccade Fixation task [6] is one such laboratory task. (A saccade is a quick eye movement that + + we make from one spot to another in order to look at an object.) In this task, Cue LQGLYLGXDOVDUHVKRZQDVFUHHQZLWKRQO\DVLQJOHÀ[DWLRQSRLQWLQWKHFHQWHU to look at (see Figure 1). Then an object suddenly appears on either the right or left side of the screen. Participants are instructed ahead of time to either Target look toward the new object (pro-saccade) or away from it (anti-saccade). The natural response is to look toward the newly appearing object because it captures our attention; looking in the other direction takes attentional control. People with greater working memory FDSDFLW\DUHEHWWHUDEOHWRYROXQWDULO\FRQWURODWWHQWLRQDQGGLUHFWLWDZD\IURPWKHÁDVKLQJREMHFW>@

Figure 1: Attentional capture in the pro-saccade and anti-saccade tasks. In the pro-saccade task, participants are always told to look toward the flashing light in anticipation of the red “X”, which flickers only briefly on the same side of the screen. But in the anti-saccade task they must look away from the flashing light in order not to miss the red “X”’s brief presentation on the opposite side of the screen. The flashing light captures attention, but resisting the urge to look in the direction of the red “X” in the anti-saccade task takes attentional control.

The Next Wave „ Vol 18 No 1 „ 2009 7 regionsDVJHQHUDOÁXLGLQWHOOLJHQFH7KH While you have probably heard of the greater working memory resources have control center of the working memory aerobic exercise spinning, you probably better attentional focus and better short- system is located in the prefrontal cortex have never heard of spanning, which is term memory; Individuals with fewer (PFC), the area at the front of the brain a term CASL researchers coined for their working memory resources are more eas- above the eyes. The primary function of new memory exercise program, designed ily distracted and are more likely to for- the PFC is to orchestrate, coordinate, and QRW IRU SK\VLFDO ÀWQHVV EXW IRU PHPRU\ get information they were trying to use. guide our thoughts and actions in concert ÀWQHVV &$6/ UHVHDUFKHUV DUH GHYHORS- 5HFDOOWKHSUHYLRXVH[DPSOHRIWKHGLIÀ- with current task goals and intentions. ing a working memory workout designed culty people sometimes have when trying Some individuals have greater control to increase working memory span—the to pick up a conversation following brief over their thoughts and actions than length of the information sequence one and unexpected interruptions. others. can hold in immediate memory—through CASL researchers hypothesize that Studies of patients with traumatic attentional control training. the proposed working memory/attention- brain injury to the PFC, Alzheimer’s de- CASL’s attention and working al control training program stimulates the mentia, Parkinson’s disease, and regular memory training program is designed to PFC. As the control center of the work- aging have shown that interruptions and stimulate brain activity and promote the ing memory system, the PFC is the brain impairment of the PFC make multi-task- sort of higher-level cognitive function- UHJLRQ UHVSRQVLEOH IRU IDVW DQG HIÀFLHQW ing required for text comprehension and working memory performance. CASL’s LQJ GLIÀFXOW RU LPSRVVLEOH ,Q D FODVVLF research scientists are challenging the example of this, Baddeley, Bressi, Della language analysis. Research in cognitive conventional wisdom that working mem- Sala, Logie, and Spinnler [8] compared psychology and cognitive neuroscience, ory capacity is innate and cannot be en- the dual-tasking performance of young including work done at CASL, suggests hanced through training. and elderly adults to that of patients with that the attentional processes that make A critical assumption of this work is $O]KHLPHU·V GHPHQWLD 7KH ÀUVW RI WZR multi-tasking possible may be critical for that cognitive abilities, and the underly- tasks tested participants’ memory for se- ÀUVW DQG IRUHLJQ ODQJXDJH WH[W DQDO\VLV ing brain structures that support them, are quences of digits. The second of the two and reading comprehension. Indeed, the amenable to change. CASL’s researchers tasks tested motor control and the ability same regions of PFC that are known to be hypothesize that engagement in cogni- to track a moving light with a hand-held involved in the control aspects of work- tively demanding tasks leads to functional stylus. The Alzheimer’s patients and the ing memory are also involved in language changes in the neurological pathways that young and elderly adults were equally processing tasks. Brain-imaging studies have demonstrated an overlap in the loca- VXVWDLQ HIIHFWLYH DQG HIÀFLHQW ZRUNLQJ good in their performance when these memory processes. tasks were completed separately. How- tion of brain activity when healthy adults One of the most important recent ever, only the Alzheimer’s patients were perform anti-saccade-like tasks (Figure 1) and certain types of language processing ÀQGLQJV WR HPHUJH IURP WKH FRJQLWLYH VLJQLÀFDQWO\LPSDLUHGZKHQWKH\WULHGWR neuroscience literature is that of neural combine the tasks by remembering digits tasks that are complex or ambiguous— and cognitive plasticity in adults [9,10]. while tracking the light at the same time. tasks that also require attentional control (see examples that follow). Likewise, pa- Previously, scientists believed only chil- &RQVLVWHQW ZLWK SUHYLRXV ÀQGLQJV tients with brain lesions to the PFC ex- dren had such plasticity. However, re- the Alzheimer’s patients had some unim- hibit a failure to correctly perform work- cently scientists have found that adults’ paired memory and motor function, but ing memory tasks that require attention neurological pathways and structures also their dual-tasking ability was severely appear to change in response to repeated control (like the anti-saccade task) as compromised. Baddeley et al.’s results engagement in particular tasks [11-13]. well as a failure to correctly understand suggest that age does not necessarily in- Importantly, changes in neural structure grammatically complex or ambiguous can result from the day-to-day engage- ÁXHQFHWKHFDSDFLW\IRUGXDOWDVNLQJEXW sentences. that the loss of neurons and synapses in ment in routine activities or through ex- cortical and sub-cortical regions associat- How do working memory training tensive training or practice [14-19]. ed with Alzheimer’s dementia does have programs work? CASL’s working memory workout is DVLJQLÀFDQWLPSDct. Working memory training pro- still in development, so we cannot convey grams capitalize on the brain’s enormous all the details of the research in this ar- CASL’s working memory capacity to adapt to changes in the envi- ticle. However, we can mention that the workout ronment. People differ in their working program consists of multiple attention Whether sweatin’ to the oldies or memory abilities, and these differences and memory exercises, some of which are tackling Tae-Bo, science has long extolled have a direct impact on their daily lives based on traditional behavioral laboratory WKH EHQHÀWV RI SK\VLFDO ÀWQess training. and job performance. Individuals with measures of working memory capacity.

8 A Working Memory Workout for Language Practitioners FEATURE

ing memory capacity, people may also be item able to improve their performance in more set general areas that rely on attention and working memory, including language. Al- + Is (7x2) + 4=20 ? Dog Is (9÷3) + 3=6 ? Smile Is (18÷6) + 1= 5 ? Wind ? though prior work has shown that cogni- tive training can improve performance on ... psychometric tests of ability, no work has Trial Onset 1st display 2nd display nth display Recall Cue tested the hypothesis that it results in real gains in real-time language comprehen- Figure 2. Graphical depiction of an n-item set in the operation span task, where n typically ranges from 2 to 6. At trial onset the participant reads the string aloud sion or the interpretations readers assign beginning at “is.” At “?” in the string, the participant says “yes” if the equation is to sentences. correct or “no” if it is incorrect. The participant reads aloud and retains each word until recall is prompted. Why is working memory training relevant to language? Language processing, comprehen- How do researchers develop working-memory training task, research- sion, and translation require attention. working memory training tasks? ers mXVWPDNHVHYHUDOPRGLÀFDWLRQV7KH During reading and spoken language pro- Researchers rely on traditional lab- WDVNQHHGVWRSURJUHVVLQGLIÀFXOW\IURP cessing, readers and listeners have to ex- oratory measures of working memory to HDV\ WR GLIÀFXOW DGDSWLQJ G\QDPLFDOO\ tract the meaning of the individual words develop working-memory training tasks. as an individual’s performance improves they encounter, and also package them A group of tasks known collectively as (i.e., as their span increases), and the task into phrases to form a coherent sentence complex span tasks are typically used to needs to offer users feedback on their per- that accurately reveals who is doing what measure verbal working memory span. formance. Repeated practice at regular to whom, that is, the general content of Complex span tasks are tasks that actu- intervals is also critical. ally involve performing two separate what the writer or speaker is intending to Does working memory training tasks concurrently, combining a memory communicate. improve performance in real- requirement with an attentional control Many factors can adversely affect world tasks? requirement. reading comprehension, including textual A commonly used complex span Yes. Recent work in WKH ÀHOG RI inconsistencies within and across sen- task is the operation span task (O-Span) cognitive neuroscience demonstrates tences; complex or unusual vocabulary [20]. Figure 2 depicts the traditional O- the effectiveness of extensive attention, within the text; the reader’s unfamiliarity Span task for a single O-Span item (top) perception, and memory training for im- with the subject matter; infrequent words and a single trial (bottom). Each item con- proving overall cognitive functioning in or unexpected propositions; and the read- sists of a mathematical equation paired both children and adults. For example, er’s cognitive skills, including memory with a to-be-remembered word. Each SHUIRUPDQFHRQWHVWVRIJHQHUDOÁXLGLQ- capacity and attentional control. These trial consists of a series of such items. telligence, as well as on mathematical factors affect reading comprehension in A trial begins with the presentation of a reasoning, has been shown to increase DSHUVRQ·VÀUVWand second language. (compared to relevant control conditions) À[DWLRQSRLQWIROORZHGE\DPDWKHPDWLFDO Prior research has found that work- equation and the to-be-remembered word. following as little as 10 to 20 hours of ing memory supports the comprehension For each item, the participant reads the cognitive training [21]. process of resolving ambiguities in text, equation aloud, indicates if the equation Strikingly, cognitive training has which arise routinely during language is correct or incorrect by saying yes or no, been shown to reverse age-related de- interpretation. Evidence from language and then reads the word aloud. After read- clines in cognitive performance, decrease processing studies indicates that readers ing the word, the experimenter advances automobile accidents in elderly adults, and listeners interpret sentence mean- to the next equation-word pair, and the and even reduce behavioral symptoms ing in real-time as words and phrases procedure repeats. At the end of the trial, associated with childhood ADHD. These are perceived. In other words, readers the participant receives a cue to recall all UHVXOWVDQGRWKHUVVSHDNWRWKHHIÀFDF\ and listeners do not wait until the end of the words in serial order; the number of of cognitive training for improving func- an utterance to begin assigning an inter- words correctly reported reveals the par- tioning across a wide variety of tasks and pretation to what they just read or heard ticipant’s span. across a wide variety of individuals. [22-26]. While economical, one impor- To transform a behavioral labora- Further, these results suggest that by tant consequence of real-time processing tory task such as the O-Span task into a increasing attention allocation and work- is that it routinely comes at the cost of

The Next Wave „ Vol 18 No 1 „ 2009 9 having to deal with temporary ambigu- WRFROOHFWÀQHJUDLQUHDOWLPHGDWDDERXW encounters complex or ambiguous mate- ity. Consider for instance sentence (A): ZKHUHLQDVHQWHQFHGLIÀFXOW\LVHQFRXQ- rial in her foreign language to translate, tered (and how eye movement patterns such as the example in example A, and $ 7KHDJHQWVÀQDOO\ change after working memory training, thus has to rely on working memory and LHKRZUHDOWLPHSURFHVVLQJGLIÀFXOW\LV attentional control. A lapse in attention discovered the crucial alleviated). may cause a foreign language practitioner evidence could not be Moreover, under such ambiguous WRDUULYHÀUVWDWWKHPRVWUHOLDEOHRUIUH- conditions, readers tend to make more quent interpretation of sentence meaning located with GPS. errors when answering comprehension (that the evidence was discovered), but questions that probe whether initially not override it (e.g., in example A) above, As the words in example A are en- misunderstood elements of an ambiguous WUDQVODWLQJWKHWH[WWRUHÁHFWWKDWFHUWDLQ countered, a reader begins to interpret the SKUDVH OLQJHU E\ FRQWLQXLQJ WR LQÁXHQFH evidence was discovered when indeed it input in a way that is consistent with his readers [28]. For instance, when asked was not. or her prior knowledge of the verb dis- questions such as, “Did the agents dis- A large-scale, intensive training covered, namely that it is highly likely cover the evidence?” many people (with regimen that targets the attentional con- to appear with some discoverable object, lower working memory capacity) will an- trol aspects of working memory may UHÁHFWLQJWKDWVRPHWKLQJZDVLQGHHGGLV- swer yes, even though the correct answer therefore have important effects on real- covered (e.g., The agents discovered the is no. Thus, the incremental processing of time language processing and the ulti- evidence and sent it to the lab.). However, sentences can place high demands on the mate resolution of ambiguity at multiple VXFK NQRZOHGJH LV LQVXIÀFLHQW WR EH DQ language processing and attentional con- levels. We predict that after cognitive exact guide to how the rest of the sentence trol systems, especially when the initial training, readers will experience less on- will unfold, as illustrated in (example A), interpretation of a temporary ambiguity OLQH UHDGLQJ GLIÀFXOW\ DQG IHZHU RYHUDOO where new input, could not be located must be overridden in light of later arriv- comprehension errors linked to complex with GPS, is encountered and unambigu- ing linguistic material. or ambiguous language—in other words, ously signals that the agents actually did The ability to deal with ambiguous we predict that the effects of complexity not discover the evidence! In other words, and complex language has long been tied and ambiguity will diminish. Streamlin- at the point in the sentence when readers to individuals’ working memory capacity ing reading processes by “greasing” the encounter the evidence, the sentence is and attentional control abilities [28-32]. cognitive wheels that support them has temporarily ambiguous. The kind of ambiguity resolution scenario obvious implications for language and in- When encountering later-arriving described in example A and the cognitive telligence analysts. information—could not be located—read- processes needed to resolve it have also Conclusion HUV WHQG WR H[KLELW SURFHVVLQJ GLIÀFXOW\ been equated with those needed to suc- UHÁHFWHG E\ HOHYDWHG UHDGLQJ WLPHV DQG cessfully complete general non-linguistic The study of language comprehen- or several re-reads, presumably because attentional control tasks (e.g., the dif- sion has a long and rich history, and much they have to “slam on the brakes”—they is known about how people process lin- ÀFXOW\PRVWSHRSOHKDYHZKHQGLUHFWLQJ must override their initial interpretation their gaze in the opposite direction of an guistic information. However, despite the of the evidence as something the agents attention-grabbing image; see Figure 1). many theoretical and empirical advances discovered, and instead recover an alter- Neurologically and cognitively speaking, that have been made, relatively little work native meaning that allows the evidence working memory, ambiguity resolution, has been devoted to developing training to refer to something that could not in fact and non-linguistic attentional control ap- SURFHGXUHV WKDW LPSURYH ÀUVW DQG IRU- EHORFDWHG>@7KLVSURFHVVLQJGLIÀFXOW\ pear to engage many of the same neural eign language processing. By increas- is known as the garden-path effect: read- systems within the PFC [33-36]. ing trainees’ working memory capacity ers (or listeners) are led down one path and attentional control through CASL’s of interpretation in light of strong devel- So, working memory helps spanning workout, language practitioners oping information that points toward one readers and listeners avoid may be able to improve their analysis misleading assumptions? OLNHO\LQWHUSUHWDWLRQRQO\WRÀQGWKDWWKLV performance, particularly when texts are initial interpretation commitment is ulti- A failure to consider alternative sen- corrupt, incomplete, ambiguous, or other- mately wrong once new information is tence meanings and fully resolve ambi- ZLVHGLIÀFXOWWRSURFHVV%HFDXVHDQDO\VWV encountered. CASL researchers are cur- guities in real-time could result in a mis- typically work with materials that are rently recording readers’ eye movements characterization of sentence meaning that QRW LQ WKHLU ÀUVW ODQJXDJH GHDOLQJ ZLWK as they make their way through ambigu- ultimately has important implications. these challenges is especially hard and ous sentences of the garden-path variety, Consider: a foreign language practitioner SODFHVVLJQLÀFDQWEXUGHQVRQWKHLUZRUN-

10 A Working Memory Workout for Language Practitioners FEATURE

If you think the structure of the universe is complex, consider how the interconnectedness of billions of neurons self-organize to produce a single spoken word, a sentence, or a lengthy conversation. Language comprehension is a multifaceted and complex process. It requires access to a mental lexicon (the repository of words stored in your mind) and implicit knowledge of syntactic rules so we know how to correctly package words and phrases WRJHWKHULQRUGHUWRÀJXUHRXWZKRLVGRLQJZKDWWRZKRPLQDVHQWHQFH7KHZRUNLQJPHPRU\V\VWHPLVRQO\RQHVPDOOSLHFHRI this process, but it is a crucial piece. Example A 7KHDJHQWVÀQDOO\GLVFRYHUHGWKHFUXFLDOHYLGHQFHFRXOGQRWEHORFDWHGZLWK*36 LVMXVWRQHRIPDQ\WKDWLOOXVWUDWHWKHQHHGIRUFDUHIXODWWHQWLRQDQGZRUNLQJPHPRU\VRDVQRWWRH[SHULHQFHUHDGLQJGLIÀFXOW\RU ultimately misinterpret a sentence. Working memory and attentional control also help readers maintain information across long spans within (and across) sentences. Consider the following: The little girl wearing a green hat and grey coat, who was sitting next to an old lady with a tiny umbrella, fell asleep on the train. A reader here has to hold the little girl in memory for a long time before he or she gets to know what happened—that she fell asleep on the train. Having a smaller working memory span and attentional control can cause someone to quickly forget (or at least be confused about) the subject of the sentence (the little girl rather than the old lady). Thus, the reader with lower working-memory capacity may be required to re-read or backtrack, which will result in a slower comprehension process and a greater susceptibility to erroneous interpretation (e.g., thinking that the old lady fell asleep, due to the physical proximity of the phrase an old lady with a tiny umbrella to the phrase fell asleep on the train). In addition, having a smaller working memory capacity can also impede the rate of learning a new language, delay reading time, decrease comprehension, and lead to errors in linguistic inferences. Low working memory capacity has been implicated as a reason for reduced comprehension of metaphoric expressions such as the masked hijackers were politicians>@LQZKLFKWKH literal meaning of a sentence cannot be taken at face value. With metaphoric expressions, readers must suppress the literal meaning to promote an alternative, contextually relevant meaning (perhaps that the hijackers were not really senators, for instance, but rather maintained politician-like qualities including negotiation, posturing, etc.). Misinterpreting such a phrase could have negative consequences. Because readers and listeners, including foreign language SUDFWLWLRQHUVIUHTXHQWO\GHDOZLWKWKLVW\SHRIPDWHULDOWKHSV\FKROLQJXLVWLFÀQGLQJWKDWUHDGHUVZLWKORZHUZRUNLQJPHPRU\ capacities tend to provide incorrect analyses of metaphoric expressions ought to inform how we think about improving language DQDO\VLV7KLVÀQGLQJLVHVSHFLDOO\UHOHYDQWJLYHQWKDWSURFHVVLQJDQGFRPSUHKHQGLQJLQSXWLQVRPHRQH·VVHFRQGODQJXDJHWD[HVWKDW SHUVRQ·VZRUNLQJPHPRU\PRUHWKDQÀUVWODQJXDJHSURFHVVLQJHVSHFLDOO\DWORZHUSURÀFLHQF\OHYHOV³XQGHUVWDQGLQJLQVXFKFDVHV is less automatic. We believe, therefore, that training working memory is fertile experimental ground. Attentional control and working memory are also required for many other aspects of language use and comprehension. Code switching—a term linguists use to describe the use of two or more languages in a conversation—is a prime example of having to control the shifting of attention among multiple tasks. Likewise, translation and interpretation from one language to another requires engaging multiple languages simultaneously. 2WKHU UHVHDUFK VKRZV WKDW ZRUNLQJ PHPRU\ FDSDFLW\ FRYDULHV ZLWK WKH UDWHV RI ÀUVW DQG IRUHLJQ ODQJXDJH YRFDEXODU\ OHDUQLQJZLWKUHDGLQJDQGOLVWHQLQJFRPSUHKHQVLRQDQGZLWKZULWLQJSURÀFLHQF\>@,QYRFDEXODU\OHDUQLQJIRUH[DPSOH working memory makes it possible to hold a newly learned word in memory while learning to associate it with known words and meanings. Taken together, prior work has demonstrated a compelling correspondence between working memory ability and the ability to engage in successful language understanding at multiple levels, including resolving ambiguity and mitigating against its OLQJHULQJHIIHFWVPHWDSKRUFRPSUHKHQVLRQDQGVZLWFKLQJEHWZHHQRQH·VÀUVWDQGIRUHLJQODQJXDJHZLWKLQDVLQJOHFRQYHUVDWLRQ Consequently, pairing this research with the research on working memory training may have important implications: enhancing the working memory system through adaptive training may have valuable applied outcomes for language comprehension, including improved performance on tasks of ambiguity, metaphor generation and comprehension, and code switching—all tasks routinely undertaken by foreign language practitioners.

The Next Wave „ Vol 18 No 1 „ 2009 11 ing memory and attentional control sys- tems. By training this system—which is critically relevant for resolving complex, incomplete, or ambiguous language— CASL researchers hope to improve the performance of language practitioners.

References >@ &ODUNH 7 %UDLQ ÀWQHVV VHHQ DV KRW industry of the future. Reuters [Internet].

2008 Mar 12. Available from: http://www. reuters.com/article/technologyNews/idUSN 1218668920080312?feedType=RSS&feedN ame=technologyNews&pageNumber=2&vir tualBrandChannel=0&sp=true [2] Baddeley AD, Hitch GJ. Working mem- ory. In: Bower GA, editor. Recent advances in learning and motivation. New York (NY): $FDGHPLF3UHVVYROS        [3] Cowan N. Working memory capacity. New York (NY): Psychology Press; 2005. [4] Engle RW, Kane MJ, Tuholski SW. Individual differences in working memory capacity and what they tell us about controlled DWWHQWLRQ JHQHUDO ÁXLG LQWHOOLJHQFH DQG functions of the prefrontal cortex. In: Miyake A, Shah P, editors. Models of working memory: Mechanisms of active maintenance and executive control. New York (NY): Cam- bridge University Press; 1999. p. 102-134.

[5] Kane MJ, Hambrick DZ, Tuholski SW, Wilhelm O, Payne TW, Engle RW. The generality of working memory capacity: A latent-variable approach to verbal and visuospatial memory span and reasoning. Journal of Experimental Psychology: *HQHUDO [6] Everling S, Fischer B. The antisaccade:         A review of basic research and clinical studies. Neuropsychologia.1998;36:885-899.

>@ 8QVZRUWK 1 6FKURFN -& (QJOH 5: Working memory capacity and the antisaccade      task: Individual differences in voluntary saccade control. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2004;30:1302-1321.       [8]

12 A Working Memory Workout for Language Practitioners FEATURE

working memory in Alzheimer’s disease: A 0HPRU\DQG/DQJXDJH Working memory constraints on the resolution longitudinal study. Brain. 1991;114:2521- [21] Jaeggi SM, Buschkuehl M, Jonides J, of lexical ambiguity: Maintaining multiple 2542. interpretations in neutral contexts. Journal of 3HUULJ:-,PSURYLQJÁXLGLQWHOOLJHQFHZLWK [9] Greenwood PM. Functional plasticity training on working memory. Proceedings 0HPRU\DQG/DQJXDJH in cognitive aging: Review and hypothesis. of the National Academy of Sciences, USA; [33] Fiebach CJ, Vos SH, Friederici AD. 1HXURSV\FKRORJ\ 2008. vol. 105, p. 6829-6833. Neural correlates of syntactic ambiguity [10] Mercado E 3rd. Neural and cognitive [22] Altmann GTM, Kamide Y. Incremental in sentence comprehension for low and plasticity: From maps to minds. Psychological interpretation at verbs: Restricting the high span readers. Journal of Cognitive %XOOHWLQ domain of subsequent reference. Cognition. 1HXURVFLHQFH [11] Huttenlocher PR. Synaptogenesis  [34] Mason RA, Just MA, Keller TA, in human cerebral cortex. In: Dawson G, [23] Ferreira F, Clifton C. The independence Carpenter PA. Ambiguity in the brain: What Fischer KW, editors. Human behavior and the of syntactic processing. Journal of Memory brain imaging reveals about the processing of syntactically ambiguous sentences. Journal developing brain. New York (NY): Guilford DQG/DQJXDJH of Experimental Psychology: Learning, 3UHVVS [24] Rayner K, Carlson M, Frazier L. The Memory, & Cognition. 2003;29:1319-1338. [12] Katz LC, Shatz CJ. Synaptic activity interaction of syntax and semantics during [35] Ye Z, Zhou X. Involvement of cognitive and the construction of cortical circuits. sentence processing: Eye movements in the control in sentence comprehension: Evidence analysis of semantically biased sentences. 6FLHQFH from ERPs. Brain Research. 2008;1203:103- Journal of Verbal Learning and Verbal [13] Kolb B, Whishaw IQ. Brain plasticity 115. and behavior. Annual Review of Psychology. %HKDYLRU [36] Novick JM, Trueswell JC, Thompson- 1998;49:43-64. [25] Trueswell JC, Tanenhaus MK, Garnsey Schill SL. Cognitive control and parsing: [14] Garlick D. Understanding the nature of 60 6HPDQWLF LQÁXHQFHV RQ SDUVLQJ 8VH Reexamining the role of Broca’s area of thematic role information in syntactic the general factor of intelligence: The role of in sentence comprehension. Cognitive, ambiguity resolution. Journal of Memory individual differences in neural plasticity as Affective, & Behavioral Neuroscience. and Language. 1994;33:285-318. an explanatory mechanism. Psychological 2005;5:263-281. Review. 2002;109:116-136. [26] Van Berkum JJA, Brown CM, Zwitserlood P, Kooijman V, Hagoort P. >@ &KLDSSH '/ &KLDSSH 3 7KH UROH RI [15] Garlick D. Integrating brain science working memory in metaphor production Anticipating upcoming words in discourse: research with intelligence research. Current and comprehension. Journal of Memory and Evidence from ERPs and reading times. Journal Directions in Psychological Science. /DQJXDJH 2003;12:185-189. of Experimental Psychology: Learning, Memory and Cognition. 2005;31:443-466. [38] Atkins PWB, Baddeley AD. Working [16] Nyberg L, Sandblom J, Jones S, memory and distributed vocabulary learning. >@ *DUQVH\ 60 3HDUOPXWWHU 1- 0\HUV Stigsdotter Neely A, Petersson KM, Ingvar $SSOLHG3V\FKROLQJXLVWLFV  M, Backman L. Neural correlates of training- E, Lotocky MA. The contributions of verb [39] Baddeley AD. Working memory and related memory improvement in adulthood bias and plausibility to the comprehension of language: An overview. Journal of Comm- and aging. Proceedings of the National temporarily ambiguous sentences. Journal of unication Disorders. 2002;36:189-208. Academy of Sciences, USA; 2003. vol. 100, 0HPRU\DQG/DQJXDJH S [28] Christianson K, Williams CC, Zacks [40] Daneman M, Hannon B. What do RT, Ferreira F. Younger and older adults’ working memory span tasks like reading >@ 0D\$+DMDN**DQ‰EDXHU66WHIIHQV span really measure? In: Osaka N, Logie RH, T, Langguth B, Kleinjung T, Eichhammer P. “good-enough” interpretations of garden-path D’Esposito M, editors. The cognitive neuro- Structural brain alterations following 5 days sentences. Discourse Processes. 2006;42:205- science of working memory. New York (NY): of intervention: Dynamic aspects of neuro- 238. 2[IRUG8QLYHUVLW\3UHVVS SODVWLFLW\&HUHEUDO&RUWH[  [29] Daneman M, Carpenter PA. Individual [41] Engle RW. What is working-memory [18] Olesen P, Westerberg H, Klingberg differences in working memory and reading. capacity? In: Roediger HL 3rd, Nairne JS, T. Increased prefrontal and parietal activity Journal of Verbal Learning & Verbal editors. The nature of remembering: Essays after training of working memory. Nature Behavior. 1980;19:450-466. in honor of Robert G. Crowder. Washington [30] Fedorenko E, Gibson E, Rohde D. 1HXURVFLHQFH (DC): American Psychological Association; The nature of working memory capacity [19] Temple E, Deutsch GK, Poldrack RA, S Miller SL, Tallal P, Merzenich MM, Gabrieli in sentence comprehension: Evidence -'(1HXUDOGHÀFLWVLQFKLOGUHQZLWKG\VOH[LD DJDLQVW GRPDLQVSHFLÀF ZRUNLQJ PHPRU\ Further Reading ameliorated by behavioral remediation: resources. Journal of Memory and Language. 2006;54:541-553. Rakauskas ME, Ward NJ, Boer ER, Bernat Evidence from functional MRI. Proceedings EM, Cadwallader M, Patrick C J. Combined of the National Academy of Sciences, USA; [31] Kemper S, Liu C. Eye movements effects of alcohol and distraction on 2003. vol. 100, p. 2860-2865. of young and older adults during reading. driving performance. Accident Analysis & [20] Turner ML, Engle RW. Is working 3V\FKRORJ\DQG$JLQJ 3UHYHQWLRQ memory capacity task dependent? Journal of [32] Miyake A, Just MA, Carpenter PA.

The Next Wave „ Vol 18 No 1 „ 2009 13 "Thinking Out of the Box" Through Cognitive Neuroscience

magine you are interpreting an utterance in a foreign language. The utterance contains an ambiguous word-let's pretend the word in the foreign language is dax-and you can only I glean the gist of what the sentence means: "onto an airplane, load kitchen and food supplies in the dax." You ask yourself, what does dax mean in this context?

You recognize that dax usually means "afternoon." However, dax can also mean "rear." Both interpretations are plausible in this case. The linguistic context that preceded this utterance might have pointed toward the location meaning; how­ ever, that part of the communication is missing.

Now, imagine that you, a foreign language practitioner, select the "afternoon" interpretation because it is the more common usage of dax. Failure to con­ sider the alternative meaning-"rear"-could lead the intelligence analyst who uses your translation to draw an incorrect and possibly perilous as­ sessment. Providing scenarios that account for all possible interpreta­ tions of ambiguous terms requires foreign language practitioners and intelligence analysts to apply their best divergent thinking skills to the task.

Divergent thinking [OT] is the cognitive ability to think of as many useful solutions as possible for open-ended problems. These solutions include both common solutions that come easily to mind, and less common ones that require flexible, creative thought. As the opening scenario illustrates, foreign language practitioners and analysts require good OT skills because the problems of interpretation they encounter are often open-ended. Linguistic and contextual ambiguities, combined with incomplete transcripts, make it difficult to provide accurate interpretations.

Other factors might contribute to the interpretive challenge, as well. For example, communications can be intentionally distorted. In the opening scenario kitchen and food supplies could be euphemisms for dangerous and illegal materials. Relevant conditions such as weather or personnel on the scene might also be in flux. If the linguist translating the utterance in the example had known that a storm was predicted to clear soon, afternoon might have seemed the better word choice [1 ].

Researchers at the University of Maryland Center for Advanced Study of Lan­ guage [CASL] are looking into ways to improve OT In this article we discuss the brain and cognitive mechanisms involved in OT We then describe CASL.:s research efforts to improve analysts' OT using two methods: (1) alpha neurofeedback training and (2) changing mental set. FEATURE

Brain and cognitive mechanisms Cognitive neuroscience is aimed at Delta ( 1-4 oyores. per second) understanding the neurobiological bases of human thought processes, including problem solving and attention. Recently, a panel from the National Academy of Sciences concluded that cognitive neu­ Theta (4-8 cycles per seoond) roscience methods are crucial for both assessing and inducing brain states that reflect expert performance of Intelligence Community (IC) and military profession­ als [2]. Two advances in cognitive neu­ Alpha (8-12 cycles per second) roscience have proven to be particularly important for our understanding of the neurocognitive mechanisms underlying creative problem solving, including DT ability: (1) recent insights about the kinds Beta (12-28 cycles pe:r second) of brain waves associated with good DT and (2) elucidation of the role of cognitive control mechanisms in the frontal parts of the brain, near the forehead.

Brain waves and DT Gamma (28-90 cycles per second) Alpha brain waves have been as­ sociated with DT. Brain activity varies in intensity and, like radio waves, can be analyzed in terms of signals in various frequency ranges (Figure 1). 0 Bra in wave cycles per second 1 Alpha waves are regular sinusoidal brain waves. When a person is in a calm Figure 1 : Brain waves and alert state of mind, alpha waves are more prominent than theta or beta waves. in the environment. difficulty by including solution words that Hans Berger, the discoverer of the Recent cognitive neuroscience stud­ have a relatively weak association with human EEG (electroencephalograph), ob­ ies have found evidence to support the each of their three related stimulus words. served that when a person closes his eyes, hypothesis that a problem solver in an In the test a participant might be given the alpha waves become easily visible in re­ elevated alpha state is much more recep­ stimulus words shoe, car, and French, and cordings focused over the vision centers tive to becoming aware of his inner trains have to come up with horn for the solu­ in the back of the brain. However, under of thought. This heightened awareness tion word. Immediately after answering, certain conditions researchers also see extends to weakly associated thoughts, the participant reports whether he arrived an increase in alpha brain wave activity which are otherwise difficult to detect. at his answer more with insight-defined when subjects have their eyes open. Three Conceiving such remotely associated as a sudden awareness of the solution-or states in which a person's alpha wave ac­ thoughts, or weak associates, is central to sheer mental processing effort. tivity is elevated are (1) while trying to DT. Just before participants solved a ignore a visible distracting stimulus, (2) Jung-Beeman et al. [3] tested par­ problem with insight, the researchers ob­ while attending to an imagined stimulus, ticipants in EEG and fMRI (functional served an increase in alpha waves over the and (3) while holding information in the magnetic resonance imaging) studies for visual-spatial regions in the back of the part of memory that stores word mean­ compound remote associates (CRA). Dur­ right brain (specifically over a sensor lo­ ing. What these different conditions have ing the CRA test, participants must think cation known as P08). Heightened alpha in common is an inward direction of at­ of a word that forms a compound word or states were followed by a burst of gamma tention that frees the mind from actively phrase when associated with each of three waves. Jung-Beeman and colleagues rea­ processing potentially distracting stimuli other words. The test is designed to add soned that alpha waves reflected a recep-

The Next Wave • Vol 1 8 No 1 • 2009 15 tive state in which the visual-spatial part nating inWKHSUHIURQWDOFRUWH[H[HFXWLYH mental rotation tasks in healthy adults [8]. of the brain was not overwhelmed with control processes enlist other parts of the ,QWKHVHVWXGLHVWKH17ZDVDLPHGDWLQ- processing potentially distracting stimuli, brain, including the areas for memory and creasing brain waves in frequency ranges thus making linked regions more recep- language, in the service of a person’s im- that were believed to be important for the tive to detect the weakly associated an- mediate goals [6]. Attention control plays assessed cognitive function. swer word. The subsequent gamma wave a crucial role in assigning meaning for di- *LYHQWKHÀQGLQJRIDQDVVRFLDWLRQ ZDV K\SRWKHVL]HG WR UHÁHFW WKH VXGGHQ recting these cognitive resources. Atten- between increased alpha brain waves and emergence of the insight solutions into tion control allows the brain to override good creative problem solving over brain awareness. biased responses in the face of ambiguity region PO8 [3,4], we hypothesize that NT Further evidence for a role of alpha and reconcile competing alternatives. aimed at teaching people how to increase brain waves in creative problem solving We believe that this meaning-based their alpha brain waves over the same was reported in an EEG study on DT by attention control is crucial for select- brain region should improve DT. Recall Grabner, Fink, and Neubauer [4]. Partici- ing weak associates over dominant ones that PO8 is a region over the right pos- pants in the study were given an alternate and thereby promotes terior part of the brain. XVHV $8 WDVNLQZKLFKWKH\KDGWRJHQ- the production of useful, CASL research aimed A related reason for erate as many uses for a given object as original solutions during increasing alpha brain they could think of. The subjects were en- DT. The same control at improving analysts’ waves over a portion of couraged to focus on creative and original system likely helps ana- divergent thinking WKH ULJKW EXW QRW OHIW  DSSOLFDWLRQV )RU H[DPSOH LQ UHVSRQVH lysts think of uncommon hemisphere of the brain to the test word brick a participant might interpretations. However, attention con- is that the right hemisphere is superior answer paper weight, door stop, weapon, WUROKDVDOLPLWHGFDSDFLW\,IWKHFRJQLWLYH LQ ÀQGLQJ FRQQHFWLRQV DPRQJ XQUHODWHG and so forth. Just before the participant demands of a task surpass the capacity ideas, which is one of the hallmarks of was ready to give another answer, he for attention, then not enough attention creative thinking. This capability prob- would press a button. After the test, the resources remain to cope with effortful DEO\VWHPVIURPWZRVRXUFHV  WKHULJKW participant judged on a rating scalea the meaning selection. As a result, weak se- hemisphere’s ability to detect overlapping originality of his answers. mantic associates may be overlooked in meanings for words that are not strongly The study found that alpha activity favor of stronger associates, which can DVVRFLDWHGDQG  LWVDELOLW\WRLQWHJUDWH over the right visual-spatial regions of the lead to incorrect translations and false in- word meanings that have been derived EUDLQ LQFOXGLQJ32 ZDVJUHDWHUZKHQ terpretations. IURPGLIIHUHQWFRQWH[WV participants generated ideas they rated as $VDÀUVWVWHSLQRXU17UHVHDUFK more original. Results from the AU study CASL research aimed at improving CASL recently completed a study with were found to be similar to those obtained analysts’ divergent thinking university students that demonstrates an by Jung-Beeman and colleagues in terms effective NT method for increasing al- Alpha neurofeedback training of brain region and cognitive process. pha brain waves over PO8 within a single Weak meaning associates in the CRA test Alpha QHXURIHHGEDFNWUDLQLQJ 17  training session [9]. Participants received and original answers on the AU test both is seen as one way to improve analysts’ three NT sessions of about half an hour had to compete with more dominant an- '7,Q17EUDLQZDYHVDUHFRQWLQXRXVO\ each. The feedback signal for the test con- swers for conscious access, indicating a monitored with one or more electrode sen- sisted of a moving bar on a display that relationship between alpha brain waves sors placed, noninvasively, on the trainee’s rose higher as alpha brainwaves grew in and receptivity to detecting less obvious VFDOS,QIRUPDWLRQDERXWZKHWKHUWDUJHWHG intensity. Trainees were instructed to try answers. The observed increase in alpha brain waves are increasing or decreasing to increase the height of the feedback bar activity is hypothesized to be due to the is continuously fed back to the learner. as much as possible during 2.5-minute subjects’ inwardly turned attention during Using this feedback information, the training segments. There were also one- an alert, distraction-free state. trainee can learn to increase or decrease minute baseline segments during which brain waves in selected frequency ranges, the bar did not move and no feedback was Executive control including alpha brain waves. Previous given. ([HFXWLYHFRQWUROSURFesses enable studies have shown that NT improves We hypothesized that this alpha goal-directed thought and action. Origi- performance on working memory [7] and brain wave training would improve the

a Originality can also be determined more objectively as the inverse of the frequency with which a useful idea is generated among a large group of problem solvers [5]

16 “Thinking Out of the Box” Through Cognitive Neuroscience FEATURE

trainees' ability to block out the alpha-de­ be shown a small set of common solu­ Incubation creasing effect of a moving visual stimu­ tions. The participants will be instructed Incubation will be a critical ma­ lus, allowing brain regions necessary for that they cannot generate solutions from nipulation in our next study with alpha creative problem solving to become more this set of well known solutions. We hy­ NT. Incubation occurs during a period sensitive to internal associations. The ma­ pothesize that this experimental manipu­ of time that a person has been distracted jor finding of the study was that our alpha lation will induce a state of mental fixa­ from a task before returning to the prob­ NT protocol allowed most trainees - ten tion, in which subjects will find it difficult lem. The premise underlying incubation of thirteen - to learn to increase the inten­ to think of uncommon solutions in addi­ is that when mental fixation occurs, peo­ sity of their alpha brain waves over P08 tion to the common ones already provid­ ple solve problems better after some time in just a single session. Twelve of the thir­ ed. These well-known solutions should has elapsed. This expected improvement teen trainees showed improvement over­ make the participants think of closely re­ in problem solving is due to the redirec­ all. Participants in a no-neurofeedback lated solutions that merely reinforce their tion of attention to something else. Redi­ control group did not increase their alpha awareness of the common solutions. rection helps reduce the influence of the levels within a session. See Figure 2 for Mental fixation is likely to arise mental state that caused the mental fixa­ study results. frequently in the workplace during both tion, making it easier to access the weak As the next step in our research, individual and group brainstorming, es­ thought associations required for uncom­ we will use our alpha NT protocol in an pecially when the analysts have already mon solutions. empirical study with government analysts generated several solutions - typically Furthermore, to optimize the acces­ and University of Maryland students. The the more obvious ones-and realize that sibility of weak thought associations, our study is designed to test the hypothesis they must also consider further solutions next study will have participants engage that increasing alpha brain waves will im­ that are less obvious but still potentially in alpha NT during the incubation period. prove performance on tests requiring DT. relevant. Indeed, in a study on DT with Incubation effects are well established in Participants will complete a DT test after government analysts, we found that peo­ studies with convergent thinking tasks NT in which they are presented short texts ple generate more of the less obvious, that present closed insight problems for conveying scenarios that pose an open­ original solutions during a later phase which there is only a single solution, after ended problem that is difficult to solve, of an eight-minute problem-solving task changing from a dominant to a less obvi­ and for which they must generate as many posing scenario-based problems [5]. ous perspective. However, only a few be­ solutions-common and uncommon-as havioral studies have examined effects of possible. Before the test, participants will

- ses:sionl 1.15 Sesosian 2 1.10 ········Se:s,sian 3 ~ .2: 1.05 * .. p < .05 ~ .0[) m ..c Cl. 0.95 <( 0.90

0.85 T1 84

Figure 2: Alpha power increased within and across sessions

The Next Wave • Vol 1 8 No 1 • 2009 17 LQFXEDWLRQRQ'7DQGZLWKPL[HGUHVXOWV creativity literature that show that people a set of intervention tasks such as the ACT. Moreover, the application of alpha NT as are more likely to solve insight problems This intervention would be relatively easy an incubation method is new. after they have performed a variant of an to implement in the workplace. An interesting aspect of alpha NT AU task [10]. as a method for improving DT is related Previous studies have shown that Conclusion to the fact that there are systematic differ- AU techniques can be used to gener- The cognitive neuroscience ap- ences in creativity as an individual trait. ate new and creative solutions to insight proach has been fruitful in uncovering the )RUH[DPSOHZHDVVHVVHGVXFKLQGLYLGX- problems. AU research has tested whether brain’s information processing mecha- al differences in a study with university insight problems were better solved by nisms that support creative problem solv- students and government analysts with a SHRSOH ZKR ÀUVW FRPSOHWHG DQ DOWHUQD- ing, including DT and its component standard and novel task of DT and found WLYH FDWHJRULHV WHVW $&7  FRPSDUHG WR functions. The detailed functioning of that performance is predicted by the par- those who did not [10]. The ACT required these mechanisms remains to be further ticipant’s ability to comprehend written generating uses for objects in alternative elucidated. Nevertheless, the time is ripe WH[WV $OSKD 17 PLJKW DOORZ SHRSOH RI meaning categories, such as thinking of a IRU D V\VWHPDWLF H[SORUDWLRQ RI ZD\V average or low creative ability to become shoe as a hammering tool instead of foot- to facilitate DT through various neuro- PRUHFUHDWLYHLQWKHLUSUREOHPVROYLQJ,I wear. This test was completed prior to at- science and behavioral techniques for after a few training sessions of alpha NT tempting to solve insight problems. The changing neurocognitive states, including people can be taught to change their alpha results reliably demonstrated improved alpha neurofeedback training and chang- waves up or down merely on cue with- problem solving performance in the ACT ing mental set. These approaches can be out the alpha NT technique being used, group, regardless of whether participants OHYHUDJHGWRHQVXUHH[FHSWLRQDOODQJXDJH WKHEHQHÀWVFRXOGEHHDVLO\DGRSWHGLQWKH ZHUH H[SOLFLWO\ DZDUH WKDW WKH WUDLQLQJ analysis through better DT, which will work place. was relevant to the subsequent problem- KHOS,&SURIHVVLRQDOVDQWLFLSDWHWKHNLQGV Developing methods for induc- solving tasks. of low-probability yet high-impact events ing brain states associated with good DT Successful problem solving may that threaten security. could increase the number of candidates therefore hinge on the process of build-

WKH,&KDVWRFKRRVHIURPIRUDVVLJQPHQWV ing goal-derived categories [11]—cate- that require higher levels of DT. With a JRULHV FRQVWUXFWHG RQ WKH Á\ WR DFKLHYH larger pool of candidates to select from, DSDUWLFXODUREMHFWLYH,QIDFWIXUWKHUUH- WKH,&FRXOGPRUHHDVLO\LGHQWLI\SHRSOH search revealed that ACT training enabled ZLWK D GLIÀFXOWWRÀQG FRPELQDWLRQ RI people to form a greater number of, and knowledge and skills needed for analysis more variability in, goal-derived catego-          RIWHFKQLFDOO\VSHFLÀFDVVLJQPHQWV UL]DWLRQVZKLOHSUREOHPVROYLQJDFURVVVL[ different tasks [12]. Presumably, the ACT     Changing mental set implicitly trained people to construct Mental set refers to the tendency novel alternative categories based on the to repeatedly think about and approach a given problems, which in turn enhanced problem in a habitual manner. Changing a problem solving performance and en- person’s mind set to one that is more ef- abled the problem solver to detect distant-       fective provides another way to improve meaning relations. performance. DT can be used to train a $W&$6/ZHDUHFXUUHQWO\H[WHQG- person to see a problem through an alter- ing this behavioral intervention approach native lens, thus bringing new candidate to test its effect on an ecologically valid solutions in view and altering the prevail- DT test in which government analysts      ing mental set. must generate as many common and CASL researchers are currently uncommon solutions to an open-ended,         adopting this approach by designing be- scenarios-based problem in which men- havioral intervention tasks aimed broadly WDOÀ[DWLRQLVLQGXFHG:HSUHGLFWWKDWWKH     DW RYHUFRPLQJ À[DWLRQ DQG JHQHUDWLQJ most useful and original solutions will be alternative solutions. This work is moti- provided by people who are trained to es- vated in part by results in the cognitive capeIURPDPHQWDOVHWE\ÀUVWFRPSOHWLQJ

18 “Thinking Out of the Box” Through Cognitive Neuroscience FEATURE

Emerging cognitive neuroscience and SHUIRUPDQFH ,QWHUQDWLRQDO -RXUQDO RI UHODWHG WHFKQRORJLHV :DVKLQJWRQ '&  3V\FKRSK\VLRORJ\   National Academic Press; 2008. [8] Hanslmayr S, Sauseng P, Doppelmayr [3] Jung-Beeman M, Bowden EM, 0 6FKDEXV 0 .OLPHVFK: ,QFUHDVLQJ Haberman J, Frymiare JL, Arambel-Liu individual upper alpha power by neuro- S, Greenblatt R, et al. Neural activity feedback improves cognitive performance when people solve verbal problems with in human subjects. Appl Psychophysiol LQVLJKW3/R6%LRO  ( %LRIHHGEDFN   [4] Grabner RH, Fink A, Neubauer AC. [9] Haarmann HJ, George T, Smaliy Brain correlates of self-rated original- A, Grunewald K, Novick JM. A method ity of ideas: Evidence from event-re- for quickly increasing alpha brain waves lated power and phase-locking changes through neurofeedback. College Park in the EEG. Behavioral Neuroscience. 0'  8QLYHUVLW\ RI 0DU\ODQG &HQWHU    for Advanced Study of Language; 2008. [5] Haarmann HJ, Platt K, Donavos D, Technical Report No.: E.3.2. )R[/%RZOHV$'LYHUJHQWWKLQNLQJLQ [10] Chrysikou EG. When shoes become intelligence analysts: an empirical study hammers: Goal-derived categorization of incubation and predictors. College Park training enhances problem-solving per- 0' 8QLYHUVLW\RI0DU\ODQG&HQWHUIRU IRUPDQFH - ([S 3V\FKRO /HDUQ 0HP Advanced Study of Language; 2008. Fi- &RJQD   nal Technical Report No.: E.3.1. [11] Barsalou LW. Ad hoc categories. [6] Miller EK, Cohen JD. An integra- 0HP&RJQLW   WLYH WKHRU\ RI SUHIURQWDO FRUWH[ IXQF- [12] Chrysikou EG, Alvarez K, Clarke tion. Annual Review of Neuroscience. JN. Do alternative categorizations im- 2001;24:167-202. prove problem solving? Evidence from a [7] Vernon DJ, Egner T, Cooper N, verbal protocol analysis. Proceedings of Compton T, Neilands C, Sheri A, et al. the 28th Annual Conference of the Cogni- The effect of training distinct neurofeed- tive Science Society; 2006b; Vancouver, back protocols on aspects of cognitive %& &DQDGD 

Haarmann HJ. Cognitive neuro- science guidelines for improving divergent WKLQNLQJ&ROOHJH3DUN 0' 8QLYHUVLW\ of Maryland Center for Advanced Study of Language; 2007. Final Technical Report No.: M.1. >@ 1DWLRQDO 5HVHDUFK &RXQFLO 86  Committee on military and intell- igence methodology for emergent neurophysiological and cognitive/ QHXUDOUHVHDUFKLQWKHQH[WWZRGHFDGHV

The Next Wave „ Vol 18 No 1 „2009 19 For a Better Dictionary, Build a Better Parser FEATURE

The problem: dictionary lookup People working with a language in which WKH\·UH QRW ÁXHQW PXVW RIWHQ UHVRUW WR ÀQGLQJ ZRUGV LQ WKH GLFWLRQDU\ 'HSHQGLQJ RQ WKH Interagency Language Roundtable ODQJXDJH GLFWLRQDU\ ORRNXS FDQ EH SUREOHPDWLF Language Skill Level Descriptions )RU H[DPSOH WKH ODQJXDJH PLJKW KDYH FRPSOH[ for Reading, Writing, Listening, and RUWKRJUDSK\³VSHOOLQJ UXOHV³DV ZLWK (QJOLVK Speaking (from http://www.govtilr.org/) ZKHUHWKHFRUUHVSRQGHQFHEHWZHHQOHWWHUDQGVRXQG LVQRWRULRXVO\IDUIURPVLPSOH8SRQKHDULQJDZRUG Shaded area indicates range of expected WKDWEHJLQVZLWKDQfVRXQGQHZ(QJOLVKOHDUQHUV proficiency upon completion of full-length DLI PD\ ZRQGHU ZKHWKHU WR ORRN LW XS XQGHU f or ph course. And how do they know whether the long aVRXQG WKH\KHDULQDZRUGLVVSHOOHGayaDVLQlay, may, day; aiDVLQpain, maid, bait; eibDVLQrein, skein, veil; PROFICIENCY ey they, hey, whey; eighc eight, weight, DESCRIPTION DVLQ DVLQ LEVEL neigh; a  RQH FRQVRQDQW d DV LQ nature, bacon, haven; or a VLOHQWe DVLQtake, save, ace? 0 No Proficiency $QRWKHU FRPPRQ SUREOHP ZLWK GLFWLRQDU\ ORRNXS LV GHWHUPLQLQJ ZKDW form RI WKH ZRUG WR 0+ Memorized Proficiency ORRNIRU,QJHQHUDOGLFWLRQDULHVOLVWRQO\RQHKHDG 1 Elementary Proficiency word, called the citation formIRUDJLYHQZRUG)RU H[DPSOH DQ (QJOLVK GLFWLRQDU\ ZLOO KDYH rake DV 1+ Elementary Proficiency, Plus WKHFLWDWLRQIRUPLQLWVHQWU\IRUWKDWYHUEZKLOHWKH YHUE·VRWKHUIRUPVraked and rakingLIWKH\DSSHDU 2 Limited Working Proficiency DWDOOZLOOEHLQFOXGHGRQO\XQGHUWKHLQIRUPDWLRQ IRUrakeUDWKHUWKDQKDYLQJWKHLURZQHQWULHVIXUWKHU 2+ Limited Working Proficiency, Plus GRZQWKHSDJH 3 General Professional Proficiency 7KH VWXG\ RI UHODWHG ZRUG IRUPV VXFK DV rake/raked/raking WKHLU FRPSRQHQW SDUWV RU 3+ General Professional Proficiency, Plus morphemes IRUH[DPSOHrak-, -ed, and -ing DQG 4 Advanced Proficiency WKHUXOHV IRU SXWWLQJWKRVH PRUSKHPHVWRJHWKHULV called morphology raked and raking  :H UHIHU WR 4+ Advanced Proficiency, Plus DV LQÁHFWHG IRUPV RI WKH YHUE rake Rak- LV D e VWHP -ed and -ing DUH VXIÀ[HV   $XWRPDWLFDOO\ 5 Functionally Native Proficiency SURGXFLQJ WKRVH LQÁHFWHG YHUEV DV D VSHDNHU DQG DQDO\]LQJWKHPDVDUHDGHURUOLVWHQHUDUHHDV\IRU HYHQDEHJLQQLQJXVHURI(QJOLVKEHFDXVHWKHYHUE rake LV UHJXODU³RU weak LQ (QJOLVK JUDPPDWLFDO WHUPLQRORJ\³DQGZHDN(QJOLVKYHUEVIROORZYHU\ VLPSOHUHJXODUUXOHVRILQÁHFWLRQ%XWVRPHPRUH 0RVW VSHDNHUV RI (QJOLVK DUH QRW H[SOLFLWO\ H[DPSOHV IURP (QJOLVK ZLOO LOOXVWUDWH KRZ HYHQ DZDUH WKDW WKH (QJOLVK SOXUDO PRUSKHPH DFWXDOO\ VLPSOH PRUSKRORJ\ LV PRUH FRPSOLFDWHG WKDQ D FRPHVLQWKUHHYHUVLRQVSURQRXQFHG-s, -z, and κ] QDWLYHVSHDNHUPD\UHDOL]H 7KXV ZH KDYH IRU H[DPSOH aardvark/aardvark-s,

a :KLFKFDQDOVRUHSUHVHQWWKHORQJeVRXQGDVLQquay b:KLFKFDQDOVRUHSUHVHQWWKHVKRUWiVRXQGDVLQweird  c:KLFKFDQDOVRUHSUHVHQWWKHORQJiVRXQGDVLQheight d:KLFKFDQDOVRUHSUHVHQWVKRUWaVRXQGDVLQsatin e0RVWRIWKHWLPHLQ(QJOLVKWKHVWHPDQGFLWDWLRQIRUPDUHWKHVDPHDVLQWKHYHUEV reach (reach-ed, reach-ing  jump (jump-ed, jump-ing play (play-ed, play-ing

The Next Wave „ Vol 18 No 1 „ 2009 21 1. -zDIWHUYRZHOVRUWKHFRQVRQDQWVRXQGVb, d, “hard” g, l, m, n, ng, r, th, or v 2. -sDIWHUWKHFRQVRQDQWVRXQGVf, k, p, or t Morpheme: the smallest unit of 3. κ]DIWHUWKHFRQVRQDQWVRXQGVch,´VRIWµg, s, sh, z, or zh meaning in a language 6LPLODUO\ WKH (QJOLVK SDVW WHQVH PRUSKHPH VSHOOHG -ed KDV WKUHH YDULDQWV LQ VSRNHQ (QJOLVK Stem: Roughly, that part of a word that carries pronounced -t, -d, and -κGDVLQrake-d (pronounced the main meaning. The stem of the English verb rake-t  live-d, and wait-ed :KLFK IRUP LV XVHG rains, for example, is rain (which also happens GHSHQGVDVZLWKWKHSOXUDOPRUSKHPHRQWKHVRXQG to be a word on its own). For the noun raincoats, DWWKHHQGRIWKHZRUGWRZKLFKLWLVDGGHG 1. -d the two morphemes rain and coat together DIWHUYRZHOVDQGWKHFRQVRQDQWVRXQGV b, ´KDUGµDQG´VRIWµ g, l, m, n, ng, r, th, constitute the stem. v, or z 2. -t ch, f, k, p, s, Prefix: A morpheme that precedes a stem, as DIWHUWKHFRQVRQDQWVRXQGV sh, or th in English re-, un-, bi-. 3. κGDIWHUWKHFRQVRQDQWVRXQGVd or t Suffix: A morpheme that follows the stem, such ,QZULWLQJWKHSOXUDOLVVSHOOHGs or es, while as the two suffixes -s in rains and raincoats. WKHSDVWWHQVHLVQHDUO\DOZD\VVSHOOHGed,I(QJOLVK While these two suffixes are spelled the same, ZHUH ZULWWHQ FORVHU WR WKH ZD\ LW LV SURQRXQFHG ZH ZRXOG LQVWHDG KDYH WKUHH VSHOOLQJV RI HDFK RI they have different meanings: one is a present WKHVH VXIÀ[HV UHÁHFWLQJ WKHLU GLIIHUHQW VSRNHQ tense ending on a verb, while the other is a IRUPV7KLVVLWXDWLRQRIPRUSKHPHVZLWKPXOWLSOH plural ending on a noun. IRUPVLVFDOOHGallomorphyDQGHDFKYDULDQWIRUPLV called an allomorph(QJOLVKDOVRKDVPDQ\FDVHVRI Infix: A morpheme that goes inside a stem. DOORPRUSK\WKDWGRQRWIROORZUXOHVDQGDUHWKHUHIRUH English does not have infixes, but later on we labeled irregularVXFKDVgoose/geese, mouse/mice, g will talk about a language that does. louse/lice, HWFDPRQJQRXQVDQGbuild/built, run/ ran, bring/broughtHWFDPRQJYHUEV:LWKWKHVH Affix: A prefix, suffix, or infix. ZRUGVWKHDOORPRUSK\OLHVLQWKHVWHPRIWKHZRUG UDWKHU WKDQ LQ DQ\ DIÀ[HV 6SHDNHUV PXVW VLPSO\ Example: re-educat-ed OHDUQWKHVHIRUPVDQGQRQÁXHQWVSHDNHUVLQFOXGLQJ \RXQJQDWLYHVSHDNHUVRIWHQJHWWKHPZURQJXQWLO re–prefix (= again); educat–stem; ed–suffix (past WKH\EHFRPHPRUHÁXHQW)RUWXQDWHO\LQWKHFDVHRI tense) (QJOLVKWKHUHDUHFRPSUHKHQVLYHGLFWLRQDULHVWKDW OLVW DOO WKH LUUHJXODU IRUPV DQG DOSKDEHWL]H WKHP ZKHUHRQHZRXOGH[SHFWWKHPWREH%XWIRUPDQ\ ODQJXDJHVWKHUHDUHQRFRPSUHKHQVLYHGLFWLRQDULHV DQGWKHLUUHJXODUIRUPVDUHRQO\OLVWHGLQWKHPDLQ HQWU\RUQRWDWDOO dog/dog-s (pronounced dog-z DQGostrich/ostrich- Inflectional morphology es (pronounced ostrichκ] :LWKRXWWKLQNLQJDERXW 7KH VHW RI LQÁHFWHG IRUPV RI D ZRUG LV LWQDWLYHVSHDNHUVFKRRVHDQHQGLQJEDVHGRQWKH UHIHUUHG WR DV LWV SDUDGLJP 3DUDGLJPV RI QRXQV IROORZLQJUXOHV ZLWKVRPHH[FHSWLRQV I DUH RIWHQ FDOOHG GHFOHQVLRQV DQG SDUDGLJPV RI

I1RWHWKDWWKHUXOHVDUHEDVHGRQWKHÀQDOsoundQRWOHWWHURIWKHZRUG g 7KHWZRSDLUVmouse/mice and louse/liceDSSHDUWRUHSUHVHQWDUHJXODUSDWWHUQWKDWVSHDNHUVFRXOGOHDUQWRSUHGLFW EXWFRPSDUHWKHIRUPVhouse/house-s, spouse/spouse-s

22 For a Better Dictionary, Build a Better Parser FEATURE

YHUEVFRQMXJDWLRQV)RU(QJOLVKWKHSDUDGLJPRI IRUH[DPSOHkiraPHDQVguess, while kira- D QRXQ FRQVLVWV RI WKH VLQJXODU DQG SOXUDO IRUPV kiraPHDQVDSSUR[LPDWHO\,QWKLVFDVHWKH DQGSHUKDSVWKHSRVVHVVLYH IRUPV WKHSDUDGLJP HQWLUHZRUGLVUHSHDWHG RIPRVW(QJOLVKYHUEVFRQVLVWVRIWKHEDUHIRUP OLNH 7DJDORJ JLYHV XV DQ H[DPSOH RI SDUWLDO walk WKHWKLUGSHUVRQVLQJXODUSUHVHQWWHQVHIRUP UHSHWLWLRQ IURP WKH YHUE sulat again, (walks WKHSUHVHQWSDUWLFLSOHIRUP walking DQG D VRUW RI IXWXUH WHQVH LV GHULYHG E\ walked h WKHSDVWWHQVHIRUP  UHGXSOLFDWLQJ WKH ÀUVW V\OODEOH WR JLYH %HOLHYH LW RU QRW (QJOLVK PRUSKRORJ\ LV susulat 5HGXSOLFDWLRQ LV QRW DOZD\V WKLV UHODWLYHO\ VLPSOH ,Q ODQJXDJHV ZLWK FRPSOH[ VLPSOH 6RPH 7DJDORJ YHUEV ERUURZHG PRUSKRORJ\ WKH SDUDGLJPV PD\ EH PXFK PRUH IURP6SDQLVKVWDUWZLWKWZRFRQVRQDQWV FRPSOLFDWHG UXQQLQJ LQWR WKRXVDQGV RI IRUPV )RU WKHVH YHUEV UHGXSOLFDWLRQ LQYROYHV 0RUHRYHUWKHFLWDWLRQIRUPVRIZRUGVZLWKSHUIHFWO\ RQO\WKHÀUVWRIWKHWZRFRQVRQDQWVSOXV UHJXODUPRUSKRORJ\DUHRIWHQQRWREYLRXVWRDQRQ WKH IROORZLQJ YRZHO trabaho, to work ÁXHQW XVHU RI WKH ODQJXDJH EHFDXVH WKH LQÁHFWHG IURP 6SDQLVK trabajo, I work  EHFRPHV IRUPVFDQEHYHU\GLIIHUHQWIURPWKHFLWDWLRQIRUPV tatrabaho, I will work )RUH[DPSOH ‡ 6WHPDOORPRUSK\$VLISUHÀ[HVVXIÀ[HV ‡ 3UHÀ[HVDQGVXIÀ[HV,Q6ZDKLOL D%DQWX LQÀ[HVDQGUHGXSOLFDWLRQZHUHQRWHQRXJK ODQJXDJHRIHDVW$IULFD YHUEVRIWHQKDYH ODQJXDJHVRIWHQPRGLI\WKHIRUPVRIVWHPV D ODUJH QXPEHU RI SUHÀ[HV DQG VXIÀ[HV 6RPHWLPHVWKLVSHUWDLQVRQO\WRLUUHJXODU 7KH LQÁHFWHG YHUE amewanunulia IRU ZRUGVLQ6SDQLVKIRUH[DPSOHUHPRYLQJ H[DPSOHFRQWDLQVWKUHHSUHÀ[HVDQGWZR the -es VXIÀ[ IURP WKH ZRUG tienes, VXIÀ[HVWKHSUHÀ[a-PHDQLQJDFODVV PHDQLQJyou haveJLYHVDVWHPtien-%XW VXEMHFW QRXQVLQ6ZDKLOLEHORQJWRRQHRI WKH IRUP OLVWHG LQ WKH GLFWLRQDU\ tener DGR]HQRUPRUHFODVVHVVLPLODUWRJHQGHU DQ LQÀQLWLYH  KDV D GLIIHUHQW VWHP ten- LQ5RPDQFHODQJXDJHVDQGYHUEVDJUHHLQ FODVVZLWKWKHLUVXEMHFWV WKHSUHÀ[me-, ,QRWKHUODQJXDJHVVWHPFKDQJHVDUHPRUH PHDQLQJSHUIHFWLYHDVSHFW VLPLODUWRhas RU OHVV UHJXODU ,Q %DQJOD D ODQJXDJH RI doneLQ(QJOLVK WKHSUHÀ[wa-PHDQLQJ %DQJODGHVKDQG,QGLDVWHPVZKLFKFRQWDLQ DFODVVREMHFWWKHVXIÀ[-liPDUNLQJWKH WZR FRQVRQDQWV GLVSOD\ DQ DOWHUQDWLRQ o and u, depending on ´DSSOLFDWLYHµIRUPRIWKHYHUE WKLVKDVWR EHWZHHQWKHYRZHOV shono, you hear, GRZLWKKRZWKHGLUHFWDQGLQGLUHFWREMHFWV WKHIROORZLQJFRQVRQDQW but shuni, I hear DSSHDULQWKHVHQWHQFH DQGWKHVXIÀ[ -a,  ZKRVH PHDQLQJ LV VRPHZKDW GLIÀFXOW WR Foreign language dictionaries SLQGRZQ7KHYHUEDWWKHKHDUWRIWKLVORQJ 6LJQLÀFDQW LQÁHFWLRQDO PRUSKRORJ\ VXFK ZRUGLVnunuPHDQLQJto buy DV WKHVH H[DPSOHV LOOXVWUDWH FDQ SUHVHQW VHULRXV ‡ ,QÀ[HV %HVLGHV SUHÀ[HV DQG VXIÀ[HV GLIÀFXOWLHV IRU ÀQGLQJ ZRUGV LQ WKH GLFWLRQDU\ VRPHODQJXDJHVHPSOR\LQÀ[HVDIÀ[HVWKDW 6LQFH GLFWLRQDULHV GR QRW QRUPDOO\ OLVW DOO VXFK JRLQVLGHVWHPV,Q7DJDORJDODQJXDJHRI IRUPV RI ZRUGV PRUSKRORJLFDO RSHUDWLRQV PD\ WKH3KLOLSSLQHVWKHLQÀ[-um-FDQEHXVHG PDNHLWGLIÀFXOWWRORRNXSDZRUG$IÀ[HVPXVWEH WRFUHDWHDQRXQIURPDYHUE)RUH[DPSOH UHPRYHGDQGRIWHQRWKHUDIÀ[HVPXVWEHDWWDFKHG the verb sulatPHDQLQJto write, can take 7DNHWKH6ZDKLOLH[DPSOHDERYHZKHUHWKHZRUG WKLV LQÀ[ WR EHFRPH sumulat PHDQLQJ D amewanunuliaFRQWDLQVWKHYHUEURRWnunu6ZDKLOL writer GLFWLRQDULHVXVXDOO\OLVWWKLVYHUEZLWKWKH-aVXIÀ[ ‡ 5HGXSOLFDWLRQ0DQ\ODQJXDJHVHPSOR\D attached to the root (but without the intervening SURFHVVFDOOHGreduplication, in which all DSSOLFDWLYH VXIÀ[ -li  JLYLQJ nunua ,Q RWKHU RUSDUWRIDZRUGLVUHSHDWHG,Q,QGRQHVLDQ VLWXDWLRQVWKHZRUGPD\EHJLQZLWKSUHÀ[HVFRQWDLQ

h,UUHJXODUYHUEVRIWHQKDYHDVHSDUDWHSDVWSDUWLFLSOHIRUPDVZHOODVLQsung, given, goneDQGVRRQDOWKRXJKIRU RWKHUYHUEVWKHSDVWSDUWLFLSOHLVWKHVDPHDVWKHSDVWIRUP

The Next Wave „ Vol 18 No 1 „ 2009 23 infixes, or the form of the stem may be modified. word reruns, for example, would parse into the Linguists who are fluent and experienced in prefix re-, the stem run, and the suffix -s. The words a language can surmount these problems. Often, being analyzed may be all the words of a text, or however, linguists work with languages in which individual words that are typed or pasted in to the they aren't sufficiently expert to know-or at least computer. to be able to determine quickly-a word's citation Researchers at the University of Maryland form. They may have completed language training Center for Advanced Study of Language (CASL) recently or been called on to work in a related are building a morphological parser by developing language they don't know well. The examples a new, more efficient methodology that also answers above demonstrate how much detail a user of a the problem of software obsolescence. Our project, morphologically complex language must know "Dual-Use Grammars in Related Languages," for efficient dictionary lookup, and that sort of has thus far focused on building morphological knowledge comes only with time. parsers for two South Asian languages: Bangla Difficulties with dictionary lookup can lead to and Urdu. We are now beginning research on the significant loss of time and efficiency in translating. Pashto language, which will pose new challenges Linguists are often forced to spend their time and due to dialectal variation and a scarcity of written resources. Three things are needed to create a morphological parser for a language: 1. a list of the language's words, as they appear in a dictionary; 2. a list of the language's grammar rules, particularly the morphological rules; and 3. a morphological parsing engine. The dictionary Obtaining an adequate dictionary can be a challenge. At a minimum, the entries must be labeled for part of speech (noun, verb, adverb, etc.). For some languages, additional information may be required: • Noun class (All nouns in Spanish are either masculine or feminine; nouns in Bantu languages belong to one of a dozen or more classes.) Conjugation or declension class (These tell energy searching through the dictionary, guessing how a verb, noun, or adjective is inflected; at the citation form of an intricately inflected word. for some languages, this can be inferred They need a tool to help them. from the form of the word listed in the dictionary, while for others it cannot.) The solution: a morphological Irregular forms (such as oxen and wept) parser Perhaps surprisingly, information about the Fortunately, tools that help us find the meaning of a word is not needed for purposes of dictionary form of a word exist; they are called morphological parsing-although of course helping morphological parsers. A morphological parser is the user find its meaning is why we're doing the an automatic tool that breaks up an inflected word parsing in the first place! into its morphemes. One of these morphemes will Dictionaries of well-known languages such as be the stem, from which the form of the word as French or German contain the information a parser it is listed in the dictionary can be produced and requires, but dictionaries of less commonly taught looked up in an electronic dictionary. The English languages are less likely to be complete.

24 For a Better Dictionary, Build a Better Parser FEATURE

Additionally, dictionaries for many lesser known languages do not contain a sufficient number Lots of words on which to base a parser. Dictionaries English X of better known languages often contain thirty or French X X X Arabic Chinese forty thousand words, which is adequate for most High-Density Languages (MSA) purposes. But for many smaller languages it may )(Japanese be difficult to find an electronic dictionary with )(··········)( more than ten thousand words -or for some written "European" Ko~an )( )( Russian Spanish languages, any electronic dictionary. And of course Medium-Density Languages the thousands of unwritten languages have no )(Hindi )(Inuit, Basque ... dictionaries at all. Low-Density )( Bangla Languages )( )(Tamil Cebuano, Amharic ... X Indonesian Borrowed words present yet another challenge. Everyone else--ff------x

109 For many dictionary compilers, it is a matter of pride Number of Speakers to exclude loanwords from other languages, even when such words may be used frequently in speech or texts (particularly texts on technical topics). A The grammar rules non-native speaker may need to look for such words in dictionaries of related languages. But because As our examples show, morphological rules the spelling (and pronunciation) of loanwords is can be quite complex. But for a morphological often quite different from the accepted spelling in parser, we need a description of the language's the word's original language, these words can be morphology that is clear and unambiguous. difficult to find in the other languages' dictionaries. Ambiguity, however, is inherent to natural language. The Tagalog word iskor, for example, is borrowed We therefore also need a grammar written in a from the English word score-but you will not find formal-that is, computer-readable-language. So iskor in an English dictionary! Chinese loanwords for each language, we require two grammars: an old­ in Tagalog, or Russian loanwords in Uzbek, can fashioned descriptive grammar in straightforward likewise be hard to find in dictionaries. English prose and a formal grammar that can be

The Next Wave • Vol 18 No 1 • 2009 25 FRUUHFWLQJ HUURUV LQ WKH JUDPPDU DGGLQJ D QHZ GLFWLRQDU\ RU DGDSWLQJ WKH JUDPPDU WR D UHODWHG ODQJXDJHZLWKRXWVWDUWLQJIURPVFUDWFK Spelling standardization Challenges of building parsers %ULQJLQJ WKHVH SLHFHV WRJHWKHU WR EXLOG D In English, as well as languages like Spanish and PRUSKRORJLFDO SDUVHU LV SUREOHPDWLF IRU VHYHUDO French, we are accustomed to one “correct” UHDVRQV)LUVWUDUHO\GRHVRQHSHUVRQFRPELQHWKH QHFHVVDU\ NQRZOHGJH RI ERWK WKH WDUJHW ODQJXDJH spelling, although different countries may have OLQJXLVWLFV DQG WKH FRPSXWDWLRQDO WHFKQRORJ\ different spellings for the same word (think of IRU EXLOGLQJ SDUVHUV 6HFRQG SDUVHUV RQFH EXLOW color and colour). But many written languages DUH OLPLWHG E\ WKH OLIH RI WKH VRIWZDUH XVHG IRU lack such standards, and words are simply LPSOHPHQWDWLRQDOLIHWKDWLVRIWHQVKRUW,QDGGLWLRQ spelled as they sound, making it difficult to look PRUSKRORJLFDO SDUVHUV³DQG WKH JUDPPDWLFDO UHVRXUFHVWKDWOLHEHKLQGWKHP³DUHFXUUHQWO\KDQG words up. There may be different ways to write EXLOWVRFUHDWLQJWKHPLVH[WUHPHO\ODERULQWHQVLYH words depending on pronunciation, which in turn )LQDOO\ZHQHHGDZD\RIWHVWLQJIRUDFFXUDF\+RZ depends on dialect, or how fast one is speaking. GRZHNQRZWKDWWKHJUDPPDURQZKLFKWKHSDUVHULV And of course there is no accepted writing EDVHGLVDQDFFXUDWHGHVFULSWLRQRIWKHODQJXDJH" system for some languages. This is particularly &$6/·V SULQFLSOHG DSSURDFK WR EXLOGLQJ D SDUVHU DGGUHVVHV WKHVH SUREOHPV LQ WKH IROORZLQJ true for many minority languages of Africa or ZD\V Asia and for many sign languages. Problem: finding the expertise :ULWLQJ VRIWZDUH UHTXLUHV WZR NLQGV RI H[SHUWLVHNQRZOHGJHRIWKHSUREOHPWREHVROYHG DQG NQRZOHGJH RI KRZ WR SURJUDP VRIWZDUH )RU SDUVHUV WKH SUREOHPVSHFLÀF NQRZOHGJH UHTXLUHV DXWRPDWLFDOO\ FRQYHUWHG WR VHUYH DV WKH JUDPPDU XQGHUVWDQGLQJWKHJUDPPDURIWKHWDUJHWODQJXDJH,W i UXOHVIRUWKHSDUVHU PLJKWVHHPWKDWÀQGLQJVRPHRQHZKRXQGHUVWDQGV The parsing engine WKHJUDPPDURIDQ\SDUWLFXODUODQJXDJHLVDVHDV\ 7KH WKLUG FRPSRQHQW WKH SDUVLQJ HQJLQH LV DV ÀQGLQJ D ÁXHQW QDWLYH VSHDNHU 8QIRUWXQDWHO\ DFRPSXWDWLRQDOWRROWKDWLQFRUSRUDWHVLQIRUPDWLRQ DV JHQHUDWLRQV RI ÀHOG OLQJXLVWV KDYH GLVFRYHUHG DERXW KRZ ODQJXDJHV ZRUN LQ JHQHUDO VXFK DV D QDWLYH VSHDNHU·V NQRZOHGJH RI D ODQJXDJH LV WKHIDFWWKDWSUHÀ[HVSUHFHGHDVWHPDQGVXIÀ[HV QRWRULRXVO\LPSOLFLW³MXVWUHFDOOWKHFRPSOH[LW\RI IROORZDVWHP7KLVJHQHULFTXDOLW\LVZK\OLQJXLVWV WKHXQFRQVFLRXVUXOHVIRUIRUPLQJSOXUDOVDQGSDVW QHHGRQO\SURYLGHDGLFWLRQDU\DQGWKHJUDPPDURI WHQVHVLQ(QJOLVK&RQYHUWLQJWKDWNQRZOHGJHLQWR WKHODQJXDJH³ZKDWWKHSDUWLFXODUDIÀ[HVDQGVWHPV H[SOLFLWUXOHVLVQRVLPSOHWDVN RIWKDWODQJXDJHDUHIRUH[DPSOHDQGKRZWKH\ÀW :KDW DERXW XVLQJ H[WDQW ZULWWHQ UHVRXUFHV" WRJHWKHU7KHUH DUH VHYHUDO DGYDQWDJHV WR NHHSLQJ 'XULQJ WKH LQLWLDO VWDJHV RI UHVHDUFK LQWR RXU ÀUVW ODQJXDJHVSHFLÀF LQIRUPDWLRQ WKH GLFWLRQDU\ DQG WDUJHW ODQJXDJH ZH ZHUH VXUSULVHG WR GLVFRYHU WKHJUDPPDU VHSDUDWHIURPWKHLQIRUPDWLRQWKDWLV WKDWQRWKRURXJKDQGUHOLDEOHGHVFULSWLYHJUDPPDU WUXH IRU DOO ODQJXDJHV LQIRUPDWLRQ ZKLFK LV EXLOW RIPRGHUQFROORTXLDO%DQJODH[LVWHGGHVSLWHRYHU GLUHFWO\ LQWR WKH SDUVLQJ HQJLQH  )LUVW VHSDUDWLQJ WZR KXQGUHG PLOOLRQ QDWLYH VSHDNHUV LW LV RQH RI WKHWZRNLQGVRILQIRUPDWLRQLQWKLVZD\PDNHVLW WKHWHQODUJHVWODQJXDJHVLQWKHZRUOG ,QVWHDGZH HDVLHUDQGIDVWHUWREXLOGWKHSDUVHU,WDOVRPDNHV KDGWRUHO\RQGHVFULSWLRQVRI%DQJODPRUSKRORJ\ ODWHUPRGLÀFDWLRQVWRWKHSDUVHUHDVLHUIRUH[DPSOH IURP D YDULHW\ RI VRXUFHV LQFOXGLQJ JUDPPDUV RI

i'XDOJUDPPDUVDUHDOVRVROXWLRQVWRRWKHUSUREOHPVGLVFXVVHGDWWKHHQGRIWKLVDUWLFOH

26 For a Better Dictionary, Build a Better Parser FEATURE

YDU\LQJTXDOLW\MRXUQDODUWLFOHVDQGGLVVHUWDWLRQV 7KH%DQJODVFULSWSUHVHQWVLWVRZQRUWKRJUDSKLF 7KHGHVFULSWLRQVLQWKHVHVRXUFHVZHUHVRPHWLPHV SHFXOLDULWLHVIRUH[DPSOHRQHRILWVYRZHOVLVQRW YDJXHGLGQRWDOZD\VDJUHHDQGHYHQFRQWDLQHGD UHSUHVHQWHG ZKHQ IROORZLQJ D FRQVRQDQW &DOOHG IHZJDSV the inherent vowelLWLVVLPSO\XQGHUVWRRGDVEHLQJ 7KH GLIÀFXOWLHV ZH HQFRXQWHUHG LQ SUHVHQW³H[FHSW ZKHQ LW·V QRW &RQVRQDQWV not XQGHUVWDQGLQJ JUDPPDWLFDO GHVFULSWLRQV SURQRXQFHGZLWKWKHLQKHUHQWYRZHODIWHUWKHPDUH UHFRQFLOLQJ GLIIHUHQW JUDPPDWLFDO DFFRXQWV DQG ZULWWHQH[DFWO\WKHVDPHDVFRQVRQDQWVSURQRXQFHG with kon which? kono any ÀOOLQJLQJDSVLQFRYHUDJHXQGHUOLQHWKDWZHFRXOG LWVRWKHZRUGV ( DQG (  QRWKDYHVLPSO\SLFNHGXSDQH[LVWLQJJUDPPDUDQG DUH VSHOOHG WKH VDPH LQ WKH %DQJOD VFULSW ZLWK o in kono ZULWWHQRXUIRUPDOJUDPPDU)RUODQJXDJHVZLWKDQ\ QR H[SOLFLW UHSUHVHQWDWLRQ RI WKH ÀQDO  GHJUHHRILQÁHFWLRQDOFRPSOH[LW\WKHFRPSOH[LWLHV .QRZLQJ ZKHWKHU WR SURQRXQFH VXFK FRQVRQDQWV SUHYHQW VXFK D VLPSOH DSSURDFK &OHDUO\ FUHDWLQJ ZLWK RU ZLWKRXW WKH LQKHUHQW YRZHO UHTXLUHV D PRUSKRORJLFDO SDUVHU UHTXLUHV OLQJXLVWLF DV IDPLOLDULW\ZLWKWKHODQJXDJH$VDUHVXOWZHKDYH ZHOO DV FRPSXWDWLRQDO H[SHUWLVH %XW ÀQGLQJ RQH FKRVHQ WR ZULWH RXU 8UGX DQG %DQJOD H[DPSOHV SHUVRQ ZKR FRPELQHV D GHWDLOHG VRSKLVWLFDWHG XVLQJERWKWKHQDWLYHVFULSWDQGDWUDQVFULSWLRQWKDW XQGHUVWDQGLQJRIDJLYHQODQJXDJH·VJUDPPDUZLWK LVDV\VWHPWKDWZULWHVDOOWKHQHFHVVDU\YRZHOV WKH FRPSXWDWLRQDO H[SHUWLVH QHFHVVDU\ WR ZULWH $IWHU JDWKHULQJ UHVRXUFHV DQG DGGUHVVLQJ D IRUPDO JUDPPDU LV GLIÀFXOW SDUWLFXODUO\ ZLWK ORJLVWLFDOTXHVWLRQVRIVFULSWDQGWUDQVFULSWLRQZH ODQJXDJHVIRUZKLFKZHGRQRWDOUHDG\KDYHODUJH WKHQ EHJLQ ZULWLQJ RXU GHVFULSWLYH JUDPPDU 7KLV DPRXQWVRIUHVRXUFHV³SUHFLVHO\WKHODQJXDJHVWKDW SURFHVV LQYROYHV XQGHUVWDQGLQJ WKH JUDPPDUV QHHGWRROVOLNHPRUSKRORJLFDOSDUVHUV ZH KDYH FROOHFWHG GHVFULELQJ LQ RXU WHUPV WKHLU FROOHFWHGZLVGRPDXJPHQWLQJZKHUHQHFHVVDU\E\ Solution: collaborative grammar writing KHOS IURP RXU FRQVXOWDQWV DQG XOWLPDWHO\ YHWWLQJ &RPELQLQJ D IRUPDO JUDPPDU ZLWK WKH IRUDFFXUDF\³DJDLQDVVLVWHGE\FRQVXOWDQWV(DFK GHVFULSWLYH JUDPPDU JLYHV XV DQ DQVZHU WR WKH JUDPPDUFRQWDLQVDFKDSWHURQWKHSKRQRORJ\DQG SUREOHP RI DPELJXLW\ LQ WUDGLWLRQDO JUDPPDUV D ZULWLQJV\VWHPRIWKHODQJXDJHDQGFKDSWHUVIRUWKH IRUPDOJUDPPDULVQHLWKHUDPELJXRXVQRUYDJXH,Q YDULRXVSDUWVRIVSHHFK(DFKFKDSWHUGHVFULEHVWKH RXUPHWKRGRORJ\RQHPHPEHURIRXUWHDPZRUNV LQÁHFWLRQDODIÀ[HVHDFKSDUWRIVSHHFKWDNHVDQGKRZ DVDGHVFULSWLYHJUDPPDULDQFROOHFWLQJJUDPPDUV WKHUHVXOWLQJLQÁHFWHGIRUPVGHÀQHWKHSDUDGLJPV DQG RWKHU UHVRXUFHV RQ WKH ODQJXDJH RI LQWHUHVW 7KHZD\WKHVHIRUPVDUHXVHGLQVHQWHQFHVLVDOVR 7KH ÀUVW VWHS LV WR UHVHDUFK H[LVWLQJ GHVFULSWLYH GHVFULEHGZLWKLOOXVWUDWLYHH[DPSOHV DQG SHGDJRJLFDO JUDPPDUV %HFDXVH VRPH RI 7KHWKLUGVWHSLQWKHJUDPPDUZULWLQJSURFHVV WKHVHVRXUFHVWXUQRXWWREHUHODWLYHO\XVHOHVVZH LVZULWLQJRXUIRUPDOJUDPPDUXVLQJWKHGHVFULSWLYH VLPXOWDQHRXVO\ VHDUFK IRU SRWHQWLDO FRQVXOWDQWV JUDPPDUDVDJXLGH,QWKHRU\WKLVPHDQVVLPSO\ WR ÀOO LQ JDSV LQ FRYHUDJH FODULI\ DPELJXLWLHV WDNLQJWKHGHVFULSWLRQFRQGHQVHGE\WKHGHVFULSWLYH SURRIUHDGH[DPSOHVDQGFKHFNWKHDFFXUDF\RIRXU JUDPPDUZULWHUIURPPXOWLSOHVRXUFHJUDPPDUVDQG JUDPPDWLFDO GHVFULSWLRQ DV ZHOO DV KHOS DQQRWDWH WXUQLQJLWLQWRDIRUPDOGHVFULSWLRQLQRXUIRUPDO;0/ WH[WIRUWHVWSXUSRVHV ODQJXDJH VHHEHORZIRUPRUHRQ;0/ ,QSUDFWLFH 7KH VHFRQG VWHS LV GHWHUPLQLQJ KRZ WR WKHQDWXUHRIIRUPDOJUDPPDUVLVWKDWQRPDWWHUKRZ UHSUHVHQW WKH ZRUGV RI WKH ODQJXDJH )RU PDQ\ FDUHIXO WKH GHVFULSWLYH JUDPPDULDQ LV WKH IRUPDO ODQJXDJHVUHSUHVHQWDWLRQLVQRWDQLVVXHDVH[LVWLQJ JUDPPDU EULQJV WR OLJKW JDSV DQG DPELJXLWLHV RUWKRJUDSKLHV DUH DGHTXDWH )RU WKH ODQJXDJHV RI :KHQDPELJXLWLHVVXUIDFHWKHGHVFULSWLYHJUDPPDU &$6/·VSURMHFWKRZHYHUUHSUHVHQWDWLRQKDVEHHQ LV FODULÀHG HLWKHU E\ UHIHUULQJ EDFN WR WKH VRXUFH DQG FRQWLQXHV WR EH DQ LVVXH 8UGX IRU H[DPSOH JUDPPDUVRUE\DVNLQJRXUFRQVXOWDQWV ,WLVZRUWK XVHV D ULJKWWROHIW ZULWLQJ V\VWHP UHODWHG WR WKDW QRWLQJWKDWZKHQZHFUHDWHGDIRUPDOJUDPPDUIRU XVHGIRU$UDELF ZKLFKPDNHVZULWLQJDQGJORVVLQJ %DQJODRXUFRQVXOWDQWIUHTXHQWO\FRPPHQWHGWKDW H[DPSOHVGLIÀFXOW,QDGGLWLRQVKRUWYRZHOVDUHQRW QRRQHKDGDVNHGWKHVHSDUWLFXODUTXHVWLRQVEHIRUH ZULWWHQLQWKH8UGXZULWLQJV\VWHPVRDQRQÁXHQW :HWDNHWKLVWREHLQSDUWDUHVXOWRIXVLQJFRPSXWHUV VSHDNHUKDVGLIÀFXOW\NQRZLQJZKHWKHUWKHYRZHO ZKLFKXQOLNHKXPDQVWROHUDWHQRDPELJXLW\EXWLW IROORZLQJDFRQVRQDQWLVa, i, or u-VWSFWUKZGIIFOW DOVRXQGHUVFRUHVWKHVWDWHRILQYHVWLJDWLRQRIWKHVH UGQJZWKWYZOVFQEZKQ\GQ·WNQZWKOQJJ ODQJXDJHV 

The Next Wave „ Vol 18 No 1 „ 2009 27 ,Q VXPPDU\ RXU GLYLVLRQ RI ODERU WRJHWKHU VXFKDVSDUVHUVPHDQVWKDWDUFKLYHGODQJXDJHGDWD ZLWK VLPXOWDQHRXVO\ GHYHORSLQJ WKH WZR NLQGV FDQ EHFRPH XQXVDEOH PXFK PRUH TXLFNO\ WKDQ RI JUDPPDU XVLQJ RXU FRPSXWDWLRQDO WRROV DQG SULQWHGGHVFULSWLRQVk,IZHZURWHIRUPDOJUDPPDUV LQFRUSRUDWLQJ LPPHGLDWH IHHGEDFN KDV PDGH XVLQJ WKH SURJUDPPLQJ ODQJXDJHV XQGHUVWRRG SRVVLEOH D PXFK EHWWHU UHVXOW WKDQ ZULWLQJ WKH E\ WRGD\·V SDUVLQJ HQJLQHV WKH JUDPPDUV ZRXOG GHVFULSWLYH JUDPPDU DQG WKHQ ZULWLQJ WKH IRUPDO VRRQQHHGWREHUHZULWWHQ7KXVDOWKRXJKDSDUVHU JUDPPDU FRQVWLWXWHVDYDOXDEOHGHVFULSWLRQRIDODQJXDJHLW Problem: obsolescence LV³XQWLOQRZ³DQHSKHPHUDOGHVFULSWLRQ 8QIRUWXQDWHO\VRIWZDUHGRHVQRWODVWIRUHYHU Solution: a generic way to write 7KH GHYHORSPHQW RI FRPSXWHUSURFHVVDEOH computer-readable grammars PRUSKRORJLFDO JUDPPDUV LV RIWHQ WLHG WR WKH :H H[SHFW WKDW (QJOLVK ZLOO EH XQGHUVWRRG SURJUDPPLQJODQJXDJHRIDSDUWLFXODUPRUSKRORJLFDO IRUDORQJWLPHMXVWDV/DWLQLVVWLOOXQGHUVWRRGE\ SDUVLQJ HQJLQH RU WR D JHQHUDO SXUSRVH FRPSXWHU H[SHUWVWRGD\6RRXUGHVFULSWLYHJUDPPDUVZULWWHQ SURJUDPPLQJODQJXDJH,IWKHSDUVLQJHQJLQHZHUH LQ(QJOLVKVKRXOGEHXQGHUVWDQGDEOHIRUFHQWXULHV JXDUDQWHHGWREHDURXQGIRUHYHURULIWKHUHZHUHDQ SHUKDSV HYHQ PLOOHQQLD IURP QRZ 1RW VR RXU DJUHHGRQGHVFULSWLYHODQJXDJHWKDWDOOVXFKSDUVHUV IRUPDOJUDPPDUV7U\LQJWRZULWHIRUPDOJUDPPDUV ZRXOGXVHWKLVPLJKWQRWEHSUREOHPDWLF%XWQHLWKHU LQ (QJOLVK KRZHYHU LV QRW D VROXWLRQ GXH WR WKH RIWKHVHFRQGLWLRQVLVWUXH,QIDFWZHHVWLPDWHWKH DPELJXLW\RI(QJOLVKDQGRWKHUQDWXUDOODQJXDJHV DYHUDJHOLIHWLPHIRUODQJXDJHEDVHGFRPSXWDWLRQDO :KDW ZH QHHG LV WR GHÀQH WKH IRUPDO JUDPPDU WRROVWREHÀYHRU\HDUV,QWKHSDVW\HDUVDW VXFK WKDW LW FDQ EH XQDPELJXRXVO\ WUDQVODWHG LQWR OHDVWDGR]HQPXWXDOO\LQFRPSDWLEOHPRUSKRORJLFDO WKHSURJUDPPLQJODQJXDJHRIERWKWRGD\·VSDUVLQJ SDUVLQJWRROVKDYHEHHQFUHDWHGHDFKZLWKLWVRZQ HQJLQHV DQG IXWXUH HQJLQHV :H FDQ·W NQRZ ZKDW SURJUDPPLQJODQJXDJH WKRVHIXWXUHHQJLQHVZLOOEHEXWZHFDQDVVXPHWKDW 2I FRXUVH WKHUH LV OLWWOH GRXEW WKDW IXWXUH WKH\ZLOOEHDWOHDVWDVFDSDEOHDVWRGD\·VHQJLQHV SDUVLQJ HQJLQHV ZLOO EH LPSURYHPHQWV RQ FXUUHQW :HKDYHWKHUHIRUHFKRVHQWRZULWHRXUIRUPDO SDUVLQJ HQJLQHV:H GR QRW QHHG WR EXLOG SDUVLQJ JUDPPDUV XVLQJ WKH REMHFWV WKDW OLQJXLVWV KDYH HQJLQHVWKDWZLOOFRQWLQXHWREHXVHGGHFDGHVIURP GLVFRYHUHG SUHÀ[HV VXIÀ[HV LQÀ[HV SKRQHPHV QRZ 5DWKHU WKH ODQJXDJHVSHFLÀF LQIRUPDWLRQ SKRQRORJLFDOUXOHVDQGVRIRUWK7KHVHREMHFWVDUH WKDW JRHV LQWR D SDUVHU³WKH JUDPPDU³VKRXOG EH GHVFULEHG LQ RXU IRUPDO JUDPPDUV XVLQJ ;0/ ZULWWHQLQVXFKDZD\WKDWLWFDQEHHDVLO\UHXVHG WKDW LV XVLQJ GHVFULSWLYH WDJV WR LQGLFDWH ZKDW ZLWKIXWXUHSDUVLQJHQJLQHV HDFKOLQJXLVWLFREMHFWLV)RUH[DPSOHDVLPSOLÀHG 2QHRIRXUPRWLYDWLRQVIRUZDQWLQJWREXLOG UHSUHVHQWDWLRQRIWKH(QJOLVK-ingVXIÀ[PLJKWORRN SDUVLQJWRROVZLWKDORQJHUOLIHLVWKHSDUWWKH\SOD\ OLNHWKLV LQWKHHQWHUSULVHRIOLQJXLVWLFVDPDMRUJRDORIZKLFK VXI¿[! LV WKH GRFXPHQWDWLRQ DQG GHVFULSWLRQ RI KXPDQ  IRUP!LQJ IRUP! ODQJXDJHV3DUVLQJWRROVFRQVWLWXWHDGHVFULSWLRQRI  JORVV!3UHVHQW3DUWLFLSOH JORVV! DODQJXDJHLQWZRVHQVHVÀUVWWKHJUDPPDUWKDWWKH VXI¿[! SDUVHUXVHVLVDIRUPDOGHVFULSWLRQRIWKHODQJXDJH·V 7KHWDJVDUHQDPHGLQZD\VWKDWPDNHVHQVH PRUSKRORJ\DQGVHFRQGWKHSDUVHUFDQEHXVHGWR WROLQJXLVWVDQGVKRXOGPDNHVHQVHHYHQLI;0/ M DQDO\]HODQJXDJHWH[WVDQGWRSURGXFHSDUDGLJPV LWVHOI EHFRPHV REVROHWH VRPH GD\ 0RUHRYHU ZH 7KDWLVDSDUVHULVDQDFWLYHGHVFULSWLRQQRWDVWDWLF HPEHG WKHVH IRUPDO JUDPPDU REMHFWV GLUHFWO\ RQH LQVLGHWKHGHVFULSWLYHJUDPPDUVRWKDWZHSURYLGH :KLOH VFKRODUV RI WRGD\ FDQ UHDG ZLWK IXWXUH UHDGHUV RI RXU IRUPDO JUDPPDU HYHQ PRUH XQGHUVWDQGLQJ JUDPPDUV DQG FRUSRUD SHQQHG LQIRUPDWLRQ DERXW WKH PHDQLQJ RI WKHVH REMHFWV WKRXVDQGVRI\HDUVDJRWKHXVHRIGLJLWDOWHFKQRORJLHV )RUH[DPSOHWKHIRUPDOJUDPPDUGHVFULSWLRQRIWKH

M7HFKQLFDOO\DSDUVHUFDQDQDO\]HLQÁHFWHGZRUGVDQGDJHQHUDWRUFDQSURGXFHLQÁHFWHGZRUGVL H DSDUDGLJP  7RGD\WKHVHWZRIXQFWLRQVDUHXVXDOO\FRPELQHGLQDVLQJOHWRROFDOOHGDWUDQVGXFHU k6HH%LUGDQG6LPRQV

28 For a Better Dictionary, Build a Better Parser FEATURE

SUHVHQW SDUWLFLSOH VXIÀ[ ZRXOG EH HPEHGGHG LQWR WKHGHVFULSWLYHJUDPPDUQH[WWRDKXPDQUHDGDEOH GHVFULSWLRQRIZKDWDSUHVHQWSDUWLFLSOHLVDQGKRZ LWLVXVHGLQFOXGLQJH[DPSOHVRILWVXVHl Endangered languages ,Q RUGHU WR EH XVHG DV D SDUVHU WKH ;0/ VWUXFWXUHV RI RXU JUDPPDUV QHHG WR EH H[WUDFWHG Some languages have hundreds of millions of IURP WKH GHVFULSWLYH JUDPPDU DQG FRQYHUWHG LQWR speakers, such as English or Mandarin Chinese. WKH SURJUDPPLQJ ODQJXDJH RI D SDUVLQJ HQJLQH $OWKRXJKWKLVFRQYHUVLRQFRXOGEHGRQHE\KDQGZH But most of the seven thousand languages KDYHEXLOWDSURJUDPWKDWGRHVWKLVDXWRPDWLFDOO\IRU in the world today are much smaller—some RXUFXUUHQWSDUVLQJHQJLQH7KLVSURJUDPFDQHDVLO\ with only a few speakers. Most of these small EHUHWDUJHWHGWRIXWXUHSDUVLQJHQJLQHV,QIDFWZH languages (and even some with thousands of SUHGLFWWKDWIXWXUHSDUVLQJHQJLQHVZLOOLQFRUSRUDWH PRUHNQRZOHGJHRIOLQJXLVWLFV³IRUH[DPSOHWKH\ speakers today) will probably die out by the end ZLOO SUREDEO\ KDYH PRUH EXLOWLQ FDSDELOLWLHV IRU of this century. Such languages are referred to KDQGOLQJ WKH FRPSOH[LWLHV RI PRUSKRORJ\ WKDW ZH as “Endangered Languages”; they can be found GHVFULEHGDERYH7KLVLPSURYHGFDSDELOLW\VKRXOG around the world, but are especially common in PDNHLWHDVLHUWRFRQYHUWRXUOLQJXLVWLFGHVFULSWLRQV Africa, South America, and parts of Asia. LQWR WKH SURJUDPPLQJ ODQJXDJHV RI WKRVH IXWXUH SDUVHUV Language death is of course not new; Latin more :KLOH WKH XVH RI ;0/ DQG D FRQYHUWHU or less died with the end of the Roman Empire, SURJUDP KHOSV XV DYRLG WKH 6F\OOD RI ZULWLQJ JUDPPDUV LQ SURJUDPPLQJ ODQJXDJHV WKDW ZLOO although the modern Romance languages are VRRQ EHFRPH REVROHWH ZH DOVR QHHG WR DYRLG WKH descended from it. Today we know Latin from &KDU\EGLV RI OLQJXLVWLF WKHRULHV WKDW ZLOO OLNHZLVH documents written in Latin, and from descriptions EHFRPHREVROHWH7KHIXWXUHRIOLQJXLVWLFWKHRULHV written by Latin grammarians. Linguists today LV MXVW DV KDUG WR VHH DV WKH IXWXUH RI SDUVLQJ HQJLQHVDQGSURJUDPPLQJODQJXDJHV:HWKHUHIRUH are trying to document endangered languages QHHG WR HQVXUH WKDW RXU ;0/EDVHG IRUPDOLVP by writing down stories in those languages, and LV VLPXOWDQHRXVO\ XQGHUVWDQGDEOH VLPSOH DQG by describing the languages with dictionaries and VXIÀFLHQWO\SRZHUIXO grammars. )RUWXQDWHO\ RYHU WKH ODVW KXQGUHG \HDUV OLQJXLVWV KDYH LQYHVWLJDWHG D ZLGH YDULHW\ RI ODQJXDJHV³ZLGH HQRXJK WKDW ZH FDQ VD\ ZLWK D GHJUHH RI FRQÀGHQFH WKDW ZH NQRZ WKH UDQJH RI RSHUDWLRQVWKDWODQJXDJHVFDUU\RXWLQPRUSKRORJ\ EH LI WKH\ ZHUH ZULWWHQ XVLQJ WKH PRVW XSWRGDWH DQG SKRQRORJ\ LI QRW LQ DUHDV VXFK DV V\QWD[ WKHRUHWLFDOIDVKLRQV DQG VHPDQWLFV  /LQJXLVWLF PHFKDQLVPV SRZHUIXO HQRXJK WR KDQGOH QHDUO\ DOO VXFK RSHUDWLRQV ZHUH Problem: labor-intensive nature of GHYHORSHGÀIW\\HDUVDJRPXFKOLQJXLVWLFUHVHDUFK grammar production VLQFH WKHQ KDV EHHQ DLPHG DW LVVXHV LUUHOHYDQW WR %XLOGLQJ D JUDPPDU LV GLIÀFXOW WLPH SDUVLQJ 7KHUHIRUH ZH FDQ ZULWH RXU GHVFULSWLRQV FRQVXPLQJ DQG H[SHQVLYH ,Q IDFW RXW RI WKH XVLQJ WKH ROGHU OLQJXLVWLF PHFKDQLVPV LQIRUPHG WKRXVDQGVRIODQJXDJHVRIWKHZRUOGOLQJXLVWVKDYH LQDIHZFDVHVE\QHZHUGLVFRYHULHV$QGLQIDFW GHVFULEHGWKHJUDPPDUVRIOHVVWKDQDWKRXVDQGDQG GHVFULSWLRQV XVLQJ WKHVH ROGHU IRUPDOLVPV DUH LQ PDQ\RIWKHVHGHVFULSWLRQVDUHIDUIURPFRPSOHWH³ PDQ\FDVHVHDVLHUWRXQGHUVWDQGWKDQWKH\ZRXOG SUHFLVHO\EHFDXVHJUDPPDUVDUHGLIÀFXOWWRZULWH

l6RPHUHDGHUVPD\UHFRJQL]HWKDWWKLVLVDQLPSOHPHQWDWLRQRIDWHFKQLTXHNQRZQDPRQJFRPSXWHUVFLHQWLVWVDV /LWHUDWH3URJUDPPLQJ

The Next Wave „ Vol 18 No 1 „ 2009 29 Solution: efficient grammar production IDUDSSURDFKOHDUQLQJZLWKRXWUHIHUHQFHWRH[LVWLQJ and portability to other languages JUDPPDUV RI UHODWHG ODQJXDJHV 7KLV OLPLWDWLRQ LV 7KHGLYLVLRQRIODERUZHKDYHFUHDWHGDPRQJ DWOHDVWLQSDUWGXHWRWKHIDFWWKDWXQWLOQRZWKHUH WKHGHVFULSWLYHJUDPPDUZULWHUWKHFRQVXOWDQWVDQG KDYH EHHQ QR VWDQGDUG IRUPDO ZD\V RI ZULWLQJ WKHIRUPDOJUDPPDUZULWHUFRXOGEHGXSOLFDWHGLQ JUDPPDUV :LWKRXW VXFK VWDQGDUGV WKHUH DUH QR TXLWH GLIIHUHQW VLWXDWLRQV 2XU PHWKRGRORJ\ DQG IRUPDOJUDPPDUVIURPZKLFKDGDSWDWLRQFRXOGEH WHPSODWHVDUHDGDSWDEOHE\RWKHUWHDPVRIWUDGLWLRQDO GRQH:HKRSHWRFKDQJHWKDWWKURXJKWKLVSURMHFW DQGFRPSXWDWLRQDOOLQJXLVWV 7KHIRUPDOJUDPPDUVWDQGDUGVGHYHORSHGKHUHDQG WKHDFWXDOIRUPDOJUDPPDUVEHLQJGHYHORSHGPHDQ )RU H[DPSOH D SRWHQWLDO JUDPPDUZULWLQJ WKDW WKHUH LV QRZ D FRQFUHWH EDVLV IRU DXWRPDWLF WHDP ZRXOG LQFOXGH D JUDPPDU ZULWHU FRQWUDFWHG JUDPPDUDGDSWDWLRQ WRZULWHWKHGHVFULSWLYHJUDPPDUWRJHWKHUZLWKDQ LQKRXVHFRPSXWDWLRQDOOLQJXLVWZKRZRXOGDWWHPSW Problem: how do we test the accuracy of WR ZULWH D FRUUHVSRQGLQJ IRUPDO JUDPPDU IURP the grammar? WKHGHVFULSWLYHJUDPPDUDQGWKHQWHVWWKDWIRUPDO 2QHRIWKHDXWKRUVZDVLQYROYHGWZHQW\\HDUV JUDPPDUE\FRQYHUWLQJLWLQWRDSDUVHU%\VWDJLQJ DJR LQ HGLWLQJ RWKHU SHRSOH·V JUDPPDUV +H ZDV WKLV SURFHVV LQ ORJLFDO VHFWLRQV IRU H[DPSOH WKH FRQWLQXDOO\IUXVWUDWHGEHFDXVHKHFRXOGXVXDOO\WHOO FDVHPDUNLQJRIQRXQVPLJKWEHRQHPRGXOH WKH ZKHWKHUDJUDPPDWLFDOGHVFULSWLRQPDGHVHQVHEXW GHVFULSWLYHJUDPPDULDQFDQEHHYDOXDWHGWKURXJKRXW LWZDVGLIÀFXOWLIQRWLPSRVVLEOHWRWHOOZKHWKHUWKH WKHSURFHVVDQGJLYHQIHHGEDFNRQKRZWRLPSURYH GHVFULSWLRQZDVFRUUHFW³WKDWLVZKHWKHULWDFWXDOO\ WKH GHVFULSWLRQ 4XDOLW\ FRXOG WKXV EH FRQWUROOHG FRYHUHGWKHODQJXDJHGDWDLQFOXGLQJWKHH[DPSOHV QRW RQO\ LQ WHUPV RI ZKHWKHU WKH GHVFULSWLRQ LV LQWKHJUDPPDU,WZDVWRRKDUGWRZRUNWKURXJKDOO XQGHUVWDQGDEOH EXW ZKHWKHU LW LV REMHFWLYHO\ WKHJUDPPDUUXOHVE\KDQGIRUHDFKH[DPSOHDQGDW FRUUHFW,QVXPZHZRXOGYLHZRXUZRUNDVEHLQJ WKDWWLPHWKHWHFKQRORJ\WRWHVWWKHJUDPPDURQWKH DVHJXHLQWRZD\VRILPSURYLQJWKHRXWVRXUFLQJRI FRPSXWHUZDVQRWUHDGLO\DYDLODEOH JUDPPDUGHYHORSPHQW Solution: incorporating verifiability *UDPPDUVPLJKWDOVREHEXLOWE\DGDSWLQJD 2XUSURMHFWPDNHVWHFKQRORJ\IRUWHVWLQJWKH JUDPPDURIRQHODQJXDJHWRVHUYHDQRWKHUFORVHO\ JUDPPDUDYDLODEOH³DQGPRUHRYHUPDNHVLWXVDEOH UHODWHGODQJXDJH0RVWODQJXDJHVRIWKHZRUOGDUH GXULQJ WKH JUDPPDU ZULWLQJ UDWKHU WKDQ ODWHU UHODWHG WR VRPH VHW RI RWKHU ODQJXDJHV 6SDQLVK 7KHUHIRUHZHFDQYHULI\WKDWRXUJUDPPDUZRUNV ,WDOLDQ3RUWXJXHVHDQG)UHQFKIRUH[DPSOHDUHDOO XVLQJWKHYHU\H[DPSOHVDQGSDUDGLJPFKDUWVWKDWLW GHVFHQGHGIURP9XOJDU/DWLQDQGWKHUHIRUHH[KLELW FRQWDLQVDVZHOODVDQ\DGGLWLRQDOHOHFWURQLFWH[WV VLPLODULWLHV LQ YRFDEXODU\ DQG JUDPPDU 2WKHU :HFDQGHPRQVWUDWHWKDWRXUJUDPPDUVPDNHVHQVH JURXSVRIPRUHRUOHVVFORVHO\UHODWHGODQJXDJHVDUH DQGWKDWWKH\ZRUNRQUHDOODQJXDJHGDWD WKH7XUNLFODQJXDJHV 7XUNLVK DQG8]EHNDPRQJ RWKHUV  ,QGR$U\DQ ODQJXDJHV VXFK DV +LQGL Conclusion 8UGX %DQJOD 'UDYLGLDQ ODQJXDJHV OLNH 7DPLO /LQJXLVWVZRUNLQJZLWKDODQJXDJHLQZKLFK DQG 0DOD\DODP VWLOO RWKHU JURXSV LQFOXGH PRVW WKH\ODFNÁXHQF\RIWHQIDFHDGDLO\FKDOOHQJHZLWK 3KLOLSSLQH ODQJXDJHV RU WKH %DQWX ODQJXDJHV RI GLFWLRQDU\ ORRNXS 8VLQJ RXU SULQFLSOHG DSSURDFK PLG$IULFD:LWKLQHDFKRIWKHVHJURXSVLWLVRIWHQ RI EXLOGLQJ D PRUSKRORJLFDO SDUVHU EDVHG RQ WKHFDVHWKDWVRPHODQJXDJHVKDYHEHHQGHVFULEHG SDUDOOHO JUDPPDUV ZH DUH FUHDWLQJ WRROV WKDW ZLOO PRUH FRPSOHWHO\ DQG FRUUHFWO\ WKDQ RWKHUV $ KHOS OLQJXLVWV ZKLOH GHYHORSLQJ PHWKRGV WKDW ZLOO SURPLVLQJ DSSURDFK ZRXOG EH WR ZULWH JUDPPDUV PDNH VXFK OLQJXLVWLF WRRO EXLOGLQJ PRUH HIÀFLHQW IRUWKHZHOOGHVFULEHGODQJXDJHVDQGWKHQWRDGDSW DQG FRVWHIIHFWLYH 7KH SD\RIIV LQFOXGH QRW RQO\ WKHIRUPDOJUDPPDUV DQGSHUKDSVWKHGLFWLRQDULHV  PRUH HIÀFLHQW DQG FDSDEOH OLQJXLVWV EXW DOVR D WRWKHOHVVZHOOGHVFULEHGODQJXDJHV7KLVSURFHVV PHWKRGRORJ\ WKDW FDQ EH XVHG WR EXLOG JUDPPDUV FRXOGEHGRQHE\KDQGEXWDXWRPDWLFDGDSWDWLRQLV IRURWKHUODQJXDJHVLQWKHIXWXUH DQRWKHUSRVVLELOLW\ 7KHUHDUHPDQ\RWKHUSRWHQWLDOEHQHÀWVIURP $XWRPDWLFDGDSWDWLRQKDVQRWEHHQDVXEMHFW RXU ZRUN )RU LQVWDQFH WKH H[DPSOH VHQWHQFHV RIJUHDWLQWHUHVWDPRQJUHVHDUFKHUVPRVWPHWKRGV DQG SDUDGLJP FKDUWV FRQWDLQHG LQ RXU GHVFULSWLYH IRUDXWRPDWLFPRUSKRORJ\OHDUQLQJGHYHORSHGWKXV JUDPPDUV PD\ KHOS ODQJXDJH DQDO\VWV QRZ 30 For a Better Dictionary, Build a Better Parser FEATURE

Dialectal variation

All languages exhibit dialectal variation, most often based on geographical or social differences. For example, English speakers use a wide variety of second person plural pronouns: you (standard American and British English), y’all (Southern American English, African American vernacular English, some Western American varieties, and South Asian English), you-uns (Western Pennsylvania and Appalachians), you guys (colloquial non-Southern American English, Australian English), yous(e) (Irish English and mid-Atlantic American English), and so on. For an even wider range of dialectal variation within English, listen to some of the sound files on the International Dialects of English Archive (http://web.ku.edu/idea/).

Similarly, some words in the languages of our project vary according to region, and we have tried to address this phenomenon in our grammars by listing alternate forms when we hear of them from our informants or in our written sources.

DOWKRXJKWKH\ZHUHLQWHQGHGWRFODULI\XVDJHLVVXHV WKHVH ODQJXDJHV GLH RXW ZLWK WKH ODVW VSHDNHUV IRU IXWXUH FRPSXWDWLRQDO OLQJXLVWV ,Q WKLV UHJDUG ZKRVHFKLOGUHQDUHWXUQLQJWRRWKHUODQJXDJHV $ XVLQJ;0/IRUPDWVDOORZVXVWRH[WUDFWWKHUHOHYDQW JRDORIPDQ\OLQJXLVWVLVWRSUHVHUYHWKLVNQRZOHGJH SRUWLRQVRIRXUJUDPPDUVDQGWXUQWKHPLQWR´KHOSµ *UDPPDUV RI HQGDQJHUHG ODQJXDJHV FRXOG EH GRFXPHQWVIRUODQJXDJHDQDO\VWV)RUH[DPSOHZH ZULWWHQ XVLQJ WKLV SURFHVV VR WKDW D KXQGUHG RU D FRXOGHDVLO\H[WUDFWDQH[SODQDWLRQRIWKHXVHRIWKH WKRXVDQG\HDUVIURPQRZWH[WVLQWKHODQJXDJHFDQ ´ORFDWLYHLQVWUXPHQWDOµ FDVH LQ %DQJOD WRJHWKHU EHVWXGLHGDQGWKHJUDPPDWLFDOV\VWHPUHFRQVWUXFWHG ZLWKH[DPSOHVRILWVXVH PRUHUHOLDEO\WKDQLVSRVVLEOHZLWKSXUHO\GHVFULSWLYH )XUWKHUPRUH D PRUSKRORJLFDO SDUVHU LV D JUDPPDUV QHFHVVDU\WRROIRUPDFKLQHWUDQVODWLRQRIODQJXDJHV ZLWK FRPSOH[ PRUSKRORJ\ :KLOH KLJKHU TXDOLW\ PDFKLQHWUDQVODWLRQUHTXLUHVUHVRXUFHVDYDLODEOHIRU        RQO\DIHZODQJXDJHVWKHUHDUHDSSOLFDWLRQVWKDWGR         QRW UHTXLUH VXFK KLJK TXDOLW\³LQGHHG ZRUGIRU          ZRUG´WUDQVODWLRQµPD\EHDGHTXDWHIRUVXFKWDVNV         DVVLIWLQJWKURXJKDODUJHQXPEHURIGRFXPHQWVWR       ÀQGWKHIHZWKDWPD\EHZRUWKWUDQVODWLQJE\KDQG          )LQDOO\ RXU PHWKRGRORJ\ FDQ EH XVHG WR          GRFXPHQWODQJXDJHVLQDQHQGXULQJZD\7KHUHDUH       PRUHWKDQVHYHQWKRXVDQGKXPDQODQJXDJHVLQWKH      ZRUOG DQG PRVW RI WKHP KDYH OLWWOH WR QR         GRFXPHQWDWLRQ ,PSRUWDQW NQRZOHGJH RI KRZ         KXPDQ ODQJXDJHV ZRUN LV EHLQJ ORVW DV PDQ\ RI         

The Next Wave „ Vol 18 No 1 „ 2009 31 References

Bird S, Simons G. Seven dimensions of portabil­ ity for language documentation and description. Language. 2003;79:557-582.

Knuth DE. Literate Programming: CSLI Lecture Notes. Stanford (CA): Center for the Study of Language and Information; 1992.

Maxwell M, David A. Interoperable grammars. Proceedings from First International Conference on Global Interoperability for Language Resources (ICGL 2008); Hong Kong; 2008.

Maxwell M, David A. Joint grammar development by linguists and computer scientists. Proceedings from Workshop on NLP for Less Privileged Languages, Third International Joint Conference on Natural Language Processing; Hyderabad, India; 2008.

Poser WJ. Making Athabaskan dictionaries us­ able. In: Holton G, editor. Proceedings of the Athabaskan Languages Conference; Fairbanks (AK). Alaska Native Language Center, University of Alaska; 2002. p. 136-147.

32 For a Better Dictionary, Build a Better Parser FEATURE

A Talent for Language

hat makes a sports champion? Is Tiger Test (MLAT), and the VORD [2]. (VORD requires Woods—the only golfer to hold all four WHVW WDNHUV WR DSSO\ UXOHV WR DUWLÀFLDO ODQJXDJH Wprofessional major championships at segments presented in context. VORD is the word the same time—the sport’s greatest because he was IRUZRUGLQWKHDUWLÀFLDOODQJXDJHZKLFKLVEDVHG born to golf, because he trained hard every day for on Turkish, a language typologically different from years, or because he had the help of an expert coach? Western European languages.) Currently, the DLAB Most people would claim that athletes need all three and the MLAT are the most widely used tests for US of these factors for success: talent, hard work, and government (USG) purposes. Although these tests good coaching. Is the same true for successful are useful for establishing cutoff scores for basic linguists? What makes a linguistic champion? Are language study and for supporting hiring decisions, KDUG ZRUN DQG JRRG FRDFKLQJ VXIÀFLHQW RU LV D they may not predict with enough accuracy which special talent for language necessary? Researchers learners will succeed and which learners will either at the University of Maryland Center for Advanced GURSRXWRUIDLOWRPHHWSURÀFLHQF\VWDQGDUGV Study of Language (CASL) are trying to answer During the years since these tests were these questions through a set of related studies on developed, cognitive psychologists have made foreign language aptitude. substantial progress in the understanding of Aptitude refers to a person’s inherent capability human memory and learning. Also, during or underlying talent. If a classroom has ten students, WKLVSHULRGVLJQLÀFDQWDGYDQFHVZHUHPDGH all with the motivation to work hard and the same in understanding how foreign languages RSSRUWXQLWLHV WR EHQHÀW IURP JRRG WHDFKLQJ RU DUHOHDUQHG7KHÀHOGRI6HFRQG/DQJXDJH coaching, students’ results will still vary based Acquisition (SLA) was established, and upon aptitude. John B. Carroll, a distinguished 6/$ UHVHDUFK ÀQGLQJV SURPSWHG D psychologist and co-developer of the Modern paradigm shift in language teaching /DQJXDJH$SWLWXGH 7HVW 0/$7  GHÀQHG IRUHLJQ methodology. Contemporary language aptitude as an “…individual’s initial state approaches emphasize developing of readiness and capacity for learning a foreign communication rather than the language, and probable degree of facility in doing study of language as object and, VRµ>@7RVWXG\DSWLWXGHVFLHQWLÀFDOO\UHVHDUFKHUV thus, incorporate interactive and PXVWGHWHUPLQHKRZWRULJRURXVO\GHÀQHFRQFHSWV experiential learning tasks. In light such as state of readiness and capacity for learning RI WKHVH VFLHQWLÀF DGYDQFHV H[LVWLQJ and how to measure these abilities reliably. aptitude tests have become outdated $WWHPSWVWRGHÀQHIRUHLJQODQJXDJHDSWLWXGH in terms of their underlying constructs and determine its subcomponents had begun by the and their congruence with how foreign 1920s in the United States [1]. During subsequent languages are learned. decades, several tests were developed, with the Based upon these advances in theories of support of the military, for purposes of personnel language learning, and upon the increasingly critical selection and placement into foreign language need for a large number of well-trained linguists, training. These include the Army Language Aptitude the USG has asked CASL to lead research and Test (ALAT), the Defense Language Aptitude development efforts to improve the measurement of Battery (DLAB), the Modern Language Aptitude foreign language aptitude in terms of (a) identifying

The Next Wave „ Vol 18 No 1 „ 2009 33 A New Aptitude-Screening Test

The Pre-DLAB is a short screening test that predicts who is likely to attain the DLIFLC entry cut-off score on the DLAB. The Pre-DLAB can be administered in less than 30 minutes, in any setting, to thousands of people (e.g., each year over 500,000 people take enlistment tests at Military Entrance Processing Stations). Since the DLAB takes 90 minutes, it will be more efficient if only those who pass the Pre- DLAB screening test move on to take the DLAB during the recruitment process. Using the test specifications that were developed during previous research, new items were written at Second Language Testing, Inc. for certain sections of DLAB, taking care to match the new items to the aptitude constructs underlying the original test. CASL then conducted a study with 500 university volunteers who took both the DLAB and the prototype Pre-DLAB. Findings showed that the initial form of the Pre-DLAB accurately predicts a cut score of 95 on the DLAB. Moreover, the Pre-DLAB test does not require audio equipment, thus it is adaptable to administration in a wide range of settings. This pre-testing greatly increases the military’s or USG’s ability to identify talented language learners.

potential for language expertise, (b) mitigating able to identify which beginning or intermediate attrition from language training, and (c) casting the level learners would be most likely to overcome net more widely to increase the number of individuals the critical period effect and achieve high-level tested for language aptitude. In response, CASL has competence in a foreign language, given extensive undertaken research on language aptitude: training. ‡ To predict the ability to reach high-level Hi-LAB, a new test currently under SURÀFLHQF\LQDIRUHLJQODQJXDJHWKURXJKWKH development at CASL, is designed to identify creation of the High-Level-Language Aptitude individuals with the capacity to reach advanced Battery (Hi-LAB), an innovative test that OHYHOV RI IRUHLJQ ODQJXDJH SURÀFLHQF\ +LJKOHYHO measures the ceiling on potential and predicts ODQJXDJHDSWLWXGHLVGHÀQHGDVDPHDVXUDEOHceiling the maximum level of language-learning on language-learning ability, holding equal all success other factors such as motivation, stable personality ‡ To update aptitude testing in and mitigate characteristics, and opportunities for instruction attrition from basic language programs at or immersion. Hi-LAB constructs (Table 1) were the Defense Language Institute motivated by theories of learning and memory Foreign Language Center (DLIFLC) by updating DLAB to DLAB 2, Constructs Brief Definitions comprising measures of aptitude, MEMORY the capacity to process and store input with active trade-offs among these components: personality, and motivation Working Memory Short-term the small amount of information that can be ‡ To manage the tremendous increase Memory kept in an accessible state in order to be used in the amount of aptitude testing Capacity in ongoing mental tasks that must be done to identify Executive a set of processes that, collectively, regulate Capacity & and direct attention and control voluntary candidates for foreign language Control processing

training at DLIFLC by developing Long-term Memory Rote Memory explicit, intentional long-term memory storage and validating Pre-DLAB, a short that results from rehearsal screening test that can be easily ACUITY Perceptual an individual’s capacity to detect difficult-to- administered to thousands of military Acuity perceive auditory or visual information recruits SPEED Processing speed of an individual’s perceptual, motor, or Speed decision responses Hi-LAB – finding language PRIMABILITY Priming the extent to which prior experience facilitates subsequent processing experts INDUCTION the process of reasoning from the specific to Existing aptitude tests were designed to the general, i.e., noticing similarities among several instances and drawing a generalization predict early rate of learning and successful based on these similarities

attainment of intermediate level language Implicit acquiring the statistical patterns contained Induction within complex input without the learner’s SURÀFLHQF\6XFKWHVWVFDQEHYHU\XVHIXO conscious awareness when managers must decide who should Explicit acquiring the patterns in input through be selected for basic language training. Induction conscious awareness and reasoning

However, many learners perform well in PRAGMATIC In Research & the ability to hypothesize connections early training, but never reach the higher SENSITIVITY Development between context and use: registering and tracking salient context cues; detecting OHYHOV RI SURÀFLHQF\ QHFHVVDU\ IRU PDQ\ miscommunication MREV 7KH VFLHQWLÀF H[SODQDWLRQ LV NQRZQ FLUENCY In Research & the ability automatically to plan and articulate as the critical period effect. Time, money, Development speech and effort could be saved if testing were Table 1: Hi-LAB Components 34 A Talent for Language FEATURE

Age and Foreign Language Learning

Studies reveal a tight correlation between age of first exposure to a foreign language and overall success to Level 4 in that language throughout the neurological critical period [3]. At around puberty, the age-success correlation falls off sharply, as shown schematically in Figure 1. Nonetheless, some individuals do appear to attain near-native expertise [4]. What characterizes the individuals who become experts in a foreign language as adults and, more importantly, can they be identified at the outset of learning? derived from research in cognitive psychology and by theories of second language acquisition. Multiple measures of each construct, developed or adapted High Lower correlation correlation No correlation by CASL, are undergoing usability, reliability, and Native-l ke validity testing with government, military, and High-level university populations. During this iterative testing process, factor analytic procedures are used to identify the most useful measures for inclusion in the operational version of the test [5]. An important innovation in Hi-LAB is that Learning Success abilities are measured behaviorally, by computerized Level 4 Language- testing. This direct-measurement methodology 0 3 6 9 12 15 18 21 24 ... provides several advantages over more traditional Age of First Exposure testing formats, such as paper and pencil tests. Computer-delivery ensures that both accuracy and Figurre 1e Schematic1: Schematic Correlat correlationon Sca scatterer Plotplot forfo reaction time (in milliseconds) are available for each age/level 4 language learning success cognitive or perceptual task included in the battery. Reaction time is a good indicator of automaticity, a hallmark of foreign language expertise. Each of the Hi-LAB constructs is hypothesized to be important for moving from intermediate to professional or high levels of foreign language SURÀFLHQF\7KXV WKH WHVW LV GHVLJQHG WR PHDVXUH “what is left to learn” at this relatively advanced stage of language learning. Some constructs, such Fixation Point as perceptual acuity, are clearly important for the Stimulus beginning stages of learning as well. For example, Stimulus beginning learners must learn to hear and produce the sounds of a new language (perceptual acuity). Stimulus Participant 2WKHU +L/$% FRQVWUXFWV PD\ UHÁHFW DVSHFWV RI Response Stimulus aptitude that are not predictive of early language Buttons success, but that are critical for moving beyond The Serial Reaction Time task asks examinees to respond to the location of an asterisk on the intermediate stages. One example of this type of computer screen. The asterisk can appear in one of a number of locations, each with a corresponding construct is pragmatic sensitivity ZKLFK UHÁHFWV response button. Unknown to the examinee, the locations follow a designated 10-trial pattern, in a learner’s ability to learn and use the aspects of which one location was never followed by the same location more than once. This means that to language that depend on context. For example, know the location of the next asterisk, one would need to know the entire 10-trial string and current serial position in the string. After a number of string repetitions, the order is switched to a randomized a person high in pragmatic sensitivity would be order which does not follow the pattern before returning to the repeating strings. Scoring of the Serial more likely to accurately learn the correct forms of Reaction Time task depends on the average reaction times during the sequence repetition condition address for persons of differing social ranks or to versus the average reaction times during the random condition. The longer reaction times after notice nonverbal signals indicating that the listener switching to a random sequence are an indication of the facilitation the examinee experiences when has misunderstood something. they have implicitly learned, or induced, the sequence of the repeating trials.

Another construct of this type is implicit Figure 2: The Serial Reaction Time task induction, which refers to an individual’s ability to measure of implicit induction

The Next Wave „ Vol 18 No 1 „ 2009 35 learn the statistical regularities in a set of complex understanding of the human memory system. data, without conscious awareness or explicit Memory comprises several sub-systems. Previous reasoning. Research has shown, for example, aptitude tests primarily measured long-term that human infants can use this sort of statistical memory, or the ability to store information in a learning to learn where word boundaries are in a permanent form and recall it after a delay. Like string of incoming syllables produced by adults [6]. these previous tests, Hi-LAB contains measures of By deriving these word boundaries from statistical long-term memory, but unlike these older tests, Hi- regularities in the language they hear, babies are LAB also includes measures of working memory able to “bootstrap” the process of learning their (WM) with its component systems, short-term ÀUVW ODQJXDJH 3V\FKRORJLFDO UHVHDUFKHUV KDYH memory and executive capacity and control. The not yet resolved to what extent adults are able to short-term memory (STM) system allows a person make use of this type of statistical learning. It is to hold information in an accessible state for a few possible, however, that individuals who maintain seconds, such as when rehearsing a phone number. this ability in adulthood will be better able to learn complex, probabilistic aspects of a foreign language +L/$%VSHFLÀFDOO\WDSVRQH670VXEFRPSRQHQW namely verbal-acoustic STM, which aids in the grammatical system that are necessary for “native- rehearsal or maintenance of unfamiliar words, such like” performance. The serial reaction time task, as vocabulary in a foreign language [7]. shown in Figure 2, measures implicit induction. The executive capacity and control system Advances in memory controls the focus of attention and includes +L/$%LVWKHÀUVWIRUHLJQODQJXDJHDSWLWXGH three separable sub-constructs [8, 9]—updating, test to incorporate important advances in the inhibition, and task-switching. Updating refers to

+ * Baseline RED Condition Fixation Point Congruent RED Condition Flash

Incongruent GREEN Congruent Arrows Participant Condition Response Buttons Incongruent BLUE Condition + Participant * Response Buttons Incongruent * Condition Fixation Point Flash The Standard Stroop Task The original Stroop task is a measure of an individual’s ability to inhibit the highly automatic skill of reading. Examinees Incongruent Arrows Participant Response view a series of color words (red, green, or blue) and color swatches (also Buttons red, green, or blue). The task is to name the “ink” color in which the color word or the color swatch is printed. For some of the color words, the color in The Attention Network Test (ANT) is a measure that combines cued which the word is displayed matches the word (example “RED” printed in red reaction time and the Flanker Task. The ANT requires participants to ink). This is referred to as the “congruent condition.” For other color words, determine whether a centrally presented arrow points to the left or to the the ink color does not match the word (example “RED” printed in blue ink). right. The arrow appears above or below a fixation point, following a brief This is referred to as the incongruent condition. When the colors are flash of asterisks that may or may not appear where the arrow is displayed. presented as swatches, this is the “baseline condition.” The degree of slowing The central arrow may be accompanied by flanker arrows to the right and left. When flanker arrows are present, they appear under one of two conditions: produced by incongruent stimuli, rather than baseline or congruent stimuli, is congruent, where the flanker arrows point in the same direction as the target measured and can be reported either absolutely (number of milliseconds arrow, and incongruent, where the flanker arrows point in the opposite slower for incongruent stimuli) or relatively (proportion of average response direction. Efficiency of attentional networks is assessed by measuring how time slower for incongruent stimuli). Individuals who are faster to respond to response times are influenced by alerting cues, spatial cues, and flankers. incongruent stimuli are more effective at inhibiting automatic processes that would otherwise interfere.

Figure 3: The Stroop task measure of inhibition [13, 14] Figure 4: The Flanker task measure of inhibition [15, 16, 17]

36 A Talent for Language FEATURE

the process of refreshing the contents of working classroom-based, military environment at DLIFLC. memory with new, more relevant information [10]. Students who succeed in this setting may differ from Inhibition is the ability to ignore a dominant or those who would do well in a language immersion automatic response when necessary, a skill that may situation or in classes at a university. Success in the be particularly necessary for effective bilingual DLIFLC environment may depend on a number of functioning, which involves inhibiting the strong attributes other than purely cognitive or perceptual ÀUVWODQJXDJHLQIDYRURIWKHIRUHLJQODQJXDJH> aptitude. Recall the three prerequisites for success 12]. See Figures 3 and 4 for test examples. discussed initially—talent, hard work, and good Finally, task-switching, the ability to shift coaching. While the Hi-LAB project focuses between multiple tasks, operations, or mental sets precisely on identifying the talent of individual learners, the DLAB 2 project aims to quantify >@LVK\SRWKHVL]HGWRUHÁHFWDQDVSHFWRIFRJQLWLYH potential for hard work. Thus, in addition to updating FRQWUROWKDWLVFULWLFDOIRUHIÀFLHQWELOLQJXDOOH[LFDO selection and for advanced language tasks such as the existing cognitive and perceptual measures, this translation or switching between two languages project incorporates a set of non-cognitive measures that may predict which learners will be most likely [11, 19, 20]. The color-shape task shown in Figure to persevere in an intensive program and achieve 5 measures the mental cost of switching from language learning success at DLIFLC. Although deciding between shapes to distinguishing colors. these are non-cognitive attributes, they may be By incorporating measures of these sub- included in a new version of the DefenseUNCLA LanguageSIFIED components of memory, along with the other constructs measured, Hi-LAB will provide the government with a more sensitive test of foreign SHAPE language aptitude and one that is geared toward identifying language expertise. While Hi-LAB SHAPE measures cognitive and perceptual measures of aptitude to predict high-level success, another CASL research and development effort, DLAB 2, SHAPE incorporates additional, non-cognitive measures COLOR to make other predictions about foreign language SHAPE learning. COLOR DLAB 2—revised Defense Language Aptitude Battery COLOR Learning a foreign language is neither easy nor fast. To succeed, a learner must stick with the COLOR program for an extended period of time, push on when progress seems to have stalled, overcome setbacks, and work on maintenance once a goal has been reached. Gaining expertise in a foreign Task-switching Task-switching refers to the process of shifting between multiple tasks, language requires that even the most talented operations, or mental sets. There are significant individual differences in task-switching learners commit years to study and practice. As a ability. Some individuals are markedly better at doing two or three things at once than result, predicting which learners will be successful others, controlling for the extent of prior practice and background knowledge relevant to may require consideration not only of cognitive and the task situation. In this test, individuals see stimuli consisting of triangles or squares in yellow or pink blocks followed by a prompt. Depending on the prompt that is displayed, perceptual abilities, but also non-cognitive factors they must pay attention to shape and ignore color or the reverse, with no prior knowledge such as motivation, interests, beliefs about learning, about when a switch in prompt will occur. Mean reaction times and accuracy are recorded and differences in personality. CASL is currently for the single-task test blocks (shape and color) as well as for the mixed condition that examining these issues for the Defense Language contain switch and non-switch items in random order. Items that involve the same task as the previous item (“non-switch”) can then be compared to those that immediately follow a Institute Foreign Language Center (DLIFLC) as task-switch (“switch”). In addition, non-switch items are compared to the baseline accuracy part of the DLAB 2 project. and reaction time information for each task when performed in the non-mixed condition. The goal of this project is to predict success in learning a foreign language within the intensive, Figure 5: Task-switching measure of attentional control [18]

The Next Wave „ Vol 18 No 1 „ 2009 37 Aptitude Battery, because they help to address such The new cognitive ability measures in DLAB a test’s main goal—to predict which learners are 2 are a small subset of those measures in Hi-LAB most likely to succeed. and will update this aspect of the test. The addition Like the original DLAB, the goal of the DLAB of non-cognitive measures is expected to directly 2 test is to predict which learners will be able to address course attrition. These measures recognize learn quickly and reach minimum levels of foreign the fact that adults bring to language study an established set of preferences attitudes, learning ODQJXDJH SURÀFLHQF\ WR PHHW '/,)/& JUDGXDWLRQ requirements, given a set amount of training time strategies, and motivations. If those selected for This time can vary from 25 weeks of full-time language training at DLIFLC have what it takes to study for a language such as Spanish or French, up persevere, plus a talent for language learning, they can be expected to work hard and graduate with at WRZHHNVRIIXOOWLPHVWXG\IRUDPRUHGLIÀFXOW language such as Arabic or Mandarin Chinese. An OHDVWEDVLFODQJXDJHSURÀFLHQF\>@ important aspect for success in such an intense The future of aptitude research program is the learner’s ability to remain motivated There are many additional questions that can be to work week after week and to overcome the asked about foreign language aptitude and a variety LQKHUHQWGLIÀFXOWLHVDQGVWUHVVHVLQYROYHGLQORQJ of new techniques that can be employed for aptitude WHUPODQJXDJHVWXG\%HFDXVHRIWKHVHGLIÀFXOWLHV research One of the most promising areas for student attrition is a major problem, and measures enhancing the measurement of aptitude is the use of that predict who is likely to “stick with” the program techniques from cognitive neuroscience³DÀHOGRI may provide important additional information to the study that combines psychology with neuroscience JRYHUQPHQW&$6/UHVHDUFKHUVKDYHLGHQWLÀHGDQG through the use of modern technologies such as piloted a set of possible new measures and tested brain imaging. (See the article “Thinking Out of WKHP(DFKRIWKHPHDVXUHVEHORQJVWRRQHRIÀYH the Box” in this issue.) These techniques allow for major categories: visualizing individual differences in brain structure ‡ Cognitive Abilities—such as general or measuring changes in brain state while a person intelligence, memory, attention, and auditory completes test tasks. How might these techniques perception be used to enhance our understanding of foreign ‡ Learning Orientation—such as preferred language aptitude? One example might be by using learning styles or activities and learning goals these technologies to provide a more sensitive measure of an individual’s ability to discriminate Personality—such as levels of ‡ foreign language sounds conscientiousness, anxiety, and openness to experience We know from research on speech perception that infants are able to distinguish sound contrasts ‡ 6HOI (IÀFDF\³VXFKDVEHOLHIVLQWKHDELOLW\WR important for any of the world’s languages [21]. succeed and the ability to cope with setbacks However, by the time an infant is one year old, his ‡ Motivation—such as motivation to achieve, to brain has “tuned in” to those contrasts necessary OHDUQDQGWRPDVWHUGLIÀFXOWPDWHULDO for the native language and learned to average

“bug...bug” “pug...bug”

1 2 1 2

Press 1 if the sounds are the same Press 1 if the sounds are the same Press 2 if the sounds are different Press 2 if the sounds are different

Figure 6: Phonemic discrimination task

38 A Talent for Language FEATURE

500

250

Magnetic Field (fT) Figure 7: MEG measurement of stimuli Changes in the magnetic field surrounding the skull measured from onset of a

0 stimulus and continuing for 500 ms. Each line represents the signal (in femtoTesla) recorded from one detector. Signals from multiple detectors are recorded over various areas of the scalp. Peaks in the signal represent purported sub-processes taking place during the brain’s recognition of the input stimulus. -250

-500 0 50 100 150 200 250 300 350 400 450 500 Time (msec) over contrasts that are not important for the native ZD\ WR LQFUHDVH HIÀFLHQF\ RI ODQJXDJH WUDLQLQJ language [22]. Thus, older children and adults are Furthermore, technology may assist instructors to often no longer able to discriminate contrasts that provide individualized learning plans and activities are not used in their native language, but that might that assist learners with their weaknesses, build upon be very important for a foreign language. Both the their strengths, and help maintain motivation and Hi-LAB and DLAB 2 projects examine the ability to interest during long-term study by accommodating hear these non-native contrasts by asking test-takers learners’ personalities and preferred learning ZKHWKHUWZRVRXQGVWKDWDUHGLIÀFXOWWRGLVFULPLQDWH activities. This type of instructional design would are “the same” or “different,” as shown in Figure 6. DOORZWKHÀQGLQJVIURPDSWLWXGHUHVHDUFKWRLQIRUP the third of our three keys for success—good The hypothesis is that some adults may retain coaching. the ability to distinguish these non-native contrasts even though most people have lost that ability. Finally, future research might address the Such learners may truly have “an ear” for foreign issue of differential aptitude, allowing for a better languages. Perhaps an even more sensitive way to matching of student to foreign language. Differential test this ability is by measuring changes in the brain’s aptitude refers to the hypothesis that learners’ DFWLYLW\ZKHQDOLVWHQHULVSUHVHQWHGZLWKWZRGLIÀFXOW SDWWHUQV RI VWUHQJWKV DQG ZHDNQHVVHV LGHQWLÀHG to-discriminate sounds. This measurement can be through aptitude testing, might make some students accomplished using either electroencephalography more likely to succeed with special challenges of (EEG), which measures changes in electrical particular languages. For example, some languages conductance, or magnetoencephalography (MEG), require mastery of a new writing system, and which measures changes in the brain’s magnetic these systems may vary in their complexity. ÀHOG³ERWKPHDVXUHGQRQLQYDVLYHO\E\VHQVRUVDW Other languages may have a particularly complex the scalp (Figure 7). Discrimination of sounds can grammatical system with the need to learn and then be measured using signals originating from correctly use a large number of case endings or verb the auditory cortex area of the brain, eliminating conjugations, or may require the learner to acquire a confounds with a learner’s use of particular response strategies and guessing procedures, and providing a SDUWLFXODUO\GLIÀFXOWVRXQGV\VWHPVXFKDVDVHULHV of tonal contrasts. Imagine the situation shown in more sensitive measure of discrimination. Figure 8, where a learner has relatively high general An additional future goal of aptitude research foreign language aptitude and must now be assigned PLJKWEHWRPDWFKDSWLWXGHSURÀOHVSHUVRQDOLWLHV or learning preferences to particular types of to study either Arabic or Mandarin Chinese. Both instructional methods, thus allowing for the DUH YHU\ GLIÀFXOW ODQJXDJHV IRU (QJOLVK VSHDNHUV customization of language courses to particular to learn, but they present very different challenges. learners. This sort of matching process is not yet Learner 1, with a high perceptual acuity, might be widely used, but with increased understanding of better suited to learn Mandarin Chinese, whereas the relationship between the variables currently /HDUQHUZLWKDQHYHQDSWLWXGHSURÀOHFRXOGEH under investigation, it may become an important assigned to learn Arabic.

The Next Wave „ Vol 18 No 1 „ 2009 39 Language Writing System Grammar Sound System Figure 8: Mandarin Very Hard Easy Very Hard Optimizing language training Arabic Hard Hard Hard

Learner 1 Learner 2

Excellent phonological memory Prefers rule-based training Good visual-spatial processing Average visual-spatial processing Musical background Exellent analytic ability        

Conclusion

Expertise in critical languages is in high

demand. Despite the intense pressure, the demand

LVGLIÀFXOWWRPHHW%\LQYHVWLJDWLQJWKHFRPSRQHQWV of language talent and ways to uncover them

through precise and sensitive measurement, CASL

researchers are contributing to meeting the need.

Eventually, through optimizing foreign language

training in terms of rapid screening, precise selection

of individuals likely to succeed, and placement into language programs suited to personal talents, it can be expected that our national language readiness will be greatly increased.

         

         

     

40 A Talent for Language FEATURE

[8] Baddeley AD, Hitch GJ. Working memory. In: Bower GA, editor. The psychology of learning and motivation. New York (NY): Academic Press; 1974. p. 47-89. [9] Baddeley A. Working memory and language: An overview. Journal of Communication Disorders. 2003;36:189-208.        [10] Morris N, Jones DM. Habituation to irrelevant speech: Effects on a visual short-term memory task. Perception & Psychophysics. 1990;47:291-297.

[11] Abutalebi J, Annoni J-M, Zimine I, Pegna AJ, Seghier ML, Lee-Jahnke H, et al. Language control and lexical competition in bilinguals: an event-related fMRI

study. Cerebral Cortex. 2008;18:1496-1505.

[12] Kroll JF, Bobb SC, Misra M, Guo T. Language selection in bilingual speech: Evidence for inhibitory

processes. Acta Psychologica. 2008;128:416-430. Notes [13] Stroop JR. Studies of interference in serial >@ &DUUROO-%7ZHQW\ÀYH\HDUVRIUHVHDUFKRQIRUHLJQ verbal reactions. Journal of Experimental Psychology. language aptitude. In: Diller KC, editor. Individual 1935;12:643-662. differences and universals in language learning aptitude. [14] MacLeod CM. The Stroop task: The “gold standard” Rowley (MA): Newbury House; 1981. of attentional measures. Journal of Experimental [2] For short descriptions of these tests, see: Bowles A, Psychology: General. 1992;121:12-14. Bauman J, Winn M. An aptitude for tone? Music, memory [15] Fan J, McCandliss BD, Sommer T, Raz A, and more. University of Maryland Center for Advanced 3RVQHU 0, 7HVWLQJ WKH HIÀFLHQF\ DQG LQGHSHQGHQFH RI Study of Language; 2007. 112-M.1. attentional networks. Journal of Cognitive Neuroscience. [3] For more detailed discussion, see: DeKeyser 2002;14:340–347. R, Larson-Hall J. What does the critical period really [16] Posner MI. Orienting of attention. Quarterly Journal mean? In: Kroll JF, De Groot AMB, editors. Handbook of Experimental Psychology. 1980;32:3-25. of Bilingualism: Psycholinguistic Approaches. Oxford [17] Eriksen BA, Eriksen CW. Effects of noise letters (UK): Oxford University Press; 2005. XSRQWKHLGHQWLÀFDWLRQRIWDUJHWOHWWHULQDQRQVHDUFKWDVN Johnson JS, Newport EL. Critical period effects in second Perception 6 Psychophysics. 1974;16:143-149. ODQJXDJHOHDUQLQJWKHLQÁXHQFHRIPDWXUDWLRQDOVWDWHRQ [18] Monsell S, Driver JS, editors. Attention and the acquisition of English as a second language. Cognitive Performance XVIII: Control of cognitive processes. Psychology. 1989;21(1):60-99. Cambridge (MA): MIT Press; 2000. Long MH. Maturational constraints on language [19] Hernandez AE, Martinez A, Kohnert K. In search development. Studies in Second Language Acquisition. of the language switch: An fMRI study of picture naming 1990;12:251-285. in Spanish-English bilinguals. Brain and Language. [4] DeKeyser R. The robustness of critical period 2000;73:421-431. effects in second language acquisition. Studies in Second [20] Segalowitz N, Frenkiel-Fishman S. Attention control Language Acquisition. 2000;22:499-533. and ability level in a complex cognitive skill: Attention [5] For descriptions of the measures, see: Doughty VKLIWLQJ DQG VHFRQGODQJXDJH SURÀFLHQF\ 0HPRU\  CJ, Bunting MF, Campbell S, Bowles AR. Internal Cognition. 2005;33:644-653. consistency and test-retest reliability of the Hi-LAB [21] Best CC, McRoberts GW. Infant perception of measures. College Park: University of Maryland, Center non-native consonant contrasts that adults assimilate in for Advanced Study of Language; 2007. different ways. Language and Speech. 2003;46:183-216. [6] Saffran JR, Aslin RN, Newport EL. Statistical [22] Werker JF, Tees RC. Cross-language speech learning by 8-month-old infants. Science. 1996;274:1926- perception: Evidence for perceptual reorganization during 1928. WKHÀUVW\HDURIOLIH,QIDQW%HKDYLRUDQG'HYHORSPHQW [7] Cowan N. Attention and memory: An integrated 1984;7:49–63. framework. New York: Oxford University Press; 1995. (Oxford Psychology Series, No. 26).

The Next Wave „ Vol 18 No 1 „ 2009 41 What’s the Bottom Line?

Development of and potential uses for the Summary Translation Evaluation Tool—STET

anguage analysts in the intelligence community (IC) confront huge amounts of foreign language material, some highly relevant to national security requirements L and some of lesser value. Language analysts cannot provide full translations of everything they process, nor do other analysts have the time or need to read full translations. Summary translation enables language analysts to identify, distill, and present English translations of the important information contained in the original foreign language material, thereby efficiently communicating the most crucial information. The term summary translationa can be used to cover a broad range of tasks that vary in their purposes and skill demands, from documenting essential elements of information contained in a single source item to writing a personality profile or situation assessment synthesizing information from many source items [1]. Despite this variability, a common attribute of summary translations produced within the IC context is that they are typically targeted summaries written in response to intelligence needs, i.e., customer- specified requirements or requests for information. In other words, writers may be looking for information about specific topics or answers to specific questions, rather than attempting to summarize all of the main points of the source item as they would in a generic summary. The Summary Translation Evaluation Tool (STET) was created primarily for assessment of targeted summaries, which are uncommon in commercial and academic environments and therefore rarely studied and ill-understood. The STET was designed not only to help researchers develop a deeper understanding of targeted summary translation, but also to establish a standard for summary translations and to provide language analysts with a vital tool for both assessing and improving summary translation performance via standardized quality control (QC) and enhanced training.

42 What’s the Bottom Line? FOCUS

What is the STET? ERWKWKHGLIÀFXOW\RIWKHWDVNDQGWKHVNLOO Potential uses of the STET with which it is executed in order to make The STET is a computerized form Quality control that offers a standardized framework a meaningful evaluation” [2]. Language analysts and supervisors for evaluating summary translation Figure 1 shows the STET form throughout the IC recognize the importance productsb. The heart of the STET is a along with descriptions of each major of quality control (QC), but there is currently set of rating scales to assess summaries component. In the interactive version of little standardization of QC procedures. In DORQJ VL[ GLPHQVLRQV 6LJQLÀFDQFH addition to its critical function of ensuring Completeness, Accuracy, Omission of the tool, pop-up windows provide more that products are accurate, QC serves as Irrelevant Information, Organization, and detailed information than is available on a mechanism for training and providing Writing. Each dimension is described in the one-page static form. For the Summary feedback to language analysts. QC may Figure 1. These dimensions were designed Assessment section on the right side of the also contribute to record-keeping and help to cover all of the important elements form, each pop-up window describes the supervisors determine work assignments. of a summary’s content, structure, and fundamental question that is addressed style, but the relative importance of each The STET is designed to facilitate all of in the rating scale, often indicating what dimension may depend on the purpose for these aspects of QC. qualities a summary should possess to which the summary is written. Help for the QC provider—a more receive a high rating on that scale. Each The STET also includes a description HIÀFLHQWDQGHIIHFWLYHPHDQVRI of the source item(s) on which the summary pop-up window also provides labels for FRQGXFWLQJ4& is based. Although users of the STET are the end-points of the rating scale. The One of the most critical functions of instructed not to adjust their ratings based labels are tailored to each dimension; for QC is ensuring that intelligence products RQWKHGLIÀFXOW\RIWKHPDWHULDOWKH6RXUFH example, the Organization scale ranges are of high quality. QC is important not ,WHP'HVFULSWLRQLGHQWLÀHVIHDWXUHVRIWKH from “The summary is extremely poorly MXVW IRU D ÀQDO UHSRUW EXW DOVR IRU WKH translations, summaries, and other possible material that may be especially challenging organized” to “The summary is extremely and helps provide context for the STET steps that may be completed along the way; well organized.” Finally, because the STET ratings. if the original source material is translated is intended to be consistent with the eight As described in the STET user’s and/or summarized inaccurately, there is a analytic standards issued by the Director of manual, the Source Item Description and KLJKULVNWKDWWKHÀQDOUHSRUWZLOOFRQWDLQ Summary Translation Assessment are National Intelligence (Table 1), each pop- incorrect information. ´DQDORJRXVWRWKHGLIÀFXOW\DQGH[HFXWLRQ up window lists the particular standards QC is seen as especially crucial for in an Olympic dive; one must describe that are addressed by that dimension. junior analysts who may have limited

Table 1: ODNI Standards of Analytic Tradecraft [3]

1. Properly describes quality and reliability of underlying sources 2. Properly caveats and expresses uncertainties or confidence in analytic judgments 3. Properly distinguishes between underlying intelligence and analysts’ assumptions and judgments 4. Incorporates alternative analysis where appropriate 5. Demonstrates relevance to US national security 6. Uses logical argumentation 7. Exhibits consistency of analysis over time, or highlights changes and explains rationale 8. Makes accurate judgments and assessments

a The term gisting is sometimes used synonymously with summary translation, but we will not use gisting here, because it often refers to a process that would not typically be evaluated with the Summary Translation Evaluation Tool (STET) For example, in some operational environments analysts use gisting to refer to the process of making brief notes about the content of an item for triage purposes, and summary translation to refer to a more formal summarization process

b The STET has also been adapted for evaluation of other translation products For example, one operational organization has created a spin-off called the Language Product Evalu- ation Tool (LPET), which can be used to evaluate summary translations, verbatim translations, or hybrids (in which some material is summarized and some is translated verbatim)

The Next Wave „ Vol 18 No 1„ 2009 43 Figure 1 : STET form

Summary Translation Evaluation Tool Language Level-Users determine the ILR level of the source item and Language Analyst~------~ indicate which characteristic(s) of the item contributed to that level. A. Source Item Description 0 voice 0 graphic 0 both In the electronic version of the STET, the language levels and characteristics include pop-up windows with A.1. Language Level (Select overall level of source; mark all characteristics descriptions and examples. of significance in source.) [I]~ rn rn D cultural information D lack of continuity D diagrams, charts, graphs D meaning beyond the literal D greater than average length D multiple objects or concepts D high density of information D rhetorical devices D highly technical subject matter D shared knowledge D inference based on overt info D spatial relationships Complicating Mode Factors-The D intentional deception D telling out of sequence presence of these factors can make an item more difficult to understand, translate, or summarize. Some A.2. Complicating Mode Factors (Mark all of significance in source.) factors are specific to one modality; D communicants speaking over one D non-standard colloquialisms other factors, such as colloquialisms another or slang or poor grammar, can be found in D corrupt source D omissions both voice and graphic items. D dialect D one-sided conversation D distortion D poor grammar D elliptical or telegraphic style D poor handwriting D heavy accent D poor spelling D more than one language or dialect D rapid speech or alphabet D sudden changes in subject D non-standard abbreviations or typographical errors Impact of Complicating Mode D specialized terminology Factors-Users assess the degree D urgency (need for time- to which the complicating factors sensitive processing) impact the ability to understand, translate, or summarize the item. A.3. Impact of Complicating Mode Factors (Mark one.) For example, typographical errors O none O inconsequential O moderate O considerable O extensive may be relatively inconsequential in one text but may render another B.8. QCer 2 Comments (Use appropriate handling and classification markings, virtually incomprehensible. if needed.)

Written Comments-Users are encouraged to provide written comments. Ratings on a single dimension (compared to an overall rating) can reflect specific problems that might benefit from different types of interventions. For example, a Writing score could be poor because the summary is full of typographical errors or because the writer is a non-native speaker of English who does not have an adequate grasp of English grammar.

44 Whats the Bottom Line? FOCUS

Significance-The summary should clearly demonstrate why the source item is relevant to national security and should indicate how the relevant information relates to what is already known about a particular requirement. Users may indicate that the dimension is not applicable if, for example, a generic summary is required rather than a targeted summary.

QCer 1

B. ary Assessment (Mark a number for each factor; add comments.)

Completeness-Users address the B.1. Significance: How well does the summary relate [8]1 2 3 4 5 degree to which the summary contains ~ the "so what" of the source item to requirements? 00000 all of the relevant information. In addition to presenting the facts that are B.2. Completeness: How much of the essential 2 3 4 5 explicitly stated within the source item, information is covered in the summary: 00 a good summary may need to include who? what? when? explanatory facts available elsewhere where? why? how? and analytic comments. relevant background? analytic comment?

B.3. Accuracy: How much of the information in the 2 3 4 Summary is accurate? Accuracy-Many consider accuracy to be the most fundamental component of B.4. Omission of Irrelevant Information: How well [8] 1 2 3 4 5 the STET. If a summary receives a low rating for accuracy, the summary will be of does the summary omit irrelevant information? 0 0 0 0 0 very limited value even if it receives high ratings for all of the other dimensions. B.5. Organization: How well organized is the summary: "bottom line" up front? logical organization? well-structured paragraphs? Omission of Irrelevant Information­ B.6. Writing: How well does the summary follow A critical feature of summarizing is conventions for: efficiently communicating the most grammar? spelling? important information in the source punctuation? word usage? material. Because of the targeted nature of summary translation within date, time, transliteration, etc.? the IC, the source material may often contain a great deal of information that B.7. QCer 1 Comment n Summary Translation (Use appr is not relevant to national security. and classification m kings, if needed.)

Writing-Poor grammar, spelling, Organization-A good summary must and punctuation can obscure be organized in such a way that the the message in an otherwise message is communicated clearly. One good summary. This dimension of the most highly valued organizing takes into account conventions principles within the IC is "bottom line up for reporting dates, times, and front'.' Sometimes language analysts will transliterations of foreign names­ need to alter the original organization particularly important because of the source material to convey the foreign names can be spelled information in a way that most directly many different ways in English. addresses the information need.

The Next Wave • Vol 18 No 1 • 2009 45 experience with the target and/or language. Over time, the STET will help one of the goals of research on translation However, most professionals realize that supervisors and managers keep track is to understand the cognitive and HYHQKLJKO\VHDVRQHGH[SHUWVFDQEHQHÀW of strengths and weaknesses in order to procedural processes involved in summary from having their work reviewed by determine work assignments and note translation, and to apply that understanding colleagues. During structured interviews opportunities for targeted training for to improve the performance and training conducted by our summary translation individuals or groups. For example, a of language analysts. With respect to both research team, nearly all language analysts supervisor who is using the STET to track aspects of this aim—understanding and noted that “nothing goes out the door an analyst’s progress may note that the application—evaluation is an essential without being seen by at least two pairs of analyst performs very well with Level component. The STET will provide eyes.” 2 material but less well with Level 3 researchers with a valuable mechanism for Because of the vast amount of material; this type of pattern can be useful evaluating the summary translations that material that is processed every day, in making appropriate work assignments. are produced in experiments. The STET QC places a heavy burden on the most In addition, the detailed nature of the is a powerful tool for experimentation experienced analysts. One goal of the STET Source Item Description may reveal that because it allows the researchers to is to facilitate the process of conducting the analyst excels in the face of certain examine summarization performance QC. Although an initial time investment challenges but struggles with others, along a variety of dimensions. may be required for QCers to learn about DOORZLQJIRULGHQWLÀFDWLRQRILQGLYLGXDOO\ In one CASL experiment using the the STET and become accustomed to tailored professional development STET, we are examining the ways in using it, the STET can ultimately make activities. Aggregated STET data may which varying amounts of time pressure QC faster and more effective by providing also help managers to determine whether impact the quality of summary translations a standardized framework for evaluating an entire shop’s performance is affected and the strategies that language analysts summary translations. by factors such as new software tools, new use to create them. In this experiment, mentoring programs, or changes to the we ask language analysts to summarize a +HOSIRUWKH4&UHFLSLHQW³PRUH physical workspace. different foreign language text in each of GHWDLOHGDQGXVHIXOIHHGEDFN three time conditions: 2 hours, 1 hour, and The “checking” aspect of QC Training ½ hour. (The order of the time conditions emphasizes forward movement in the Not only will on-the-job training and the assignment of text to condition are sense that each piece of work is checked EHQHÀWIURPWKH67(7YLDLPSURYHG4& counterbalanced across participants.) and passed forward to the next person in and feedback, but classroom training the chain. For example, a language QCer With a holistic rubric, we would only will also be able to capitalize on the be able to determine whether summary may check a translation and pass the STET. Perhaps most critically, the STET corrected version forward to a reporter. translations were “better” in one condition will provide instructors with a coherent than in another. With the STET, however, Ideally, QC also involves a framework for teaching students about we can look at the effects of time pressure “backward” step in which feedback is the components of a good summary. on different aspects of the summarization provided to the language analyst who wrote Instructors can also use the STET to process. For example, we might see that the initial translation or summary. Such provide standardized feedback on student feedback is vital for improving the junior 6LJQLÀFDQFH LV UHODWLYHO\ XQDIIHFWHG E\ assignments, which will make the grading time pressure if language analysts prioritize analysts’ language skills and helping them SURFHVVPRUHHIÀFLHQWIRULQVWUXFWRUVDQG the need to identify the critical intelligence to avoid similar mistakes in the future. more useful for students. Using the STET value of the source item; Completeness, on Unfortunately, the fast operational tempo in the classroom will also help students the other hand, might suffer under extreme VRPHWLPHVPDNHVLWGLIÀFXOWIRU4&HUVWR become accustomed to the way they will time pressure if the language analyst does provide feedback in a timely manner. be evaluated on the job. QRW KDYH VXIÀFLHQW WLPH WR LQFOXGH DOO The structure of the STET will allow The Source Item Description section important details. Similarly, Organization QCers to generate feedback at the same of the STET may also be helpful for might be relatively stable, but Writing time that they are checking the work, instructors in guiding the selection of texts might be vulnerable to time pressure when thus helping both the QC provider and that are at an appropriate level for the class language analysts do not have time to recipient, and the multidimensional nature and that present students with particular proofread or check their work. of the STET will ensure that the feedback is challenges that are relevant to the lesson. This detailed level of analysis detailed enough to help language analysts enabled by the STET will help researchers Research SLQSRLQWVSHFLÀFDUHDVIRULPSURYHPHQW better understand the various components +HOSIRUWKHVXSHUYLVRU³DPHFKDQLVP At the University of Maryland Center of the summary translation task and guide IRUWUDFNLQJSURJUHVV for Advanced Study of Language (CASL), the development of interventions to help 46 What’s the Bottom Line? FOCUS

analysts maintain key components of H[LVWLQJ VFLHQWLÀF OLWHUDWXUH DQG E\ DQ the source text. According to Jonassen, summary quality under trying conditions. analysis of the original holistic rubric. Beissner, and Yacci [7], the meaning of a Following standards in educational and source text comes from its structure—the Development of the STET psychological measurement [6], we strove way its propositions are related to each $VSDUWRI&$6/·VÀUVWH[SHULPHQWDO to produce an analytic rubric characterized other and organized. In other words, the study of summary translation, our research by the following qualities: reader must derive structural knowledge team developed a “holistic” rubric, which of the text and be able to integrate each ‡ All elements worthy of evaluation successive part of the text with his or assigned a single qualitative rating to are included. each summary. The scale was developed her prior representation of it. Related to Each element is unidimensional in ‡ this point, learning and understanding XVLQJDPRGLÀHGHPSLULFDOO\EDVHGELQDU\ that it cannot be further separated boundary approach [4], meaning that we of the text take place by assimilating or partitioned. (Given this quality, relied on collaboration and consensus of new knowledge with prior background we refer to each element as a knowledge. TXDOLÀHG SURIHVVLRQDOV XVLQJ DQ LWHUDWLYH dimension.) process of categorizing and characterizing For a language analyst to produce a Ratings are distinct, comprehensive, salient features to arrive at descriptors for ‡ quality summary, he or she must have (1) and descriptive in that they cover HDFK OHYHO RI SURÀFLHQF\ >@ 7KLV W\SH VXIÀFLHQWODQJXDJHSURÀFLHQF\WRDFKLHYH the range of expected performance. of scale is probably similar to informal a discourse-level understanding of the text evaluations used in the (IC) and is ‡ Each element of the rubric and not just a word-by-word glossing and typically fairly quick to use. However, a communicates clearly to the user. (2) adequate background knowledge to holistic evaluation provides only a single ‡ The rating score on each element assimilate the text with prior structural rating of the summary, and two summaries covers the range of performance, knowledge. Preliminary CASL research could receive the same rating for very perhaps in the range of 3-7 levels. on summary translation demonstrated different reasons (e.g., one due to poor WKDW DQ LQVXIÀFLHQW GLVFRXUVHOHYHO comprehension of the source item and one Characteristics of a good understanding of the text often led to due to poor English writing skills). summary gross misunderstandings and eventually We ultimately decided that an To develop the appropriate to inadequate summaries [8]. A solid “analytic” rubric would be more useful for dimensions for the STET, we needed to discourse-level understanding of the text both experimental and applied purposes, as identify the most important components is assessed most directly by the Accuracy described above, i.e., summary translation of summary translation based on existing dimension of the STET. In addition, a evaluation via an analytic rubric returns science and operational needs. From thorough understanding of the text and much more informative feedback D VFLHQWLÀF SHUVSHFWLYH ZH EHJDQ E\ its relation to the relevant background about performance, allowing for more H[DPLQLQJ OLWHUDWXUH LQ ÀHOGV VXFK DV knowledge are also necessary for the powerful experimentation as well as more language processing, memory, translation, reader to determine and explain how and individualized on-the-job evaluation and reading comprehension, and spoken- why the text relates to intelligence needs, training. The current version of the STET discourse comprehension. To determine DVDVVHVVHGE\WKH6LJQLÀFDQFHGLPHQVLRQ was developed as a collaborative effort what constitutes a good summary for of the STET. between CASL researchers and our United operational purposes, we conducted Endres-Niggemeyer [9] emphasizes States Government (USG) colleagues, structured interviews with language the importance of representing and capitalizing on scholarly literature, analysts, intelligence analysts, QCers, understanding discourse via schemata and instructors. We asked interviewees to VFLHQWLÀF PHWKRGV DQG WKH RSHUDWLRQDO (structured groups of concepts used to expertise of many language analysts. describe the ways in which they write or organize knowledge). This emphasis use summary translations on the job and implies that summarizers must be able to Characteristics of an analytic what they look for in a good summary identify the schematic elements of the text rubric translation. In addition to these individual and the relationships and actions between Our development of an analytic structured interviews, focus groups were them. Some of these elements will be rubric had the goal of making transparent convened to determine the qualities that crucial to the summary, and the success the component processes involved in users would look for in an evaluation tool RIWKHLULGHQWLÀFDWLRQVKRXOGEHHYDOXDWHG summary translation while also ensuring and to solicit comprehensive feedback on by the rubric. For example, does the that the STET would be easily understood preliminary drafts of the STET. summarizer correctly identify the relevant and used. Preliminary effort sought to One of the most important participants, their roles in the text, and the derive a set of unidimensional evaluatory characteristics of a good summary is DFWLRQVWDNLQJSODFH",GHQWLÀFDWLRQRIWKH elements, and was informed both by the WKDW LW DFFXUDWHO\ UHÁHFWV WKH PHDQLQJ RI key pieces of information in the text is

The Next Wave „ Vol 18 No 1„ 2009 47 assessed by the Completeness dimension description of each rating of each dimension UHÁHFWHGLQUDWLQJVIRUWKHFRUUHVSRQGLQJ of the STET. of the STET. We discovered, however, that dimension (validity) and if the STET Another fundamental skill required most of the important information appeared adequately assesses the full range of for summarization is information reduction. in the descriptor for the highest rating, performance along each dimension Ultimately, in a quality summary only which listed the characteristics required (sensitivity). Statistical testing will also the relevant information should remain, for a high quality summary. By putting all help to determine if the dimensions are and the irrelevant information should be of the desired characteristics up front after treated independently or if, for example, discarded, as measured by the Omission each question, the endpoints of the scale grammatical errors affect Accuracy scores of Irrelevant Information dimension of the emerged naturally without the need for as well as Writing scores. Lastly, the STET STET. descriptions of the intermediate ratings. will be tested thoroughly to establish its Finally, the summary translation This decision was validated by research consistency across users and conditions must be appropriately communicated, suggesting that the labels for intermediate (reliability). CASL researchers are which might be evaluated as having macro- scale intervals are not as critical as the currently conducting a set of experiments structural and microstructural dimensions FKRLFHRIHQGSRLQWODEHOV>@ to accomplish these goals. (Organization and Writing in the STET). Refining the STET Conclusion - 7KHIRUPHULVFOHDUO\UHÁHFWHGLQWKHRU Once the dimensions and rating ganization of the summary and the latter The STET is the result of a needs- system were established, we used several in its grammar. Important macrostructure based approach to research in which a approaches (some of which are still in components include clear, coherent over- multidisciplinary team of scientists col- all organization and a structure that makes SURJUHVV WRUHÀQHWKH67(7 laborated with members of the operational evident how the summary responds to the 5LJRURXVSUDFWLFDOWHVWLQJ workforce to develop a product that ad- relevant information need(s). Important Since a major aim of rubric GUHVVHV ERWK RSHUDWLRQDO DQG VFLHQWLÀF microstructure components include proper development is the application of the problems. This analytic rubric for evaluat- grammar, spelling, and word choice, as rubric to the training and evaluation ing summary translations was developed well as a writing style that is clear at both of language analysts working within WR EHQHÀW ODQJXDJH DQDO\VWV 4&HUV DQG the sentence and clause level. government agencies, practical testing their managers, resulting in better reports by representatives of those agencies is and better language analysts. The STET Properties of the rating system crucial. This “beta testing” will allow will also allow CASL scientists to deepen Once the dimensions were agencies to report experiences using the our understanding of the summary transla- established, we had to decide on other rubric and to provide feedback so we can tion process so we can continue to conduct properties of the rating system, including make the rubric maximally user-friendly rigorous research to enhance language the appropriate number of points on the and useful for their needs. For example, performance in the IC. rating scale and the descriptors of the it will likely be valuable to build into different levels. the rubric a degree of modularity, so that We ultimately decided to use a particular dimensions can be added or   SRLQW UDWLQJ VFDOH $OWKRXJK VRPH RI omitted as needs dictate. Similarly, the the potential users we consulted felt that size of the rating scale might be collapsed a smaller number of points would make or expanded according to practical needs.       the scale faster and easier to use, research In coordination with practical testing, has demonstrated that reliability and VWDWLVWLFDOWHVWLQJZLOOEHXVHGWRFRQÀUP YDOLGLW\ DUH EHWWHU LQ  WR SRLQW VFDOHV that adapted versions of the rubric are than in scales with fewer points [10-13]. valid, sensitive, and reliable. Also, respondents tend to avoid using 5LJRURXVVWDWLVWLFDOWHVWLQJ the endpoints of a scale [14], so having a Rigorous statistical testing is required 4-point scale could potentially concentrate to examine the validity, sensitivity, and        most of the responses on only two points, reliability of the STET against gold which may not be sensitive enough to standards, i.e., summaries pre-established detect subtle differences in summary by experts to represent certain levels quality. of performance along the different Another important decision was dimensions. This testing will allow us KRZWRODEHOWKHÀYHSRLQWVRIWKHVFDOH to determine if variations in each aspect We initially attempted to write a detailed of summary quality are appropriately

48 What’s the Bottom Line? FOCUS

References [1] Michael EB, Bailey B, Gannon-Kurowski S, Pinckney K. Description of summary translation task requirements for specific jobs and the uses of the summaries. College Park (MD): University of Maryland Center for Advanced Study of Language; 2007. Technical Report No.: M.19. [2] University of Maryland Center for Advanced Study of Language. The Summary Translation Evaluation Tool: What it is and how to use it. [User’s manual]. College Park (MD); 2008. p. 3. [3] McConnell J. Intelligence community directive number 203: Analytic standards; 2007. [4] Upshur J, Turner C. Constructing rating scales for second language tests. ELT Journal. 1995;49:3-12. [5] Turner J. Report on working session: Rubric development and rater orientation. In: B. Bailey B, Turner J, Pinckney K, de Terra D, Michael E. Technical Report No. M.16: Compilation of milestones examining evaluation of translation summaries. College Park (MD): University of Maryland Center for Advanced Study of Language; 2007. [6] Crocker L, Algina J. Introduction to classical and modern test theory. Mason (OH): Thomson Wadsworth; 2006. [7] Jonassen DH, Beissner K, Yacci M. Structural knowledge: Techniques for representing, conveying, and acquiring structural knowledge. Hillsdale (NJ): Lawrence Erlbaum Associates; 1993. [8] Michael EB, Allison T, Danks J, de Terra D, Massaro D, Donavos D, Graham K, Klavans J. Does verbatim translation help summary translation? Paper presented at the 6th International Symposium on Bilingualism; Hamburg, Germany; May 2007. [9] Endres-Niggemeyer B. Summarizing information. Berlin (Germany): Springer- Verlag; 1998. [10] Bandalos DL, Enders CK. The effects of nonnormality and number of response categories on reliability. Applied Measurement in Education. 1996;9:151-160. [11] Dawes J. Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research. 2008;50:61-77. [12] Jenkins GD Jr, Taber TD. A Monte Carlo study of factors affecting three indices of composite scale reliability. Journal of Applied Psychology. 1977;62:392-398. [13] Lissitz RW, Green SB. Effect of the number of scale points on reliability: A Monte Carlo approach. Journal of Applied Psychology, 1975;60:10-13. [14] Beal DJ, Dawson JF. On the use of Likert-type scales in multilevel data: Influence on aggregate variables. Original Research Methods. 2007;10:657-672. [15] Gannon KM, Ostrom TM. How meaning is given to rating scales: The effects of response language on category activation. Journal of Experimental Social Psychology. 1996;32:337-360. [16] Klocktars AJ, Yamishi M. The influence of labels and position in rating scales. Journal of Educational Measurement. 1988;25:85-96.

Further Reading Jonassen DH, Howland J, Moore J, Marra RM. Learning to solve problems with technology: A constructivist perspective. 2nd ed. Columbus (OH): Merrill/Prentice- Hall; 2003.

The Next Wave „ Vol 18 No 1„ 2009 49