DOCSLIB.ORG

Resonance Analysis

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Resonance Analysis Working Paper Resonance Analysis A Method for Detecting Author Affiliations Through Distinctive Patterns of Word Choice Joshua Mendelsohn, William Marcellino, Jaime L. Hastings, and Steven J. Morgan RAND Center for Network Analysis and System Science WR-A1008-1 January 2021 Prepared for the Pardee RAND Graduate School RAND working papers are intended to share researchers’ latest findings and to solicit informal peer review. This paper has been approved for circulation by the RAND Pardee Graduate School. Unless otherwise indicated, working papers can be quoted and cited without permission of the author, provided the source is clearly referred to as a working paper. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. is a registered trademark. For more information on this publication, visit www.rand.org/pubs/working_papers/WRA1008-1.html Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2021 RAND Corporation R® is a registered trademark Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions.html. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Abstract Resonance analysis is an evolving text mining method for estimating how much affinity exists in a broader population for a specific group. The method involves a multistage procedure for scoring words according to how distinctive they are of authors in the specific group, and then scoring a broader population according to whether they make similar distinctive choices. While likely applicable to many different forms of written content, resonance analysis was developed for short-form social media posts and has been tested primarily on Twitter and Twitter-like data. In this working paper, we describe resonance analysis and provide detailed guidance for using it effectively. We then conduct an empirical test of real-world Twitter data to demonstrate and validate the method using tweets from Republican Party members, Democratic Party members, and members of the news media. The results show that the method is able to distinguish Republicans’ tweets from Democrats’ tweets with 92-percent accuracy. We then demonstrate and validate the method using simulated artificial language to create controlled experimental conditions. The method is very accurate when minimum data requirements have been met. In an operational field test, resonance analysis generated results that mirror real-world conditions and achieved statistically significant agreement with double-blind human analyst judgments of the same users. Finally, we provide examples of resonance analysis usage with social media data and identify opportunities for future research. ii Contents Abstract ........................................................................................................................................... ii Figures............................................................................................................................................ iv Tables .............................................................................................................................................. v Executive Summary ....................................................................................................................... vi Acknowledgments…………………………………………………………………………..…....xi Abbreviations ................................................................................................................................ xii 1. Context and Conceptual Framework .......................................................................................... 1 Motivating Problem .................................................................................................................................. 1 Methodological Foundations .................................................................................................................... 3 Conceptual Foundations ........................................................................................................................... 7 2. Methodology ............................................................................................................................... 9 Chapter Summary ..................................................................................................................................... 9 Procedure ................................................................................................................................................ 10 Assorted Guidance on Research Design Decisionmaking ..................................................................... 23 3. Empirical Validation Test Results ............................................................................................ 29 Chapter Summary ................................................................................................................................... 29 Data ......................................................................................................................................................... 30 Results .................................................................................................................................................... 32 4. Simulation-Based Validation Test Results ............................................................................... 44 Chapter Summary ................................................................................................................................... 44 Procedure ................................................................................................................................................ 44 Results .................................................................................................................................................... 49 5. Resonance Analysis in Practice: Closing Thoughts .................................................................. 53 Chapter Summary ................................................................................................................................... 53 Previous Work Using Resonance Analysis on Social Media ................................................................. 53 Opportunities for Further Research ........................................................................................................ 55 Conclusion .............................................................................................................................................. 57 References ..................................................................................................................................... 58 iii Figures Figure 2.1. Example Dual Calibration Chart from Performance Testing of Equation 2.2 ........... 22 Figure 2.2. Percentage of Tweeters in Top 13 Twitter Languages Meeting Minimum Volume Requirements over Time ....................................................................................................... 28 Figure 3.1. Correspondence Between Resonance Scores and NOMINATE Scores .................... 39 Figure 3.2. Correspondence Between Resonance Scores and Congressional Mentions Network 43 Figure 4.1. Word Distributions for Tweets in 15 Languages, Plus Our Synthetic Language ...... 47 Figure 4.2. Generating Distinctive Word Use Patterns ................................................................. 48 Figure 4.3. Average Accuracy ...................................................................................................... 51 Figure 4.4. Expected Accuracy Rates Given Moderate Topical Focus ........................................ 52 iv Tables Table 2.1. Example of Text Regularization—World Series Outcomes ........................................ 14 Table 2.2. Example of Signature Creation—Cities in Which Major League Baseball World Series Occurred, Before and After 1967 ............................................................................... 17 Table 2.3. Resonance Scoring Text Vignettes and Authors ........................................................ 18 Table 2.4. Resonance Detection Rates from the Test Simulation Underlying Figure 2.3 ............ 23 Table 2.5. Proposed Scoring Table for Population of Interest Language Distinctiveness ........... 25 Table 2.6. Guidance Table for Determining Minimum Number of Training Examples .............. 26 Table 3.1. Congressional Tweet Data ........................................................................................... 30 Table 3.2. New Media Data .........................................................................................................
Load more

Recommended publications
  • The ISIS Bride
    American University National Security Law Brief Volume 11 Issue 2 Article 4 2021 The Right of Return: The ISIS Bride Casey Hare-Osifchin Follow this and additional works at: https://digitalcommons.wcl.american.edu/nslb Part of the National Security Law Commons Recommended Citation Casey Hare-Osifchin "The Right of Return: The ISIS Bride," American University National Security Law Brief, Vol. 11, No. 2 (2021). Available at: https://digitalcommons.wcl.american.edu/nslb/vol11/iss2/4 This Response or Comment is brought to you for free and open access by the Washington College of Law Journals & Law Reviews at Digital Commons @ American University Washington College of Law. It has been accepted for inclusion in American University National Security Law Brief by an authorized editor of Digital Commons @ American University Washington College of Law. For more information, please contact [email protected]. The Right of Return: The ISIS Bride Casey Hare-Osifchin* ABSTRACT With the migration of Westerners to fight as a part of the Islamic State of Iraq and the Levant (“ISIS” or “ISIL”), there is an often-overlooked group of others who flock to the region in order to do what they consider to be their part. This is a group comprised of women, often extremely young, some even minors, who have felt compelled to fulfill their perceived duty to the Caliphate. They are usually branded ISIS brides. These girls and women are frequently radicalized online, and they subsequently travel to war-torn areas to be married to a member of ISIS and provide children for the Caliphate.
    [Show full text]
  • U.S. Citizens Kidnapped by the Islamic State John W
    CRS Insights U.S. Citizens Kidnapped by the Islamic State John W. Rollins, Specialist in Terrorism and National Security ([email protected], 7-5529) Liana Rosen, Specialist in International Crime and Narcotics ([email protected], 7-6177) February 13, 2015 (IN10167) Overview On February 10, 2015, President Barack Obama acknowledged that U.S. citizen Kayla Mueller was killed while held in captivity by the terrorist group known as the Islamic State (IS). This was the fourth death of an American taken hostage by the Islamic State: Abdul-Rahman Kassig (previously Peter Kassig), James Foley, and Steven Sotloff were also killed. The death of Mueller and the graphic videos depicting the deaths of the other three Americans have generated debate about the U.S. government's role and capabilities for freeing hostages. In light of these deaths, some policymakers have called for a reevaluation of U.S. policy on international kidnapping responses. Questions include whether it is effective and properly coordinated and implemented, should be abandoned or modified to allow for exceptions and flexibility, or could benefit from enhancements to improve global adherence. Scope The killing of U.S. citizens by the Islamic State may be driven by a variety of underlying motives. Reports describe the group as inclined toward graphic and public forms of violence for purposes of intimidation and recruitment. It is unclear whether the Islamic State would have released its Americans hostages in exchange for ransom payments or other concessions. Foley's family, for example, disclosed that the Islamic State demanded a ransom of 100 million euros ($132 million).
    [Show full text]
  • “State of Civil Society Report: 2015
    the year in review State of Civil Society report 2015: THE YEAR IN REVIEW ...these stories tell us that only civil introduction society, in its broadest sense, is taking a It has been another year of hard work and high achievement for civil society. The story of the year since the stance against the 2014 State of Civil Society Report was published has partly been one of a continuing series of attacks on civil concentration of society in the many countries where, when civil society asks difficult questions about power, the powerful seek to silence it. But is has also been a story of impressive and sustained civil society response, in a world that has power in the hands of become more turbulent and contested. a tiny, global, super- rich elite, and against As we show below, civil society faces challenges - of lack of space, under-resourcing and limited access to the attempts of many decision-makers. Civil society also needs continually to prove its connection with and relevance to citizens, political leaders and and it needs to demonstrate its ability to stay ahead of trends and innovate. When civil society groups do not corporate interests do these, they fail. But so often, we see civil society leading the response to crisis, taking on difficult issues, contributing to change, and winning arguments for social justice. to undermine human rights and This year in review section of the 2015 CIVICUS State of Civil Society Report is complemented by our report’s the value of people’s special thematic section on the resourcing for civil society, and the 27 guest contributions, from civil society participation.
    [Show full text]
  • Mining Pro-ISIS Radicalisation Signals from Social Media Users Conference Or Workshop Item
    Open Research Online The Open University’s repository of research publications and other research outputs Mining Pro-ISIS Radicalisation Signals from Social Media Users Conference or Workshop Item How to cite: Rowe, Matthew and Saif, Hassan (2016). Mining Pro-ISIS Radicalisation Signals from Social Media Users. In: Proceedings of the Tenth International AAAI Conference on Web and Social Media (ICWSM 2016) pp. 329–338. For guidance on citations see FAQs. c 2016 Association for the Advancement of Artificial Intelligence https://creativecommons.org/licenses/by-nc-nd/4.0/ Version: Version of Record Link(s) to article on publisher’s website: http://www.aaai.org/ocs/index.php/ICWSM/ICWSM16/paper/view/13023/12752 Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyright owners. For more information on Open Research Online’s data policy on reuse of materials please consult the policies page. oro.open.ac.uk Proceedings of the Tenth International AAAI Conference on Web and Social Media (ICWSM 2016) Mining Pro-ISIS Radicalisation Signals from Social Media Users Matthew Rowe Hassan Saif School of Computing and Communications Knowledge Media Institute Lancaster University The Open University Lancaster, United Kingdeom Milton Keynes, United Kingdeom [email protected] [email protected] Abstract work, to recruit Westerners - in particular Muslims from Eu- ropean countries to join them in Syria - and to carry out ter- The emergence and actions of the so-called Islamic State rorist activities in western countries. There have been nu- of Iraq and the Levant (ISIL/ISIS) has received widespread merous reports of people from European countries, in par- news coverage across the World, largely due to their cap- ticular the United Kingdom, France and Belgium, moving ture of large swathes of land across Syria and Iraq, and the publishing of execution and propaganda videos.
    [Show full text]
  • TITLE Ll—AUTHORITY for the USE of MILITARY FORCE AGAINST
    DAV15E09 S.L.C. AMENDMENT NO.llll Calendar No.lll Purpose: To authorize the use of the United States Armed Forces against the Islamic State of Iraq and the Levant. IN THE SENATE OF THE UNITED STATES—114th Cong., 1st Sess. (no.) lllllll (title) llllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllll lllllllllllllllllllllllllllllllll Referred to the Committee on llllllllll and ordered to be printed Ordered to lie on the table and to be printed AMENDMENT intended to be proposed by Mr. KAINE (for himself and Mr. FLAKE) Viz: 1 At the appropriate place, insert the following: 2 TITLE ll—AUTHORITY FOR 3 THE USE OF MILITARY FORCE 4 AGAINST THE ISLAMIC STATE 5 OF IRAQ AND THE LEVANT 6 SEC. l1. SHORT TITLE. 7 This title may be cited as the ‘‘Authority for the Use 8 of Military Force Against the Islamic State of Iraq and 9 the Levant Act’’. 10 SEC. l2. FINDINGS. 11 Congress makes the following findings: DAV15E09 S.L.C. 2 1 (1) The terrorist organization that has referred 2 to itself as the Islamic State of Iraq and the Levant 3 and various other names (in this resolution referred 4 to as ‘‘ISIL’’) poses a grave threat to the people and 5 territorial integrity of Iraq and Syria, regional sta- 6 bility, and the national security interests of the 7 United States and its allies and partners. 8 (2) ISIL holds significant territory in Iraq and 9 Syria and has stated its intention to seize more ter- 10 ritory and demonstrated the capability to do so. 11 (3) ISIL leaders have stated that they intend to 12 conduct terrorist attacks internationally, including 13 against the United States, its citizens, and interests.
    [Show full text]
  • MENA Women News Briefdownload
    MENA Women News Brief August 11, 2015-August 25, 2015 Egypt August 12: Egypt's prosecutors take aim at pop stars “Rida al-Fouly, also called Salma, is known for her video clip ‘Sib Idi’ (‘Let Go of my Hand’). Suha Muhammad Ali, known as Egypt's Shakira, has a famous video clip called ‘Al-Kamoun’ (‘Cumin’), and Dalia Kamal Youssef, whose stage name is Radis, is the dancer in ‘Ya Wad ya Tqil’ (‘You Cold- Hearted Man’). On June 2, lawyer Mohammed al-Nimr filed charges with the general prosecutor's office against the three Egyptian dancers for incitement, immorality, and indecency, crimes punishable by three months to more than three years in prison.” (Al-Monitor) August 24: Anti-harassment campaign stirs controversy in Egypt “A Facebook campaign called ‘Mat3brhash’ (‘Don’t Give Her Attention’) stirred controversy July 19, when it attacked Egypt’s women with harsh words such as ‘She’s not worth it, it’s not like she’s Angelina Jolie,’ angering women and men alike. Many women found this campaign insulting and worried it could lead to a rise in harassment, while men feared it could affect their efforts to fight harassment.” (Al-Monitor) Iran August 2015: 50 Iranian Women You Should Know The Iran Wire is profiling “50 Iranian Women You Should Know.” Entries are published every few days and include profiles on Shirin Ebadi and Fatemeh Karroubi most recently, among others. (Iran Wire) August 24: In Iran, a Women’s Soccer Revolution “Women, and women’s sports, still face stiff winds of resistance in the Islamic Republic of Iran.
    [Show full text]
  • Answering Christ's Call to Witness in the Middle East
    PRESBYTERIAN MISSION AGENCY / Spring 2015 pcusa.org/missioncrossroads mission ANSWERING CHRIST’S CALL TO WITNESS IN THE MIDDLE EAST AT THE CROSSROADS | By Amgad Beblawi, World Mission Area Coordinator for the Middle East and Europe Mission Crossroads is a Seeing the Middle East Presbyterian Mission Agency publication about the church’s through Christ’s call mission around the world. As reports of turmoil and conflict in the Middle East continue Presbyterian World Mission is to make news headlines, Western governments continue to committed to sending mission deliberate and strategize how to protect their national interests in personnel, empowering the the region. global church, and equipping On the other hand, the church’s outlook and response to events the Presbyterian Church (USA.) in the world is diametrically different. Compelled by the love for mission as together we address of God, the church responds to Christ’s call – “you will be my the root causes of poverty, work for witnesses” (Acts 1:8). reconciliation amidst cultures Christians in the Middle East have, in fact, been Christ’s witnesses since the Day of Pentecost. Successive generations of Christ’s followers proclaimed the of violence, and share the good Gospel to the region’s inhabitants for the past two millennia. However, since the dawn of Islam in news of God’s saving love the seventh century, Christians gradually became a small minority. By the beginning of the 19th through Jesus Christ. century, Orthodox, Assyrian, Maronite, and Eastern Catholic churches that trace their origin to the Apostolic Era were in a state of decline. EDITOR Presbyterian churches in the US and Scotland heard God’s call to send missionaries to Kathy Melvin strengthen indigenous churches in the land where Christianity had its cradle.
    [Show full text]
  • Gw Extremism Tracker Terrorism in the United States
    GW EXTREMISM TRACKER TERRORISM IN THE UNITED STATES INDIVIDUALS HAVE BEEN CHARGED IN THE U.S. ON OFFENSES RELATED 217 to the Islamic State (also known as IS, ISIS, and ISIL) since March 2014, when the first arrests occurred. Of those: Their activities were located in 30 states and the District of Columbia the average age of are male 90% 28 those charged. have pleaded or * the average length 157 were found guilty 13.2 of sentence in years. *Uses 470 months for life sentences per the practice of the U.S. Sentencing Commission were accused of attempting 39% to travel or successfully traveled abroad. were accused of being 31% involved in plots to carry out attacks on U.S. soil. were charged in an operation MALE 58% involving an informant and/or an undercover agent. FEMALE indicates law enforcement operation Acknowledgement Disclaimer This material is based upon work supported by the U.S. Department of Homeland Security The views and conclusions contained in this document are those of the authors and should not be interpreted as under Grant Award Number 20STTPC00001‐01 necessarily representing the official policies, either expressed or implied, of the U.S. Department of Homeland Security. Apprehensions & Charges for Other conspirators were involved in the IS “hostage-taking scheme,” which resulted in the kidnapping and subsequent death of Jihadist Groups James Foley, Kayla Mueller, Steven Sotloff, Peter Kassig, as well as British and Japanese nationals. The pair, who also face material support charges, were captured in January of 2018 by OCT 21 VA the Syrian Democratic Forces and transferred to American custody in 2019.
    [Show full text]
  • Congressional Record—House H772
    H772 CONGRESSIONAL RECORD — HOUSE February 5, 2020 rare to encounter those who are truly back. As Eddie said to the Wilmington are their successes. But just because selfless, truly dedicated to a cause Star-News last January: ‘‘It’s about they came out of the mouth of Presi- larger than themselves, and who truly much more than me. It honors the 1 dent Donald Trump, the Democrats op- care about preserving the best of our million men, women, and children who pose them. natural resources for future genera- hunt and fish and inject more than $1.3 Folks, that is chaos in government. tions. Greensboro’s Eddie Bridges is billion into North Carolina’s economy Ever since the Democrats took con- such a rare person. every year.’’ trol of this Chamber, they have had Madam Speaker, Eddie is an unself- An accomplished athlete at Elon Uni- one mission, one mission alone, and ish leader whose love of the outdoors versity, a leader in the sportsman com- that is to remove the duly elected 45th and sportsmen’s community has led munity, and a hunter and angler leg- President of the United States, Donald him to become one of the most effec- end, conservationist Eddie Bridges has J. Trump. tive conservation leaders in the history made a positive impact on North Caro- Their mission wasn’t about lowering of North Carolina. lina’s natural resources like no other the cost of prescription drugs. Their On behalf of North Carolina’s con- before him. mission was not getting government gressional delegation, I want the world Madam Speaker, on behalf of the en- off the backs of small businesses, fam- to know what an impact Eddie has tire delegation, I wish to thank Eddie ily farmers, and individuals.
    [Show full text]
  • Reporting ISIS
    Shorenstein Center on Media, Politics and Public Policy Discussion Paper Series #D-101, July 2016 The Pen and the Sword: Reporting ISIS By Paul Wood Joan Shorenstein Fellow, Fall 2015 BBC Middle East Correspondent Licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License. May 2013: The kidnapping started slowly. 1 At first, it did not feel like a kidnapping at all. Daniel Rye delivered himself to the hostage-takers quite willingly. He was 24 years old, a freelance photographer from Denmark, and he had gone to the small town of Azaz in northern Syria. His translator, a local woman, said they should get permission to work. So on the morning of his second day in Azaz, only his second ever in Syria, they went to see one of the town’s rebel groups. He knocked at the metal gate to a compound. It was opened by a boy of 11 or 12 with a Kalashnikov slung over his shoulder. “We’ve come to see the emir,” said his translator, using the word – “prince” – that Islamist groups have for their commanders. The boy nodded at them to wait. Daniel was tall, with crew-cut blonde hair. His translator, a woman in her 20s with a hijab, looked small next to him. The emir came with some of his men. He spoke to Daniel and the translator, watched by the boy with the Kalashnikov. The emir looked through the pictures on Daniel’s camera, squinting. There were images of children playing on the burnt-out carcass of a tank. It was half buried under rubble from a collapsed mosque, huge square blocks of stone like a giant child’s toy.
    [Show full text]
  • Kayla Mueller
    The JackCentral.org Pg. 7-9 Pg. 16 Pg. 19-20 Pg. 30 INSIDE LNORTHERNUMBER ARIZONA UNIVERSITY’S STUDENT J VOICEACK SINCE 1914 • VOL 101 • ISSUE 20 • FEB. 12 - FEB. 18, 2015 KAYLA AID WORKER’S MUELLER: DEATH ORDINARY WORK, CONFIRMED EXTRAORDINARY LIFE BY SCOTT BUFFON BY KEVIN MORROW he White House has confi rmed the & MITCHELL FORBES death of humanitarian aid worker and ayla Mueller was a humanitarian from NAU alumna Kayla Mueller while beingT held hostage by the Islamic State (ISIS) Prescott, Ariz., who wanted to impact the world as she traveled it. Mueller had in Syria. aK deep care for others — friends recall Mueller ISIS fi rst reported her death Feb. 6, trying to teach Syrian prison guards origami. As claiming a Jordanian airstrike struck the Mueller saw poverty and injustices in the world, facility she was being held in. Th e Offi ce of she asked herself what she could do to improve Public Aff airs at NAU sent out a campus-wide the situations of those less fortunate. email on the morning of Feb. 10 spreading Mueller grew up in Prescott and in 2007 news of the White House confi rmation. graduated from Tri-City College Prep High “Th e Northern Arizona University family School, a local charter school where Mueller was is deeply saddened by the loss of alumna already interested in activism and other cultures. Kayla Jean Mueller,” said NAU President Rita She was part of the Youth Count volunteer Cheng. “Th is is absolutely heartbreaking and program in Prescott Valley, according to an our thoughts are with her family and friends.” article from Th e Daily Courier.
    [Show full text]
  • Flyer News, Vol. 62, No. 03
    TUESDAY, SEPT. 16, 2014 NEWS // Scotland decides on the A&E // Urban Nights showcases OPINIONS // NFL policy causes SPORTS // A-10 poised to contin- VOL. 62 NO. 3 future of their nation, pg. 4. local culture, nightlife, pg. 8. controversy, pg. 11. ue success, growth pgs. 14-15. Hamster Ball Soccer keeps students FLYER NEWS rolling, pg. 3. Wright State protest joins national debate New apps ROGER HOKE Staff Writer update UD’s culture Following an incident in which a lo- cal man was shot and killed by police in MEGAN WELCH a Beavercreek, Ohio, Wal-Mart, Wright Staff Writer State University students gathered for a protest march initiated by a “Theatre of Justice” and in solidarity with the New apps such as Uber, Crunch- nation-wide “It’s Bigger Than You” button and Yik Yak have recently movement that arose in the wake of the reached the University of Dayton controversy surrounding the events in community and are challenging Ferguson, Missouri. conventional aspects of student Hundreds of students gathered at life at UD an outside venue known as “the Quad” Uber, an app-based taxi service on WSU’s campus Sept. 4 to watch a that allows users to request rides dramatic performance put on by the of varying comfort and price and to rally’s organizing group, the Ethnic view the location of their reserved Theatre Alliance, according to local driver, announced expansion of news outlets. their operations into Dayton and ETA members at the protest said the 21 other cities starting Aug. 28, ac- group was formed in response to the cording to the Uber website.
    [Show full text]