THE DAWN OF THE HUMAN-MACHINE ERA A FORECAST OF NEW AND EMERGING LANGUAGE TECHNOLOGIES The Dawn of the Human-Machine Era A forecast of new and emerging language technologies This work is licenced under a Creative Commons Attribution 4.0 International Licence https://creativecommons.org/licenses/by/4.0/ This publication is based upon work from COST Action ‘Language in the Human-Machine Era’, supported by COST (European Cooperation in Science and Technology). COST (European Cooperation in Science and Technology) is a funding agency for research and innovation networks. Our Actions help connect research initiatives across Europe and enable scientists to grow their ideas by sharing them with their peers. This boosts their research, career and innovation. www.cost.eu Funded by the Horizon 2020 Framework Programme of the European Union To cite this report Sayers, D., R. Sousa-Silva, S. Höhn et al. (2021). The Dawn of the Human- Machine Era: A forecast of new and emerging language technologies. Report for EU COST Action CA19102 ‘Language In The Human-Machine Era’. https://doi.org/10.17011/jyx/reports/20210518/1 Contributors (names and ORCID numbers) Sayers, Dave • 0000-0003-1124-7132 Höckner, Klaus • 0000-0001-6390-4179 Sousa-Silva, Rui • 0000-0002-5249-0617 Láncos, Petra Lea • 0000-0002-1174-6882 Höhn, Sviatlana • 0000-0003-0646-3738 Libal, Tomer • 0000-0003-3261-0180 Ahmedi, Lule • 0000-0003-0384-6952 Jantunen, Tommi • 0000-0001-9736-5425 Allkivi-Metsoja, Kais • 0000-0003-3975-5104 Jones, Dewi • 0000-0003-1263-6332 Anastasiou, Dimitra • 0000-0002-9037-0317 Klimova, Blanka • 0000-0001-8000-9766 Beňuš, Štefan • 0000-0001-8266-393X Korkmaz, Emin Erkan • 0000-0002-7842-7667 Bowker, Lynne • 0000-0002-0848-1035 Maučec, Mirjam Sepesy • 0000-0003-0215-513X Bytyçi, Eliot • 0000-0001-7273-9929 Melo, Miguel • 0000-0003-4050-3473 Catala, Alejandro • 0000-0002-3677-672X Meunier, Fanny • 0000-0003-2186-2163 Çepani, Anila • 0000-0002-8400-8987 Migge, Bettina • 0000-0002-3305-7113 Chacón-Beltrán, Rubén • 0000-0002-3055-0682 Mititelu, Verginica Barbu • 0000-0003-1945-2587 Dadi, Sami • 0000-0001-7221-9747 Névéol, Aurélie • 0000-0002-1846-9144 Dalipi, Fisnik • 0000-0001-7520-695X Rossi, Arianna • 0000-0002-4199-5898 Despotovic, Vladimir • 0000-0002-8950-4111 Pareja-Lora, Antonio • 0000-0001-5804-4119 Doczekalska, Agnieszka • 0000-0002-3371-3803 Sanchez-Stockhammer, C. • 0000-0002-6294-3579 Drude, Sebastian • 0000-0002-2970-7996 Şahin, Aysel • 0000-0001-6277-6208 Fort, Karën • 0000-0002-0723-8850 Soltan, Angela • 0000-0002-2130-7621 Fuchs, Robert • 0000-0001-7694-062X Soria, Claudia • 0000-0002-6548-9711 Galinski, Christian • (no ORCID number) Shaikh, Sarang • 0000-0003-2099-4797 Gobbo, Federico • 0000-0003-1748-4921 Turchi, Marco • 0000-0002-5899-4496 Gungor, Tunga • 0000-0001-9448-9422 Yildirim Yayilgan, Sule • 0000-0002-1982-6609 Guo, Siwen • 0000-0002-6132-6093 A note on the contributors This report began life in October 2020 at the start of the Language In The Human-Machine Era net- work (lithme.eu). Several online co-writing workshops followed, working together in Google Docs while video-conferencing. The list of contributors was recorded automatically in the Google Doc activity log. The content of the report was finalised on 12 May 2021, at which point this activity log was copied into a Google spreadsheet, and a ‘table chart’ automatically rendered to weigh contributions. On this basis LITHME’s Chair, Dave Sayers, is the named first author. He is very closely followed in the activity log by Rui Sousa Silva, Chair of LITHME Working Group 1, and then by Sviatlana Höhn, LITHME’s Vice- Chair. All three contributed significantly and consistently. The other named contributors all made the report what it is: authoritative, clear, diverse, and future-oriented. We look forward to working together on future editions of this important forecast. Contents 1 Introduction: speaking through and to technology ..............................................................................5 1.1 Speaking through technology ..........................................................................................................................6 1.2 Speaking to technology .....................................................................................................................................7 1.3 The variety of languages, tools and use-cases ...............................................................................................7 1.3.1 Non-standard language (data) ................................................................................................................8 1.3.2 Minority and under-resourced languages .............................................................................................9 1.3.3 Sign languages ..........................................................................................................................................9 1.3.4 Haptic language ......................................................................................................................................10 1.4 Endless possibilities vs boundless risks, ethical challenges .......................................................................10 1.5 The way ahead ..................................................................................................................................................11 2 Behind the scenes: the software powering the human-machine era .................................................. 12 Summary and overview .........................................................................................................................................12 2.1 Text Technology...............................................................................................................................................14 2.1.1 Translation of texts ...............................................................................................................................14 2.1.2 Sentiment, bias .......................................................................................................................................17 2.1.3 Text-based conversation .......................................................................................................................18 2.2 Speech Technology ..........................................................................................................................................19 2.2.1 Automatic speech recognition, speech-to-text, and speech-to-speech ..........................................19 2.2.2 Voice Synthesis .......................................................................................................................................21 2.3 Visual and tactile elements of interaction ....................................................................................................23 2.3.1 Facial expression, gesture, sign language ...........................................................................................23 2.3.2 Tactile expression and haptic technology ..........................................................................................24 2.4 Pragmatics: the social life of words ..............................................................................................................24 2.5 Politeness ...........................................................................................................................................................27 3 Gadgets & gizmos: human-integrated devices ................................................................................. 28 Summary and overview .........................................................................................................................................28 3.1 Augmented Reality ...........................................................................................................................................30 3.1.2 The visual overlay ..................................................................................................................................30 3.1.3 Augmenting the voices we hear ...........................................................................................................31 3.1.4 Augmenting the faces we see ...............................................................................................................32 3.2 Virtual reality ....................................................................................................................................................34 3.3 Automated conversation (chatbots) ..............................................................................................................35 3.4 Second Language Learning and Teaching ....................................................................................................37 3.4.1 Intelligent Computer-Assisted Language Learning (ICALL) .........................................................37 3.4.2 Communicative ICALL ........................................................................................................................38 3.4.3 VR-based Language learning ...............................................................................................................39 3.4.4 AR-based Language Learning ..............................................................................................................40 3.5 Law and Order .................................................................................................................................................42