The Increased Scientific Activity in Artificial Intelligence During the Pandemic
Prof. Guntis Bārzdiņš University of Latvia, IMCS AI-Lab
The 17th Baltic Conference on Intellectual Co-operation: Mathematics for Society 28 June 2021, Tallinn, Estonia ~80% Complied with COVID-19 Lockdowns SAAS Enabled Relocation Cities CountrySide
Increased activity Decreased activity Data source: Latvian Mobile Telephone (LMT) A new class of high-skill service workers, the digital nomads, is emerging Outline
• There are two "pandemies" ongoing o COVID-19: efficient mRNA vaccines available, will be forgotten soon o SAAS (Software As A Service): will genuinely change the course of history
• COVID-19 pandemy accelerated the SAAS adoption o 80% of population can work/study from home (using SAAS), and economy survives o Online shopping and delivery services cover basic needs (depend on SAAS) o International travel and physical meetings can be avoided (thanks to SAAS) o Consequences: SAAS providers accumulated BIG DATA on all activities of 80% of population
• BIG DATA drives the progress of Artificial Inteligence o Google, FaceBook, Apple, Amazon, Tesla, Microsoft, Twitter all are SAAS companies (↑52% in 2020) o These companies realized they can grow faster by sharing AI research results (fortunately) o Shared AI research enables cloud-in-the-box alternative to SAAS: PiniTree.com from Latvia Transformers & Knowledge 28 May, 2020 (OpenAI) 175B parameters GPT-3 0.5T words training text Graphs 10,000 GPUs + 300,000 CPUs E=mc2 GPT-3 Scaling Laws
Data Compute
Test error diminishes by increasing any of these
Richard Sutton "bitter lesson": Model 28 October, 2020 (OpenAI) GPT-3 has no inductive bias, scale wins over biases. size $1Bn $1Bn Tesla Microsoft Only small data required biases. (overparameterisation)
https://youtu.be/__Kj9HeU5tE https://arxiv.org/abs/2105.12806 Henighan T., et.al., Scaling Laws for Autoregressive https://ml.berkeley.edu/blog/posts/dalle2/ Generative Modeling . 2020, https://parl.ai/projects/params_vs_compute/ https://arxiv.org/pdf/2010.14701.pdf
GPT-3: Translation
Eduards Mukans, PhD student: Trained only on "Farewell to Arms" (300KB, 7490 parallel sentences) achieves BLEU 12.5 for LV-EN translation https://github.com/emukans/minGPT 5 January, 2021 (OpenAI) GPT-3: Text-to-Image 12B parameters 250M text-image pairs Translation (DALL-E)
Unsupervised Visual concept discovery Supervised via text Visual concept ranking Dāgs Ādams Grīnbergs, MSc student: DALL-E and CLIP replicated https://github.com/NomadicDaggy/DALLE-pytorch KnowledgeGraph Editor31 May, 2020 (PiniTree) with TextComprehesion
Human-in-the-loop AI suggestions for: (a) linking to the KnowledgeGraph (b) enriching the KnowledgeGraph (c) inverse URL text comprehension Guntis Barzdins, Didzis Gosko, Karlis Cernas, et.al.: Pini Language and PiniTree Ontology Editor: Annotation and Verbalisation for Atomised Journalism https://pinitree.com Open Problems in AI
Knowledge Transformers (language models) Graphs (schemaless database) The missing links of AI to achieve the CriticalMass
Concept Creation (discrete & compositional)
AI breakthroughs nowadays almost exclusively happen in industry and require vast resources not available in the academic environment Google FaceBook Twitter etc.
Data SAAS Money from users companies targeted advertising Google FaceBook Twitter etc.
Data SAAS Money from users companies targeted advertising
Fortune 500 companies represent 2/3 of U.S. GDP and have $22.6trn in market cap
Top 5 SAAS companies have $7.8trn in market cap Google FaceBook Twitter etc.
Data SAAS Money from users companies targeted advertising Misuse of SAAS Google FaceBook Twitter etc.
Data SAAS Money from users companies targeted advertising
FLoC Source: https://www.eff.org/deeplinks/2021/03/googles[Flock - a group of sheep,-floc goats,-terrible or- ideabirds ] Misuse of SAAS Google FaceBook Twitter etc.
Data SAAS Money from users Make companies targeted advertising free shareing Creative"SAAS Data Pandemy"easy and Regulation:CambridgeAnalytica (used to be monetised) • Music voter profiling • Publishing• GDPR • Software • Personal Data protection • Movies Brexit Trump • Thedemocrats E.U. AI Act autocrats undecided • A Risk-Based Policy Approach to AI Application COVID• -19The E.U. DigitalFew get Markets Act & Digital Sevices Act vaccines, $ and • Gatekeeperspower (large online platforms) Regulation creativity • Elections reduced to arithmentics, 3rd world Digitalintellectuals Europe business Programme low income (9.2B € ) countries lack If you are so smart, targeted advertising • COVID-19 pandemic highlighted20% ... for Europe not to be dependent on this then why are not to this group "inconvenient" yousystems rich? and solutions coming from other regions of the world group and 80% capacity• US&EUTax Trade / Universal and Basic Technology Income Inequality ofCouncil income Rousevelt plan for infrastructure • Focus(starting on cooperation during COVIDand cybersecurity-19) to create of middleclass platforms Three AI / SAAS uses
Misuse Productivity (targeted advertising /flock (secure, trusted) social division pandemy)
an EU company
Dual-use (police-state, AI cold-war)
Yearly % of productivity growth (AI boom anticipated) Cloud-in-the-box alternative *) Decentralised **) Good-enough to global SAAS ***) Eternal archiving PiniTreePlatform Ecosystem
selma-project.eu Video subtitling, translation, synchronous voice-over
LETA.lv Tezaurs.lv News enriched with Documents enriched with EnterpriseRegister data Dictionary data
STT / ASR KnowledgeGraph Editor Editor
Aslimnica.lv Translation NLPStation.ee Docker Medical speech Editor LUMII.lv NewsWorkers translation and transcription and editing DeepLearning lifecycle clustering in 100 languages management Cloud-in-the-box: generic data&state backend + kubernetes AI containers = REST API frontends Thank You for Your Attention