and Fundamental Physics Research Tommaso Dorigo, INFN – Padova

5th USERN Congress, Workshop 2 November 8, 2020 - Tehran Contents

• Artificial Intelligence: Where It Comes From, What It Is, Where It Is Going

• Applications in Fundamental Physics Research

• Quantum Computing: Physics Research for AI

• The Future What is Intelligence ? Before we discuss artificial intelligence, we should agree on what intelligence is – and what isn’t. • The term «intelligence» comes from Latin “intelligo”  to comprehend, to perceive …but that does not help much. We can consider the literature for help. We find that notable definitions differ significantly, also in relation to what can be general and what is specific of human beings:

“The aggregate or global capacity of the individual to act purposefully, to think rationally, and to deal effectively with his environment” [Wechsler 1944] “The unique propensity of human beings to change or modify the structure of their cognitive functioning to adapt to the changing demands of a life situation” [Feuerstein 1990]

But also, and more useful to us, are more abstract definitions such as:

“Intelligence measures an agent's ability to achieve goals in a wide range of environments” [Legg 2007] “Intelligence is goal-directed adaptive behavior” [Sternberg 1982] What is Artificial Intelligence ?

Curiously, a precise definition of Artificial Intelligence is not less challenging than that of Intelligence at large • AI: “Intelligence demonstrated by machines” – devices that have the ability to perceive the environment and take actions toward achieving some goal The above puts automatic wipers and thermostats in the AI category…

Patricia McCorduck describes the AI effect: «AI is what hasn’t been done yet» [Mc Corduck 2004] We continue to raise the bar as we get accustomed to new progress in artificial intelligence research: - Self-driving cars ? Just a smart NN with lots of input data - ? Only an algorithm, not real AI - Facebook targeted ads, gmail text completion ? Specialized data-crunching, little more than nearest- neighbor techniques

At the basis of this is the perception that we, as sentient beings, «know» what we are doing. But we, too, are machines! A Five-Slide History of AI : 1 – the Origins

The idea of artificial intelligence in ancient history is only found connected with that of creating life from unanimated matter: • Talos, an automated guardian of Crete • Galatea, a statue made by king and sculptor Pygmalion • The Golem, from Jewish folklore A different path emerged much later, with the idea of automata and real examples built by craftsmen

A significant development of the underpinnings of AI was produced by SciFi writers in the late XIX and early XX centuries The study of mathematical logic provided even more concrete bases, with a demonstration that any form of mathematical reasoning could be mechanized [Church 1936; Turing 1936] AI History: 2 – Before the Birth

Important milestones toward a recognition of the possibility of artificial thinking:

• In 1943 Walter Pitts and Warren McCullock showed how artificial neurons could be arranged in networks to perform simple logic [Pitts 1943](the basic unit term «psychon» never caught up)

• In 1950 Turing laid down the bases to discuss about AI, proposing the («imitation game»)to express how intelligent thinking could be defined [Turing 1950]

• took inspiration from Pitts and McCullock to build the first neural network in 1951 AI History: 3 – The Modern Origins

In 1956 a landmark workshop was organized in Dartmouth by Minsky and John McCarthy. At the workshop were laid the bases of fundamental research in artificial intelligence –a name coined there

The paradigm that was born in Dartmouth was that “every aspect of learning or any other feature of intelligence can be so precisely described that a machine can be made to simulate it”

Simulation! - A sham object: counterfeit [Merriam-Webster] - The action of pretending: deception [Oxford Languages] - The production of a computer model of something [OL] AI History: 4 – Summers …

Following Dartmouth, AI received large funding and ideas thrived. New applications were found and problems solved; enthusiasm reigned: • 1958: “Within ten years a digital computer will be the world's chess champion; […] within ten years a digital computer will discover and prove an important new mathematical theorem.” [Simon 1958] • 1967: “Within a generation ... the problem of creating 'artificial intelligence' will substantially be solved.” [Minsky 1967]

Frank Rosenblatt invented in 1958 the perceptron, a single-layer NN element [Rosenblatt 1958], and predicted this would bring a breakthrough: «a perceptron may eventually be able to learn, make decisions, and translate languages”. … And Winters

A 1969 book by Minsky and Papert [Minsky 1969] threw serious doubt on the capability of such systems. This had a huge influence on the general perception of AI. In the early 1970s the lack of results given the high expectations, and the lack of sufficient computing power to handle anything but toy applications started a period of almost zero funding and academic interest (the «AI Winter»).

A short revival of interest in AI research came in the 1980s, with the focus on «expert systems» - computer programs that answer questions or solve problems about a specific domain of knowledge, using logical rules derived from the knowledge of experts. But that was also short-lasting, due also to the high cost of specialized hardware and a unfavourable economic juncture  Second AI winter AI History 5:  AI

From the 90ies the path to AI took a different direction eased by increases in cheap CPU, with development of Statistical Learning techniques (e.g. decision trees) and breakthroughs in neural networks research • [Mitchell 1997] provides a succinct definition of what Machine Learning is: “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P , if its performance at tasks in T , as measured by P , improves with experience E“ In general, machine learning can be divided into supervised (classification or regression tasks) and unsupervised (autonomous learning of data structures; clustering; anomaly detection) The development of complex neural networks brought many successful applications, where we may now identify artificial intelligence at work 1997: Deep Blue defeats the chess World Champion G. Kasparov BTW - Can We Agree on What "Learning" Is?

Maybe this really is the relevant question. Not idle to answer it, as by clarifying what our goal is we take a step in the right direction. Artificial intelligence requires machines to learn new tricks from experience!

Some definitions of Learning: • the process of acquiring new, or modifying existing, knowledge, behaviors, skills, values, or preferences • the acquisition of knowledge or skills through study, experience, or being taught. • becoming aware of (something) by information or from observation. • Also, still valid is Mitchell's definition in previous slide

In general: learning involves adapting our response to stimuli via a continuous inference. The inference is done by comparing new data to data already processed. How is that done? By the use of analogy. Analogies: The Heart of Thinking

Analogies: arguably the building blocks of our learning process. We learn new features in a unknown object by analogy to known features in similar known objects This is true for language, tools, complex systems – EVERYTHING WE INTERACT WITH

... But before we even start to use analogy for inference or guesswork, we have to classify unknown elements in their equivalence class! (Also, one has to define a metric along which analogous objects align)

In a sense, Classification is even more "fundamental" in our learning process than is Analogy. Or maybe we should say that classification is the key ingredient in building analogies. Classification and the Scientific Method Countless examples may be made of how breakthroughs in the understanding of the world were brought by classifying observations - Mendeleev's table, 1869 (150 years ago!) - Galaxies: Edwin Hubble, 1926 - Evolution of species: Darwin, 1859 (classification of organisms was instrumental in allowing a model of the evolution, although he used genealogy criteria rather than similarity) - Hadrons: Gell-Mann and Zweig, 1964 It is thus no surprise that the advancements in supervised learning are bringing huge benefits to fundamental physics research today The Status of AI Research Today

Applications of are today ubiquitous, although we sometimes fail to identify them

More market-driven: • Speech and • Search engines, advertising • Stock market analysis, predictive models, fraud detection • Self-driving cars

More basic-research-oriented (also feeding into above class): • Mastering closed systems (go, chess, computer games) simpler to define tasks and monitor progress • Natural language processing allows computers to understand, interpret and manipulate human language • Object and image recognition, robotics  methods to give spatial awareness to machines The Future: Is A(G)I desirable?

Although a general artificial intelligence (AGI) is generally believed to be still far in the future (20? 40? 60 years from now?), the possibility to create it must be contemplated now

Humanity is going to face in the next few decades the biggest "growing pain" of its history: how to deal with the explosive force of artificial intelligence

Once we create an AGI - a superintelligence - there is no turning back – in terms of safeguards we have to "get it right the first time", or we risk to become extinct in a very short time. This is the misalignment problem: a superintelligence may give itself a purpose where humanity does not play the role we wish to have…

But we cannot stop AI research, no more than we could stop nuclear weapons proliferation

Interesting times ahead! Particle Physics Applications

In fundamental physics research we study the inner structure of matter by colliding elementary particles at high kinetic energy The resulting reactions produce very complex «events» - the information is stored in millions of electronic channels We need to reconstruct the reactions from that information pattern recognition We need to separate small signals from background supervised classification We want to measure things with high precision supervised regression We may want to find unpredicted structure in the data  clustering, anomaly detection, unsupervised learning The New Mendeleev Table

Through decades of studies, particle physicists have formulated a successful description of matter and its interactions: the Standard Model

The theory allows to predict the outcome of reactions with high precision But new physics might be hiding in rare processes – the LHC is still searching for them Artificial Intelligence is used for collecting, reconstructing, and interpreting the data The use cases are special enough that such effort is drawing the interest of computer scientists  tight collaboration between AI and Physics research is underway Neural Networks work by learning structure in input data through a process called backpropagation, to find parameters maximizing some utility function In supervised classification the utility is the maximum separation of a signal from One Example backgrounds In particle physics analyses, the separation is an intermediate step: with the data selected by the procedure we want to measure some physics parameter as well as possible Modeling imperfections enter at this stage and may ruin the accuracy  systematic uncertainties In a recent development [Dorigo 2019] we developed a feedback loop to let the network know what our real objective is, injecting in the utility function a notion of the effects of mismodeling  Large improvements in accuracy! A New Avenue: Optimization of Experiments Particle detectors are hugely complex. Their design follows robust paradigms and experience from previous experiments, but today we have AI on our side What if we teach a machine to find the best configuration of our detection elements, after considering what our final goals are, and how information extraction procedures are affected by construction choices? Potential gains are enormous, but creating the right infrastructure is a very tall order!

The capability of creating differentiable models of a system today enables deep learning programs to construct the derivative of some high-level objective function with respect to all the system’s parameters We may conceive a pipeline where all the ingredients of a complex experimental task may play in, which may be navigated with feedback loops to find the optimal configuration

MODE (https://mode-collaboration.github.io/) is a collaboration born to explore these exciting new possibilities for applications to physics research as well as industry Quantum Computing: Physics for AI

Investigating matter at the shortest distance scales, and the interactions it withstands, can be argued to be an indirect way to support research in AI coming from pure research in Physics A more direct «fifth column» is the use of quantum phenomena to create a new computational paradigm: quantum computing (QC) - the use of superposition and entanglement of quantum states for computational purposes Some QC models like quantum circuits work using the qubit, the QC analogue of The state of a qubit, |ψ>, projects the bit. Qubits have value 0, 1, or a superposition of the two states; their reading a 0/1 measurement along z axis however always produces a definite 0/1 answer, with probabilities related to the quantum state of the qubit.

Quantum computers may in principle solve any problem as a classical computer, and they also withstand the Church-Turing theorem – but the time to solve problems may be vastly different. Hence QC can be important for solving certain problems that are practically unfeasible classically  quantum supremacy Google AI claimed one year ago [IBM 2019] to have produced one such example with a quantum computer. The use of QC to speed up calculations for AI applications is an intensive area of research at present The Future In a not-so-distant future, humankind will be facing bigger and bigger challenges and opportunities from AI technology developments

Breakthrough Timescale Opportunities Challenges Automatization of jobs 10-20 y More free time, more effective Change the economy, provide crops, reduction of famine living salary to human beings Globalization of data 0-20 y Easier access to information Security, privacy issues; protect Self empowerment from hacking of democracies Internet of things 10-30 y Augmentation of human Prevent a growing divide faculties between rich and poor countries Neural implants 20-40 y Avoid a dystopian society, castes AGI 40-80 y A crucial step toward a multi- Avoid misalignment problem, planet, evolved civilization integrate machines

AI provides more power, and with it bigger responsibilities are coming. Will we be prepared? As usual, fundamental research in physics will provide tools and understanding, but the crucial ingredient is humanity’s union in pursuing the common good. THANK YOU FOR YOUR ATTENTION! References

• Wechsler 1944: D. Wechsler, The measurement of adult intelligence, Baltimore: Williams & Wilkins (1944) ISBN 978-0-19-502296-4 • Feuerstein 1990: R. Feuerstein et al., Dynamic assessments of cognitive modifiability, ICELP Press (1990) • Legg 2007: S. Legg and M. Hutter, A Collection of Definitions of Intelligence , Proc. AGI Workshop 206, 157 (2007). ISBN 9781586037581 • Sternberg 1982: R.J. Sternberg and W. Salter, Handbook of human intelligence, Cambridge University Press (1982) ISBN 978-0-521-29687-8 • McCorduck 2004: P. McCorduck, Machines who think, AK Peters (2004), ISBN 1-56881-205-1 • Church 1936: A. Church, An Unsolvable Problem of Elementary Number Theory, American Journal of Mathematics 58 (2) 345 (1936), doi:10.2307/2371045 • Turing 1937: A.M. Turing, On Computable Numbers, with an Application to the Entscheidungsproblem, Proceedings of the London Mathematical Society, 2, 42, 230 (1936), doi:10.1112/plms/s2-42.1.230 • Pitts 1943: W.S. McCulloch and W. Pitts, A logical calculus of the ideas immanent in nervous activity, Bulletin of Mathematical Biophysics. 5 (4) 115 (1943), doi:10.1007/bf02478259 • Turing 1950: A.M. Turing, Computing Machinery and Intelligence, Mind LIX (236) 433 (1950), doi:10.1093/mind/LIX.236.433 • Simon 1958: H.A. Simon and A. Newell, Heuristic Problem Solving: The Next Advance in Operations Research, Operations Research, 6 (1958), doi:10.1287/opre.6.1.1 • Minsky 1967: M. Minsky, Computation: Finite and Infinite Machines, Englewood Cliffs, N.J.: Prentice-Hall (1967) • Rosenblatt 1958: F. Rosenblatt, The perceptron: A probabilistic model for information storage and organization in the brain, Psychological Review 65 (6) 386–408 (1958), doi:10.1037/h0042519 • Minsky 1969: M. Minsky and S. Papert, Perceptrons, an introduction to computational geometry, MIT press (1969), ISBN: 9780262630221. • Mitchell 1997: T. Mitchell, Machine Learning, McGraw Hill. (1997). ISBN 978-0-07-042807-2. • Dorigo 2019: P. de Castro Manzano and T. Dorigo, INFERNO: Inference-Aware Neural Optimization, Computer Physics Comm. 244 (2019) 170, 10.1016/j.cpc.2019.06.007 • IBM 2019: On "Quantum Supremacy“, IBM Research Blog. 2019-10-22.