Robert B. Lisek 1. a Statement Outlining What You
Total Page:16
File Type:pdf, Size:1020Kb
Robert B. Lisek 1. A statement outlining what you plan to work on during the residency period (300 word max) Title of the project: SIBYL SIBYL is Artificial Intelligence that proposes new stories and makes predictions. The project offers an innovative fusion between three domains: storytelling, linguistics and artificial intelligence. The project creates and tests new methods of generating stories and new types of audio-visual performance through interactions with autonomous AI agent. The project investigates the storytelling potential of human beings interacting with an AI agent in a natural language as a possible foundation for establishing the best way for humans and machines to interact overall. SIBYL is presented as an immersive installation that uses AI deep learning software and analog sound-video synthesizers. The main objective of this residency will be to research and develop new forms of storytelling by developing a new dimension of artificial intelligence (AI) and machine learning which will lead to the development of new techniques in the area of storytelling, media arts and performance. The project proposes precise solutions for the understanding of story, automatic creation of stories and the creation of interactional narrative experiences. This proposal investigates the storytelling potential of human beings interacting with an AI agent in a natural language as a possible foundation for establishing the best way for humans and machines to interact overall. Communication with an AI agent, accompanied by an understanding of human activities, is an important area within the future of storytelling, creative communication and media arts. However, our knowledge about language and the nature of thinking is limited by the possibilities of implementing this knowledge in a computer programming language. Formalisation of this problem and solving the challenges related with interactive stories will encourage scholars to research this area, and in the future will allow AI systems and people to communicate in a natural way. We observe the success of artificial neural networks in simulating human performance on a number of tasks: such as image recognition, natural language processing, etc. However, There are limits to state-of-the- art AI that separate it from human-like intelligence. Humans can learn a new skill without forgetting what they have already learned and they can improve their activity and gradually become better learners. Today’s AI algorithms are limited in how much previous knowledge they are able to keep through each new training phase and how much they can reuse. In practice this means that it is necessary to build and adjust new algorithms to every new particular task. This is closer to a sophisticated data processing than to real intelligence. This is why research concerning generalisation are becoming increasingly important. Processes such as intuition, emotions, planning, thinking and abstraction are a part of processes, which occur in the human brain. A generalization in AI means that system can generate new compositions or find solutions for new tasks that are not present in the training corpus. There is domain called AGI where will be possible to find solutions for this problems. Artificial general intelligence (AGI) describes research that aims to create machines capable of general intelligent action. "General" means that one AI program realizes number of different tasks and the same code can be use in many applications. We must focus on self-improvement techniques e.g. reinforcement learning and integrate it with deep learning, recurrent neural networks, probability valuation. O1. Analysis of creating analogies and metaphors. O2. Preparation of a Story Database, building a Model describing composition of stories based on formal methods from computer science and creating AI program for generation of new stories. O3. Creation of new narrative methods and new type of audio-visual installation/performance through the use of AI. O4. Application of proposed methods and technologies in society. The project will use formal methods from computer science to create a model and AI program for the generation of new stories. The project will create a Story Database on the basis of different story archives. Different AI techniques will be tested from the perspective of their usefulness for media art. The Model will be created with the use of Recurrent Neural Networks and Deep Learning. RNN are an excellent tool for creating new stories and predictions. DL simulates the process of generalisation. In the next phase, RNN will be self-improved by Reinforcement Learning in order to simplify the patterns. I will incorporate the RL process into DL for creating a system that will have an ability to learn and self- improve. 2. A short artist's statement describing your overall artistic practice (250 word max) I research at the cutting-edge vanguard of electronic arts. The fields of my activity covering conceptual art, tactical media, environmental arts, new media and video art with a critical consideration of social problems. I am a pioneer of art based on Artificial Intelligence and Machine Learning. I'm using Reinforcement Learning and Recurrent Neural Networks techniques. Some of my projects deal with human - machine interaction and building of new interactive. It is a user-centric development process, new modes of interaction that integrate physiological human sensing, gesture and body language, and smart information analysis and adaptation. I deal also with sound and video synthesis. I'm researching new methods and algorithms for sound and video synthesis based on high order algebras and category theory. The part of my theoretical research in focus on the mind uploading problem which means emulation of brain functions by computing. My projects received numerous awards, distinctions and grants: ARCO Art Fair Award Madrid, Harvestworks Arts Center, Lower Manhattan Cultural Council, CEC ArtsLink Award. I am going to continue research and develop new forms of storytelling by developing a new dimension of artificial intelligence (AI) and machine learning which will lead to the development of new techniques in the area of storytelling, media arts and performance. 3. Please describe what you can contribute to and what do you need from a dynamic, international, multidisciplinary, small scale artist resid ency program? (500 word max) I will contribute to local community by making some artistic talks, exhibition and workshop for artists and composers. How human brain functions and the processes behind the creative inspiration are still unclear for the most part. In this workshop you will approach creativity in the field of visual art, music and film starting from the extraction of randomness. Hacking devices and coding will be used for this purpose. Unexpected applications and methods will leave you with a completely new perception of the subject. This workshop uses randomness and machine learning methods for creation of sound, visuals, performance, installation, interactive media, physical computing, and networking. This is practical course with the use of Supercollider, Pure data/Max Msp, Fluxus, Python, Lisp and hacking analogue devices for detection, creation and amplification of noise. To obtain interesting results in music and art we need randomness. Randomness is important when you want the Neural Network from the same input to create different possibilities as an output, without generating the same output over and over again. Therefore it is different than in science, which is all about grouping and clustering. In art and music we don't want to endlessly obtain the same result. When you are composing sound or images, you don't want the neural network program to create the same sets of sounds and images; instead, you need creativity and variability. One of the solutions is to parametrize NN outputs with the use of probability distribution. A different way is to add noise directly to NN, instead of modeling the distribution parameters. Paradoxically, in such NNs, the more randomness during training, the better the results. Good random generators allow to avoid situations, when NN gets fixed in local minima. The workshop consists of: • work with three main types of learning algorithms: Neural Networks and Deep Learning, Reinforcement Learning, Genetic Algorithms. Supervised and unsupervised learning, data sampling, backpropagation, explorations of environment, generalization, experimentation, markov models and processes (speech recognition), support vector machines (image recognition) • practical application of different types of random generators for constructing visual works, music compositions (random generators, random walks, monte carlo, etc), dealing with large systems of events: probability distributions (average, spread, deviation) • extraction of randomness from a physical processes through hacking analogue devices for detection, amplification, sonic and visual analysis (electromagnetic fields, gas particles, photons and decay of radioactive materials) • applications RG and NN in building of new prototypes, artworks and network app 4. Selected projects Primes engine The main goal of the project was create a vehicle that enable experience foundations of reality and tests limitations of computing by dealing with the largest know prime number. The installation produces a very high radiation in several modes having different frequencies (wavelengths) and polarizations based on digits sequence of the largest know