Artificial Intelligence Katarzyna Mazur Information Contact: I [email protected], [email protected] I +48 (81) 537-29-38 I Maria Curie Sklodowska University, Akademicka 9 Street, Department of Informatics, room 412 (IV floor) WWW: I https://cybersecurity.umcs.lublin.pl/ai-wsei/ Assignment: I ... Course Content I Introduction I Multilayer Perceptrons I Convolutional Neural Networks I Recurrent Neural Networks I ... Introduction: what is AI? I Artificial intelligence - or AI for short - is technology that enables a computer to think or act in a more ’human’ way. It does this by taking in information from its surroundings, and deciding its response based on what it learns or senses Introduction: what is AI? I What is the relationship between AI, machine learning, neural networks and deep learning? I You can think of deep learning, machine learning and artificial intelligence as a set of Russian dolls nested within each other, beginning with the smallest and working out I Deep learning is a subset of machine learning, and machine learning is a subset of AI, which is an umbrella term for any computer program that does something smart I In other words, all machine learning is AI, but not all AI is machine learning, and so forth Introduction: what is AI? Introduction: what is AI? Introduction: what is AI? I Let us start with the basics, and describe a basic unit of an AI, that is, neurons and neural networks I The artificial neuron has a similar structure to the neurons of the human brain - so at first let us focus on biological neurons, in order to understand their operation Introduction: biological neuron I There are 4 parts of a typical nerve cell (neuron): dendrites, axon, synapses, and nucleus I Dendrites accept the inputs, nucleus processes the inputs, axon turns the processed inputs into outputs and synapses are the electrochemical contact between the neurons Introduction: artificial neuron (perceptron) I Artificial neuron consists of: I inputs (which correspond to synapses in biological neuron), I weights (which correspond to dendrites in biological neuron), I activation function (or transfer function, activation function (or transfer function, which corresponds to nucleus in bioloical neuron) I outputs (which correspond to axon in biological neuron) Introduction: artificial neuron (perceptron) I Inputs to the network are represented by the xi , where i 2 f1; :::; ng I Each of these inputs are multiplied by a connection weight, wi , where i 2 f1; :::; ng, weight determines the importance of incoming value I These products are simply summed, fed through the transfer function to generate a result and then output I Outputs to the network are represented by the yj , where j 2 f1; :::; mg Introduction: artificial neuron (perceptron) I The activation function determines the value of the neuron’s output. The simplest form of activation function is a certain type of step function. It mimics the biological neuron firing upon reaching its firing threshold by outputting a 1 if the total input exceeds a given threshold quantity, and outputting a 0 otherwise I However, for a more realistic result, one needs to use a non-linear activation function. One of the most 1 commonly used is the sigmoid function: f (x) = (1+e−x ) Introduction: biological neuron vs artificial neuron Introduction: exercises I Exercise 1: using programming language Python, implement, without any additional modules and libraries, a simple single layer perceptron (tip: perceptron is a single neuron) with the threshold activation function. Inputs, weights and threshold should be read from a text file. I Exercise 2: using programming language Python, implement, without any additional modules and libraries, a simple single layer perceptron (tip: perceptron is a single neuron) with the sigmoid activation function. Inputs, weights and threshold should be read from a text file. Introduction: Artificial Neural Networks (ANN) I For understanding single layer perceptron, it is important to understand Artificial Neural Networks (ANN) I Artificial neural networks is the information processing system the mechanism of which is inspired with the functionality of biological neural circuits I An artificial neural network possesses many processing units connected to each other Introduction: Artificial Neural Networks (ANN) Artificial Neural Networks (ANN) is: I An information processing architecture loosely modelled on the brain I Consists of a large number of interconnected processing units (neurons) I Works in parallel to accomplish a global task I Generally used to model relationships between inputs and outputs or find patterns in data Introduction: Artificial Neural Networks (ANN) Artificial Neural Networks (ANN) has 3 types of layers: input layer, hidden layer or layers, and the output layer Introduction: Artificial Neural Networks (ANN) Building blocks of each of these layers are neurons. Neurons are single processing unit and are built as follows: Introduction: Artificial Neural Networks (ANN) I There are 2 types of artificial neural networks architecture: feed-forward network and recurrent network I Feed-forward networks can be single layered or multi-layered I Recurrent networks can be single layered or multi-layered An example of a single layer ANN: Introduction: Artificial Neural Networks (ANN) An example of a multi layer ANN: Introduction: Artificial Neural Networks (ANN) A feed-forward artificial neural networks have the following characteristics: I Perceptrons are arranged in layers, with the first layer taking in inputs and the last layer producing outputs. The middle layers have no connection with the external world, and hence are called hidden layers. I Each perceptron in one layer is connected to every perceptron on the next layer. Hence information is constantly "fed forward" from one layer to the next., and this explains why these networks are called feed-forward networks. I There is no connection among perceptrons in the same layer. Introduction: Artificial Neural Networks (ANN) A recurrent artificial neural networks have the following characteristics: I Recurrent nets are a powerful set of artificial neural network algorithms especially useful for processing sequential data such as sound, time series (sensor) data or written natural language I Recurrent nets differ from feedforward nets because they include a feedback loop, whereby output from step n-1 is fed back to the net to affect the outcome of step n, and so forth for each subsequent step Introduction: Artificial Neural Networks (ANN) Introduction: Artificial Neural Networks (ANN) A perceptron is: I An algorithm for supervised learning of binary classifiers I A binary classifier is a function which can decide whether or not an input, represented by a vector of numbers, belongs to some specific class I It is a type of linear classifier, i.e. a classification algorithm that makes its predictions based on a linear predictor function combining a set of weights with the feature vector A neural network is really just a composition of perceptrons, connected in different ways and operating on different activation functions Introduction: exercises I Exercise 3: using programming language Python, implement, without any additional modules and libraries, a simple multi layer perceptron with the threshold activation function and one hidden layer. Inputs, weights and threshold should be read from a text file. I Exercise 4: using programming language Python, implement, without any additional modules and libraries, a simple multi layer perceptron with the sigmoid activation function and two hidden layers. Inputs, weights and threshold should be read from a text file. Introduction: Artificial Neural Networks (ANN) Introduction: Artificial Neural Networks (ANN) Input 1: 001 Weights for input 1: 1 2 3 Input 2: 101 Weights for input 2: 4 5 6 Input 3: 111 Weights for input 3: 7 8 9 Input 4: 000 Weights for input 4: 1 2 3 Input 5: 110 Weights for input 5: 4 5 6 Treshold for neruon 1 in hidden layer: 3 Treshold for neruon 2 in hidden layer: 5 Treshold for neruon 3 in hidden layer: 6 Introduction: Artificial Neural Networks (ANN) Input 1:001 Weights for input 1:123 Input 2:101 Weights for input 2:456 Input 3:111 Weights for input 3:789 Input 4:000 Weights for input 4:123 Input 5:110 Weights for input 5:456 Input for neruon 1 in hidden layer: 1*0+4*1+7*1+1*0+4*1 = 15 Output from neruon 1: 1, because 15 > 3 Introduction: Artificial Neural Networks (ANN) Input 1: 001 Weights for input 1: 123 Input 2: 101 Weights for input 2: 456 Input 3: 111 Weights for input 3: 789 Input 4: 000 Weights for input 4: 123 Input 5: 110 Weights for input 5: 456 Input for neruon 2 in hidden layer: 2*0+5*0+8*1+2*0+5*1 = 13 Output from neruon 2: 1, because 13 > 5 Introduction: exercises I Exercise 5: using programming language Python, implement, without any additional modules and libraries, a network from the image. The inputs, weights and treshold values should be taken from image too. Introduction: what is deep learning? I Deep learning is a machine learning technique that teaches computers to do what comes naturally to humans: learn by example I Deep learning is a key technology behind driverless cars, enabling them to recognize a stop sign, or to distinguish a pedestrian from a lamppost. It is the key to voice control in consumer devices like phones, tablets, TVs, and hands-free speakers I In deep learning, a computer model learns to perform classification tasks directly from images, text, or sound Introduction: what is deep learning? I Models are trained by using a large set of labeled data and neural network architectures that contain many layers I Most deep learning methods use neural network architectures, which is why deep learning models are often referred to as deep neural networks I The term “deep” usually refers to the number of hidden layers in the neural network.