Computational Methods for Data Analysis – 2014/15

Lab 5: Neural Networks in R

In this lab we’ll see how to train a neural net for classification, using the package nnet.1

Starting

In this lab we’ll use the nnet library; you should start by loading it into your R environment using library (and installing the package using install.packages if you haven’t installed them yet).

Loading the cheese dataset

In this exercise we’ll use the cheese dataset, containing information about taste, acetic component, lactic component, etc of several types of cheese.

Exercise: Download the cheese data from the Lab5 directory on the course website.

Exercise: Load the cheese data into a variable called cheese in your workspace.

Exploring the data

Exercise: Use the commands cheese names(cheese) summary(cheese) to get a feel for the data.

Create and analyze a perceptron

Fit a simple neural net to predict with no hidden layer and linear activation function using the function nnet(). nnet() takes as arguments - a formula specifying output and input variables (just as in linear regression) - a parameter data specifying the data frame to be used 1 The lab follows closely the presentation in http://www.louisaslett.com/Courses/Data_Mining/ST4003-Lab5- Introduction_to_Neural_Networks.pdf - a parameter size specifiying the number of hidden units (0 in this case) - a parameter skip specifying whether the inputs are directly connected with the outputs or now - a parameter linout specifying whether a linear activation function is desired. (see http://cran.r-project.org/web/packages/nnet/nnet.pdf for full details of all the arguments)

For example, the following command creates a perceptron with three input variables, one output variable, and using a linear activation function instead of the logistic function.

> fitnn1 = nnet(taste ~ Acetic + H2S + Lactic, cheese, size=0, skip=TRUE, linout=TRUE) > summary(fitnn1)

In the output of summary(), i1, i2 and i3 are the input nodes (so acetic, H2S and lactic respectively); o is the output node (taste); and b is the bias. The values under the links are the weights.

Evaluating the predictions of a perceptron

Let us now use the J cost function implemented in Lab 1 to evaluate the model:

J = function (model) { return(mean(residuals(model) ^ 2) / 2) }

Exercise Compute J of fitnn1

Let us now compare this to the fit we get by doing standard least squares regression: fitlm = lm(taste ~ Acetic + H2S + Lactic, cheese) summary(fitlm)

Exercise Compute J of fitlm

Adding a Hidden Layer With a Single Node

We now add a hidden layer in between the input and output neurons, with a single node.

> fitnn3 = nnet(taste ~ Acetic + H2S + Lactic, cheese, size=1, linout=TRUE)

During backpropagation you should see values like the following: # weights: 6 initial value 25372.359476 final value 7662.886667 converged

You can visualize the structure of the network created by nnet using the function plot.nnet(), in the devtools library library(devtools)

The following encantation enables devtools: source_url('https://gist.githubusercontent.com/fawda123 /7471137/raw/466c1474d0a505ff044412703516c34f1a4684a5/n net_plot_update.r')

We can now plot the architecture of fitnn3 with the following command: plot.nnet(fitnn3)

(NB plot.nnet() doesn’t seem to work with perceptrons)

Train and Test subsets

In this exercise we are going to use part of our data for training, part for testing.

First of all, download the Titanic.csv dataset into a variable called data. This dataset provides information on the fate of passengers on the maiden voyage of the ocean liner ‘Titanic’. Explore the dataset.

Then split the dataset into a training set consisting of 2/3 the data and a test set consisting of the other 1/3 using the method seen in Lab 4. n <- length(data[,1]) indices <- sort(sample(1:n, round(2/3 * n))) train <-data[indices,] test <- data[-indices,]

The nnet package requires that the target variable of the classification (in this case, Survived) be a two-column matrix: one column for No, the other for Yes — with a 1 in the No column for the items to be classified as No, and a 1 in the Yes column for the items to be classified as 0:

No Yes Yes  0 1 Yes  0 1 No  1 0 Yes  0 1

Etc. Luckily, the class.ind utility function in nnet does this for us: train$Surv = class.ind(train$Survived) test$Surv = class.ind(test$Survived)

(Use head to see what the result is on train for instance.) We can now use train to fit a neural net. The softmax parameter specifies a logistic activation function. fitnn = nnet(Surv~Sex+Age+Class, train, size=1, softmax=TRUE) fitnn summary(fitnn)

And then test its performance on test: table(data.frame(predicted=predict(fitnn, test)[,2] > 0.5, actual=test$Surv[,2]>0.5))