Matlab Tutorial 6

Introduction to Matlab

MATLAB Tutorial Amir massoud Farahmand http://www.cs.ualberta.ca/~amir

Ela Pek! alska, Marjolein van der Glas [CMPUTPattern Recognit651] PrionobabilisticGroup, Faculty Graof Appliephicald Scienc Modelses Russ DelftGreinerUniversit andy of T Mattechnology Brown

VersionJan 0.6:uary September2002 24, 2008

Send comments to [email protected]

The MATLAB logo is a trademark of MathWorks, Inc. What is MATLAB?

• A scripting/programming language for numerical calculations • Pros • Fast prototyping for your [numerical] algorithms • Drawing graphs easily • Fast matrix computation • Cons • Slow for many other things (e.g. for loops are terribly slow) • Not a general-purpose programming language • Not so cheap How to Run MATLAB @ U of A? login to bonanza, pipestone, eureka, etc.

or matlab -nodesktop You need to type here! I Will Talk About ...

• vectors and matrices in MATLAB • several useful predefined functions • graphics • writing your own functions • tricks for writing efficient codes • ... Vectors >> a = 2

a =

2

>> b = [1 2 3]

b =

1 2 3

>> c = [-1 1.1 2]'

c =

-1.0000 1.1000 2.0000 Vectors >> a = 2 >> a*b a = ans =

2 2 4 6

>> b = [1 2 3] >> c*b b = ans =

1 2 3 -1.0000 -2.0000 -3.0000 1.1000 2.2000 3.3000 >> c = [-1 1.1 2]' 2.0000 4.0000 6.0000 c = >> b*c

-1.0000 ans = 1.1000 2.0000 7.2000 Vectors >> sin(b) >> a = 2 >> a*b ans = a = ans = 0.8415 0.9093 0.1411

2 2 4 6 >> exp(c)

>> b = [1 2 3] >> c*b ans = b = ans = 0.3679 3.0042 1 2 3 -1.0000 -2.0000 -3.0000 7.3891 1.1000 2.2000 3.3000 >> c = [-1 1.1 2]' 2.0000 4.0000 6.0000 >> b + c ??? Error using ==> plus c = >> b*c Matrix dimensions must agree.

-1.0000 ans = >> b + c' 1.1000 2.0000 7.2000 ans =

0 3.1000 5.0000 Another way to generate vectors

>> x = -2*pi:0.01:2*pi; >> y = sin(x); >> y2 = sin(x) + cos(2*x) + 0.1*sin(10*x); >> plot(x,y,'b') >> hold Current plot held >> plot(x,y2,'k') >> x = -2*pi:0.01:2*pi; >> y = sin(x); >> y2 = sin(x) + cos(2*x) + 0.1*sin(10*x); >> plot(x,y,'b') >> hold Current plot held >> plot(x,y2,'k') xlabel('Time'); ylabel('Amplitude'); title('My sinusoid wave') >> A = [1 2 3;4 5 6] Matrices

A =

1 2 3 4 5 6

>> B = ones(3,5)

B =

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

>> C = zeros(3)

C =

0 0 0 0 0 0 0 0 0 >> A = [1 2 3;4 5 6] Matrices

A =

1 2 3 4 5 6

>> B = ones(3,5) size(D) >> D = A*B B = ans = D = 1 1 1 1 1 1 1 1 1 1 2 5 6 6 6 6 6 1 1 1 1 1 15 15 15 15 15 >> C = zeros(3)

C =

0 0 0 0 0 0 0 0 0 Accessing Elements of Vectors and Matrices a = [1 2 3 4 5 6 7 8 9 10] A = [1 2 3 4;5 6 7 8; 9 10 11 12;13 14 15 16] a = A =

1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 a(3) 9 10 11 12 13 14 15 16 ans = >> B = A(2:3,2:4) 3 B =

6 7 8 10 11 12 Random Number Generators rand(5,1) Uniform ans =

0.0193 0.7683 0.4579 0.0436 0.0996

>> randn(5,1) Normal (Gaussian)

ans =

-0.6223 -0.8081 -0.8724 0.1395 0.6093 Random Numbers and Histograms X1 = randn(2000,1); >> X2 = rand(2000,1); >> hist(X1,20) sums, means, var

X = randn(1000,1); >> Y = rand(1000,1); >> sum(X), mean(X), var(X) min(X), max(X) ans = -14.6247 ans = ans = -3.5851 -0.0146 ans = ans = 3.7476 1.0325 >> min(Y), max(Y) >> sum(Y), mean(Y), var(Y) ans = ans = 0.0010 491.2878 ans = ans = 0.9973 0.4913 ans = 0.0863 Matrices Again! A = [1 2 3; 4 5 6; 7 8 9] A*B A = ans = 1 2 3 3 2 4 4 5 6 9 2 10 7 8 9 15 2 16 >> B = [1 -1 1;1 0 0;0 1 1] B = >> A.*B ans = 1 -1 1 1 0 0 1 -2 3 0 1 1 4 0 0 >> rank(A) 0 8 9 ans = B./A 2 ans = >> eig(A) ans = 1.0000 -0.5000 0.3333 0.2500 0 0 16.1168 0 0.1250 0.1111 -1.1168 -0.0000 How to Write a Program?

• Script • Anything that you do in command line except in a single file • Functions • Get inputs, return outputs MATLAB for Machine Learning: Regression

Assume that y = f(x) + ! where E ! = 0. The goal is estimating the { } regressor, f(.), using samples (Xi, Yi) , i = 1, , n. There are books written on thi{ s topic,}but for·n· o· w, we consider a simple (but efficient) method called Kernel regression estimator. Let h > 0 be a positive number called the bandwidth. The Nadaraya-Watson kernel estimator is defined by

n 1 x xi fˆ(x) = K − Yi n x xj h K − i=1 j=1 h $ % & ! " # Kernels can have different forms. An example of them is Gaussian kernel: 1 K(x) = exp( x2/2). √2π −

See L. Wasserman, All of Nonparametric Statistics (Section 5.4) for more information. Let’s implement it! What Do We Need?

• Some function f(x) • Noisy samples from f(x) • Kernel regressor • kernel function (e.g. Gaussian) A Script File as the Skeleton of the Program edit KernelRegressionTest First Attempt

KernelRegressionTest.m KernelRegressor.m

% Number of samples function yhat = KernelRegressor(xQuery,XTrain,YTrain,h) n = 1000;

% Noise variance sigma = 1; SizeOfTrainingSet = size(XTrain,2);

% Bandwidth yhat = 0; h = 0.05; Normalizer = 0; for m=1:SizeOfTrainingSet Kxi = exp( -(xQuery - XTrain(:,m))^2/h ); xSamples = (rand(1,n)-0.5)*8; yhat = yhat + Kxi*YTrain(m); xGrid = linspace(-4,4,2000); Normalizer = Normalizer + Kxi; end ySamples = 2*sin(2*xSamples) + 0.5*cos(5*xSamples) + sigma^2 * randn(1,n); yTrue = 2*sin(2*xGrid) + 0.5*cos(5*xGrid); yhat = yhat/Normalizer; plot(xSamples,ySamples,'.') hold on; plot(xGrid,yTrue,'r','Linewidth',3);

Let h > 0 be a positive number called the bandwidth. The Nadaraya-Watson

% Evaluating the regressor on a set of test points kernel estimator is defined by for m=1:length(xGrid) n yTest(m) = KernelRegressor(xGrid(:,m),xSamples,ySamples,h); 1 x xi fˆ(x) = K − Yi end n x xj h K − i=1 j=1 h $ % & " # plot(xGrid,yTest,'g','Linewidth',2); ! First Attempt tic; KernelRegressionTest; toc @ Command line Elapsed time is 31.262343 seconds. profile on KernelRegressionTest profile report

Source of slowness: for loop. Vectorization

Avoid loops; vectorize! FOR loop solution: tic; i = 1; for x=0:0.001:8*pi; y(i) = sin(x); i = i+1; end; toc Elapsed time is 9.641997 seconds. Sin #2: Incremental growing of vectors Sin #1: for loop Vectorized solution: >> tic; x = 0:0.001:8*pi; y = sin(x); toc A = [1 2 3;4 5 6] Elapsed time is 0.013867 seconds. A =

1 2 3 4 5 6 >> repmat(A,3,2) Related trick: ans = 1 2 3 1 2 3 4 5 6 4 5 6 1 2 3 1 2 3 4 5 6 4 5 6 1 2 3 1 2 3 4 5 6 4 5 6 Vectorization

What can be done for the previous code?

for m=1:SizeOfTrainingSet Kxi = exp( -((xQuery - XTrain(:,m))^2)/h ); yhat = yhat + Kxi*YTrain(m); Normalizer = Normalizer + Kxi; end

scalar XTrain is originally a vector, but we de-vecotorize it to a scalar here. Let’s vectorize xQuery.

xQueryRepeated = repmat(xQuery,1,SizeOfTrainingSet); and do all calculations at once.

KX = exp( -((xQueryRepeated - XTrain).^2) / h); yhat = sum(KX.*YTrain)/sum(KX); Second Attempt

function yhat = KernelRegressor(xQuery,XTrain,YTrain,h) tic; KernelRegressionTest; toc

Elapsed time is 0.514618 seconds.

SizeOfTrainingSet = size(XTrain,2);

% First solution (slow) if 1>2 yhat = 0; Normalizer = 0; for m=1:SizeOfTrainingSet Kxi = exp( -((xQuery - XTrain(:,m))^2)/h ); yhat = yhat + Kxi*YTrain(m); Normalizer = Normalizer + Kxi; end

% Second solution (faster) else

xQueryRepeated = repmat(xQuery,1,SizeOfTrainingSet); KX = exp( -((xQueryRepeated - XTrain).^2) / h); yhat = sum(KX.*YTrain)/sum(KX); end Remarks on Kernel Regressor

• Easily extendable to multi-dimensions • Selecting the bandwidth is important • Model selection • Theoretical results Other Useful Functions, Commands, ... • inv(A), pinv(A), det(A), cond(A), svd(A) • & (logical AND), | (logical OR), ~ (logical NOT) • clear, whos, load, save • help • surf, mesh, plot3, comet, ... • Lots of other functions (ODE, optimization, control toolbox, etc.) Resources

• MATLAB’s help (command line’s help or the manual) • E. Pekalska and M. van der Glas, Introduction to MATLAB, 2002. (http://www.cs.ualberta.ca/ ~dale/cmput466/w06/matlab_manual.pdf) • MATLAB 7.6 demos (http:// www.mathworks.com/products/matlab/ demos.html) • Many others