Program: Try Your Own Pivoting Scheme

MCS143M Program: Try your own pivoting scheme L. Foster Due October 17

Modify my pseudocode for complete pivoting to try out your own pivoting scheme. In your code you will change the section of the algorithm below commented as “select the pivot element”. You might try worst partial pivoting (the element in column k closest to zero but not equal to zero), worst complete pivoting, rook pivoting, double partial pivoting (search the kth row and kth column), diagonal pivoting (search the diagonal on or below a(k,k)), abbreviated pivoting (search only s < n elements in column k, if they are all zero search the next s, etc.) or some other pivoting scheme (knight pivoting, pawn pivoting, ??). The scheme you try must be different from partial, complete or no pivoting.

Implement and test the code with one of three approaches (you select which approach you prefer): (1) As a Matlab function. You will need to translate the pseudocode into Matlab syntax. (2) In C and use Matlab’s mex utility to call your code from Matlab. To help you implement the mex utility I have posted four files below at http://www.math.sjsu.edu/~foster/m143m/math143mf12.html . The files have been tested on Microsoft SDK 7.1, Microsoft Visual C++ 2005, lcc (which is a C compiler provided with 32 bit Matlab version 7) and gcc 4.4. The computers in MH 221 have SDK 7.1 installed. If you use your own 64 bit computer it is probably easiest to install SKD 7.1 (see http://www.mathworks.com/support/compilers/R2012a/win64.html.) With a 32 bit computer and Matlab 7.0 or later is it easiest to use lcc, which comes with 32 bit Matlab. With Matlab 5.3i t may be hard to locate a suitable C compiler for this old version of Matlab (see http://www.mathworks.com/support/sysreq/release11/index.html,)  gecp.m - comments the describe how to use the files  gecp.c – A Matlab mex “gateway” function that facilitates calling C subroutines from Matlab  gecpf.c – A C function implementing the attached pseudocode. Note: knowledge of Matlab mex utility can be a marketable skill. The mex utility allows one to combine the nice Matlab environment with more efficient compiled code. If you do part (2) include a short paragraph in your report discussing the purpose of the Matlab gateway function (to force you to read the comments in the gateway function). (3) Another language of your choice. Only C, C++, and Fortran can be linked to Matlab. If you use a language and do not link it to Matlab using the mex utility, you will need to write your own utilities to test your code. For example you will want a function to multiply a matrix by a vector, a function to fill up a matrix with random entries and a function to find the norm of a vector. For example the key code to fill up a matrix with random entries between 0 and 1 in C++ is: srand( (int) time(0)) // sets the random number seed ( include the cstdlib and ctime libraries) for (i = 0; i <= n ; i++ ) { for (j = 0; j <= n; j++ ) { a[i][j] = rand( ) / RAND_MAX; // in Java Math.random( ) generates random numbers } }

You should try small examples where you know the answer and also random examples. I suggest that you choose b so that the solution x is known. For example for a matrix A you might choose a vector xtrue all of whose components are one (or choose xtrue = randn(n,1), a vector of random values) and calculate b by multiplying A by xtrue. You can then see how close the solution x that you calculate is to xtrue by calculating norm( x – xtrue).

I would like you to choose examples that test your code. I would suggest examples from class and perhaps other sources. A requirement is that some of your matrices be larger (larger than 100 by 100) random matrices. If you use Matlab it is easy to create a loop that can generate many random matrices to try.

Write a report that discusses your test runs in three parts:

 Discuss examples that show that your code works. Clearly write up (in English paragraphs) a discussion of these examples that you did and why they indicate that your code works. Include a discussion of why you chose the examples you did. The reader should not have to refer to your print out to understand what testing you did and why it indicates that your code is correct. Include a test run with the matrix whose first row is 1 1 1, second row is 1 1 0 and third row is 3 2 1. Your program should work correctly for this 3 by 3 matix. This section of your code should convince the reader that you code successfully implements your pivoting scheme on these small examples. Note that if you want to print out an intermediate result, such as the final A, in a Matlab function, perhaps for debugging or to illustrate the pivoting, leave the semicolon off of the end of a line – e.g. a line with just “A” and no ending semicolon.. One can print out results from inside gecpf.c using the mexprintf command. There is an example of mexprintf in gecp.c. Also gecp allows one to return the final A after Gaussian elimination.  Discuss examples that show unusual error growth in the solutions that you calculate, if there is any. For example you can compare the error in your calculated solution to the error in the solution calculated by complete pivoting (implement my psuedocode below unchanged). If xtrue is the true solution, x is your solution and xoracle is the solution with a known stable algorithm (I recommend complete pivoting for your oracle) you are interested in the ratio norm(x – xtrue) / norm(xoracle - xtrue). I suggest that you make a table of these ratios for a variety of matrices including some larger random matrices. If this ratio is substantially bigger than one then your solution would appear to have error growth substantially worse than xoracle.. If you get large error growth try to explain why. If you can not get large error growth, discuss the runs that you did and try to explain why your algorithm did not get unusual error growth. In this section, among other examples, I suggest that you try the matrix generated by the Matlab code “ n=60; A=gallery('triw',n); A = A' ; A(:,n)=ones(n,1); xtrue = randn(n,1); b = A*xtrue;”. This is an example where Gaussian elimination with partial pivoting and the Matlab code “x = A \ b” will fail (see problem 2 item 21, hwgrowthandAinverse.doc , at http://www.math.sjsu.edu/~foster/m143m/math143mf12.html).

 Compare the run times for your pivoting method, complete pivoting (the attached pseudocode or gecpf.c implemented without changing the pivoting) and Matlab’s backslash. To time code with some reliability you will need to use matrices that are large enough so that the run takes at least a few seconds and you should also average the time of a few runs with the same matrix. For example : “tic, for i=1:5 , x =gecp(A,b); end, t =toc; time = t/5” with calculate the average time for calling gecp. Make sure that there are no other applications active while you time a routine. Discuss the relative times, including reasons that one code is faster or slower than the others.

Again write your report in good English. The report should have an introduction (usually with the purpose of your investigation), body and conclusion. Paragraphs should have topic sentences, etc. Your report can have tables summarizing your results but should not have printouts of matrices or vectors unless they are relatively small. In an appendix include a listing of your code. If you feel that you need details of any runs they should be in the appendix.

Extra credit (+2): implement your pivoting scheme in Matlab code and C (or another language). Focus your report on only one of these implementations, but include a section that briefly compares the run times of the two implementations. Pseudocode (to be changed to Matlab or other code) for Gaussian elimination with complete pivoting function x=gecp(a,b) (or function [x,a]=gecp(a,b) to return the final A)

// This function receives an n by n square matrix A and a vector b. // It returns the solution x calculated by Gaussian elimination with // complete pivoting. If the matrix is singular x will have components // equal to infinity (inf) or not a number (NaN). n = length(a) // the dimension of A

// initialize vectors to keep track of the pivot elements for i = 1 to n lambda(i) = i // lambda will keep track of the row switches omega(i) = i // omega will keep track of the col switches endfor for k = 1 to n // zero out column k

// select the pivot element amax = abs( a(k,k) ) // note C++ would use fabs not abs imax = k jmax = k for i = k to n for j = k to n temp = abs( a(i,j) ) if ( temp > amax) amax = temp imax = i jmax = j endif endfor endfor

// switch rows imax and k for j = 1 to n temp = a(k,j) a(k,j) = a(imax,j) a(imax,j) = temp endfor // switch column jmax and k for i = 1 to n temp = a(i,k) a(i,k) = a(i,jmax) a(i,jmax) = temp endfor

// change lambda and omega to remember the pivots temp = lambda(k) lambda(k) = lambda(imax) lambda(imax ) = temp

temp = omega(k) omega(k) = omega(jmax) omega(jmax) = temp

// now we start the elimination if ( a(k,k) is not equal to 0) // when a(k,k) is zero, column k already 0 below the diagonal

// calculate the multipliers and store in a(i,k) for i = k+1 to n a(i,k) = a(i,k) / a(k,k) endfor

// update a for j = k+1 to n for i = k+1 to n a(i,j) = a(i,j) – a(i,k) * a(k,j) endfor endfor endif endfor

// The matrix is now factored, now solve the system

// Permute the right hand side as indicated by lambda for i = 1 to n bh(i) = b( lambda(i) ) endfor

// Solve the lower triangular system Lc = b for i = 1 to n sum = bh(i) for j = 1 to i-1 sum = sum – a(i,j)*c(j) endfor c(i) = sum endfor

// Solve the upper triangular system, Uz = c for i = n downto 1 // i = n:-1:1 in Matlab sum = c(i) for j = i+1 to n sum = sum – a(i,j)*z(j) endfor z(i) = sum / a(i,i) // if a(i,i) is 0, z(i) is NaN or inf endfor

// permute the z vector to get the solution x (this // step is required if there were any column switches for i = 1 to n x( omega(i) ) = z(i) endfor

// That’s all folks!

A suggestion for C or C++. The indices in all the for loops should be reduced by one since C and C++ start array indices at 0. Matlab and Fortran start indices at 1.